Repository navigation
Add TFG automated assessments analysis site and tools - #10
Merged
Merged
Conversation
Copies tfg-rules-and-checks-site into
docs/analysis/2026-08-automating-technical-hardening-guidance/ with no
edits, so the next commit's relocation changes are reviewable on their own.
Excluded from the copy:
*:Zone.Identifier, *:sec.endpointdlp 129 NTFS alternate-data-stream
files that materialised on the
WSL side; not content
__pycache__/, *.pyc, build/ generated
The site is unbuilt in this state: tools/manifest.yaml still points at
../../tfg-automated-assessments, which does not resolve from this depth.
The site was written to sit at a repository root with the corpora as its
sibling. Nested under docs/analysis/ neither holds, so:
corpora tools/extract.py grows corpora_root(), which reads TFG_CORPORA
first and falls back to the manifest path. Four call sites
resolved this independently, two of them hardcoded; they now
share one. No relative path reaches every caller's checkout,
because the corpora are not this repository's sibling either.
CI .github/workflows/verify.yml moves to the repository root as
verify-2026-08-hardening-guidance.yml, scoped by a paths filter
so it runs on the analysis folder rather than on every push.
The repository is checked out under site/ so the corpora stay
outside the working tree the byte-identical check git-diffs.
Two defects found by moving, both pre-existing:
- verify.py never created build/grayscale before writing 16 renders to
it. build/ is gitignored, so --diagrams failed on any fresh clone.
- CI installed pyyaml only. verify.py imports svgrender imports PIL at
module scope, so the harness could not have started. Pillow and
fonts-dejavu-core, which svgrender loads by absolute path, are now
installed.
Verified from the new location: 2631/2639, all 8 generators byte-identical.
The 8 failures are 5 NIST schema 404s, 2 unreachable namespace URIs and a
missing local oscal-cli; all 8 reproduce against the unmoved original.
The site shipped its CIS material and its OSCAL tree between the README
being written and the build ending, and the README still described the
state before that.
- 17 documents under sources/ is 16, and 29.3 MB is 29.0 MB
- sources/oscal/ and sources/cis/ are no longer gitignored. oscal has
no row because sources_files.py excludes it from the inventory, not
because it is absent
- the paragraph telling a reader to download the Benchmarks from
cisecurity.org to rebuild described a repository that no longer
needs it. Replaced with the basis BUILD-LOG records, and with the
fact that a fork outside that conversation does not inherit it
- compare.html has been scenario.html since before the build ended
(README twice, CORRECTIONS.md template once)
Also brought forward from the migration: pillow is required, TFG_CORPORA
is how the corpora are found, and the workflow has moved and been renamed.
The Foundation's primary, #4E79D8, is 4.16:1 on the light background and
4.36:1 on the dark one. verify.py --a11y holds a link to 4.5 in both, so
adopting the brand colour as the accent would have failed the check this
site's stylesheet is written around.
Its hue and saturation are adopted instead, hue 221 and 0.64, and only
lightness moves — to wherever the contrast the site already had comes
back:
was now
light #0B5FA5 6.57:1 #2958BC 6.53:1
dark #6EB5F5 8.29:1 #96B0E8 8.36:1
Every accent pair is within 0.07 of where it was, and the hue is the
Foundation's rather than the cyan-leaning 205 it had been. --focus tracks
--accent as it did before.
The six question hues, the three approach hues, the layer hues and the
code surface are untouched: each is held to a measured property that a
brand match would have cost. --css and --a11y pass 158/158; the diagrams
carry var() rather than baked values, so nothing regenerates.
Both variants the Foundation ships, downscaled from 1024x472 to 208x96 and served at 2.25rem: the blue mark on the light theme, the reversed white one on dark. Two <img> rather than <picture>, because the theme here is a data-theme attribute a reader toggles rather than a media query, and <picture> cannot follow that. The mark is the one thing on the page drawn in the brand blue at full strength, 4.16:1. WCAG exempts a logotype from the ratio it holds text to, which is the reason --accent had to move off that colour and this did not. alt="" on both: the link is already named by the title beside them, and a second accessible name would be announced twice. The header is written in six places -- assets/shell.html, the copy in tools/approach_pages.py, and the four pages no generator owns -- so all six carry it and the four generated pages were rebuilt. All 8 pages now show the mark. 2631/2639, unchanged from before the theme work.
docs/ is the site and there is no build step. The deploy workflow adds
one thing: summit/ copied to patterns/summit/, so the pages that link the
artifacts are same-origin with them rather than pointing at a
branch-pinned raw.githubusercontent URL.
docs/ -> _site/
summit/ -> _site/patterns/summit/
Three areas, each with its own lifecycle:
patterns/ the worked examples. Names all seven models and says
which two are published, because which models an
example is missing is part of what it is
analysis/ one dated area per effort, rendered from
analyses.json. Adding an effort means adding an area
and appending to that file; no page is edited
recommendations/ an empty registry and a page that says so. Built now
rather than when the first one lands, so the shape is
proven while it costs nothing
The recommendations page states that the open analysis makes no
recommendation by design rather than leaving a reader to infer one is
pending. analysis.json carries makesRecommendation: false for the same
reason.
Two guards run before deploy: every path is relative, because a project
site is served under /Pattern-Library/ and an absolute path resolves
against the wrong root; and every analysis the registry names is
published with its metadata, because a registry entry pointing at
nothing renders as an ordinary card with a dead link.
The shell follows prefers-color-scheme rather than carrying a toggle.
An analysis keeps its own toggle and its own stored choice, so crossing
between them can change the theme; that is the cost of each analysis
owning its look and being frozen with it.
Verified against the assembled tree: 16 paths resolve, both registries
render, the empty state shows, and the analysis site serves from its
nested location with no broken asset. The harness is unmoved at
2631/2639.
- Introduce `serve.py` to serve the Pattern Library site locally. - Create VS Code launch configurations for site and analysis debugging. - Add tasks for serving the site and managing analysis workflows. - Update `.gitignore` to exclude the assembled preview tree. - Enhance README with instructions for running the site.
Contributor
|
After conversation with @pjavan , we agreed to publish this as-is so that it can be published on Pages and people can review online. this is a work in progress and additional PRs will be issued with updates. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add analysis and recommendation areas
Important
Work in progress, not final. Opened for review of the structure and the lifecycle rules. Details below will still move.
The repository currently holds worked examples only. There is no place to record the analysis behind them, or the recommendations that analysis produces.
This PR adds support for analysis, recommendations with the first analysis included.
The lifecycle
Analyses. An analysis is an effort that debates one question about OSCAL. Each gets its own dated area under
docs/analysis/and is retained after it finishes, so a later reader can see what was weighed.Status:
draft,active,concluded,superseded.Recommendations. A recommendation is what an analysis produces when it reaches one. It lives in its own area under
docs/recommendations/so it can be cited directly and superseded independently of the analysis behind it.Status:
proposed,accepted,superseded.Retention. An area is never renamed, moved or deleted, and a concluded analysis is not edited to match a later view. A superseding effort gets its own area, and the old one is marked as superseded by it.
Linking. An analysis names the recommendation that settled it, and a recommendation names the analysis it came from. Concluding an analysis sets its
recommendationfield and stampsconcluded, and its page then renders a banner pointing forward.Adding an analysis
Each index renders from a registry file, so adding an effort means adding a folder and appending one object. No page is edited and no build runs.
docs/analysis/analyses.jsondocs/recommendations/recommendations.jsonEach analysis also carries an
analysis.jsonbeside itsindex.htmlholding the question, status, dates, the approaches compared, and the recommendation it produced.Recommendations ships empty
There are no recommendations yet. The area is built anyway, with an empty registry and a page that states the position, because the open analysis makes no recommendation by design rather than because it is unfinished. A blank page would invite a reader to assume a verdict is pending.
analysis.jsoncarriesmakesRecommendation: falsefor the same reason.The first analysis
Automating Technical Hardening Guidance with OSCAL, from the OSCAL Foundation Technical Focus Group on Automated Assessments. Opened 2026-08-12, active.
Three organizations have published working OSCAL content answering this three different ways. The site puts them side by side against the same six questions (Catalog-first, Component-first, Assessment-first) and writes the same two rules out in all three shapes, so the only thing differing between the columns is the modelling rather than the subject matter.
Eight pages. Published guidance from CIS, DISA, CISA and AWS is inventoried as input: 16 documents, 29.0 MB, one row per benchmark with its provenance.
VS Code
Press F5. The
Pattern Libraryconfiguration runs thesite: servetask, which servesdocs/on port 4173, then opens the landing page in Edge with the debugger attached. Breakpoints indocs/assets/site.jsor in an analysis's scripts bind to the served file. Ending the session stops the server.Pattern Library(Edge / Chrome)Debug the 2026-08 analysis harnessverify.pysite: serve/site: stopanalysis 2026-08: verifyanalysis 2026-08: regenerateThe serve task is a stock
python3 -m http.serveroverdocs/, so nothing in this repository serves the site. Any static server works the same way. Live Preview is recommended inextensions.jsonwith its server root already set todocs/; installing it replaces the task.Opening a page from the filesystem does not work, because each index reads its registry over
fetchand a browser blocks that on afile://origin. The pages report it rather than appearing empty.CI/CD
pages.ymlruns on a push tomainthat touchesdocs/, and publishesdocs/to GitHub Pages. There is no build step: what is in the folder is what is served. Two guards run first:/Pattern-Library/, so an absolute path resolves against the wrong root.verify-2026-08-hardening-guidance.ymlruns that analysis's own harness, and apaths:filter scopes it to that analysis's folder. It runs twice: an offline pass covering what a contributor can reproduce on a laptop, then--strict, which counts every skipped check as a failure so a skip cannot go unnoticed. The schedule reruns it weekly, since the external schemas and links it checks can change without anyone touching the repository.Each analysis brings its own workflow on the same naming convention.
Notes for reviewers
github-pagesenvironment lint warning clears on first deploy.BUILD-LOG.md, and each file carries the basis in its own terms line. A fork made outside that conversation does not inherit it.not_shippedentry for CIS indata/source-files.json(nothing renders it, but it is a stale licensing assertion), and no favicon.