Skip to content

Add Executable-first, a fourth approach to the hardening-guidance analysis - #15

Open
christoph-puppe wants to merge 8 commits into
OSCAL-Foundation:mainfrom
christoph-puppe:claude/practical-franklin-6rcnnr
Open

christoph-puppe wants to merge 8 commits into
OSCAL-Foundation:mainfrom
christoph-puppe:claude/practical-franklin-6rcnnr

Conversation

@christoph-puppe

Copy link
Copy Markdown

Summary

The hardening-guidance analysis compared three published approaches to getting security checks into OSCAL. This PR adds Executable-first as a fourth. In it, the check's code sits in the OSCAL document as an executable assessment-method part on the control, next to an assessment-objective part that holds the rule. It also reworks section 6 of the start page around the question of what each approach's OSCAL says about the check.

  • Fourth approach. It comes from the executable assessment methods concept note. An earlier version was called "profile-first". It is now named for what it carries, and three open questions have been added.

  • Generated corpus. tools/executable_first_corpus.py builds OSCAL files from the CIS Benchmark and the DISA STIG for Ubuntu 24.04 LTS that the repo already holds:

    • a catalog with 312 recommendations, each with an objective and an executable method
    • four profiles and a mapping collection
    • a STIG profile adding 194 rules to 79 controls in the NIST Rev 5 catalog

    All the files validate against OSCAL 1.2.1, and every relative reference is percent-encoded so that jsonschema validation passes.

  • Check axis. A new table in section 6 shows whether each of the four approaches carries a reference an engine resolves, a description that may hold a script, or a declared executable. It includes side-by-side pairs for the same rule: CIS 1.1.1.1 and STIG V-270747.

  • Three decisions. Section 6 now presents code in OSCAL as three decisions: what the document carries, which construct carries it, and where it sits. The construct decision compares Rules, a typed assembly and the released method part aspect by aspect. The placement decision is a grid with six cells. The Rules shape is generated from the CIS benchmark as the site's own encoding.

  • Verification. verify.py gets new --corpus, --carrier and --decisions phases. These recompute every figure and check every table cell against the files. pagecheck.js asserts what was rendered.

The branch has upstream main merged in, so it includes #13. It replaces #14, which I closed before those merges.

Test plan

  • tools/verify.py, all phases pass (including --corpus, --carrier, --decisions)
  • All the generated files validate against the OSCAL 1.2.1 schema
  • Page check passes on the rebuilt site; section 6 and the Executable-first page render correctly

🤖 Generated with Claude Code

…methods concept note

The hardening-guidance analysis compared three published approaches. A concept
note proposes a fourth: the check as an executable assessment-method part on
the control, beside an assessment-objective part that is the rule, carried by
the catalog when the requirement's author owns the check and added by a profile
otherwise, with findings targeting the objective.

Option D, Profile-first, now sits on every page the other three are on: its own
approach page, a column on the six questions page with the two rules written in
the note's shape, a chain figure, a scenario figure and file set, an open
questions section, an entry on the artifacts page, and a card on the start page.

The approach has no corpus, so its cells declare no extracts, its status
annotation says it is a concept note, the corpus-reading checks skip it by
name, and the note itself is held under examples/profile-first and linked. A
fourth hue at the same lightness and a fourth marker shape keep the palette
rules; the checks that counted to three now read the count from the data or
name the fourth. The build log records every decision.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
The fourth approach was described from a concept note and no published
OSCAL. This writes a corpus in the note's shape from guidance the
repository already holds, and brings the site onto it.

tools/profile_first_corpus.py builds seven files under
examples/profile-first/oscal from the CIS Benchmark and the DISA STIG for
Ubuntu 24.04 LTS. The benchmark becomes a catalog whose 312
recommendations each carry an assessment objective, an assessment method
with the note's props (platform, language, evaluation, pass condition),
links to the Script Check Engine scripts as back-matter resources, and the
exported values as parameters; its four profiles and its own SP 800-53
references become four profiles and a mapping collection. The STIG becomes
a profile that imports the NIST Revision 5 catalog and adds an objective
and a method for each of its 194 rules to the 79 controls its CCIs map to.
All seven validate against OSCAL 1.2.1.

The site reads the corpus like any other: ten extracts at pointers into it
via a site: manifest source, the inventory page lists its files under the
note that says whose they are, and the conformance check validates every
file. A new --corpus phase recomputes the corpus's structure, links,
parameters and identifiers, fetching the NIST catalog for the last of
these when the network allows. Prose, README, build log and corrections
file say the corpus is the site's, not a proponent's.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
The site compared the four approaches by where the rule lives and never
said, in one place, that their answers to the check question are three
kinds of thing: a reference an engine resolves, a description that may
hold a script, or a declared executable with props saying what runs it
and how the output is judged. Section 6 of the start page now tables the
four on that axis, shows the case where rule and script sit on one
control, and puts the same rule from two corpora side by side: CIS
recommendation 1.1.1.1 as an assessment plan step and as a method part,
and STIG rule V-270747 the same way.

data/check-carrier.json carries the rows, the pairs, the consequences
per kind and the figures. A new --carrier phase checks that each row's
kind is visible in the extract it cites, that both sides of a pair hold
the same rule, and recomputes every figure from the plan and the catalog
held under examples/. The page check asserts what the renderer made of
it. The six questions page points at the section.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
Section 6 of the start page now leads with the proponent's thesis, that
the check's code belongs in the OSCAL document, and states the three
decisions under it: what the document carries about the check, which
construct carries it, and where it sits. The construct decision puts
Rules, a typed assembly with the published rules and checks and the
April automation-scripts as one row, against the released
assessment-method part, aspect by aspect with consequences and no
ranking. The placement decision crosses construct with control,
component and activity in a six-cell grid whose cells name the held
documents by inventory key and carry a status the harness checks.

The generator writes the CIS benchmark a second time in the Rules shape:
a validation component with automation-scripts objects on requirements
keyed to CIS Controls v8, expected to fail validation on the proposed
field. Conformance asserts that the assessment-method files use no
undefined assembly and that this one uses only the proposed one; the
corpus phase asserts its audit scripts are the catalog's inline scripts
as a set. A --decisions phase recomputes the figures and checks every
cell against the files. Two extracts from the new file feed the page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
Every reference to a document the site does not hold is removed. The
Rules-shape component definition the generator writes is described as
the site's encoding of the Rules construct carrying code, written from
the CIS benchmark; the unheld placeholder leaves the Rules row and the
grid cell; the thesis states the idea without a history; the generator's
remarks, the harness comments and the READMEs follow.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
The fourth option is named by what it carries rather than by the model
the rule first appears in: the check's code, in the OSCAL, on a control
in a catalog or a profile or in a typed assembly on a component. The
key, the label, the colour token, the marker's owner, the page, the
corpus directory, the generator and the two diagrams are renamed, and
the start page's section 4 states the exception to the naming. The
primary model stays the profile, since the mechanism the note turns on
and everything built on it is unchanged.

Three open questions join the option's section, first in the list: a
typed assembly or standardised props on the released part; which
assemblies may carry an executable; and how an executable on a control
reaches the plan of record. Each points at the section 6 decision it
comes from.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
Upstream replaced the external corpora directory with a source lock and an
explicit public fetch, added the progress banner, added regression tests
for the verifier, the source resolver and the workflow, dropped the
quotation register, and rewrote the three approaches' prose shorter.

The fourth approach is carried across: site-held extracts keep their site:
prefix ahead of the lock, the inventory scans the generated corpus by
path, conformance validates every generated file with the published
schemas and expects the Rules-shape file to fail, the corpus phase reads
the committed NIST catalog offline, the markdown extract of the concept
note is dropped because the manifest is JSON only, the snippet count in
the tests is 43, and the fourth page's prose is trimmed to the equal
budget the rewritten pages set. Every page and the shell carry the banner
and the new footer with the count corrected to four; the workflow keeps
the corpus generator and its outputs in the generated-file check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
The published schema types every rlink href as a URI reference, and the
STIG profile's reference to the assessment plan it took its mapping from
runs through a directory name with spaces. The jsonschema path CI now
validates with rejects it where trestle did not, so the generator
percent-encodes every relative reference it writes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants