Repository navigation
Add Executable-first, a fourth approach to the hardening-guidance analysis - #15
Open
christoph-puppe wants to merge 8 commits into
Open
christoph-puppe wants to merge 8 commits into
christoph-puppe wants to merge 8 commits into
Conversation
…methods concept note The hardening-guidance analysis compared three published approaches. A concept note proposes a fourth: the check as an executable assessment-method part on the control, beside an assessment-objective part that is the rule, carried by the catalog when the requirement's author owns the check and added by a profile otherwise, with findings targeting the objective. Option D, Profile-first, now sits on every page the other three are on: its own approach page, a column on the six questions page with the two rules written in the note's shape, a chain figure, a scenario figure and file set, an open questions section, an entry on the artifacts page, and a card on the start page. The approach has no corpus, so its cells declare no extracts, its status annotation says it is a concept note, the corpus-reading checks skip it by name, and the note itself is held under examples/profile-first and linked. A fourth hue at the same lightness and a fourth marker shape keep the palette rules; the checks that counted to three now read the count from the data or name the fourth. The build log records every decision. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
The fourth approach was described from a concept note and no published OSCAL. This writes a corpus in the note's shape from guidance the repository already holds, and brings the site onto it. tools/profile_first_corpus.py builds seven files under examples/profile-first/oscal from the CIS Benchmark and the DISA STIG for Ubuntu 24.04 LTS. The benchmark becomes a catalog whose 312 recommendations each carry an assessment objective, an assessment method with the note's props (platform, language, evaluation, pass condition), links to the Script Check Engine scripts as back-matter resources, and the exported values as parameters; its four profiles and its own SP 800-53 references become four profiles and a mapping collection. The STIG becomes a profile that imports the NIST Revision 5 catalog and adds an objective and a method for each of its 194 rules to the 79 controls its CCIs map to. All seven validate against OSCAL 1.2.1. The site reads the corpus like any other: ten extracts at pointers into it via a site: manifest source, the inventory page lists its files under the note that says whose they are, and the conformance check validates every file. A new --corpus phase recomputes the corpus's structure, links, parameters and identifiers, fetching the NIST catalog for the last of these when the network allows. Prose, README, build log and corrections file say the corpus is the site's, not a proponent's. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
The site compared the four approaches by where the rule lives and never said, in one place, that their answers to the check question are three kinds of thing: a reference an engine resolves, a description that may hold a script, or a declared executable with props saying what runs it and how the output is judged. Section 6 of the start page now tables the four on that axis, shows the case where rule and script sit on one control, and puts the same rule from two corpora side by side: CIS recommendation 1.1.1.1 as an assessment plan step and as a method part, and STIG rule V-270747 the same way. data/check-carrier.json carries the rows, the pairs, the consequences per kind and the figures. A new --carrier phase checks that each row's kind is visible in the extract it cites, that both sides of a pair hold the same rule, and recomputes every figure from the plan and the catalog held under examples/. The page check asserts what the renderer made of it. The six questions page points at the section. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
Section 6 of the start page now leads with the proponent's thesis, that the check's code belongs in the OSCAL document, and states the three decisions under it: what the document carries about the check, which construct carries it, and where it sits. The construct decision puts Rules, a typed assembly with the published rules and checks and the April automation-scripts as one row, against the released assessment-method part, aspect by aspect with consequences and no ranking. The placement decision crosses construct with control, component and activity in a six-cell grid whose cells name the held documents by inventory key and carry a status the harness checks. The generator writes the CIS benchmark a second time in the Rules shape: a validation component with automation-scripts objects on requirements keyed to CIS Controls v8, expected to fail validation on the proposed field. Conformance asserts that the assessment-method files use no undefined assembly and that this one uses only the proposed one; the corpus phase asserts its audit scripts are the catalog's inline scripts as a set. A --decisions phase recomputes the figures and checks every cell against the files. Two extracts from the new file feed the page. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
Every reference to a document the site does not hold is removed. The Rules-shape component definition the generator writes is described as the site's encoding of the Rules construct carrying code, written from the CIS benchmark; the unheld placeholder leaves the Rules row and the grid cell; the thesis states the idea without a history; the generator's remarks, the harness comments and the READMEs follow. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
The fourth option is named by what it carries rather than by the model the rule first appears in: the check's code, in the OSCAL, on a control in a catalog or a profile or in a typed assembly on a component. The key, the label, the colour token, the marker's owner, the page, the corpus directory, the generator and the two diagrams are renamed, and the start page's section 4 states the exception to the naming. The primary model stays the profile, since the mechanism the note turns on and everything built on it is unchanged. Three open questions join the option's section, first in the list: a typed assembly or standardised props on the released part; which assemblies may carry an executable; and how an executable on a control reaches the plan of record. Each points at the section 6 decision it comes from. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
Upstream replaced the external corpora directory with a source lock and an explicit public fetch, added the progress banner, added regression tests for the verifier, the source resolver and the workflow, dropped the quotation register, and rewrote the three approaches' prose shorter. The fourth approach is carried across: site-held extracts keep their site: prefix ahead of the lock, the inventory scans the generated corpus by path, conformance validates every generated file with the published schemas and expects the Rules-shape file to fail, the corpus phase reads the committed NIST catalog offline, the markdown extract of the concept note is dropped because the manifest is JSON only, the snippet count in the tests is 43, and the fourth page's prose is trimmed to the equal budget the rewritten pages set. Every page and the shell carry the banner and the new footer with the count corrected to four; the workflow keeps the corpus generator and its outputs in the generated-file check. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
The published schema types every rlink href as a URI reference, and the STIG profile's reference to the assessment plan it took its mapping from runs through a directory name with spaces. The jsonschema path CI now validates with rejects it where trestle did not, so the generator percent-encodes every relative reference it writes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013h1RZCurTffvziZAcmU4kM
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The hardening-guidance analysis compared three published approaches to getting security checks into OSCAL. This PR adds Executable-first as a fourth. In it, the check's code sits in the OSCAL document as an executable assessment-method part on the control, next to an assessment-objective part that holds the rule. It also reworks section 6 of the start page around the question of what each approach's OSCAL says about the check.
Fourth approach. It comes from the executable assessment methods concept note. An earlier version was called "profile-first". It is now named for what it carries, and three open questions have been added.
Generated corpus.
tools/executable_first_corpus.pybuilds OSCAL files from the CIS Benchmark and the DISA STIG for Ubuntu 24.04 LTS that the repo already holds:All the files validate against OSCAL 1.2.1, and every relative reference is percent-encoded so that jsonschema validation passes.
Check axis. A new table in section 6 shows whether each of the four approaches carries a reference an engine resolves, a description that may hold a script, or a declared executable. It includes side-by-side pairs for the same rule: CIS 1.1.1.1 and STIG V-270747.
Three decisions. Section 6 now presents code in OSCAL as three decisions: what the document carries, which construct carries it, and where it sits. The construct decision compares Rules, a typed assembly and the released method part aspect by aspect. The placement decision is a grid with six cells. The Rules shape is generated from the CIS benchmark as the site's own encoding.
Verification.
verify.pygets new--corpus,--carrierand--decisionsphases. These recompute every figure and check every table cell against the files.pagecheck.jsasserts what was rendered.The branch has upstream
mainmerged in, so it includes #13. It replaces #14, which I closed before those merges.Test plan
tools/verify.py, all phases pass (including--corpus,--carrier,--decisions)🤖 Generated with Claude Code