Skip to content

feat: implement mapping integrity validator for CWE and MITRE ATLAS - #147

Open
rashim27us wants to merge 1 commit into
GenAI-Security-Project:mainfrom
rashim27us:feature/mapping-integrity-validator
Open

rashim27us wants to merge 1 commit into
GenAI-Security-Project:mainfrom
rashim27us:feature/mapping-integrity-validator

Conversation

@rashim27us

Copy link
Copy Markdown

Summary

This PR adds the first phase of automated cross-framework mapping integrity validation for the 2026 release, starting with CWE 4.20 and MITRE ATLAS 2026.06 as requested in #132.

  • Adds a local and CI validator that checks mapping catalogs against version-pinned primary datasets, including identifier existence, duplicate IDs, title/metadata consistency, source version checks, pinned ATLAS source enforcement, and SHA-256 verification via new source_sha fields.
  • Updates cwe-4.20.json and mitre-atlas-2026.06.json to pin machine-readable source URLs and record the verified source digests, replacing the CWE homepage URL with the version-specific XML archive.
  • Adds offline negative tests, a GitHub Actions workflow, and mapping documentation so future framework validators can follow the same pattern.

Test plan

  • Run python .github/scripts/test_validate_mappings.py
  • Run python .github/scripts/validate_mappings.py --download
  • Confirm both 2026/final/mappings/cwe-4.20.json and 2026/final/mappings/mitre-atlas-2026.06.json pass validation
  • Verify negative cases fail for invalid IDs, duplicate IDs, title mismatches, deprecated ATLAS source paths, version mismatches, and SHA mismatches

@avp9-nexus

Copy link
Copy Markdown

I ran the seven files of this PR at ce3efca in a sandbox, since the two notes from #132 are what it implements.

What holds, measured. The 8 offline tests pass with no network. With the network on, both catalogs pass, and the two fetched sources hash to the recorded source_sha values (ATLAS at 625,790 bytes, the size the mapping notes already give for the July read). As a control on the real file, I set the title of AML.TA0000 in mitre-atlas-2026.06.json to the stale "ML Model Access": the validator fails with a title mismatch, and passes again once the file is restored. So the stale default path and the unpinned bytes are both closed.

One thing does not match the premise of #132, "avoiding dependence on live external APIs during CI". The workflow runs validate_mappings.py --download, .cache/ is ignored and no step restores it, so every run fetches both sources. With the network off and an empty cache, that command exits 1 and prints FAIL for both catalogs with "could not download". An unreachable host and an invalid mapping get the same verdict and the same exit code, so an outage at cwe.mitre.org would turn every PR that touches the mappings red.

Two small ways out. A separate outcome for "source unavailable", with its own message and exit code, so CI can tell "not checked" from "wrong". Or a cache step for .cache/mapping-sources, so runs reuse the pinned bytes: the digest check already makes a cached copy safe to trust. The first one fixes the meaning, the second only makes it rarer.

Avp9.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants