Skip to content

feat(data): verification methods and links data model (refs #191) - #215

Merged
emmanuelgjr merged 3 commits into
GenAI-Security-Project:mainfrom
a-moskvin:feat/verification-data-model
Oct 9, 2026
Merged

emmanuelgjr merged 3 commits into
GenAI-Security-Project:mainfrom
a-moskvin:feat/verification-data-model

Conversation

@a-moskvin

Copy link
Copy Markdown

What this PR changes

Implements the verification-methods data model agreed in #191 (follow-up to #101 / #186). Data, schemas and validator only; no methods yet. The pilot follows in a separate PR.

  • data/verification-methods-schema.json, data/verification-links-schema.json: Draft-07 schemas
  • data/verification-methods.json: empty methods array
  • data/verification-links/: one file per framework (none yet)
  • scripts/verification.js: schema-derived checks plus cross-file rules; called from scripts/validate.js on full runs
  • scripts/verification.test.mjs: 34 tests; each rule has a breaking fixture, cross-checked against the schemas with Ajv
  • docs/VERIFICATION_METHODS.md: specification; data/README.md and CHANGELOG.md updated

No changes to generate.js, exports, webapp or the npm package contents. Regenerating produces no data changes.

Type of change

  • New mapping file
  • Update to existing mapping (content, controls, CVE refs)
  • Bug fix (broken link, typo, incorrect cross-ref)
  • New recipe (shared/RECIPES.md)
  • New tool (shared/TOOLS.md)
  • Infrastructure (scripts, CI, templates)
  • Translation (i18n/)

Source / evidence

#191 (follow-up to #101 / #186)

Checklist

  • npm run build passes: 0 errors, warnings unchanged from main
  • npm test passes: 129/129
  • CHANGELOG.md updated

Content

  • Follows the file template structure (header comment, H1, Why section, quick-reference table, audience
    tags, detailed per-entry mappings, references, changelog)
  • Severity ratings consistent with AIVSS / OWASP definitions in shared/SEVERITY.md
  • Cross-references are bidirectional — if this file mentions Agentic_X.md, that file mentions this one back
  • All referenced vulnerability IDs are valid (LLM01–LLM10, ASI01–ASI10, DSGAI01–DSGAI21)
  • License header present: CC BY-SA 4.0

Links & data

  • All internal .md links resolve to real files
  • All external URLs return 200 (checked manually or via lychee)
  • data/schema.json compatible (if adding a new entry type)

Project hygiene

  • Changelog entry added at bottom of every modified file (YYYY-MM-DD format)
  • CHANGELOG.md updated if this is a new mapping file (include in correct version section)
  • README.md counts updated if file count changed (badge + summary table + section heading + repo tree)
  • Ran node scripts/validate.js --file <path> locally and it passes

For new mapping files only

  • Added to correct section in README.md mapping table with "Standout content" description
  • Added to CROSSREF.md primary frameworks column where relevant
  • Framework coverage matrix in README.md updated (✅ for the new cell)
  • File named correctly: SourceList_Framework.md (e.g., Agentic_SAMM.md)

Notes for reviewers

…e text

Review follow-ups on GenAI-Security-Project#215:

- validate.yml: run on changes to scripts/verification.js (both filters),
  so an edit to the rules alone no longer skips validation and tests.
- verification.js: type-check the top-level version/description of both
  files and require review_date to be a real calendar day. The schema
  already rejected version: 1, version: "" and 2026-13-45; the validator
  that CI runs did not.
- Schema + validator: source.text is an error when source.license names
  itself proprietary or "all rights reserved" (case-insensitive). The
  pattern lives in the schema and the validator reads it from there.
  Every other licence is still judged in review.

8 new fixtures, each checked against both the validator and Ajv; all 8
fail on the previous validator. validate 0 errors / 92 warnings,
test:scripts 137/137, stats:check current, no generate drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@emmanuelgjr emmanuelgjr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Thanks, @a-moskvin. The model is the one agreed on #191, and all six points are met. The one deliberate deviation, leaving licence compatibility to review, is stated openly.

I pushed one commit to your branch (100a15a) with the three review follow-ups, so this didn't need another round trip:

  1. CI path filter. validate.yml didn't list scripts/verification.js, so a PR that edited only the rules would have skipped validation and unit tests. It's now in both the push and pull_request filters.
  2. The validator now agrees with the schemas. CI runs the hand-rolled checks, and they accepted version: 1, version: "", description: 5 and review_date: "2026-13-45", all of which Ajv rejects. Both files now get their top-level fields type-checked, and review_date must be a real calendar day.
  3. Proprietary source text. source.text is now an error when source.license names itself proprietary or "all rights reserved" (case-insensitive). That's the AIUC-1 case from #186/#190. The pattern sits in the schema as an if/then, and the validator reads it from there, so the schemas stay the single source. Every other licence is still judged in review, as you proposed.

That's 8 new fixtures, each run against both the validator and Ajv. All 8 fail on the previous validator. Locally the result is validate 0 errors / 92 warnings (same as main), test:scripts 137/137, stats:check current, and no drift from generate.js. The docs and the CHANGELOG entry are updated to match.

For #216: it's stacked on your original commit, so please rebase it onto this branch, or onto main once this lands.

@emmanuelgjr
emmanuelgjr merged commit b0b85c5 into GenAI-Security-Project:main Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants