Repository navigation
feat(data): verification methods and links data model (refs #191) - #215
emmanuelgjr merged 3 commits into
Conversation
…e text Review follow-ups on GenAI-Security-Project#215: - validate.yml: run on changes to scripts/verification.js (both filters), so an edit to the rules alone no longer skips validation and tests. - verification.js: type-check the top-level version/description of both files and require review_date to be a real calendar day. The schema already rejected version: 1, version: "" and 2026-13-45; the validator that CI runs did not. - Schema + validator: source.text is an error when source.license names itself proprietary or "all rights reserved" (case-insensitive). The pattern lives in the schema and the validator reads it from there. Every other licence is still judged in review. 8 new fixtures, each checked against both the validator and Ajv; all 8 fail on the previous validator. validate 0 errors / 92 warnings, test:scripts 137/137, stats:check current, no generate drift. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
emmanuelgjr
left a comment
There was a problem hiding this comment.
Approved. Thanks, @a-moskvin. The model is the one agreed on #191, and all six points are met. The one deliberate deviation, leaving licence compatibility to review, is stated openly.
I pushed one commit to your branch (100a15a) with the three review follow-ups, so this didn't need another round trip:
- CI path filter.
validate.ymldidn't listscripts/verification.js, so a PR that edited only the rules would have skipped validation and unit tests. It's now in both thepushandpull_requestfilters. - The validator now agrees with the schemas. CI runs the hand-rolled checks, and they accepted
version: 1,version: "",description: 5andreview_date: "2026-13-45", all of which Ajv rejects. Both files now get their top-level fields type-checked, andreview_datemust be a real calendar day. - Proprietary source text.
source.textis now an error whensource.licensenames itself proprietary or "all rights reserved" (case-insensitive). That's the AIUC-1 case from #186/#190. The pattern sits in the schema as anif/then, and the validator reads it from there, so the schemas stay the single source. Every other licence is still judged in review, as you proposed.
That's 8 new fixtures, each run against both the validator and Ajv. All 8 fail on the previous validator. Locally the result is validate 0 errors / 92 warnings (same as main), test:scripts 137/137, stats:check current, and no drift from generate.js. The docs and the CHANGELOG entry are updated to match.
For #216: it's stacked on your original commit, so please rebase it onto this branch, or onto main once this lands.
What this PR changes
Implements the verification-methods data model agreed in #191 (follow-up to #101 / #186). Data, schemas and validator only; no methods yet. The pilot follows in a separate PR.
data/verification-methods-schema.json,data/verification-links-schema.json: Draft-07 schemasdata/verification-methods.json: empty methods arraydata/verification-links/: one file per framework (none yet)scripts/verification.js: schema-derived checks plus cross-file rules; called fromscripts/validate.json full runsscripts/verification.test.mjs: 34 tests; each rule has a breaking fixture, cross-checked against the schemas with Ajvdocs/VERIFICATION_METHODS.md: specification;data/README.mdandCHANGELOG.mdupdatedNo changes to
generate.js, exports, webapp or the npm package contents. Regenerating produces no data changes.Type of change
Source / evidence
#191 (follow-up to #101 / #186)
Checklist
npm run buildpasses: 0 errors, warnings unchanged frommainnpm testpasses: 129/129CHANGELOG.mdupdatedContent
tags, detailed per-entry mappings, references, changelog)
shared/SEVERITY.mdAgentic_X.md, that file mentions this one backLLM01–LLM10,ASI01–ASI10,DSGAI01–DSGAI21)CC BY-SA 4.0Links & data
.mdlinks resolve to real fileslychee)data/schema.jsoncompatible (if adding a new entry type)Project hygiene
YYYY-MM-DDformat)CHANGELOG.mdupdated if this is a new mapping file (include in correct version section)README.mdcounts updated if file count changed (badge + summary table + section heading + repo tree)node scripts/validate.js --file <path>locally and it passesFor new mapping files only
README.mdmapping table with "Standout content" descriptionCROSSREF.mdprimary frameworks column where relevantREADME.mdupdated (✅for the new cell)SourceList_Framework.md(e.g.,Agentic_SAMM.md)Notes for reviewers
source.textis left to review rather than a validator rule (see point 1 on Data model for verification methods (follow-up to #101 / #186) #191).frameworkis declared once per links file and must equal the file name; a link has 4 fields.