Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,22 @@ Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
scope changes (LLM01 cross-modal 2.2.3/2.2.4, LLM04 artifact provenance 3.1.1/3.1.3, LLM05 fine-tuning subversion
6.1.2/3.5.1, LLM08 hidden context 10.2.4, LLM10 generated code 9.3.7). AISVS has no requirement for scanning
generated code itself; the LLM10 section says so.
- **Verification methods data model** (#191, follow-up to #101 / #186): how to check that a control is
implemented, held as data. `data/verification-methods.json` holds each method once (`VM-NNNN` ids,
`examine` / `interview` / `test`, procedure, expected result, provenance, review state).
`data/verification-links/<framework-id>.json` links methods to controls, at control level or on one risk–control
row, each link with its own review. Both have schemas (`verification-methods-schema.json`,
`verification-links-schema.json`), and `scripts/validate.js` enforces them plus the cross-file rules through
`scripts/verification.js` (tested in `scripts/verification.test.mjs`). A source's original wording is optional and
included only where its licence allows; otherwise the method cites the source by id and url. Licence
compatibility is judged in review, not by the validator. v1 is data, schemas and validator only: no generator,
export or webapp changes. Documented in `docs/VERIFICATION_METHODS.md`. No methods yet; the pilot follows in a
separate PR.
- **Verification methods pilot: NIST AI RMF 1.0** (#191). Ten draft methods in `data/verification-methods.json`,
linked in `data/verification-links/nist-ai-rmf.json`: six at control level and four on risk–control rows
(ASI01, ASI02, ASI03, ASI10), covering GV-1.6, GV-1.7, MP-2.3, MP-4.1, MP-5.1 and MS-2.6. Eight are adapted from
the NPW Agentic AI Control Catalogue v2.4.0 (CC BY-SA 4.0); two are project-authored. All are `draft`, pending
independent review.

### Changed

Expand Down
4 changes: 4 additions & 0 deletions data/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ dashboard integration, and downstream tooling without scraping Markdown.
| `incidents-schema.json` | JSON Schema for incident entries |
| `incidents.json` | 50 real-world + research AI security incidents with MAESTRO layer attribution |
| `tools-supplement.json` | Supplemental tool entries merged into entries at generation time |
| `verification-methods.json` | Verification methods: how to check a control is implemented (see `docs/VERIFICATION_METHODS.md`) |
| `verification-methods-schema.json` | JSON Schema for verification methods |
| `verification-links/` | Links from verification methods to controls, one file per framework registry id |
| `verification-links-schema.json` | JSON Schema for a verification links file |
| `entries/` | 41 machine-readable JSON files — one per OWASP entry (LLM01–LLM10, ASI01–ASI10, DSGAI01–DSGAI21) |

---
Expand Down
45 changes: 45 additions & 0 deletions data/verification-links-schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://github.com/GenAI-Security-Project/crosswalk/blob/main/data/verification-links-schema.json",
"title": "GenAI Security Crosswalk — Verification Links Schema",
"description": "Links between verification methods and controls, one file per framework: data/verification-links/<framework-id>.json. See docs/VERIFICATION_METHODS.md and issue #191.",
"type": "object",
"required": ["version", "framework", "links"],
"additionalProperties": false,
"properties": {
"version": { "type": "string", "minLength": 1 },
"framework": {
"type": "string",
"pattern": "^[a-z0-9]+(-[a-z0-9]+)*$",
"description": "Registry id (data/frameworks/<id>.json). Must equal the file name."
},
"links": {
"type": "array",
"description": "Sorted by control_id, then entry_id (control-level links first), then method_id.",
"items": { "$ref": "#/definitions/Link" }
}
},
"definitions": {
"Link": {
"type": "object",
"required": ["method_id", "control_id", "entry_id", "reviewed_by"],
"additionalProperties": false,
"properties": {
"method_id": { "type": "string", "pattern": "^VM-\\d{4}$" },
"control_id": { "type": "string", "minLength": 1, "description": "A control_id in the framework's registry." },
"entry_id": {
"description": "null: the link applies to the control wherever it is mapped. An entry id: only to that risk-control row.",
"oneOf": [
{ "type": "null" },
{ "type": "string", "pattern": "^(LLM\\d{2}|ASI\\d{2}|DSGAI\\d{2}|AST\\d{2})$" }
]
},
"reviewed_by": {
"type": "array",
"items": { "type": "string", "pattern": "\\S" },
"description": "Named humans who confirmed the method verifies this control or row. Empty means draft."
}
}
}
}
}
9 changes: 9 additions & 0 deletions data/verification-links/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Verification links

One file per framework, named by its registry id: `<framework-id>.json`, matching
`data/frameworks/<framework-id>.json`. Each file links verification methods from
`data/verification-methods.json` to that framework's controls.

The format is defined in `data/verification-links-schema.json` and explained in
[docs/VERIFICATION_METHODS.md](../../docs/VERIFICATION_METHODS.md).
`scripts/validate.js` checks every file here.
66 changes: 66 additions & 0 deletions data/verification-links/nist-ai-rmf.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
{
"version": "1.0",
"framework": "nist-ai-rmf",
"links": [
{
"method_id": "VM-0001",
"control_id": "GV-1.6",
"entry_id": null,
"reviewed_by": []
},
{
"method_id": "VM-0002",
"control_id": "GV-1.6",
"entry_id": "ASI03",
"reviewed_by": []
},
{
"method_id": "VM-0003",
"control_id": "GV-1.7",
"entry_id": null,
"reviewed_by": []
},
{
"method_id": "VM-0004",
"control_id": "GV-1.7",
"entry_id": "ASI02",
"reviewed_by": []
},
{
"method_id": "VM-0005",
"control_id": "GV-1.7",
"entry_id": "ASI10",
"reviewed_by": []
},
{
"method_id": "VM-0006",
"control_id": "MP-2.3",
"entry_id": null,
"reviewed_by": []
},
{
"method_id": "VM-0007",
"control_id": "MP-2.3",
"entry_id": "ASI01",
"reviewed_by": []
},
{
"method_id": "VM-0008",
"control_id": "MP-4.1",
"entry_id": null,
"reviewed_by": []
},
{
"method_id": "VM-0009",
"control_id": "MP-5.1",
"entry_id": null,
"reviewed_by": []
},
{
"method_id": "VM-0010",
"control_id": "MS-2.6",
"entry_id": null,
"reviewed_by": []
}
]
}
96 changes: 96 additions & 0 deletions data/verification-methods-schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://github.com/GenAI-Security-Project/crosswalk/blob/main/data/verification-methods-schema.json",
"title": "GenAI Security Crosswalk — Verification Methods Schema",
"description": "Verification methods: how to check that a control is implemented. Each method is held once and linked to controls through data/verification-links/. See docs/VERIFICATION_METHODS.md and issue #191.",
"type": "object",
"required": ["version", "methods"],
"additionalProperties": false,
"properties": {
"version": { "type": "string", "minLength": 1 },
"description": { "type": "string" },
"methods": {
"type": "array",
"items": { "$ref": "#/definitions/Method" }
}
},
"definitions": {
"Method": {
"type": "object",
"required": ["id", "name", "method_type", "procedure", "expected_result", "source", "status", "reviewed_by"],
"additionalProperties": false,
"properties": {
"id": {
"type": "string",
"pattern": "^VM-\\d{4}$",
"description": "Project ID. Sequential, opaque, never reused or renumbered."
},
"name": { "type": "string", "minLength": 1, "maxLength": 80 },
"method_type": {
"enum": ["examine", "interview", "test"],
"description": "Assessment method (NIST SP 800-53A)."
},
"procedure": { "type": "string", "minLength": 1, "description": "What the assessor does." },
"expected_result": { "type": "string", "minLength": 1, "description": "What a pass looks like. Must be observable." },
"source": {
"description": "Source the method is derived from; null for a method authored in the project.",
"oneOf": [{ "type": "null" }, { "$ref": "#/definitions/Source" }]
},
"status": { "enum": ["draft", "reviewed", "deprecated"] },
"reviewed_by": {
"type": "array",
"items": { "type": "string", "pattern": "\\S" },
"description": "Named humans who reviewed the method, as for schema v2 mapping rows. Empty while draft."
},
"frequency": { "$ref": "#/definitions/Frequency" },
"evidence": { "type": "string", "minLength": 1, "description": "Artefact(s) the check produces or consumes." },
"review_date": { "type": "string", "format": "date", "pattern": "^\\d{4}-\\d{2}-\\d{2}$" },
"status_note": {
"type": "string",
"minLength": 1,
"description": "Status context. Required when deprecated: the reason, and the replacement ID if there is one."
}
},
"allOf": [
{
"if": { "properties": { "status": { "const": "reviewed" } }, "required": ["status"] },
"then": { "properties": { "reviewed_by": { "minItems": 1 } } }
},
{
"if": { "properties": { "status": { "const": "deprecated" } }, "required": ["status"] },
"then": { "required": ["status_note"] }
}
]
},
"Source": {
"type": "object",
"required": ["name", "version", "id", "url", "license"],
"additionalProperties": false,
"properties": {
"name": { "type": "string", "minLength": 1 },
"version": { "type": "string", "minLength": 1 },
"id": { "type": "string", "minLength": 1, "description": "Identifier within the source, e.g. C02." },
"url": { "type": "string", "pattern": "^https?://" },
"license": {
"type": "string",
"minLength": 1,
"description": "SPDX identifier where one exists (e.g. CC-BY-SA-4.0); otherwise a short name such as Proprietary."
},
"text": {
"type": "string",
"minLength": 1,
"description": "The source's original wording, unchanged. Include only if the licence allows reproduction in this CC BY-SA 4.0 repository."
}
}
},
"Frequency": {
"type": "object",
"required": ["mode"],
"additionalProperties": false,
"description": "Recommended minimum frequency, not a requirement. An object so that interval or triggers can be added later.",
"properties": {
"mode": { "enum": ["continuous", "periodic", "event_driven"] }
}
}
}
}
Loading
Loading