Central, deduplicated home for Paramify validators — the regex/attestation
checks that assert a fetcher's evidence structurally shows a control is being
implemented. Validators used to be sketched inline in fetcher.yaml; they now
live here as first-class objects.
- One validator = one file:
validators/<category>/<key>.yaml, where the basename equals the file'skey. - The link lives on the validator. Each validator lists every evidence set
it applies to under
evidence_sets(byreference_id). A validator shared across fetchers is a single file with multipleevidence_setsentries — we never copy a validator. This mirrors how Paramify itself models a validator (it owns its evidence-set list). - A fetcher's validators = reverse lookup: every registry file whose
evidence_setsincludes that fetcher'sevidence_set.reference_id. - Three roles (
role:): each evidence set has exactly onecompletenessvalidator (did the collection succeed, is the population real), any number ofconfigurationchecks (is the posture right), and oneintegrityvalidator beside every configuration check that concludes from a count of violations (was the counted field actually there). A set passes only when all of its validators pass, so expect 2–5 per set.
Copy _template/validator.yaml to
validators/<category>/<key>.yaml and fill it in. _-prefixed directories
(like _template) are not part of the registry.
Three rules for the regex of an AUTOMATED validator — each guards a failure
mode that is otherwise silent:
-
Write ECMAScript, not Python. Paramify's engine is JavaScript. It applies
gandsautomatically but neverm, so span lines with[\s\S]*?. Named groups are(?<name>…)— the Python(?P<name>…)is a hard compile error. Test withnode, notpython -c 'import re'. -
Name every capture group, in
snake_casefrom the key it captures. Naming does not renumber anything, sovalidation_rulesstill references groups by number. NoteMATCH_GROUPreads the first match only, so you cannot use groups to assert something about every item in a list — invert it into "the count of bad items is zero". -
Guard the read. A rule that reads nothing passes: a
MATCH_GROUPcomparison over a regex that missed compares nothing to nothing and evaluates true, and a count-of-violations rule reads a renamed key as zero violations. So open every validator with a presence rule:- regexOperation: { type: MATCH_COUNT } criteria: NOT_EQUALS value: { type: CUSTOM_TEXT, customText: "0" }
A validator that reads a value guards itself this way. A validator that counts violations cannot — zero matches is its pass condition — so it needs a separate
role: integrityvalidator proving the structural key exists.
Collection health is not part of a compliance validator. It lives once, in
common/collection_succeeded.yaml, which
lists every evidence set that wants it. Add your reference_id there rather than
re-anchoring on exit_code in your own file. Its guards are written positively
(MATCH_COUNT NOT_EQUALS 0, exit_code EQUALS 0) because the REST API accepts
disposition and discards it — the negative ERROR form self-contradicts today
and yields a validator that can never pass.
See aws/alb_encryption_in_transit.yaml
for a worked compliance validator.
Prove it before you commit. Write the three directions down in
_cases/<key>.yaml — compliant, non-compliant, and the key
renamed — then:
paramify validators check --select <your key>That evaluates them with the real ECMAScript engine, including the semantics
that bite: MATCH_GROUP reads only the first match, rules combine with AND, and
a rule that read nothing still holds. The renamed-key direction is the one that
catches silent false passes and the one that gets skipped.
It is an authoring aid rather than a merge gate — nothing stops you committing a validator without cases, but nothing then proves it can fail either.