diff --git a/.agents/summary/architecture.md b/.agents/summary/architecture.md index 787dc57..ed472d1 100644 --- a/.agents/summary/architecture.md +++ b/.agents/summary/architecture.md @@ -61,7 +61,7 @@ Consequences of that choice, as implemented: Every OSCAL top-level document carries `metadata` and a body-level `uuid`, but the path to them differs per type (`doc.catalog.metadata` vs `doc.system_security_plan.metadata`). `OscalDoc` collapses that dispatch behind uniform properties. A class-level dict `_WRAPPER_TO_BODY_ATTR` maps each of the 8 wrapper types to its body attribute name, so `body` is a single `getattr` rather than an 8-way `if`/`elif` chain. ### Adapter / Convenience constructors — builders -`make_hash`, `make_rlink`, `make_resource` wrap the models in kwarg-friendly constructors with sensible defaults (e.g. auto UUID4, string `algorithm` coercion). Because `OscalBaseModel` accepts field-name kwargs, they construct directly (`BackMatterResourceRlink(media_type=...)`, `BackMatterResource(document_ids=...)`), so every argument, `document_ids` included, is validated at construction. +`make_hash`, `make_rlink`, `make_resource` wrap the models in kwarg-friendly constructors with sensible defaults (e.g. auto UUID4, an identifier from a `key`, string `algorithm` coercion). Each builder validates a JSON-shaped object with one `model_validate` call, so every argument, also an item of a list input that the caller made with `model_construct`, is validated. A failure raises `OscalBuilderError`. ### Hand-written base class — `OscalBaseModel` `v1/_base.py` is the one piece of hand-written behavior generated code inherits, wired in by codegen's `--base-class` rather than by editing output. It makes default serialization valid OSCAL (`serialize_by_alias=True` plus a wrap serializer that drops None) and maps field-name constructor kwargs to aliases. Its `__init__` carries `__pydantic_base_init__` so `model_validate*` never routes through it, keeping OSCAL input alias-only. It imports only `pydantic` and stdlib, so `models.py → _base` can't cycle. diff --git a/.agents/summary/components.md b/.agents/summary/components.md index 3a29f54..633a483 100644 --- a/.agents/summary/components.md +++ b/.agents/summary/components.md @@ -79,9 +79,10 @@ graph LR | Function | Notes | |----------|-------| -| `make_hash(value, algorithm=HashAlgorithm.SHA_256)` | Accepts enum or string (`"SHA-256"`); coerces string via `HashAlgorithm(...)` | -| `make_rlink(href, *, media_type=None, hashes=None)` | `BackMatterResourceRlink(href=..., media_type=..., hashes=...)` — by-name kwargs via `OscalBaseModel` | -| `make_resource(*, title, description, rlinks, uuid, document_ids)` | Generates a UUID4 when `uuid` omitted; one `BackMatterResource(...)` call, so `document_ids` is validated at construction (`[]` or non-`DocumentId` items raise `ValidationError`) | +| `OscalBuilderError(message, errors)` | A `ValueError` that is not an `OscalParseError`; `errors` holds the Pydantic-shaped entries | +| `make_hash(value, algorithm=HashAlgorithm.SHA_256)` | Accepts enum or string; a string that is a `HashAlgorithm` value gives the member, any other string goes to the model as is; one `model_validate` call | +| `make_rlink(href, *, media_type=None, hashes=None)` | One `model_validate` of a JSON-shaped object; items of `hashes` are validated, also an item made with `model_construct` | +| `make_resource(*, title, description, rlinks, uuid, key, document_ids)` | `key` gives the UUID version 5 of the key (an empty key, or a key with `uuid`, raises `OscalBuilderError`); with neither, a UUID4; one `model_validate` call, so `rlinks` and `document_ids` items are validated (`[]` or non-`DocumentId` items raise `OscalBuilderError`) | ## 5. `v1/extensions/validate_element.py` — element-level validation diff --git a/.agents/summary/interfaces.md b/.agents/summary/interfaces.md index a0f0b05..edb43ab 100644 --- a/.agents/summary/interfaces.md +++ b/.agents/summary/interfaces.md @@ -61,7 +61,7 @@ OscalDoc.assessment_period() -> tuple[date | None, date | None] # AP/AR only ```python make_hash(value: str, algorithm: str | HashAlgorithm = HashAlgorithm.SHA_256) -> Hash make_rlink(href: str, *, media_type: str | None = None, hashes: list[Hash] | None = None) -> BackMatterResourceRlink -make_resource(*, title=None, description=None, rlinks=None, uuid=None, document_ids=None) -> BackMatterResource +make_resource(*, title=None, description=None, rlinks=None, uuid=None, key=None, document_ids=None) -> BackMatterResource ``` ### Element validation @@ -128,7 +128,7 @@ sequenceDiagram | `assessment_period()` on non-AP/AR | `OscalAccessError` | | `OscalDoc(...)` with unrelated object | `TypeError` | | `validate_element` unrecognized element type | `OscalUnknownElementTypeError` (outside the parse errors) | -| Builders with invalid input | `pydantic.ValidationError` | +| Builders with invalid input (also an empty `key`, or a `key` with a `uuid`) | `OscalBuilderError` (a `ValueError`, outside the parse errors; `.errors` = entries) | | `validate_oscal` on invalid input | returns `False` (never raises) | ## Code-Generation Interface (build-time) diff --git a/.agents/summary/workflows.md b/.agents/summary/workflows.md index cb24abf..a1373d2 100644 --- a/.agents/summary/workflows.md +++ b/.agents/summary/workflows.md @@ -144,12 +144,12 @@ sequenceDiagram C->>H: make_hash(digest) %% default HashAlgorithm.SHA_256 H-->>C: Hash C->>L: make_rlink(href, media_type=, hashes=[Hash]) - L->>L: BackMatterResourceRlink(href=.., media_type=.., hashes=..) - L-->>C: BackMatterResourceRlink - C->>R: make_resource(title=, description=, rlinks=[BackMatterResourceRlink]) - R->>R: uuid = uuid4() if not provided - R->>R: BackMatterResource(uuid=.., ..., document_ids=..) - R-->>C: BackMatterResource (validated) + L->>L: BackMatterResourceRlink.model_validate({href, media-type, hashes}) + L-->>C: BackMatterResourceRlink (or OscalBuilderError) + C->>R: make_resource(title=, description=, rlinks=[BackMatterResourceRlink], key=) + R->>R: uuid = uuid5(namespace, key) if key, else uuid4() if not provided + R->>R: BackMatterResource.model_validate({uuid, ..., document-ids}) + R-->>C: BackMatterResource (validated, or OscalBuilderError) ``` ## 5. Element-level Validation diff --git a/AGENTS.md b/AGENTS.md index 2d2f2db..ad95d8d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -97,7 +97,7 @@ See `.agents/summary/workflows.md` for the flow diagram. - **A file-path parse finds the error category from the read itself.** `_read_oscal_file` (`v1/parser.py`) makes no `Path.exists()` call: `read_bytes` raises, and `ENOENT`, `ENOTDIR`, or a `ValueError` (a NUL byte) gives `FileNotFoundError`; each other `OSError` gives `OscalReadError(OSError)`, which is not a parse error and not a not-found error (ERR-012). `exists()` hides the `errno` and changes its answer with the Python version. `parse_oscal_file`, `parse_oscal_yaml_file`, and `OscalDoc.from_file` all share this path. - **Typed parsers return the wrapper, not the body.** `parse_catalog()` returns `CatalogDocument` (access `.catalog.metadata...`). It `hasattr`-checks `doc.root` and raises `OscalDocumentTypeMismatchError("Document is not a Catalog")` (a subclass of `OscalParseError`) on mismatch. - **`OscalDoc` collapses per-type dispatch.** Use it when you don't want to branch on document type — `doc.metadata`, `doc.uuid`, `doc.oscal_version`, `doc.body` work uniformly via the `_WRAPPER_TO_BODY_ATTR` map. `assessment_period()` is defined **only** for Assessment Plan / Assessment Results (raises `OscalAccessError` otherwise). -- **Valid OSCAL by default; construct by name, validate by alias.** `OscalBaseModel` (`v1/_base.py`) gives every object model alias keys (`media-type`, `document-ids`) and no `null` on all serialization paths (`model_dump*`, `to_json`, `TypeAdapter`); `exclude_none=False` is a no-op, `by_alias=False` emits names. Constructor kwargs accept field names or aliases (`BackMatterResourceRlink(media_type=...)`), so the builders construct directly. OSCAL input stays alias-only: dict input rejects name keys with `extra_forbidden` at every depth (`extra='forbid'`). **Don't add `validate_by_name`/`populate_by_name` to the base config**, and don't remove `__pydantic_base_init__` from its `__init__`: either makes `model_validate*` accept name-keyed OSCAL (DEVELOPING "Valid OSCAL by default"; tripwire tests guard both). JSON input rejects name keys too, which needs the pydantic ≥2.14.2 floor (2.13.x–2.14.1 silently dropped them, [#6](https://github.com/cairn-proofs/oscal-bindings-python/issues/6)). +- **Valid OSCAL by default; construct by name, validate by alias.** `OscalBaseModel` (`v1/_base.py`) gives every object model alias keys (`media-type`, `document-ids`) and no `null` on all serialization paths (`model_dump*`, `to_json`, `TypeAdapter`); `exclude_none=False` is a no-op, `by_alias=False` emits names. Constructor kwargs accept field names or aliases (`BackMatterResourceRlink(media_type=...)`), so a caller constructs directly. The builders validate a JSON-shaped object and raise `OscalBuilderError` (a `ValueError`, not an `OscalParseError`). OSCAL input stays alias-only: dict input rejects name keys with `extra_forbidden` at every depth (`extra='forbid'`). **Don't add `validate_by_name`/`populate_by_name` to the base config**, and don't remove `__pydantic_base_init__` from its `__init__`: either makes `model_validate*` accept name-keyed OSCAL (DEVELOPING "Valid OSCAL by default"; tripwire tests guard both). JSON input rejects name keys too, which needs the pydantic ≥2.14.2 floor (2.13.x–2.14.1 silently dropped them, [#6](https://github.com/cairn-proofs/oscal-bindings-python/issues/6)). - **`validate_element` type names are derived from model class names** (PascalCase → kebab-case, introspected at import), so they follow the manifest names (`back-matter-resource-rlink`). Each manifest type has one, whatever its kind: object/document classes, enums (`hash-algorithm`, validated from the string value), and unions (`parameter`). Use `get_supported_element_types()` for the live list; an unrecognized type raises `OscalUnknownElementTypeError`, which is not an `OscalParseError`. The corpus case `element-types/all` pins the list. The introspection keeps only `BaseModel` subclasses, `Enum` subclasses, and `types.UnionType` aliases that `models` defines, and skips pydantic parametrizations (`__pydantic_generic_metadata__["origin"]` set): since 2.14, pydantic registers the `RootModel[...]` bases of `OscalDocument` and `MappingCollectionMappings` in `models` under non-identifier names. Module scans in tests skip them the same way. - **Expected conformance failures are local.** `tests/support/expected_failures.py` (case id → issue URL) drives the strict xfails in `tests/test_conformance.py`; the runner ignores any `known-defects` in the corpus. The spec store holds no binding state, so fixing a listed defect means deleting its entry here, never a store change. - **Closed enums are `Enum` classes.** A closed-enum field holds its enum class (`ParameterSelectionHowMany`) and rejects any other value. An open-enum field is the union of its base datatype alias and its enum class. diff --git a/DEVELOPING.md b/DEVELOPING.md index 021a002..0fe185d 100644 --- a/DEVELOPING.md +++ b/DEVELOPING.md @@ -220,7 +220,7 @@ changes only for a new OSCAL **major** version. - **Major-version namespace** — the bindings live in `oscal_bindings.v1`, named for the OSCAL major version with no minor or patch component. OSCAL is backward-compatible within a major version (newer schemas accept older documents; minor bumps have not added required fields), so the compatibility boundary is the major version and one binding set covers every 1.x producer. A minor-version namespace (`v1_2`, `v1_3`) was considered and rejected: it would encode a break at each routine refresh that does not exist. `__oscal_schema_version__` records the exact release the models were generated from. This mirrors NIST's own schema `$id` (`ns/oscal/1.0/1.2.2/...`), which scopes the namespace to the major line and hangs the release off it. OSCAL 2.0 is the trigger for a second version package; nothing is factored out between version packages before then, because the seam is unknowable until 2.0's differences are in hand. - **Strict models tie coverage to the release of the pin** — every model carries `extra='forbid'`, so bindings generated from 1.2.3 reject a later 1.x document that uses a new field. "Supports OSCAL 1.x" therefore means "supports the release of the pin and earlier"; `__oscal_schema_version__` makes that gap visible. `extra='ignore'` was rejected because `serialize_oscal` would silently drop unknown fields. An opt-in lenient mode is deferred, not rejected. - **Valid OSCAL by default** — `v1/_base.py` defines `OscalBaseModel`, wired in through codegen's `--base-class` so `models.py` stays pristine generator output. It sets `serialize_by_alias=True` (needs pydantic ≥2.11; the floor is now 2.14.2, below) and adds a wrap `model_serializer` that drops None values, so every serialization path, including `pydantic_core.to_json` and `TypeAdapter` calls that never see our kwargs, emits OSCAL keys and no `null`. `serialize_oscal` keeps its explicit `by_alias=True, exclude_none=True`. The wrap serializer costs a Python callback per model node (benchmarks below). Per-field `Field(exclude_if=lambda v: v is None)` measured about 1.35× the pre-fix time in a prototype (the wrap serializer was about 2.4× there), with identical output, but it isn't implemented: the post-processor would have to rewrite all 698 optional fields, and `exclude_if` needs pydantic ≥2.12. It's the documented fallback if serialize time becomes a problem. - - **Construction by name, validation by alias.** `OscalBaseModel.__init__` maps field-name kwargs to aliases, so `BackMatterResourceRlink(media_type=...)` works and the builders construct directly. The override carries `__pydantic_base_init__ = True`, the marker `RootModel.__init__` uses. Without it pydantic sets `__pydantic_custom_init__` and routes `model_validate*` through `__init__`: a probe of that variant accepted and kept name-keyed input at every depth, in dict and JSON mode, even with `by_alias=True, by_name=False`. The marker is undocumented, so tests pin it structurally and behaviorally. + - **Construction by name, validation by alias.** `OscalBaseModel.__init__` maps field-name kwargs to aliases, so `BackMatterResourceRlink(media_type=...)` works. The builders do not construct directly: each validates a JSON-shaped object with `model_validate` one time and raises `OscalBuilderError`. The override carries `__pydantic_base_init__ = True`, the marker `RootModel.__init__` uses. Without it pydantic sets `__pydantic_custom_init__` and routes `model_validate*` through `__init__`: a probe of that variant accepted and kept name-keyed input at every depth, in dict and JSON mode, even with `by_alias=True, by_name=False`. The marker is undocumented, so tests pin it structurally and behaviorally. - **Not config `validate_by_name`.** Setting `validate_by_name=True` in the base config, with per-call `by_alias=True, by_name=False` at our validation sites (Option A), was rejected. Alias-only would become opt-in per call, so a user's direct `Model.model_validate` would accept and populate name-keyed OSCAL. Nested plain dicts in constructor kwargs would accept names too. Don't add `validate_by_name`, `validate_by_alias`, or `populate_by_name` to the base config. Revisit if pydantic adds per-mode by-name validation (pydantic#12056). - **Pydantic floor 2.14.2: JSON mode rejects name keys.** pydantic 2.13.x through 2.14.1 had `model_validate_json` silently discard a key equal to the Python name of one of the model's alias-differing fields, instead of reporting `extra_forbidden` ([#6](https://github.com/cairn-proofs/oscal-bindings-python/issues/6)). pydantic#13275, released in 2.14.2 (GHSA-qpmf-fgh3-ff25), made JSON mode match dict mode, so the runtime range is `pydantic[email]>=2.14.2,<3` and the JSON-mode tests require `extra_forbidden`. The base doesn't change JSON validation either way. - **Parametrized bases in the models namespace.** Since 2.14, pydantic registers a module-scope parametrization such as `RootModel[Mapping | list[Mapping]]` in the module that writes it, under a non-identifier name (pydantic#13243; 2.14.2 also sets its `__module__` there, pydantic#13994). So the bases of `OscalDocument` and `MappingCollectionMappings` appear in `vars(models)`. `validate_element`'s element map and the module scans in the tests skip any class whose `__pydantic_generic_metadata__["origin"]` is set, and `test_each_parametrized_generic_of_the_module_is_a_manifest_base` keeps that skip limited to those bases. `models.py` is unchanged. diff --git a/README.md b/README.md index 6de2971..d312a48 100644 --- a/README.md +++ b/README.md @@ -159,6 +159,22 @@ resource = make_resource( ], ) # resource.uuid is a freshly generated UUID4 + +# A key gives a stable identifier: the UUID version 5 of the key. +make_resource(key="https://example.org/policies/access-control").uuid +# '51beb3e9-30bb-535f-ba49-e68d4661aaa9' +``` + +An invalid input raises `OscalBuilderError`, a `ValueError` that is not an `OscalParseError`. Its `errors` attribute +holds the structured entries. An empty `key`, or a `key` with a `uuid`, is invalid. + +```python +from oscal_bindings import OscalBuilderError, make_hash + +try: + make_hash(digest, algorithm=" SHA-256") +except OscalBuilderError as e: + print(e.errors) ``` ### Serialization diff --git a/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/.openspec.yaml b/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/.openspec.yaml new file mode 100644 index 0000000..bbe8b39 --- /dev/null +++ b/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-11-17 diff --git a/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/design.md b/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/design.md new file mode 100644 index 0000000..dbf86d1 --- /dev/null +++ b/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/design.md @@ -0,0 +1,108 @@ +# Design + +## Context + +See [proposal.md - Why](proposal.md#why) for the motivation. These facts constrain the approach. + +1. The `Hash.algorithm` field is the union of the string datatype alias and `HashAlgorithm`. The model accepts + `SHA-999` and `MD5`, and rejects ` SHA-256` with two entries, one for each member of the union. +2. Pydantic does not validate a model instance again when the instance is the value of a field + (`revalidate_instances` is `never`). Thus `DocumentId.model_construct(identifier="")` goes into a + `BackMatterResource` with no validation. +3. The corpus README says that the runner gives each item of `hashes` and `document-ids` to the builder as the element + of the binding, with no validation. In Python, that element is `model_construct` of the item JSON. +4. The conformance runner (`category` in `tests/test_conformance.py`) maps a class to a category. + `test_every_category_is_mapped` is a strict expected failure until the builder-error category has a class. +5. A name in `v1.__all__` must be in the surface snapshot (`tests/support/version_package.py`) and in + `openspec/traceability.json`. + +## Goals / Non-Goals + +**Goals:** + +1. Each invalid input to a builder gives one `OscalBuilderError`, with the structured entries of the failure. + +**Non-Goals:** + +1. The missing patterns of the generated models ([#77]). The case `build-resource/invalid-document-id` stays an + expected failure until that issue is fixed. + +## Decisions + +### D1: `OscalBuilderError` is a subclass of `ValueError` + +`pydantic.ValidationError` is a subclass of `ValueError`. With `OscalBuilderError(ValueError)`, code that catches +`ValueError` from a builder continues to catch each builder error. The class is not a subclass of `OscalParseError` +([ERR-013.02]). + +The class has the same constructor and `errors` attribute as `OscalParseError`: `OscalBuilderError(message, errors)`. +`errors` holds the entries in the Pydantic entry shape (`type`, `loc`, `msg`, `input`). A builder raises it +`from` the `ValidationError`, so `__cause__` keeps the original error. The input of a builder is small and comes from +the caller, so the error does not omit `input` as `OscalParseError` does. + +The class is in `builders.py`. It goes through the re-export chain: `v1/extensions/__init__.py` and `v1/__init__.py`. +The top level takes it from `v1.__all__`. + +Alternative: a subclass of `pydantic.ValidationError`. Code that catches `ValidationError` then continues to work. But +each key error must be made with `ValidationError.from_exception_data` and a custom line error, and the public error +class then depends on the internals of pydantic-core. Rejected. + +### D2: The builders validate a JSON-shaped object + +Each builder makes one object with the JSON keys of the element and calls `model_validate` on it one time. An item of +a list input that is a model instance goes into the object as `item.model_dump()`. An item of another type goes in as +is. Thus the model validates each item, also an item that the caller made with `model_construct`. An item that is not +a model gives the current `model_type` entry. + +`make_hash` converts a string that is a `HashAlgorithm` value to the member, and puts any other string into the object +as is. Thus `make_hash(d, "SHA-256").algorithm is HashAlgorithm.SHA_256`, as today. `SHA-999` stays a string, which +the model accepts, and ` SHA-256` stays a string, which the model rejects. `HashAlgorithm` is not a `str` enum, so +with no conversion the string would stay a `str` and would not equal the member. + +A `ValidationError` from `model_validate` becomes an `OscalBuilderError` as [D1](#d1-oscalbuildererror-is-a-subclass-of-valueerror) shows. + +Alternative: set `revalidate_instances='always'` on `OscalBaseModel`. This validates each nested instance again on each +construction in each part of the binding, not only in the builders. Rejected. + +### D3: The key gives the identifier before validation + +`make_resource` gets the keyword argument `key: str | None = None`. Before it makes the object of +[D2](#d2-the-builders-validate-a-json-shaped-object), it validates the key and the `uuid`: + +1. A key that is not a `str` raises `OscalBuilderError` with one entry of type `string_type` at `loc == ("key",)`. + Without this check, `uuid.uuid5` raises `TypeError` for an `int`, and for `bytes` it raises on 3.11 but not on 3.12 + and later. +2. A key with a `uuid` raises `OscalBuilderError` with one entry of type `key_with_uuid` at `loc == ("key",)`. +3. An empty key raises `OscalBuilderError` with one entry of type `empty_key` at `loc == ("key",)`. +4. A key and no `uuid` give `str(uuid.uuid5(NAMESPACE, key))`. `uuid.uuid5` encodes a `str` name as UTF-8, as + [BMB-004] requires. +5. No key and no `uuid` give `str(uuid.uuid4())`. + +`NAMESPACE` is a module constant, `uuid.UUID("3942257f-e1cf-5e06-9c6f-eeb97ecabe77")`, the namespace that [BMB-004] +states. The binding does not derive it from the URL in `docs/back-matter-keys.md` of the spec artifact. + +### D4: The runner runs the three builder operations + +The runner gets `build-hash`, `build-rlink`, and `build-resource`. A builder case has no input. Each operation: + +1. Maps the JSON keys of `arguments` to the keyword arguments of the builder. `media-type` gives `media_type`, + `document-ids` gives `document_ids`, and `identifier` gives `uuid`. +2. Gives each item of `hashes` or `document-ids` as `Hash.model_construct(...)` or `DocumentId.model_construct(...)` + of the item JSON (see [Context](#context) item 3). +3. Validates `expected-category` with `_expect_error`, compares the `uuid` with `expected-uuid`, or compares the + canonical JSON of the result with `expected-element`. For a `build-resource` case with no `key` and no `identifier`, + it validates the `uuid` of the result against the UUID pattern of the schema, and removes `uuid` before the compare. + +`category` maps `OscalBuilderError` to `builder-error`. + +## Risks / Trade-offs + +1. [Code that catches `pydantic.ValidationError` from a builder does not catch the new error.] → The proposal marks + the change as breaking. `OscalBuilderError` is a `ValueError`, so `except ValueError` continues to work. +2. [`model_dump` of an item that the caller made with `model_construct` and values of the wrong type can emit a + Pydantic serialization warning before the validation error.] → The builder still raises `OscalBuilderError`. The + warning comes only from an item that bypassed validation. + +[BMB-004]: https://github.com/cairn-proofs/oscal-bindings-specs/blob/v0.8.0%2Boscal.1.2.3/openspec/specs/backmatter-builders/spec.md +[ERR-013.02]: https://github.com/cairn-proofs/oscal-bindings-specs/blob/v0.8.0%2Boscal.1.2.3/openspec/specs/error-model/spec.md +[#77]: https://github.com/cairn-proofs/oscal-bindings-python/issues/77 diff --git a/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/proposal.md b/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/proposal.md new file mode 100644 index 0000000..709ce95 --- /dev/null +++ b/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/proposal.md @@ -0,0 +1,74 @@ +# Proposal + +## Why + +The pinned spec artifact `v0.8.0+oscal.1.2.3` has these store requirements for the back-matter builders: + +1. [BMB-004]: `make_resource` accepts a key. When the caller gives a key and no identifier, the identifier is the UUID + version 5 of the key. An empty key, or a key with an identifier, is a validation error ([#62]). +2. [ERR-013]: each validation error of a builder is in the builder-error category of [ERR-001]. It is not a parse + error ([#62]). +3. [BMB-001]: the hash builder accepts each algorithm that the schema accepts, also an algorithm that is not a named + algorithm ([#63]). + +The binding does not meet these requirements: + +1. `make_resource` in `src/oscal_bindings/v1/extensions/builders.py` has no key input. +2. The binding has no builder-error category. The builders raise `pydantic.ValidationError`, and `make_hash` raises + `ValueError` for an unknown algorithm string ([#53]). +3. `make_hash` converts every string with `HashAlgorithm(algorithm)`, so it rejects `SHA-999`. The `Hash` model accepts + `SHA-999`. +4. `make_resource` does not validate an item of `document_ids` or `rlinks` that is a model instance. An instance that + the caller makes with `model_construct` goes into the resource with no validation. +5. The conformance runner has no `build-hash`, `build-rlink`, or `build-resource` operation. Thus 14 corpus cases are + expected failures, and `test_every_category_is_mapped` is an expected failure. + +The `facade` operation of [#63] is not in this change. + +## What Changes + +1. A new exception `OscalBuilderError`, a subclass of `ValueError`. It is not a subclass of `OscalParseError`. Each + builder raises it for each invalid input. Its `errors` attribute holds the structured entries. +2. `make_resource` gets a keyword argument `key`. A key and no `uuid` give the UUID version 5 of [BMB-004]. An empty + key, or a key with a `uuid`, raises `OscalBuilderError`. +3. `make_hash` converts a string that is a named algorithm to its `HashAlgorithm` member, as today, and gives any + other string to the `Hash` model as is. Thus `SHA-999` is accepted, and ` SHA-256` raises `OscalBuilderError`. +4. The builders validate each item of a list input, also an item that is a model instance. +5. **BREAKING**: the builders do not raise `pydantic.ValidationError`. Code that catches `ValidationError` from a + builder does not catch the new error. Code that catches `ValueError` catches it. +6. The conformance runner runs `build-hash`, `build-rlink`, and `build-resource`, and maps `OscalBuilderError` to the + category `builder-error`. The expected-failure entries of 13 `build-*` cases and the expected failure of + `test_every_category_is_mapped` are removed. +7. The case `build-resource/invalid-document-id` stays an expected failure. Its entry points to [#77]: the generated + `DocumentId.identifier` has no pattern, so it accepts `""`. + +## Capabilities + +### New Capabilities + +None. + +### Modified Capabilities + +1. `error-tests`: the list of categories and classes gets the builder-error category and its class + `OscalBuilderError`. + +## Impact + +1. Code: `src/oscal_bindings/v1/extensions/builders.py`, and the re-exports in + `src/oscal_bindings/v1/extensions/__init__.py` and `src/oscal_bindings/v1/__init__.py`. +2. Tests: `tests/test_conformance.py`, `tests/support/expected_failures.py`, `tests/test_builders.py`, and + `tests/test_serialization_defaults.py`. +3. Docs: `README.md`, `AGENTS.md`, `DEVELOPING.md`, and the `.agents/summary/` pages that name the builder errors, + the builder construction, or the `make_resource` signature. +4. Records: `openspec/traceability.json` and the surface snapshot `tests/support/version_package.py`. +5. Public API: one new exception class, one new keyword argument, and the exception that each builder raises. + +[BMB-001]: https://github.com/cairn-proofs/oscal-bindings-specs/blob/v0.8.0%2Boscal.1.2.3/openspec/specs/backmatter-builders/spec.md +[BMB-004]: https://github.com/cairn-proofs/oscal-bindings-specs/blob/v0.8.0%2Boscal.1.2.3/openspec/specs/backmatter-builders/spec.md +[ERR-001]: https://github.com/cairn-proofs/oscal-bindings-specs/blob/v0.8.0%2Boscal.1.2.3/openspec/specs/error-model/spec.md +[ERR-013]: https://github.com/cairn-proofs/oscal-bindings-specs/blob/v0.8.0%2Boscal.1.2.3/openspec/specs/error-model/spec.md +[#53]: https://github.com/cairn-proofs/oscal-bindings-python/issues/53 +[#62]: https://github.com/cairn-proofs/oscal-bindings-python/issues/62 +[#63]: https://github.com/cairn-proofs/oscal-bindings-python/issues/63 +[#77]: https://github.com/cairn-proofs/oscal-bindings-python/issues/77 diff --git a/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/specs/error-tests/spec.md b/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/specs/error-tests/spec.md new file mode 100644 index 0000000..42657b5 --- /dev/null +++ b/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/specs/error-tests/spec.md @@ -0,0 +1,45 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: The class of an error is its category + +For each category in this list, an error SHALL be in the category exactly when it is an instance of the class: + +1. Parse error: `OscalParseError`. +2. Document-type mismatch: `OscalDocumentTypeMismatchError`, a subclass of `OscalParseError`. +3. Unknown element type: `OscalUnknownElementTypeError`, which is not a subclass of `OscalParseError`. +4. Access error: `OscalAccessError`. +5. Not-found: `FileNotFoundError`. +6. Read error: `OscalReadError`, a subclass of `OSError`, which is not a subclass of `OscalParseError` or of + `FileNotFoundError`. +7. Builder error: `OscalBuilderError`, a subclass of `ValueError`, which is not a subclass of `OscalParseError`. + +A caller tests a category with `isinstance`. + +#### Scenario: A document-type mismatch is also a parse error + +- **WHEN** the caller parses a profile with the catalog parser +- **THEN** the error is an instance of `OscalDocumentTypeMismatchError` and an instance of `OscalParseError` + +#### Scenario: A parse error of another kind is not a document-type mismatch + +- **WHEN** the caller parses input that is not valid JSON +- **THEN** the error is an instance of `OscalParseError` and not an instance of `OscalDocumentTypeMismatchError` + +#### Scenario: An unknown element type is not a parse error + +- **WHEN** the caller validates an element with an element type that the binding does not recognize +- **THEN** the error is an instance of `OscalUnknownElementTypeError` and not an instance of `OscalParseError` + +#### Scenario: A read error is not a parse error or a not-found error + +- **WHEN** the caller parses a file at a path that is a directory +- **THEN** the error is an instance of `OscalReadError` and an instance of `OSError` +- **AND** the error is not an instance of `OscalParseError` and not an instance of `FileNotFoundError` + +#### Scenario: A builder error is a value error and not a parse error + +- **WHEN** the caller builds a hash with the algorithm ` SHA-256`, which has a leading space +- **THEN** the error is an instance of `OscalBuilderError` and an instance of `ValueError` +- **AND** the error is not an instance of `OscalParseError` diff --git a/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/tasks.md b/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/tasks.md new file mode 100644 index 0000000..1333ea7 --- /dev/null +++ b/openspec/changes/archive/2026-11-17-add-builder-error-and-resource-key/tasks.md @@ -0,0 +1,85 @@ +# Tasks + +## 1. Baseline before the change + +- [x] 1.1 Run `hatch run python scripts/bench_serialization.py` and `hatch run python scripts/bench_memory.py` on + `main`. Record the medians, GC on and GC off, in the scratchpad. Verify that the record is not empty. + +## 2. Builder error and validation of a JSON-shaped object + +- [x] 2.1 Add `OscalBuilderError(ValueError)` to `src/oscal_bindings/v1/extensions/builders.py`, as + [design D1](design.md#d1-oscalbuildererror-is-a-subclass-of-valueerror) states. Its docstring says that it is not a + parse error and that `except ValueError` catches it. Re-export it from `src/oscal_bindings/v1/extensions/__init__.py` + and `src/oscal_bindings/v1/__init__.py`. Add it to `tests/support/version_package.py`, and add an entry with + `ERR-001` and `ERR-013` to `openspec/traceability.json`. Add `BMB-004` to the `make_resource` entry of that file. + Verify that `oscal_bindings.OscalBuilderError` is + `oscal_bindings.v1.OscalBuilderError`. +- [x] 2.2 Change `make_hash`, `make_rlink`, and `make_resource` to call `model_validate` one time on a JSON-shaped object and to raise + `OscalBuilderError` from the `ValidationError`, as [design D2](design.md#d2-the-builders-validate-a-json-shaped-object) + states. Update the module docstring and the docstrings of the three builders. Verify with `tests/test_builders.py`: + 1. Each test that expects `ValidationError` expects `OscalBuilderError`, and the error is not an instance of + `OscalParseError`. This includes the `make_resource` test at `tests/test_serialization_defaults.py:184`. + 2. `make_hash("ab", "MD5")` and `make_hash("ab", "SHA-999")` return a hash whose algorithm is the string as given. + The test that expects `ValueError` for `MD5` is removed. + 3. `make_hash("ab", " SHA-256")` raises `OscalBuilderError` whose `errors` is not empty, and the error is an instance + of `ValueError` (the scenario of the `error-tests` delta). + 4. `make_hash("ab", HashAlgorithm.SHA_256)` and `make_hash("ab", "SHA-256")` give equal hashes, and the algorithm of + each is `HashAlgorithm.SHA_256`. `test_algorithm_by_string`, `tests/test_prop_builders.py`, and + `tests/test_prop_serialization_preservation.py` pass with no change. + 5. `make_rlink` with a hash from `Hash.model_construct(algorithm=" SHA-256", value="ab")` raises `OscalBuilderError`. +- [x] 2.3 Add a test in `tests/test_conformance.py`, or change one, to show that `category` maps `OscalBuilderError` to + `builder-error`. Add the class to `_CATEGORIES` and to `test_every_category_is_mapped`, and remove the `xfail` of + that test. Verify that `hatch test tests/test_conformance.py` passes. + +## 3. Resource identifier from a key + +- [x] 3.1 Add the keyword argument `key` and the module constant of the namespace to `make_resource`, as + [design D3](design.md#d3-the-key-gives-the-identifier-before-validation) states. Document `key` in the docstring of + `make_resource`, with a link to `docs/back-matter-keys.md` of the spec store. Verify with `tests/test_builders.py`: + 1. Two resources with the same key and different titles have the same `uuid` ([BMB-004.02]). + 2. The key `https://example.org/policies/données` gives `daa01937-2576-5dc8-9fb0-4da240946a5e` ([BMB-004.03]). + 3. A key with a `uuid` raises `OscalBuilderError` with one entry of type `key_with_uuid` at `("key",)`. + 4. An empty key raises `OscalBuilderError` with one entry of type `empty_key` at `("key",)`. + 5. No key and no `uuid` give a UUID version 4. + 6. `key=5` and `key=b"abc"` each raise `OscalBuilderError` with one entry of type `string_type` at `("key",)`, on + each Python version of the test matrix. + +## 4. Builder operations of the conformance runner + +- [x] 4.1 Add the operations `build-hash`, `build-rlink`, and `build-resource` to `OPERATIONS` in + `tests/test_conformance.py`, as [design D4](design.md#d4-the-runner-runs-the-three-builder-operations) states. + Remove from `tests/support/expected_failures.py` the entries of the five `build-resource` cases of #62, and of the + `build-hash`, `build-rlink`, `build-resource/generated-identifier`, and `build-resource/omitted-inputs` cases of #63. + Change the entry of `build-resource/invalid-document-id` to point to #77. Verify that `hatch test + tests/test_conformance.py` passes, with `build-resource/invalid-document-id` as a strict expected failure. + +## 5. Docs + +- [x] 5.1 Update these builder docs: + 1. The builders section of `README.md`: add a `key` example and `OscalBuilderError`. + 2. The builders bullet of "Valid OSCAL by default" in `AGENTS.md`. + 3. The `make_resource` signature and the error row of `.agents/summary/interfaces.md`. + 4. The `make_hash`, `make_rlink`, and `make_resource` rows of `.agents/summary/components.md`. + 5. The builders section of `.agents/summary/architecture.md`. + 6. The builder flow of `.agents/summary/workflows.md`. + 7. The builders sentence of "Valid OSCAL by default" in `DEVELOPING.md`. + + Verify that + `grep -rn "ValidationError" README.md AGENTS.md .agents/summary` names no builder error, and that `hatch run docs` + passes. + +## 6. Integration + +- [x] 6.1 Run `hatch run release`. Verify that it passes, and that `git diff --exit-code -- src/oscal_bindings/v1/models.py` + shows no change. +- [x] 6.2 Run the two benchmarks of task 1.1 again. Record the before and after numbers, GC on and GC off, for the pull + request. Update `PERFORMANCE.md` only if a number moves. +- [x] 6.3 Run `openspec validate add-builder-error-and-resource-key --strict`. Verify that it passes. + +## Workflow follow-up + +- Archive the change after the pull request merges. +- Close #53 and #62 with the pull request. Update #63 to say that only the `facade` operation stays open. + +[BMB-004.02]: https://github.com/cairn-proofs/oscal-bindings-specs/blob/v0.8.0%2Boscal.1.2.3/openspec/specs/backmatter-builders/spec.md +[BMB-004.03]: https://github.com/cairn-proofs/oscal-bindings-specs/blob/v0.8.0%2Boscal.1.2.3/openspec/specs/backmatter-builders/spec.md diff --git a/openspec/specs/error-tests/spec.md b/openspec/specs/error-tests/spec.md index 7a19bf1..a8d8e9e 100644 --- a/openspec/specs/error-tests/spec.md +++ b/openspec/specs/error-tests/spec.md @@ -17,6 +17,7 @@ For each category in this list, an error SHALL be in the category exactly when i 5. Not-found: `FileNotFoundError`. 6. Read error: `OscalReadError`, a subclass of `OSError`, which is not a subclass of `OscalParseError` or of `FileNotFoundError`. +7. Builder error: `OscalBuilderError`, a subclass of `ValueError`, which is not a subclass of `OscalParseError`. A caller tests a category with `isinstance`. @@ -41,6 +42,12 @@ A caller tests a category with `isinstance`. - **THEN** the error is an instance of `OscalReadError` and an instance of `OSError` - **AND** the error is not an instance of `OscalParseError` and not an instance of `FileNotFoundError` +#### Scenario: A builder error is a value error and not a parse error + +- **WHEN** the caller builds a hash with the algorithm ` SHA-256`, which has a leading space +- **THEN** the error is an instance of `OscalBuilderError` and an instance of `ValueError` +- **AND** the error is not an instance of `OscalParseError` + ### Requirement: A file-path parse finds the category of a read failure from the read A file-path parse SHALL find the category of a read failure from the error of the read itself: diff --git a/openspec/traceability.json b/openspec/traceability.json index 6fb4a80..3710329 100644 --- a/openspec/traceability.json +++ b/openspec/traceability.json @@ -26,7 +26,8 @@ { "name": "ElementValidationError", "requirements": ["ELV-001"] }, { "name": "make_hash", "requirements": ["BMB-001"] }, { "name": "make_rlink", "requirements": ["BMB-002"] }, - { "name": "make_resource", "requirements": ["BMB-003"] }, + { "name": "make_resource", "requirements": ["BMB-003", "BMB-004"] }, + { "name": "OscalBuilderError", "requirements": ["ERR-001", "ERR-013"] }, { "name": "OscalParseError", "requirements": ["ERR-001", "ERR-003", "ERR-009", "ERR-010"] }, { "name": "OscalDocumentTypeMismatchError", "requirements": ["ERR-005", "ERR-009"] }, { "name": "OscalReadError", "requirements": ["ERR-001", "ERR-012"] }, diff --git a/src/oscal_bindings/v1/__init__.py b/src/oscal_bindings/v1/__init__.py index 062db2d..3416c7f 100644 --- a/src/oscal_bindings/v1/__init__.py +++ b/src/oscal_bindings/v1/__init__.py @@ -39,6 +39,7 @@ ElementValidationResult, OscalAccessError, OscalBody, + OscalBuilderError, OscalDoc, OscalUnknownElementTypeError, OscalWrapper, @@ -100,6 +101,7 @@ "MappingCollectionDocument", "OscalAccessError", "OscalBody", + "OscalBuilderError", "OscalDoc", "OscalDocument", "OscalDocumentTypeMismatchError", diff --git a/src/oscal_bindings/v1/extensions/__init__.py b/src/oscal_bindings/v1/extensions/__init__.py index b039a5b..cdecf4d 100644 --- a/src/oscal_bindings/v1/extensions/__init__.py +++ b/src/oscal_bindings/v1/extensions/__init__.py @@ -1,4 +1,5 @@ from oscal_bindings.v1.extensions.builders import ( + OscalBuilderError, make_hash, make_resource, make_rlink, @@ -33,6 +34,7 @@ "ElementValidationResult", "OscalAccessError", "OscalBody", + "OscalBuilderError", "OscalDoc", "OscalUnknownElementTypeError", "OscalWrapper", diff --git a/src/oscal_bindings/v1/extensions/builders.py b/src/oscal_bindings/v1/extensions/builders.py index fa0b31e..c22ca7c 100644 --- a/src/oscal_bindings/v1/extensions/builders.py +++ b/src/oscal_bindings/v1/extensions/builders.py @@ -1,14 +1,23 @@ """Convenience constructors for OSCAL `back-matter` resource elements. These helpers wrap the underlying Pydantic models with the field idioms -already filled in (UUID4 default, kwarg-style construction, no need to -remember the camelCase aliases). Each helper returns a validated model -instance and raises `pydantic.ValidationError` on invalid input. +already filled in (UUID4 default, an identifier from a key, kwarg-style +construction, no need to remember the camelCase aliases). Each helper returns a +validated model instance. + +Each helper validates a JSON-shaped object with `model_validate` one time, so +the model validates each item of a list input, also an item that the caller +made with `model_construct`. Each helper raises `OscalBuilderError` for each +invalid input. It never raises `pydantic.ValidationError`. """ from __future__ import annotations +import contextlib import uuid as _uuid +from typing import Any, TypeVar + +from pydantic import BaseModel, ValidationError from oscal_bindings.v1.models import ( BackMatterResource, @@ -18,6 +27,56 @@ HashAlgorithm, ) +_M = TypeVar("_M", bound=BaseModel) + +#: The namespace of the UUID version 5 that `make_resource` derives from a key. +KEY_NAMESPACE = _uuid.UUID("3942257f-e1cf-5e06-9c6f-eeb97ecabe77") + + +class OscalBuilderError(ValueError): + """Raised when an input to a builder is not valid. + + It is not an `OscalParseError`. It is a `ValueError`, so `except ValueError` + catches it. `errors` holds the structured entries, in the Pydantic entry + shape (`type`, `loc`, `msg`, `input`). When the failure comes from model + validation, `__cause__` is the original `pydantic.ValidationError`. + """ + + def __init__(self, message: str, errors: list[Any] | None = None) -> None: + super().__init__(message) + self.errors = errors or [] + + +def _key_error(error_type: str, msg: str, value: Any) -> OscalBuilderError: + entry = {"type": error_type, "loc": ("key",), "msg": msg, "input": value} + return OscalBuilderError(f"Invalid key: {msg}", [entry]) + + +def _validate(model: type[_M], obj: dict[str, Any]) -> _M: + try: + return model.model_validate(obj) + except ValidationError as exc: + entries = exc.errors(include_url=False) + count = len(entries) + raise OscalBuilderError( + f"Invalid {model.__name__}: {count} validation " + f"error{'s' if count != 1 else ''}", + list(entries), + ) from exc + + +def _json_item(item: Any) -> Any: + """Give a model instance as a JSON-shaped object, and any other item as is.""" + return item.model_dump() if isinstance(item, BaseModel) else item + + +def _json_list(items: list[Any] | None) -> Any: + return items if items is None else [_json_item(item) for item in items] + + +def _compact(**fields: Any) -> dict[str, Any]: + return {name: value for name, value in fields.items() if value is not None} + def make_hash( value: str, @@ -27,12 +86,19 @@ def make_hash( Args: value: The hex-encoded digest. - algorithm: Either a `HashAlgorithm` enum member or its string value - (e.g., `"SHA-256"`). Defaults to SHA-256. + algorithm: Either a `HashAlgorithm` enum member or a string. A string + that is the value of a member (e.g., `"SHA-256"`) gives that + member. Any other string goes to the model as is, and the model + accepts it when the schema does (e.g., `"SHA-999"`). Defaults to + SHA-256. + + Raises: + OscalBuilderError: if the model rejects the algorithm or the value. """ if isinstance(algorithm, str): - algorithm = HashAlgorithm(algorithm) - return Hash(algorithm=algorithm, value=value) + with contextlib.suppress(ValueError): + algorithm = HashAlgorithm(algorithm) + return _validate(Hash, {"algorithm": algorithm, "value": value}) def make_rlink( @@ -41,16 +107,25 @@ def make_rlink( media_type: str | None = None, hashes: list[Hash] | None = None, ) -> BackMatterResourceRlink: - """Build a `BackMatterResourceRlink` element pointing at an external resource.""" - return BackMatterResourceRlink(href=href, media_type=media_type, hashes=hashes) + """Build a `BackMatterResourceRlink` element pointing at an external resource. + + Raises: + OscalBuilderError: if the model rejects an input, also an item of + `hashes` that the caller made with `model_construct`. + """ + return _validate( + BackMatterResourceRlink, + _compact(href=href, **{"media-type": media_type}, hashes=_json_list(hashes)), + ) -def make_resource( +def make_resource( # noqa: PLR0913 -- the public signature names each element field *, title: str | None = None, description: str | None = None, rlinks: list[BackMatterResourceRlink] | None = None, uuid: str | None = None, + key: str | None = None, document_ids: list[DocumentId] | None = None, ) -> BackMatterResource: """Build a `BackMatterResource` element for `back-matter.resources`. @@ -60,16 +135,43 @@ def make_resource( description: Optional short summary. rlinks: Optional list of `BackMatterResourceRlink`s pointing at equivalent URIs. - uuid: Resource UUID. A fresh UUID4 is generated if not provided. + uuid: Resource UUID. When neither `uuid` nor `key` is given, a fresh + UUID4 is generated. + key: Optional stable key of the resource, such as a URL. When `key` is + given and `uuid` is not, the UUID is the UUID version 5 of the + key in the namespace of the back-matter keys, so the same key + always gives the same UUID. An empty key, or a key with a `uuid`, + raises `OscalBuilderError`. See `docs/back-matter-keys.md` in + https://github.com/cairn-proofs/oscal-bindings-specs. document_ids: Optional list of `DocumentId`s recording document identities qualified by scheme. Omitted from the resource when not provided. Validated like `rlinks`: an empty list or items - that are not valid `DocumentId`s raise `pydantic.ValidationError`. + that are not valid `DocumentId`s raise `OscalBuilderError`. + + Raises: + OscalBuilderError: for an invalid key, or if the model rejects an + input, also an item of `rlinks` or `document_ids` that the caller + made with `model_construct`. """ - return BackMatterResource( - uuid=uuid if uuid is not None else str(_uuid.uuid4()), - title=title, - description=description, - rlinks=rlinks, - document_ids=document_ids, + if key is not None: + if not isinstance(key, str): + raise _key_error("string_type", "Input should be a valid string", key) + if uuid is not None: + raise _key_error( + "key_with_uuid", "A key and a uuid cannot both be given", key + ) + if key == "": + raise _key_error("empty_key", "The key cannot be empty", key) + uuid = str(_uuid.uuid5(KEY_NAMESPACE, key)) + elif uuid is None: + uuid = str(_uuid.uuid4()) + return _validate( + BackMatterResource, + _compact( + uuid=uuid, + title=title, + description=description, + rlinks=_json_list(rlinks), + **{"document-ids": _json_list(document_ids)}, + ), ) diff --git a/tests/support/expected_failures.py b/tests/support/expected_failures.py index cf2b0a9..62895a9 100644 --- a/tests/support/expected_failures.py +++ b/tests/support/expected_failures.py @@ -24,13 +24,9 @@ "assessment-period/mixed-offsets": f"{ISSUES}/58", "assessment-period/equal-instant": f"{ISSUES}/58", "assessment-period/seventh-fraction-digit": f"{ISSUES}/58", - # BMB-004, ERR-013: resource identifier from a key, builder-error category. - "build-resource/key": f"{ISSUES}/62", - "build-resource/key-with-title": f"{ISSUES}/62", - "build-resource/non-ascii-key": f"{ISSUES}/62", - "build-resource/key-and-identifier": f"{ISSUES}/62", - "build-resource/empty-key": f"{ISSUES}/62", - # FAC-001 to FAC-003, BMB-001 to BMB-003: the runner lacks these operations. + # BMB-003: the generated DocumentId.identifier has no pattern, so it accepts "". + "build-resource/invalid-document-id": f"{ISSUES}/77", + # FAC-001 to FAC-003: the runner lacks the facade operation. **dict.fromkeys( ( "facade/catalog", @@ -42,15 +38,6 @@ "facade/plan-of-action-and-milestones", "facade/mapping-collection", "facade/missing-file", - "build-hash/default-algorithm", - "build-hash/algorithm-string", - "build-hash/off-list-algorithm", - "build-hash/invalid-algorithm", - "build-rlink/reference-only", - "build-rlink/media-type-and-hashes", - "build-resource/generated-identifier", - "build-resource/omitted-inputs", - "build-resource/invalid-document-id", ), f"{ISSUES}/63", ), diff --git a/tests/support/version_package.py b/tests/support/version_package.py index 5816ed6..77d1855 100644 --- a/tests/support/version_package.py +++ b/tests/support/version_package.py @@ -39,6 +39,7 @@ "MappingCollectionDocument", "OscalAccessError", "OscalBody", + "OscalBuilderError", "OscalDoc", "OscalDocument", "OscalDocumentTypeMismatchError", diff --git a/tests/test_builders.py b/tests/test_builders.py index 15fa957..bc3f67f 100644 --- a/tests/test_builders.py +++ b/tests/test_builders.py @@ -5,9 +5,14 @@ import re import pytest -from pydantic import ValidationError -from oscal_bindings import make_hash, make_resource, make_rlink +from oscal_bindings import ( + OscalBuilderError, + OscalParseError, + make_hash, + make_resource, + make_rlink, +) from oscal_bindings.models import ( BackMatterResource, BackMatterResourceRlink, @@ -37,9 +42,22 @@ def test_algorithm_by_string(self): h = make_hash(DIGEST, algorithm="SHA-384") assert h.algorithm == HashAlgorithm.SHA_384 - def test_invalid_algorithm_string_raises(self): - with pytest.raises(ValueError, match="'MD5' is not a valid HashAlgorithm"): - make_hash(DIGEST, algorithm="MD5") + def test_algorithm_string_equals_enum(self): + by_string = make_hash(DIGEST, algorithm="SHA-256") + by_enum = make_hash(DIGEST, algorithm=HashAlgorithm.SHA_256) + assert by_string == by_enum + assert by_string.algorithm is HashAlgorithm.SHA_256 + + @pytest.mark.parametrize("algorithm", ["MD5", "SHA-999"]) + def test_algorithm_that_is_not_named_is_kept_as_given(self, algorithm): + assert make_hash(DIGEST, algorithm=algorithm).algorithm == algorithm + + def test_invalid_algorithm_raises_builder_error(self): + with pytest.raises(OscalBuilderError) as info: + make_hash(DIGEST, algorithm=" SHA-256") + assert info.value.errors + assert isinstance(info.value, ValueError) + assert not isinstance(info.value, OscalParseError) class TestMakeRlink: @@ -59,6 +77,11 @@ def test_with_hashes(self): link = make_rlink("https://example.com/a.pdf", hashes=[h]) assert link.hashes == [h] + def test_unvalidated_hash_item_raises(self): + bad = Hash.model_construct(algorithm=" SHA-256", value="ab") + with pytest.raises(OscalBuilderError): + make_rlink("https://example.com/a.pdf", hashes=[bad]) + class TestMakeResource: def test_minimal_generates_uuid4(self): @@ -83,7 +106,7 @@ def test_with_title_and_description(self): def test_invalid_title_raises(self): # Resource.title is constrained to one-line strings (^[^\n]+$). - with pytest.raises(ValidationError): + with pytest.raises(OscalBuilderError): make_resource(title="line one\nline two") def test_no_document_ids_by_default(self): @@ -99,15 +122,15 @@ def test_no_document_ids_by_default(self): def test_empty_document_ids_raises(self): # Resource.document_ids has min_length=1 ("document-ids": [] is # invalid OSCAL). - with pytest.raises(ValidationError): + with pytest.raises(OscalBuilderError): make_resource(document_ids=[]) def test_non_document_id_items_raise(self): - with pytest.raises(ValidationError): + with pytest.raises(OscalBuilderError): make_resource(document_ids=["x"]) # type: ignore[list-item] def test_mixed_document_id_items_raise(self): - with pytest.raises(ValidationError): + with pytest.raises(OscalBuilderError): make_resource( document_ids=[DocumentId(identifier="x"), "y"] # type: ignore[list-item] ) @@ -133,6 +156,55 @@ def test_valid_document_ids_equal_alias_dict_construction(self): assert r.document_ids == ids +class TestMakeResourceKey: + def test_same_key_gives_same_uuid(self): + a = make_resource(key="https://example.org/a", title="One") + b = make_resource(key="https://example.org/a", title="Two") + assert a.uuid == b.uuid + + def test_key_gives_uuid_version_5(self): + r = make_resource(key="https://example.org/policies/données") + assert r.uuid == "daa01937-2576-5dc8-9fb0-4da240946a5e" + + def test_key_with_uuid_raises(self): + with pytest.raises(OscalBuilderError) as info: + make_resource(key="k", uuid="f47ac10b-58cc-4372-a567-0e02b2c3d479") + assert [(e["type"], e["loc"]) for e in info.value.errors] == [ + ("key_with_uuid", ("key",)) + ] + + def test_empty_key_raises(self): + with pytest.raises(OscalBuilderError) as info: + make_resource(key="") + assert [(e["type"], e["loc"]) for e in info.value.errors] == [ + ("empty_key", ("key",)) + ] + + def test_no_key_and_no_uuid_gives_uuid_version_4(self): + assert UUID_RE.match(make_resource().uuid) + + @pytest.mark.parametrize("key", [5, b"abc"]) + def test_key_that_is_not_a_string_raises(self, key): + with pytest.raises(OscalBuilderError) as info: + make_resource(key=key) # type: ignore[arg-type] + assert [(e["type"], e["loc"]) for e in info.value.errors] == [ + ("string_type", ("key",)) + ] + + +class TestBuilderErrorIsNotParseError: + def test_resource_error_is_value_error_not_parse_error(self): + with pytest.raises(OscalBuilderError) as info: + make_resource(title="line one\nline two") + assert isinstance(info.value, ValueError) + assert not isinstance(info.value, OscalParseError) + + def test_unvalidated_document_id_item_raises(self): + bad = DocumentId.model_construct() + with pytest.raises(OscalBuilderError): + make_resource(document_ids=[bad]) + + class TestComposability: def test_full_composition(self): resource = make_resource( diff --git a/tests/test_conformance.py b/tests/test_conformance.py index ce41e2a..4c05adf 100644 --- a/tests/test_conformance.py +++ b/tests/test_conformance.py @@ -12,22 +12,25 @@ from __future__ import annotations import json +import re from typing import TYPE_CHECKING, Any import pytest from support.canonical_json import canonicalize from support.expected_failures import EXPECTED_FAILURES -from support.spec_artifact import corpus_root +from support.spec_artifact import corpus_root, schema_path import oscal_bindings as ob from oscal_bindings import ( OscalAccessError, + OscalBuilderError, OscalDoc, OscalDocumentTypeMismatchError, OscalParseError, OscalReadError, OscalUnknownElementTypeError, ) +from oscal_bindings.models import DocumentId, Hash if TYPE_CHECKING: from collections.abc import Callable @@ -50,6 +53,7 @@ (FileNotFoundError, "not-found"), (OscalReadError, "read-error"), (OscalAccessError, "access-error"), + (OscalBuilderError, "builder-error"), (OscalUnknownElementTypeError, "unknown-element-type"), (OscalDocumentTypeMismatchError, "document-type-mismatch"), (OscalParseError, "parse-error"), @@ -184,6 +188,59 @@ def _element_types(case: dict[str, Any]) -> None: assert ob.get_supported_element_types() == expected +def _build( + case: dict[str, Any], builder: Callable[..., Any], arguments: dict[str, Any] +) -> None: + """Run a builder with the keyword arguments of the case, then check the result. + + The result is compared with ``expected-uuid`` or ``expected-element``, or the + builder must report ``expected-category``. + """ + if "expected-category" in case: + _expect_error(case, lambda: builder(**arguments)) + return + result = builder(**arguments) + element = json.loads(result.model_dump_json()) + if "expected-uuid" in case: + assert element["uuid"] == case["expected-uuid"] + if "expected-element" in case: + generated = ( + case["operation"] == "build-resource" + and "uuid" not in case["expected-element"] + ) + if generated: + pattern = json.loads(schema_path().read_text("utf-8"))["definitions"][ + "UUIDDatatype" + ]["pattern"] + assert re.fullmatch(pattern, element.pop("uuid")) + assert element == case["expected-element"] + + +def _build_hash(case: dict[str, Any]) -> None: + _build(case, ob.make_hash, dict(case["arguments"])) + + +def _build_rlink(case: dict[str, Any]) -> None: + arguments = dict(case["arguments"]) + if "media-type" in arguments: + arguments["media_type"] = arguments.pop("media-type") + if "hashes" in arguments: + # The runner gives the builder the element with no validation. + arguments["hashes"] = [Hash.model_construct(**h) for h in arguments["hashes"]] + _build(case, ob.make_rlink, arguments) + + +def _build_resource(case: dict[str, Any]) -> None: + arguments = dict(case["arguments"]) + if "identifier" in arguments: + arguments["uuid"] = arguments.pop("identifier") + if "document-ids" in arguments: + arguments["document_ids"] = [ + DocumentId.model_construct(**d) for d in arguments.pop("document-ids") + ] + _build(case, ob.make_resource, arguments) + + # Each operation takes the case, and the input path when the case has ``input``. OPERATIONS: dict[str, Callable[..., None]] = { "round-trip": _round_trip, @@ -195,6 +252,9 @@ def _element_types(case: dict[str, Any]) -> None: "element-types": _element_types, "yaml-round-trip": _yaml_round_trip, "yaml-parse": _yaml_parse, + "build-hash": _build_hash, + "build-rlink": _build_rlink, + "build-resource": _build_resource, } @@ -229,11 +289,6 @@ def test_every_expected_failure_names_a_corpus_case() -> None: assert stale == [], f"expected failures not in the pinned corpus: {stale}" -@pytest.mark.xfail( - strict=True, - reason="builder-error is not mapped yet: " - "https://github.com/cairn-proofs/oscal-bindings-python/issues/62", -) def test_every_category_is_mapped() -> None: """Each category in the corpus has a branch in ``category``.""" index = json.loads((corpus_root() / "cases.json").read_text("utf-8")) @@ -241,6 +296,7 @@ def test_every_category_is_mapped() -> None: category(FileNotFoundError()), category(OscalReadError()), category(OscalAccessError()), + category(OscalBuilderError("Invalid Hash")), category(OscalDocumentTypeMismatchError("Document is not a Catalog")), category(OscalUnknownElementTypeError("Unknown element type: x")), category(OscalParseError("Failed to parse")), diff --git a/tests/test_serialization_defaults.py b/tests/test_serialization_defaults.py index 62467c0..adf365b 100644 --- a/tests/test_serialization_defaults.py +++ b/tests/test_serialization_defaults.py @@ -12,7 +12,7 @@ ``json.loads(serialize_oscal(doc))``. The same check over the whole corpus is in ``tests/test_prop_serialization_defaults.py``. - Keyword construction by Python field name equals construction by alias. -- ``make_resource`` validates ``document_ids`` and raises ``ValidationError``. +- ``make_resource`` validates ``document_ids`` and raises ``OscalBuilderError``. Fix-checking unit tests (task 4.1) follow: opt-outs (``by_alias=False``, ``exclude_none=False``, ``include``/``exclude``/``exclude_unset``), construction @@ -37,6 +37,7 @@ from support.oscal_shape import DEFAULT_PATHS, assert_valid_oscal_shape from oscal_bindings import ( + OscalBuilderError, make_resource, parse_catalog, parse_oscal, @@ -182,7 +183,7 @@ def test_rlink_media_type_by_name_equals_alias() -> None: [pytest.param([], id="empty"), pytest.param(["x"], id="non-DocumentId")], ) def test_make_resource_rejects_invalid_document_ids(document_ids: list[Any]) -> None: - with pytest.raises(ValidationError): + with pytest.raises(OscalBuilderError): make_resource(document_ids=document_ids)