diff --git a/.agents/summary/components.md b/.agents/summary/components.md index b160740..3a29f54 100644 --- a/.agents/summary/components.md +++ b/.agents/summary/components.md @@ -53,7 +53,8 @@ graph LR | Symbol | Kind | Purpose | |--------|------|---------| | `parse_oscal(data)` | function | `str \| bytes` → `OscalDocument` via `model_validate_json` | -| `parse_oscal_file(path)` | function | Reads file as **bytes**, delegates to `parse_oscal` | +| `parse_oscal_file(path)` | function | Reads file as **bytes**, delegates to `parse_oscal`; `FileNotFoundError` for a missing path, `OscalReadError` for another read failure | +| `OscalReadError` | exception | Subclass of `OSError`; a file-path parse raises it for a read failure that is not a missing path | | `serialize_oscal(model, *, indent=2)` | function | `model_dump_json(by_alias=True, exclude_none=True)` | | `validate_oscal(data)` | function | Returns `bool`, swallows errors | | `parse_catalog`, `parse_profile`, `parse_component_definition`, `parse_system_security_plan`, `parse_assessment_plan`, `parse_assessment_results`, `parse_plan_of_action_and_milestones`, `parse_mapping_collection` | functions | Typed parsers; each `hasattr`-checks `doc.root` for the expected body attr, else raises `OscalParseError` | diff --git a/.agents/summary/interfaces.md b/.agents/summary/interfaces.md index a6d3f88..a0f0b05 100644 --- a/.agents/summary/interfaces.md +++ b/.agents/summary/interfaces.md @@ -122,7 +122,8 @@ sequenceDiagram |-----------|----------| | Invalid JSON / invalid UTF-8 bytes | `OscalParseError` (from `parse_oscal`) | | Schema validation failure | `OscalParseError` with `.errors` = Pydantic error dicts | -| File missing | `FileNotFoundError` (from `parse_oscal_file`, includes path) | +| File missing (`ENOENT`, `ENOTDIR`, or an unrepresentable path) | `FileNotFoundError` (from a file-path parse, includes path) | +| File that cannot be read for another reason (directory, permission, symlink loop, I/O error) | `OscalReadError` (a subclass of `OSError`, outside the parse errors; `filename` and message name the path) | | Typed parser, wrong document type | `OscalDocumentTypeMismatchError("Document is not a …")` (subclass of `OscalParseError`) | | `assessment_period()` on non-AP/AR | `OscalAccessError` | | `OscalDoc(...)` with unrelated object | `TypeError` | diff --git a/.agents/summary/review_notes.md b/.agents/summary/review_notes.md index c6c35e3..08dfa98 100644 --- a/.agents/summary/review_notes.md +++ b/.agents/summary/review_notes.md @@ -17,7 +17,7 @@ Cross-document facts were verified against the source files (`v1/parser.py`, `v1 | Top level is a pure re-export; `__all__` derived from `v1.__all__` | ✅ | codebase_info, architecture, components, interfaces, index agree; asserted by `tests/test_oscal_bindings.py` | | Active schema release `1.2.3` / `__oscal_schema_version__` | ✅ | index, codebase_info, architecture, data_models, workflows, dependencies agree; tests assert it against the release of the pin and the artifact schema `$id` | | Names come only from `names.json` (hoist + post-processor guards; no naming rule in the binding) | ✅ | architecture, workflows, interfaces, components, data_models, index agree | -| Exceptions `OscalParseError` (with `.errors`) / `OscalAccessError` | ✅ | Consistent; `validate_element` raises `OscalUnknownElementTypeError` for an unrecognized type, a separate category | +| Exceptions `OscalParseError` (with `.errors`) / `OscalAccessError` / `OscalReadError` (read failure, beside `FileNotFoundError`) | ✅ | Consistent; `validate_element` raises `OscalUnknownElementTypeError` for an unrecognized type, a separate category | | `serialize_oscal` defaults (`indent=2`, `by_alias`, `exclude_none`) | ✅ | Matches source | | Two `RootModel`s: `OscalDocument`, `MappingCollectionMappings` | ✅ | architecture, components, data_models, workflows agree | | Single runtime dependency (Pydantic v2) | ✅ | dependencies.md matches `pyproject.toml` | diff --git a/.agents/summary/workflows.md b/.agents/summary/workflows.md index dacbf21..cb24abf 100644 --- a/.agents/summary/workflows.md +++ b/.agents/summary/workflows.md @@ -103,7 +103,7 @@ flowchart TD F -->|no| E ``` -`parse_oscal_file` reads the file as **bytes** (`read_bytes()`) and delegates to `parse_oscal`, so file parsing skips a decode step. A missing file raises `FileNotFoundError` before parsing. +`parse_oscal_file` reads the file as **bytes** (`read_bytes()`) and delegates to `parse_oscal`, so file parsing skips a decode step. A missing file raises `FileNotFoundError` and any other read failure raises `OscalReadError`, both before parsing; the read itself finds the category, with no `exists()` call. ## 3. Uniform Access via `OscalDoc` diff --git a/AGENTS.md b/AGENTS.md index 2b99758..2d2f2db 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -94,6 +94,7 @@ See `.agents/summary/workflows.md` for the flow diagram. ## Repo-Specific Patterns & Gotchas - **Parsers accept `str | bytes`.** Passing `bytes` (from HTTP/S3/file) skips a UTF-8 decode; `parse_oscal_file` reads bytes deliberately. Invalid UTF-8 surfaces as `OscalParseError`, not `UnicodeDecodeError`. +- **A file-path parse finds the error category from the read itself.** `_read_oscal_file` (`v1/parser.py`) makes no `Path.exists()` call: `read_bytes` raises, and `ENOENT`, `ENOTDIR`, or a `ValueError` (a NUL byte) gives `FileNotFoundError`; each other `OSError` gives `OscalReadError(OSError)`, which is not a parse error and not a not-found error (ERR-012). `exists()` hides the `errno` and changes its answer with the Python version. `parse_oscal_file`, `parse_oscal_yaml_file`, and `OscalDoc.from_file` all share this path. - **Typed parsers return the wrapper, not the body.** `parse_catalog()` returns `CatalogDocument` (access `.catalog.metadata...`). It `hasattr`-checks `doc.root` and raises `OscalDocumentTypeMismatchError("Document is not a Catalog")` (a subclass of `OscalParseError`) on mismatch. - **`OscalDoc` collapses per-type dispatch.** Use it when you don't want to branch on document type — `doc.metadata`, `doc.uuid`, `doc.oscal_version`, `doc.body` work uniformly via the `_WRAPPER_TO_BODY_ATTR` map. `assessment_period()` is defined **only** for Assessment Plan / Assessment Results (raises `OscalAccessError` otherwise). - **Valid OSCAL by default; construct by name, validate by alias.** `OscalBaseModel` (`v1/_base.py`) gives every object model alias keys (`media-type`, `document-ids`) and no `null` on all serialization paths (`model_dump*`, `to_json`, `TypeAdapter`); `exclude_none=False` is a no-op, `by_alias=False` emits names. Constructor kwargs accept field names or aliases (`BackMatterResourceRlink(media_type=...)`), so the builders construct directly. OSCAL input stays alias-only: dict input rejects name keys with `extra_forbidden` at every depth (`extra='forbid'`). **Don't add `validate_by_name`/`populate_by_name` to the base config**, and don't remove `__pydantic_base_init__` from its `__init__`: either makes `model_validate*` accept name-keyed OSCAL (DEVELOPING "Valid OSCAL by default"; tripwire tests guard both). JSON input rejects name keys too, which needs the pydantic ≥2.14.2 floor (2.13.x–2.14.1 silently dropped them, [#6](https://github.com/cairn-proofs/oscal-bindings-python/issues/6)). diff --git a/README.md b/README.md index 98f8705..6de2971 100644 --- a/README.md +++ b/README.md @@ -302,7 +302,7 @@ Each OSCAL document type has a dedicated parser: - `parse_plan_of_action_and_milestones(json)` → `PlanOfActionAndMilestonesDocument` - `parse_mapping_collection(json)` → `MappingCollectionDocument` -A typed parser raises `OscalDocumentTypeMismatchError` for a document of another type. It is a subclass of `OscalParseError`, so `except OscalParseError` also catches it. `validate_element` raises `OscalUnknownElementTypeError` for an element type that the library does not recognize. That error is a category of its own, outside the parse errors. Test the category of an error with `isinstance`. +A typed parser raises `OscalDocumentTypeMismatchError` for a document of another type. It is a subclass of `OscalParseError`, so `except OscalParseError` also catches it. `validate_element` raises `OscalUnknownElementTypeError` for an element type that the library does not recognize. That error is a category of its own, outside the parse errors. A file-path parse raises `FileNotFoundError` when no file can exist at the path, and `OscalReadError` for each other read failure (a directory, a permission failure, a loop of symbolic links). `OscalReadError` is a subclass of `OSError`, not of `OscalParseError` or of `FileNotFoundError`, so `except OSError` catches both file categories and `except OscalParseError` catches neither. Test the category of an error with `isinstance`. ## Building diff --git a/openspec/changes/archive/2026-11-17-report-read-error/.openspec.yaml b/openspec/changes/archive/2026-11-17-report-read-error/.openspec.yaml new file mode 100644 index 0000000..bbe8b39 --- /dev/null +++ b/openspec/changes/archive/2026-11-17-report-read-error/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-11-17 diff --git a/openspec/changes/archive/2026-11-17-report-read-error/design.md b/openspec/changes/archive/2026-11-17-report-read-error/design.md new file mode 100644 index 0000000..4a60d2a --- /dev/null +++ b/openspec/changes/archive/2026-11-17-report-read-error/design.md @@ -0,0 +1,91 @@ +# Design + +## Context + +See [proposal.md - Why](proposal.md#why) for the motivation. These facts constrain the approach. + +1. Three file-path parses read the file through `_read_oscal_file` in `parser.py`: `parse_oscal_file`, + `parse_oscal_yaml_file` (through `_yaml_loader.py`), and `OscalDoc.from_file` (through `parse_oscal_file`). Thus one + edit to `_read_oscal_file` gives the same behavior to all three. +2. `Path.read_bytes` raises a subclass of `OSError` for each read failure. The subclass follows the `errno`: `ENOENT` + gives `FileNotFoundError`, `ENOTDIR` gives `NotADirectoryError`, `EISDIR` gives `IsADirectoryError`, and `EACCES` + gives `PermissionError`. `ELOOP` gives a plain `OSError`. +3. The conformance runner (`category` in `tests/test_conformance.py`) maps a class to a category. + `test_every_category_is_mapped` is a strict expected failure until each category of the corpus has a class. The + builder-error category ([#62]) has no class yet. +4. A name in `v1.__all__` must be in the surface snapshot (`tests/support/version_package.py`) and in + `openspec/traceability.json`. + +## Goals / Non-Goals + +**Goals:** + +1. The category of a read failure is the same on each supported Python version. + +**Non-Goals:** + +1. A change to the message of the not-found error. + +## Decisions + +### D1: `OscalReadError` is a subclass of `OSError` + +The not-found category is `FileNotFoundError`, which is a subclass of `OSError`. A read error is also a failure of the +operating system to give the bytes of a file. With `OscalReadError(OSError)`, the two file categories have one parent +in the Python hierarchy: + +``` +OSError + +-- FileNotFoundError (not-found) + +-- OscalReadError (read error) +``` + +1. A caller catches each failure to read a file with `except OSError`, and catches no parse error with it. +2. `OscalReadError` has `errno`, `strerror`, and `filename` from `OSError`, with no new code. +3. The binding raises it as `OscalReadError(e.errno, e.strerror, str(path)) from e`. `OSError.__str__` then gives + `[Errno 21] Is a directory: 'documents/valid'`, which names the path (ERR-012). +4. Python maps an `errno` to a subclass only when the code calls `OSError` itself. A call to a subclass does not get + this mapping. Thus `OscalReadError(errno.EISDIR, ...)` stays an `OscalReadError`. + +The class is in `parser.py`, below `OscalDocumentTypeMismatchError`. It goes through the re-export chain: `v1/__init__.py` +and the explicit import list and `__all__` of the alias module `src/oscal_bindings/parser.py`. + +Alternative: `OscalReadError(Exception)`, beside `OscalAccessError`. A caller then needs two `except` clauses to catch +each failure to read a file, and the error has no `errno`. Rejected. + +### D2: The read itself finds the category, with no `exists()` call + +`_read_oscal_file` calls `read_bytes` with no check before it, and maps the exception: + +1. `FileNotFoundError` or `NotADirectoryError` gives `FileNotFoundError` with the current message, + `OSCAL file not found: `, raised `from` the original error. +2. Each other `OSError` gives `OscalReadError`, as [D1](#d1-oscalreaderror-is-a-subclass-of-oserror) shows. +3. A `ValueError` (a path that the operating system cannot represent, such as one with a NUL byte or an + unencodable surrogate) gives `FileNotFoundError` with the current message, raised `from` the original error. No + file can exist at such a path, and the failure has nothing to do with OSCAL. This keeps the current behavior. + +The exception that the read raises is the only fact that decides. `Path.exists()` hides the `errno`, and its result for a failure +that is not `ENOENT` changes with the Python version. The check before the read also allowed a race: the path can change +between `exists()` and `read_bytes`. A single read removes the race. + +Alternative: keep `exists()` and add a `try` around it. The result of `exists()` is still a bool, so the `errno` stays +hidden. Rejected. + +### D3: Tests of read failures that the corpus cannot hold + +The corpus holds only the directory case. The unit tests in `tests/test_parser.py` create each other case in `tmp_path`: + +1. A file with mode `0o000` (ERR-012.02). +2. A loop of two symbolic links. +3. A file in a directory with mode `0o000`. + +A process with root privileges reads each of these paths. Thus each of these tests is skipped when `os.geteuid() == 0`. +Each test sets the mode back in a `finally` block, so that pytest can delete `tmp_path`. + +## Risks / Trade-offs + +1. [Code that catches `OscalParseError` around a file-path parse stops catching a read failure.] → The commit is a + `feat!` with a `BREAKING CHANGE` footer, and the README says that `except OSError` catches both file categories. +2. [A test run as root skips the permission tests.] → The CI runner is not root, so CI runs them. + +[#62]: https://github.com/cairn-proofs/oscal-bindings-python/issues/62 diff --git a/openspec/changes/archive/2026-11-17-report-read-error/proposal.md b/openspec/changes/archive/2026-11-17-report-read-error/proposal.md new file mode 100644 index 0000000..2ebbd45 --- /dev/null +++ b/openspec/changes/archive/2026-11-17-report-read-error/proposal.md @@ -0,0 +1,56 @@ +# Proposal + +## Why + +The store requirement ERR-012 (cairn-proofs/oscal-bindings-specs#41, PR cairn-proofs/oscal-bindings-specs#55) says that a +file-path parse that cannot read the file at a path reports the read-error category of ERR-001. A path that the platform +reports as missing is the exception. It is not-found (ERR-006). A read error names the path. It is not a parse error and +not a not-found error ([#61]). + +The binding does not meet ERR-012: + +1. The binding has no read-error category. `_read_oscal_file` in `src/oscal_bindings/v1/parser.py` raises + `OscalParseError` for each read failure. Thus a directory gives a parse error, and the corpus case + `parse-file/directory` is an expected failure. +2. `_read_oscal_file` calls `Path.exists()` before the read, and raises `FileNotFoundError` when `exists()` is false. On + Python 3.12, `exists()` is false for `ELOOP` and `EBADF`. Thus a loop of symbolic links gives not-found. On Python 3.14, + `exists()` is false for each `OSError`. Thus a path with a directory that the caller cannot search gives not-found. The + category of a read failure changes with the Python version. + +The pin is at `v0.8.0+oscal.1.2.3` (#70), the release that has ERR-012. Thus the fix is now possible. + +## What Changes + +1. A new exception `OscalReadError`, a subclass of `OSError`. It is not a subclass of `OscalParseError` or of + `FileNotFoundError`. Each file-path parse raises it for a read failure that is not a missing path. Its `errno`, + `strerror`, and `filename` come from the failure, and its message names the path. +2. A file-path parse raises `FileNotFoundError` only when the read fails with `ENOENT` or `ENOTDIR`. The binding does not + call `Path.exists()` to find the category. +3. **BREAKING**: a file-path parse does not raise `OscalParseError` for a read failure. Code that catches + `OscalParseError` for a directory, a permission failure, or an I/O error does not catch the new error. Code that + catches `OSError` catches both the read error and not-found. +4. The conformance runner maps `OscalReadError` to the category `read-error`. The corpus case `parse-file/directory` + passes. + +## Capabilities + +### New Capabilities + +None. + +### Modified Capabilities + +1. `error-tests`: the list of categories and classes gets the read-error category and its class `OscalReadError`. + +## Impact + +1. Code: `src/oscal_bindings/v1/parser.py` and the re-exports in `src/oscal_bindings/v1/__init__.py` and the alias + module `src/oscal_bindings/parser.py`. The YAML file parser and `OscalDoc.from_file` use `_read_oscal_file`, so they + get the same behavior with no edit to their code. +2. Tests: `tests/test_conformance.py`, `tests/support/expected_failures.py`, `tests/test_parser.py`, and + `tests/test_yaml_io.py`. +3. Docs: the `Raises` sections of `parse_oscal_file` and `parse_oscal_yaml_file`, `README.md`, `AGENTS.md`, and + `.agents/summary/interfaces.md`. +4. Public API: one new exception class, and the exception that a file-path parse raises for a read failure. + +[#61]: https://github.com/cairn-proofs/oscal-bindings-python/issues/61 diff --git a/openspec/changes/archive/2026-11-17-report-read-error/specs/error-tests/spec.md b/openspec/changes/archive/2026-11-17-report-read-error/specs/error-tests/spec.md new file mode 100644 index 0000000..45ceb79 --- /dev/null +++ b/openspec/changes/archive/2026-11-17-report-read-error/specs/error-tests/spec.md @@ -0,0 +1,80 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: The class of an error is its category + +For each category in this list, an error SHALL be in the category exactly when it is an instance of the class: + +1. Parse error: `OscalParseError`. +2. Document-type mismatch: `OscalDocumentTypeMismatchError`, a subclass of `OscalParseError`. +3. Unknown element type: `OscalUnknownElementTypeError`, which is not a subclass of `OscalParseError`. +4. Access error: `OscalAccessError`. +5. Not-found: `FileNotFoundError`. +6. Read error: `OscalReadError`, a subclass of `OSError`, which is not a subclass of `OscalParseError` or of + `FileNotFoundError`. + +A caller tests a category with `isinstance`. + +#### Scenario: A document-type mismatch is also a parse error + +- **WHEN** the caller parses a profile with the catalog parser +- **THEN** the error is an instance of `OscalDocumentTypeMismatchError` and an instance of `OscalParseError` + +#### Scenario: A parse error of another kind is not a document-type mismatch + +- **WHEN** the caller parses input that is not valid JSON +- **THEN** the error is an instance of `OscalParseError` and not an instance of `OscalDocumentTypeMismatchError` + +#### Scenario: An unknown element type is not a parse error + +- **WHEN** the caller validates an element with an element type that the binding does not recognize +- **THEN** the error is an instance of `OscalUnknownElementTypeError` and not an instance of `OscalParseError` + +#### Scenario: A read error is not a parse error or a not-found error + +- **WHEN** the caller parses a file at a path that is a directory +- **THEN** the error is an instance of `OscalReadError` and an instance of `OSError` +- **AND** the error is not an instance of `OscalParseError` and not an instance of `FileNotFoundError` + +## ADDED Requirements + +### Requirement: A file-path parse finds the category of a read failure from the read + +A file-path parse SHALL find the category of a read failure from the error of the read itself: + +1. When no file can exist at the path (`ENOENT`, `ENOTDIR`, or a `ValueError` for a path that the operating system + cannot represent), the error is not-found. +2. For each other read failure, the error is a read error. Its `filename` is the path, and its message names the path. + +A file-path parse does not test whether the path exists before the read. + +#### Scenario: A missing path is not-found + +- **WHEN** the caller parses a file at a path that does not exist +- **THEN** the error is an instance of `FileNotFoundError` + +#### Scenario: A path below a regular file is not-found + +- **WHEN** the caller parses a file at a path whose parent is a regular file +- **THEN** the error is an instance of `FileNotFoundError` and not an instance of `OscalReadError` + +#### Scenario: A path that the operating system cannot represent is not-found + +- **WHEN** the caller parses a file at a path that contains a NUL byte +- **THEN** the error is an instance of `FileNotFoundError` and not an instance of `OscalReadError` + +#### Scenario: A file without read permission is a read error + +- **WHEN** the caller parses a file that the caller has no permission to read +- **THEN** the error is an instance of `OscalReadError`, and its message names the path + +#### Scenario: A loop of symbolic links is a read error + +- **WHEN** the caller parses a file at a path that is a loop of symbolic links +- **THEN** the error is an instance of `OscalReadError`, and its message names the path + +#### Scenario: A directory that the caller cannot search is a read error + +- **WHEN** the caller parses a file at a path through a directory that the caller has no permission to search +- **THEN** the error is an instance of `OscalReadError`, and its message names the path diff --git a/openspec/changes/archive/2026-11-17-report-read-error/tasks.md b/openspec/changes/archive/2026-11-17-report-read-error/tasks.md new file mode 100644 index 0000000..a00995b --- /dev/null +++ b/openspec/changes/archive/2026-11-17-report-read-error/tasks.md @@ -0,0 +1,66 @@ +# Tasks + +## 1. Baseline before the change + +- [x] 1.1 Run `hatch run python scripts/bench_serialization.py` and `hatch run python scripts/bench_memory.py` on + `main`. Record the medians, GC on and GC off, in the scratchpad. Verify that the record is not empty. + +## 2. Read error + +- [x] 2.1 Add `OscalReadError(OSError)` to `src/oscal_bindings/v1/parser.py`, below `OscalDocumentTypeMismatchError`, + as [design D1](design.md#d1-oscalreaderror-is-a-subclass-of-oserror) states. Its docstring says that it is not a + parse error and not a not-found error, and that `except OSError` catches it and `FileNotFoundError`. Re-export it from + `src/oscal_bindings/v1/__init__.py`, and add it to the import list and `__all__` of the alias module + `src/oscal_bindings/parser.py`. Add it to `tests/support/version_package.py`, and add an entry with `ERR-001` and + `ERR-012` to `openspec/traceability.json`. Verify that `from oscal_bindings.parser import OscalReadError` gives the + same class as `oscal_bindings.v1.OscalReadError`. +- [x] 2.2 Change `_read_oscal_file` to read with no `exists()` call, and map the exception as + [design D2](design.md#d2-the-read-itself-finds-the-category-with-no-exists-call) states. Update the `Raises` sections + of `_read_oscal_file`, `parse_oscal_file`, and `parse_oscal_yaml_file` (`src/oscal_bindings/v1/extensions/yaml_io.py`). + Verify with new tests in `tests/test_parser.py`: + 1. A directory raises an error that is an instance of `OscalReadError` and of `OSError`, and not an instance of + `OscalParseError` or of `FileNotFoundError`. Its `filename` is the path, and `str` of the error contains the path. + 2. A file with mode `0o000`, a loop of two symbolic links, and a file in a directory with mode `0o000` each raise + `OscalReadError` whose `str` contains the path, as [design D3](design.md#d3-tests-of-read-failures-that-the-corpus-cannot-hold) + states. Each test is skipped when `os.geteuid() == 0`. + 3. A path below a regular file (`ENOTDIR`) raises `FileNotFoundError` whose `str` contains the path. + 4. A path that contains a NUL byte raises `FileNotFoundError`, chained from the `ValueError`, and not + `ValueError` itself. + 5. The existing not-found tests in `tests/test_parser.py` and `tests/test_yaml_io.py` pass with no edit. +- [x] 2.3 Add a test in `tests/test_yaml_io.py`: `parse_oscal_yaml_file` of a directory raises `OscalReadError`. Verify + that it passes. + +## 3. Conformance runner + +- [x] 3.1 In `tests/test_conformance.py`, map `OscalReadError` to `read-error` in `category`, and add + `category(OscalReadError())` to the set in `test_every_category_is_mapped`. Change the reason of that strict expected + failure to cite only [#62]. Remove `parse-file/directory` from `tests/support/expected_failures.py`. Verify with + `hatch test tests/test_conformance.py` that `parse-file/directory` passes and that + `test_every_category_is_mapped` is still an expected failure. + +## 4. Docs + +- [x] 4.1 Update the error docs for the read-error category. Verify with a `grep` for `FileNotFoundError` in each file + that each statement about a file-path failure names both categories: + 1. `README.md`: the error paragraph at line 305 names `OscalReadError` and says that `except OSError` catches both + file categories. The comment at line 196 stays. + 2. `AGENTS.md`: a note in "Repo-Specific Patterns & Gotchas" that a file-path parse finds the category from the read, + with no `exists()` call. + 3. `.agents/summary/interfaces.md`: a row for the read failure in the error table, below the row at line 125. + 4. `.agents/summary/workflows.md` line 106, `.agents/summary/components.md`, and `.agents/summary/review_notes.md` + line 20: the read error beside not-found. + +## 5. Integration + +- [x] 5.1 Run `hatch run release`. Verify that it passes, and that `git diff --exit-code -- src/oscal_bindings/v1/models.py` + shows no change. +- [x] 5.2 Run the two benchmark scripts of task 1.1 again. Record the before and after medians, GC on and GC off, for + the PR description. Verify that the record has both runs. + +## Workflow follow-up + +- Commit as `feat!:` with a `BREAKING CHANGE` footer that says a file-path parse raises `OscalReadError`, not + `OscalParseError`, for a read failure. Close #61. +- Archive the change after review. + +[#62]: https://github.com/cairn-proofs/oscal-bindings-python/issues/62 diff --git a/openspec/specs/error-tests/spec.md b/openspec/specs/error-tests/spec.md index 0974f2d..7a19bf1 100644 --- a/openspec/specs/error-tests/spec.md +++ b/openspec/specs/error-tests/spec.md @@ -15,6 +15,8 @@ For each category in this list, an error SHALL be in the category exactly when i 3. Unknown element type: `OscalUnknownElementTypeError`, which is not a subclass of `OscalParseError`. 4. Access error: `OscalAccessError`. 5. Not-found: `FileNotFoundError`. +6. Read error: `OscalReadError`, a subclass of `OSError`, which is not a subclass of `OscalParseError` or of + `FileNotFoundError`. A caller tests a category with `isinstance`. @@ -33,6 +35,52 @@ A caller tests a category with `isinstance`. - **WHEN** the caller validates an element with an element type that the binding does not recognize - **THEN** the error is an instance of `OscalUnknownElementTypeError` and not an instance of `OscalParseError` +#### Scenario: A read error is not a parse error or a not-found error + +- **WHEN** the caller parses a file at a path that is a directory +- **THEN** the error is an instance of `OscalReadError` and an instance of `OSError` +- **AND** the error is not an instance of `OscalParseError` and not an instance of `FileNotFoundError` + +### Requirement: A file-path parse finds the category of a read failure from the read + +A file-path parse SHALL find the category of a read failure from the error of the read itself: + +1. When no file can exist at the path (`ENOENT`, `ENOTDIR`, or a `ValueError` for a path that the operating system + cannot represent), the error is not-found. +2. For each other read failure, the error is a read error. Its `filename` is the path, and its message names the path. + +A file-path parse does not test whether the path exists before the read. + +#### Scenario: A missing path is not-found + +- **WHEN** the caller parses a file at a path that does not exist +- **THEN** the error is an instance of `FileNotFoundError` + +#### Scenario: A path below a regular file is not-found + +- **WHEN** the caller parses a file at a path whose parent is a regular file +- **THEN** the error is an instance of `FileNotFoundError` and not an instance of `OscalReadError` + +#### Scenario: A path that the operating system cannot represent is not-found + +- **WHEN** the caller parses a file at a path that contains a NUL byte +- **THEN** the error is an instance of `FileNotFoundError` and not an instance of `OscalReadError` + +#### Scenario: A file without read permission is a read error + +- **WHEN** the caller parses a file that the caller has no permission to read +- **THEN** the error is an instance of `OscalReadError`, and its message names the path + +#### Scenario: A loop of symbolic links is a read error + +- **WHEN** the caller parses a file at a path that is a loop of symbolic links +- **THEN** the error is an instance of `OscalReadError`, and its message names the path + +#### Scenario: A directory that the caller cannot search is a read error + +- **WHEN** the caller parses a file at a path through a directory that the caller has no permission to search +- **THEN** the error is an instance of `OscalReadError`, and its message names the path + ### Requirement: Each documented limit has its own identifier Each documented limit SHALL have the identifier in this list: diff --git a/openspec/traceability.json b/openspec/traceability.json index 99a7377..6fb4a80 100644 --- a/openspec/traceability.json +++ b/openspec/traceability.json @@ -29,6 +29,7 @@ { "name": "make_resource", "requirements": ["BMB-003"] }, { "name": "OscalParseError", "requirements": ["ERR-001", "ERR-003", "ERR-009", "ERR-010"] }, { "name": "OscalDocumentTypeMismatchError", "requirements": ["ERR-005", "ERR-009"] }, + { "name": "OscalReadError", "requirements": ["ERR-001", "ERR-012"] }, { "name": "OscalUnknownElementTypeError", "requirements": ["ERR-008", "ERR-009"] } ] } diff --git a/src/oscal_bindings/parser.py b/src/oscal_bindings/parser.py index 49d23db..a300e6d 100644 --- a/src/oscal_bindings/parser.py +++ b/src/oscal_bindings/parser.py @@ -22,6 +22,7 @@ OscalDocument, OscalDocumentTypeMismatchError, OscalParseError, + OscalReadError, PlanOfActionAndMilestonesDocument, ProfileDocument, SystemSecurityPlanDocument, @@ -48,6 +49,7 @@ "OscalDocument", "OscalDocumentTypeMismatchError", "OscalParseError", + "OscalReadError", "PlanOfActionAndMilestonesDocument", "ProfileDocument", "SystemSecurityPlanDocument", diff --git a/src/oscal_bindings/v1/__init__.py b/src/oscal_bindings/v1/__init__.py index 1bf0815..062db2d 100644 --- a/src/oscal_bindings/v1/__init__.py +++ b/src/oscal_bindings/v1/__init__.py @@ -61,6 +61,7 @@ OscalDocument, OscalDocumentTypeMismatchError, OscalParseError, + OscalReadError, PlanOfActionAndMilestonesDocument, ProfileDocument, SystemSecurityPlanDocument, @@ -103,6 +104,7 @@ "OscalDocument", "OscalDocumentTypeMismatchError", "OscalParseError", + "OscalReadError", "OscalUnknownElementTypeError", "OscalWrapper", "PlanOfActionAndMilestonesDocument", diff --git a/src/oscal_bindings/v1/extensions/yaml_io.py b/src/oscal_bindings/v1/extensions/yaml_io.py index 3f53404..0bf3abb 100644 --- a/src/oscal_bindings/v1/extensions/yaml_io.py +++ b/src/oscal_bindings/v1/extensions/yaml_io.py @@ -145,9 +145,10 @@ def parse_oscal_yaml_file(path: Path | str) -> OscalDocument: An :class:`~oscal_bindings.v1.models.OscalDocument` for the document. Raises: - FileNotFoundError: The file does not exist. - OscalParseError: The file cannot be read, or its content is not valid - OSCAL YAML, as :func:`parse_oscal_yaml` defines. + FileNotFoundError: No file can exist at the path. + OscalReadError: The file cannot be read for another reason. + OscalParseError: The content of the file is not valid OSCAL YAML, as + :func:`parse_oscal_yaml` defines. ImportError: The ``yaml`` extra is not installed. """ _require_yaml() diff --git a/src/oscal_bindings/v1/parser.py b/src/oscal_bindings/v1/parser.py index b73e587..309dd7d 100644 --- a/src/oscal_bindings/v1/parser.py +++ b/src/oscal_bindings/v1/parser.py @@ -97,6 +97,20 @@ class OscalDocumentTypeMismatchError(OscalParseError): """ +class OscalReadError(OSError): + """Raised when a file-path parse cannot read the file at a path (ERR-012). + + A subclass of :class:`OSError`, so ``except OSError`` catches it and + :class:`FileNotFoundError`, the not-found error. It is not a parse error + and not a not-found error: it is not an instance of + :class:`OscalParseError` or of :class:`FileNotFoundError`. A path that the + platform reports as missing raises :class:`FileNotFoundError`. Each other + read failure (a directory, a permission failure, a loop of symbolic links, + an I/O error) raises this error. Its ``errno``, ``strerror``, and + ``filename`` come from the failure, and its message names the path. + """ + + #: The JSON depth limit (PRS-010). A value that more than this many arrays and #: objects hold is a parse error. pydantic-core's JSON parser sets the value, #: and tests pin it at the boundary. @@ -225,9 +239,10 @@ def parse_oscal_file(path: Path | str) -> OscalDocument: An OscalDocument instance representing the parsed document. Raises: - OscalParseError: If the file cannot be read or its contents do not - conform to the OSCAL schema. - FileNotFoundError: If the file does not exist. + OscalParseError: If the contents of the file do not conform to the + OSCAL schema. + FileNotFoundError: If no file can exist at the path. + OscalReadError: If the file cannot be read for another reason. """ return parse_oscal(_read_oscal_file(path)) @@ -236,20 +251,22 @@ def _read_oscal_file(path: Path | str) -> bytes: """Read an OSCAL file for a file parser (PRS-002). Shared by :func:`parse_oscal_file` and the YAML file parser, so both report - a missing file and a read failure the same way. + a missing file and a read failure the same way. The error of the read + itself finds the category, with no ``exists()`` call before it. Raises: - FileNotFoundError: The file does not exist. - OscalParseError: The file exists but cannot be read. + FileNotFoundError: No file can exist at the path (``ENOENT``, + ``ENOTDIR``, or a path that the operating system cannot + represent). + OscalReadError: The read failed for another reason. """ file_path = Path(path) - if not file_path.exists(): - raise FileNotFoundError(f"OSCAL file not found: {file_path}") - try: return file_path.read_bytes() + except (FileNotFoundError, NotADirectoryError, ValueError) as e: + raise FileNotFoundError(f"OSCAL file not found: {file_path}") from e except OSError as e: - raise OscalParseError(f"Failed to read file: {e}") from e + raise OscalReadError(e.errno, e.strerror, str(file_path)) from e def serialize_oscal(model: OscalDocument, *, indent: int | None = 2) -> str: diff --git a/tests/support/expected_failures.py b/tests/support/expected_failures.py index fc9147e..cf2b0a9 100644 --- a/tests/support/expected_failures.py +++ b/tests/support/expected_failures.py @@ -20,8 +20,6 @@ "yaml-round-trip/string-scalars": f"{ISSUES}/19", # PRS-014: only failures of the selected nested-union branch. "parse/closed-enum-out-of-set": f"{ISSUES}/69", - # ERR-012: read-error category for a file path that cannot be read. - "parse-file/directory": f"{ISSUES}/61", # FAC-007: datetime bounds from assessment_period(). "assessment-period/mixed-offsets": f"{ISSUES}/58", "assessment-period/equal-instant": f"{ISSUES}/58", diff --git a/tests/support/version_package.py b/tests/support/version_package.py index cd8faa8..5816ed6 100644 --- a/tests/support/version_package.py +++ b/tests/support/version_package.py @@ -43,6 +43,7 @@ "OscalDocument", "OscalDocumentTypeMismatchError", "OscalParseError", + "OscalReadError", "OscalUnknownElementTypeError", "OscalWrapper", "PlanOfActionAndMilestonesDocument", diff --git a/tests/test_conformance.py b/tests/test_conformance.py index 8232488..ce41e2a 100644 --- a/tests/test_conformance.py +++ b/tests/test_conformance.py @@ -25,6 +25,7 @@ OscalDoc, OscalDocumentTypeMismatchError, OscalParseError, + OscalReadError, OscalUnknownElementTypeError, ) @@ -44,21 +45,25 @@ } +#: Each class with its category. A subclass comes before its parent. +_CATEGORIES: tuple[tuple[type[BaseException], str], ...] = ( + (FileNotFoundError, "not-found"), + (OscalReadError, "read-error"), + (OscalAccessError, "access-error"), + (OscalUnknownElementTypeError, "unknown-element-type"), + (OscalDocumentTypeMismatchError, "document-type-mismatch"), + (OscalParseError, "parse-error"), +) + + def category(error: BaseException) -> str | None: """Map an exception to a corpus error category, or None for any other error. The class of the error is its category. A subclass comes before its parent. """ - if isinstance(error, FileNotFoundError): - return "not-found" - if isinstance(error, OscalAccessError): - return "access-error" - if isinstance(error, OscalUnknownElementTypeError): - return "unknown-element-type" - if isinstance(error, OscalDocumentTypeMismatchError): - return "document-type-mismatch" - if isinstance(error, OscalParseError): - return "parse-error" + for error_class, name in _CATEGORIES: + if isinstance(error, error_class): + return name return None @@ -226,14 +231,15 @@ def test_every_expected_failure_names_a_corpus_case() -> None: @pytest.mark.xfail( strict=True, - reason="read-error and builder-error are not mapped yet: " - "https://github.com/cairn-proofs/oscal-bindings-python/issues/61, /62", + reason="builder-error is not mapped yet: " + "https://github.com/cairn-proofs/oscal-bindings-python/issues/62", ) def test_every_category_is_mapped() -> None: """Each category in the corpus has a branch in ``category``.""" index = json.loads((corpus_root() / "cases.json").read_text("utf-8")) mapped = { category(FileNotFoundError()), + category(OscalReadError()), category(OscalAccessError()), category(OscalDocumentTypeMismatchError("Document is not a Catalog")), category(OscalUnknownElementTypeError("Unknown element type: x")), diff --git a/tests/test_parser.py b/tests/test_parser.py index 830ee6b..2f2f6dc 100644 --- a/tests/test_parser.py +++ b/tests/test_parser.py @@ -3,6 +3,7 @@ import copy import gc import json +import os import pickle # nosec B403 - the test pickles its own error entries import tempfile import tracemalloc @@ -18,6 +19,7 @@ OscalDocument, OscalDocumentTypeMismatchError, OscalParseError, + OscalReadError, parse_catalog, parse_component_definition, parse_oscal, @@ -180,6 +182,78 @@ def test_parse_file_reads_as_bytes(self): assert result.root.catalog.uuid == ("f47ac10b-58cc-4372-a567-0e02b2c3d479") +class TestParseOscalFileReadFailures: + """A file-path parse finds a read failure category from the read (ERR-012).""" + + needs_non_root = pytest.mark.skipif( + os.geteuid() == 0, reason="root reads files that deny permission" + ) + + def test_directory_is_a_read_error(self, tmp_path): + with pytest.raises(OscalReadError) as info: + parse_oscal_file(tmp_path) + error = info.value + assert isinstance(error, OSError) + assert not isinstance(error, OscalParseError) + assert not isinstance(error, FileNotFoundError) + assert error.filename == str(tmp_path) + assert str(tmp_path) in str(error) + + @needs_non_root + def test_file_without_read_permission_is_a_read_error(self, tmp_path): + path = tmp_path / "locked.json" + path.write_text(MINIMAL_CATALOG, encoding="utf-8") + path.chmod(0o000) + try: + with pytest.raises(OscalReadError) as info: + parse_oscal_file(path) + finally: + path.chmod(0o600) + assert str(path) in str(info.value) + + def test_symbolic_link_loop_is_a_read_error(self, tmp_path): + first, second = tmp_path / "first.json", tmp_path / "second.json" + first.symlink_to(second) + second.symlink_to(first) + with pytest.raises(OscalReadError) as info: + parse_oscal_file(first) + assert str(first) in str(info.value) + + @needs_non_root + def test_unsearchable_directory_is_a_read_error(self, tmp_path): + directory = tmp_path / "closed" + directory.mkdir() + path = directory / "catalog.json" + path.write_text(MINIMAL_CATALOG, encoding="utf-8") + directory.chmod(0o000) + try: + with pytest.raises(OscalReadError) as info: + parse_oscal_file(path) + finally: + directory.chmod(0o700) + assert str(path) in str(info.value) + + def test_path_below_a_regular_file_is_not_found(self, tmp_path): + regular = tmp_path / "regular.json" + regular.write_text(MINIMAL_CATALOG, encoding="utf-8") + path = regular / "catalog.json" + with pytest.raises(FileNotFoundError) as info: + parse_oscal_file(path) + assert not isinstance(info.value, OscalReadError) + assert str(path) in str(info.value) + + def test_path_with_a_nul_byte_is_not_found(self): + with pytest.raises(FileNotFoundError) as info: + parse_oscal_file("catalog\x00.json") + assert isinstance(info.value.__cause__, ValueError) + assert not isinstance(info.value, OscalReadError) + + def test_alias_module_exports_the_same_class(self): + import oscal_bindings.v1 + + assert OscalReadError is oscal_bindings.v1.OscalReadError + + class TestSerializeOscal: """Tests for the serialize_oscal function.""" diff --git a/tests/test_yaml_io.py b/tests/test_yaml_io.py index 7100e4f..4c95d91 100644 --- a/tests/test_yaml_io.py +++ b/tests/test_yaml_io.py @@ -26,7 +26,7 @@ import yaml import oscal_bindings as ob -from oscal_bindings import OscalParseError +from oscal_bindings import OscalParseError, OscalReadError from oscal_bindings.v1.extensions import _yaml_loader from oscal_bindings.v1.extensions._yaml_loader import _SCHEMA, to_json_text from oscal_bindings.v1.extensions.yaml_io import ( @@ -605,6 +605,12 @@ def test_missing_file_reports_not_found(self, tmp_path) -> None: assert str(missing) in str(info.value) assert not isinstance(info.value, OscalParseError) + def test_directory_reports_read_error(self, tmp_path) -> None: + # ERR-012: a read failure that is not a missing path is a read error. + with pytest.raises(OscalReadError) as info: + parse_oscal_yaml_file(tmp_path) + assert str(tmp_path) in str(info.value) + # --- Task 3.3: serialize_oscal_yaml -------------------------------------------