From f66bc31a0c5c2a306f0d31227abe23e1875decfd Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Mon, 22 Jun 2026 09:23:07 -0400 Subject: [PATCH 1/4] fix: reclassify FileAssert from OTS to Shared Package and fix test names FileAssert cannot treat its own published package as an OTS item. Per the software-items standard, a package produced by the same program consumed in CI is a Shared Package. Additionally, the self-validation test names FileAssert_Exists and FileAssert_Contains no longer exist; they were renamed to FileAssert_File and FileAssert_Text when validation was refactored into per-assertion-type files. Changes: - Move docs/reqstream/ots/fileassert.yaml to docs/reqstream/shared/ - Rename requirement ID from FileAssert-OTS-FileAssert to FileAssert-Shared-FileAssert with tag [shared] - Replace FileAssert_Exists and FileAssert_Contains with FileAssert_File, FileAssert_Text, FileAssert_Html, and FileAssert_Pdf to cover the assertion types actually used by this project - Move design and verification docs from ots/ to shared/ folders - Add shared.md overview documents for design and verification sections - Update docs/design/ots.md to remove FileAssert and add clarifying note - Update docs/verification/ots.md to remove FileAssert references - Update both definition.yaml files for correct document assembly - Update docs/design/introduction.md to document Shared Package artifacts - Update .reviewmark.yaml review set from FileAssert-OTS-FileAssert to FileAssert-Shared-FileAssert with corrected paths - Update requirements.yaml include path Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .reviewmark.yaml | 12 +-- docs/design/definition.yaml | 3 +- docs/design/introduction.md | 8 ++ docs/design/ots.md | 8 +- docs/design/shared.md | 28 ++++++ docs/design/{ots => shared}/fileassert.md | 13 +-- .../reqstream/{ots => shared}/fileassert.yaml | 25 ++++-- docs/verification/definition.yaml | 3 +- docs/verification/ots.md | 11 +-- docs/verification/ots/fileassert.md | 80 ----------------- docs/verification/shared.md | 47 ++++++++++ docs/verification/shared/fileassert.md | 87 +++++++++++++++++++ requirements.yaml | 2 +- 13 files changed, 211 insertions(+), 116 deletions(-) create mode 100644 docs/design/shared.md rename docs/design/{ots => shared}/fileassert.md (83%) rename docs/reqstream/{ots => shared}/fileassert.yaml (50%) delete mode 100644 docs/verification/ots/fileassert.md create mode 100644 docs/verification/shared.md create mode 100644 docs/verification/shared/fileassert.md diff --git a/.reviewmark.yaml b/.reviewmark.yaml index 9c49805..1588d0a 100644 --- a/.reviewmark.yaml +++ b/.reviewmark.yaml @@ -335,13 +335,13 @@ reviews: - "docs/design/ots/buildmark.md" # OTS design - "docs/verification/ots/buildmark.md" # OTS verification - # FileAssert-OTS-FileAssert Review - - id: FileAssert-OTS-FileAssert - title: Review FileAssert OTS FileAssert Requirements and Verification + # FileAssert-Shared-FileAssert Review + - id: FileAssert-Shared-FileAssert + title: Review FileAssert Shared Package FileAssert Requirements and Verification paths: - - "docs/reqstream/ots/fileassert.yaml" # OTS requirements - - "docs/design/ots/fileassert.md" # OTS design - - "docs/verification/ots/fileassert.md" # OTS verification + - "docs/reqstream/shared/fileassert.yaml" # Shared Package requirements + - "docs/design/shared/fileassert.md" # Shared Package design + - "docs/verification/shared/fileassert.md" # Shared Package verification # FileAssert-OTS-Pandoc Review - id: FileAssert-OTS-Pandoc diff --git a/docs/design/definition.yaml b/docs/design/definition.yaml index ec7f78c..d698f6b 100644 --- a/docs/design/definition.yaml +++ b/docs/design/definition.yaml @@ -29,7 +29,6 @@ input-files: - docs/design/file-assert/ots-dependencies.md - docs/design/ots.md - docs/design/ots/buildmark.md - - docs/design/ots/fileassert.md - docs/design/ots/pandoc.md - docs/design/ots/reqstream.md - docs/design/ots/reviewmark.md @@ -38,6 +37,8 @@ input-files: - docs/design/ots/versionmark.md - docs/design/ots/weasyprint.md - docs/design/ots/xunit.md + - docs/design/shared.md + - docs/design/shared/fileassert.md template: template.html table-of-contents: true number-sections: true diff --git a/docs/design/introduction.md b/docs/design/introduction.md index 6c63896..ad70b0c 100644 --- a/docs/design/introduction.md +++ b/docs/design/introduction.md @@ -149,6 +149,14 @@ FileAssert integrates the third-party library; their artifacts sit parallel to s - Design: `docs/design/ots/{ots-name}.md` - Verification: `docs/verification/ots/{ots-name}.md` +Shared packages (earlier releases of the same program's own packages consumed in CI) have +integration/usage design docs at `docs/design/shared/{package-name}.md`; their artifacts also +sit parallel to system and OTS folders: + +- Requirements: `docs/reqstream/shared/{package-name}.yaml` +- Design: `docs/design/shared/{package-name}.md` +- Verification: `docs/verification/shared/{package-name}.md` + Review-sets: defined in `.reviewmark.yaml` ## References diff --git a/docs/design/ots.md b/docs/design/ots.md index 01cd593..8fee35d 100644 --- a/docs/design/ots.md +++ b/docs/design/ots.md @@ -25,7 +25,6 @@ builds and provides an audit record of which tool version produced each release | OTS Item | Role | | :---------- | :--------------------------------------------------------------------------- | | BuildMark | Generates build-notes documentation from GitHub Actions metadata | -| FileAssert | Validates generated HTML and PDF documents against acceptance criteria | | Pandoc | Converts Markdown source documents to HTML for each document collection | | ReqStream | Enforces requirements traceability against TRX test-result files | | ReviewMark | Generates review plan and review report from the review configuration | @@ -35,12 +34,15 @@ builds and provides an audit record of which tool version produced each release | WeasyPrint | Converts HTML documents to PDF for release artifact archiving | | xUnit | Discovers, executes, and reports unit tests; produces TRX output | +> **Note**: FileAssert is not an OTS item for this project. Because this project consumes an +> earlier released version of its own package in CI, it is classified as a **Shared Package**. +> See the _Shared Package Integration Design_ section for details. + ## Per-Item Design Detailed design for each OTS item is provided in the following sections of this document: - See _BuildMark OTS Design_ for the build-notes generation tool. -- See _FileAssert OTS Design_ for the document assertion tool. - See _Pandoc OTS Design_ for the Markdown-to-HTML conversion tool. - See _ReqStream OTS Design_ for the requirements traceability enforcement tool. - See _ReviewMark OTS Design_ for the review plan and report generation tool. @@ -83,7 +85,7 @@ manifest outside of the design artifact set. OTS items in this project fall into two integration categories: -- **dotnet global tools** (BuildMark, FileAssert, Pandoc, ReqStream, ReviewMark, SarifMark, +- **dotnet global tools** (BuildMark, Pandoc, ReqStream, ReviewMark, SarifMark, SonarMark, VersionMark, WeasyPrint) — installed globally in the CI environment via `dotnet tool restore` from `.config/dotnet-tools.json` and invoked as command-line executables within GitHub Actions workflow steps. No wrapper code is written; tools are invoked directly diff --git a/docs/design/shared.md b/docs/design/shared.md new file mode 100644 index 0000000..73b534a --- /dev/null +++ b/docs/design/shared.md @@ -0,0 +1,28 @@ +# Shared Package Integration Design + +FileAssert uses one shared package from within the same program: an earlier released version of +itself. This section describes the integration strategy and usage design for this package. + +## Integration Strategy + +A **Shared Package** is a software package produced by the same program (repository), consumed as +a pinned released version rather than built from source. Unlike an OTS item, a shared package is +developed in-house; unlike in-scope source code, the consuming repository references its advertised +features rather than its internal design. + +FileAssert consumes a previous release of itself as a dotnet global tool installed via +`.config/dotnet-tools.json`. This is a deliberate dogfooding practice: the CI pipeline uses a +stable released build of the tool to validate the in-development build's documentation outputs, +while the in-development build is what produces the next release. + +## Shared Package Summary + +| Package | Role | +| :---------- | :--------------------------------------------------------------------------- | +| FileAssert | Validates generated HTML and PDF documents against acceptance criteria | + +## Per-Package Design + +Detailed integration design is provided in the following section of this document: + +- See _FileAssert Shared Package Design_ for the document assertion tool. diff --git a/docs/design/ots/fileassert.md b/docs/design/shared/fileassert.md similarity index 83% rename from docs/design/ots/fileassert.md rename to docs/design/shared/fileassert.md index f6fe021..bb3f23a 100644 --- a/docs/design/ots/fileassert.md +++ b/docs/design/shared/fileassert.md @@ -1,15 +1,16 @@ -## FileAssert OTS Design +## FileAssert Shared Package Design -DemaConsulting.FileAssert is the tool developed by this project and is also consumed as an OTS -item within its own CI pipeline to validate the correctness of the documents it generates and to -produce self-validation evidence. +DemaConsulting.FileAssert is the tool developed by this project. An earlier released version of it +is also consumed within this project's own CI pipeline to validate generated documents and to +produce self-validation evidence. Because this repository uses its own published package, it is +classified as a **Shared Package** rather than OTS. ### Purpose FileAssert validates HTML and PDF documents produced during the build pipeline, asserting that each document exists, has a non-trivial size, is structurally valid, and contains expected content. Its built-in self-validation suite (`--validate`) is run as a CI step to produce TRX -test evidence satisfying the `FileAssert-OTS-FileAssert` requirement. +test evidence satisfying the `FileAssert-Shared-FileAssert` requirement. FileAssert is chosen because it directly implements the project's own document-assertion capability, making its CI use a natural dogfooding exercise that simultaneously validates the @@ -42,7 +43,7 @@ The project uses the following FileAssert command-line interfaces: | `dotnet fileassert --validate --results ` | Runs self-validation and writes TRX results | | `dotnet fileassert --config --results ` | Runs document assertions from a YAML configuration | -The TRX output from `--results` is consumed by ReqStream to satisfy `FileAssert-OTS-FileAssert`. +The TRX output from `--results` is consumed by ReqStream to satisfy `FileAssert-Shared-FileAssert`. ### Dependencies diff --git a/docs/reqstream/ots/fileassert.yaml b/docs/reqstream/shared/fileassert.yaml similarity index 50% rename from docs/reqstream/ots/fileassert.yaml rename to docs/reqstream/shared/fileassert.yaml index 3f5ea6b..ada2e05 100644 --- a/docs/reqstream/ots/fileassert.yaml +++ b/docs/reqstream/shared/fileassert.yaml @@ -1,28 +1,35 @@ --- -# FileAssert OTS Software Requirements +# FileAssert Shared Package Software Requirements # # Requirements for the FileAssert document assertion tool functionality. +# FileAssert is classified as a Shared Package because this repository uses an earlier +# released version of itself in CI - a project cannot treat its own package as OTS. sections: - - title: OTS Software Requirements + - title: Shared Package Software Requirements sections: - title: FileAssert Requirements requirements: - - id: FileAssert-OTS-FileAssert + - id: FileAssert-Shared-FileAssert title: FileAssert shall validate generated documents against acceptance criteria. justification: | DemaConsulting.FileAssert validates HTML and PDF documents produced during the build, asserting that each document exists, is structurally valid, has correct - metadata, and contains expected content. It provides OTS evidence for Pandoc and - WeasyPrint and independently confirms file assertion is functioning. Self-validation - proves the tool itself is operational before ReqStream consumes the results. - tags: [ots] + metadata, and contains expected content. It provides verification evidence for + Pandoc and WeasyPrint and independently confirms file assertion is functioning. + Self-validation proves the tool itself is operational before ReqStream consumes + the results. + tags: [shared] # Test IDs below are sourced from the tool's `--validate` self-validation TRX output # (artifacts/-self-validation*.trx in CI). Each ID matches a TestCase Name in # the TRX; ReqStream consumes the artifacts/**/*.trx glob. + # Tests are selected to cover the specific assertion types used by this project: + # file existence (count), text content, HTML structure, and PDF metadata/pages/text. tests: - FileAssert_VersionDisplay - FileAssert_HelpDisplay - FileAssert_Results - - FileAssert_Exists - - FileAssert_Contains + - FileAssert_File + - FileAssert_Text + - FileAssert_Html + - FileAssert_Pdf diff --git a/docs/verification/definition.yaml b/docs/verification/definition.yaml index 1095c2f..313eab9 100644 --- a/docs/verification/definition.yaml +++ b/docs/verification/definition.yaml @@ -29,7 +29,6 @@ input-files: - docs/verification/file-assert/selftest/validation.md - docs/verification/ots.md - docs/verification/ots/buildmark.md - - docs/verification/ots/fileassert.md - docs/verification/ots/pandoc.md - docs/verification/ots/reqstream.md - docs/verification/ots/reviewmark.md @@ -38,6 +37,8 @@ input-files: - docs/verification/ots/versionmark.md - docs/verification/ots/weasyprint.md - docs/verification/ots/xunit.md + - docs/verification/shared.md + - docs/verification/shared/fileassert.md template: template.html table-of-contents: true number-sections: true diff --git a/docs/verification/ots.md b/docs/verification/ots.md index 9bd45c8..1b25538 100644 --- a/docs/verification/ots.md +++ b/docs/verification/ots.md @@ -11,8 +11,8 @@ OTS software items used by FileAssert are verified by one of two complementary a depending on the nature of the item: - **Authored integration tests**: Items whose correct behaviour cannot be inferred purely from - pipeline success (xUnit, FileAssert) are verified by named test scenarios that exercise the - specific features required by this project. Each scenario identifies a test method, the expected + pipeline success (xUnit) are verified by named test scenarios that exercise the specific + features required by this project. Each scenario identifies a test method, the expected outcome, and the requirement it covers. - **CI pipeline evidence**: Items whose primary function is to produce artefacts consumed by @@ -32,10 +32,6 @@ The following evidence is collected and retained for each OTS item: - **xUnit**: TRX result files generated by `dotnet test` during CI. Each TRX file is consumed by `reqstream --enforce` to confirm that all named test scenarios have a recorded passing result. -- **FileAssert**: The self-validation TRX file (`artifacts/fileassert-self-validation.trx`) - produced by `fileassert --validate --results` during CI. This file is also consumed by - `reqstream --enforce`. - - **All other OTS items**: A passing CI pipeline run, including Pandoc HTML generation, WeasyPrint PDF rendering, FileAssert assertions on every generated document, and `reqstream --enforce` with no unmet requirements. The combined CI log and artefacts constitute the qualification record. @@ -63,9 +59,6 @@ The following OTS software items are used by FileAssert and are verified in this - **BuildMark** — generates a markdown build-notes document from GitHub Actions workflow metadata, verified by CI pipeline success and downstream document validation. -- **FileAssert** — validates generated HTML and PDF documents for existence, size, structure, and - content; verified by self-validation tests and transitive pipeline evidence. - - **Pandoc** — converts Markdown source documents to HTML as part of the documentation build; verified by FileAssert assertions on each generated HTML document. diff --git a/docs/verification/ots/fileassert.md b/docs/verification/ots/fileassert.md deleted file mode 100644 index 39abf02..0000000 --- a/docs/verification/ots/fileassert.md +++ /dev/null @@ -1,80 +0,0 @@ -## FileAssert Verification - -This document provides the verification evidence for the FileAssert OTS software item. Requirements -for this OTS item are defined in the FileAssert OTS Software Requirements document. - -### Required Functionality - -DemaConsulting.FileAssert validates HTML and PDF documents produced during the build, asserting that -each document exists, has a non-trivial size, is structurally valid, and contains expected content. -Self-validation proves the tool itself is operational before ReqStream consumes the results. - -### Verification Approach - -FileAssert is verified by two complementary layers of evidence. First, the CI pipeline runs -`fileassert --validate --results artifacts/fileassert-self-validation.trx` after all documents -have been generated, exercising FileAssert's built-in self-validation suite and recording -functional test results for ReqStream. - -Second, FileAssert is used throughout the pipeline to validate every generated document. If -FileAssert were non-functional, these validation steps would fail, causing `reqstream --enforce` -to report missing test coverage and fail the build. A passing CI build therefore constitutes -transitive evidence that FileAssert correctly asserted document content at each stage of the -pipeline. - -### Test Scenarios - -#### FileAssert_VersionDisplay - -**Scenario**: FileAssert self-validation exercises the `--version` flag. - -**Expected**: Exits 0 and displays a version string. - -#### FileAssert_HelpDisplay - -**Scenario**: FileAssert self-validation exercises the `--help` flag. - -**Expected**: Exits 0 and displays usage information. - -#### FileAssert_Results - -**Scenario**: FileAssert self-validation exercises the `--results` flag by running a configuration with one -passing test and one deliberately failing test, then verifying that a TRX results file is written. - -**Expected**: Exits non-zero (due to the failing test) and creates a TRX results file at the specified path. - -#### FileAssert_Exists - -**Scenario**: FileAssert self-validation exercises file-existence checking by matching a glob pattern against -a temporary directory containing a single `.txt` file. - -**Expected**: Exits 0, confirming that the glob-based file-existence assertion passes. - -#### FileAssert_Contains - -**Scenario**: FileAssert self-validation exercises file-content checking by asserting that a temporary `.txt` -file contains a known string. - -**Expected**: Exits 0, confirming that the text `contains` assertion passes. - -#### FileAssert_StructuralValidity - -**Scenario**: FileAssert self-validation exercises structural validity by parsing a temporary -HTML document with `html:` queries and a temporary PDF document with `pdf:` page-count -constraints. - -**Expected**: Exits 0, confirming that both documents parse successfully and that the -structured-document queries evaluate against the parsed model. - -#### FileAssert_Metadata - -**Scenario**: FileAssert self-validation exercises metadata assertions by reading PDF metadata -fields (Title, Author) and matching them against expected values. - -**Expected**: Exits 0, confirming that the metadata assertions resolve and pass. - -### Acceptance Criteria - -N/A - Acceptance criteria are managed at the system integration level. This OTS item is -considered verified when the integration test scenarios that exercise its functionality -pass in the CI pipeline. diff --git a/docs/verification/shared.md b/docs/verification/shared.md new file mode 100644 index 0000000..220f53d --- /dev/null +++ b/docs/verification/shared.md @@ -0,0 +1,47 @@ +# Shared Package Verification + +This section documents the verification evidence for the shared package used by the FileAssert +build pipeline. The shared package is an earlier released version of FileAssert itself, consumed +as a dogfooding dependency in CI. + +## Verification Strategy + +The shared package is verified by two complementary layers of evidence. First, the CI pipeline +runs `fileassert --validate --results artifacts/fileassert-self-validation.trx` after all +documents have been generated, exercising FileAssert's built-in self-validation suite and +recording functional test results for ReqStream. + +Second, FileAssert is used throughout the pipeline to validate every generated document. If +FileAssert were non-functional, these validation steps would fail, causing `reqstream --enforce` +to report missing test coverage and fail the build. A passing CI build therefore constitutes +transitive evidence that FileAssert correctly asserted document content at each stage of the +pipeline. + +Per-package verification details, test scenarios, and requirements mappings are documented in the +individual files under `docs/verification/shared/`. + +## Qualification Evidence + +The self-validation TRX file (`artifacts/fileassert-self-validation.trx`) produced by +`fileassert --validate --results` during CI is consumed by `reqstream --enforce` to confirm +that all named test scenarios have a recorded passing result. + +All CI artifacts are retained as GitHub Actions run artifacts and are accessible from the +FileAssert releases page. + +## Regression Approach + +When the shared package version is updated, the following steps apply: + +1. Update the version reference in the tool manifest (`.config/dotnet-tools.json`). +2. Run the full CI pipeline. All self-validation tests and FileAssert assertions must pass. +3. Confirm that `reqstream --enforce` reports no unmet requirements. +4. If any step fails, investigate whether the version change introduced a breaking change + affecting the features used by this project, and resolve before merging the update. + +## Shared Packages + +The following shared package is used by FileAssert and is verified in this section: + +- **FileAssert** — validates generated HTML and PDF documents for existence, structure, and + content; verified by self-validation tests and transitive pipeline evidence. diff --git a/docs/verification/shared/fileassert.md b/docs/verification/shared/fileassert.md new file mode 100644 index 0000000..188f08d --- /dev/null +++ b/docs/verification/shared/fileassert.md @@ -0,0 +1,87 @@ +## FileAssert Verification + +This document provides the verification evidence for the FileAssert shared package. Requirements +for this shared package are defined in the FileAssert Shared Package Software Requirements document. + +### Required Functionality + +DemaConsulting.FileAssert validates HTML and PDF documents produced during the build, asserting that +each document exists, has a non-trivial size, is structurally valid, and contains expected content. +Self-validation proves the tool itself is operational before ReqStream consumes the results. + +The assertion types used by this project are: file existence and count (`FileAssert_File`), text +content assertions (`FileAssert_Text`), HTML structure assertions (`FileAssert_Html`), and PDF +metadata, page count, and body text assertions (`FileAssert_Pdf`). + +### Verification Approach + +FileAssert is verified by two complementary layers of evidence. First, the CI pipeline runs +`fileassert --validate --results artifacts/fileassert-self-validation.trx` after all documents +have been generated, exercising FileAssert's built-in self-validation suite and recording +functional test results for ReqStream. + +Second, FileAssert is used throughout the pipeline to validate every generated document. If +FileAssert were non-functional, these validation steps would fail, causing `reqstream --enforce` +to report missing test coverage and fail the build. A passing CI build therefore constitutes +transitive evidence that FileAssert correctly asserted document content at each stage of the +pipeline. + +### Test Scenarios + +#### FileAssert_VersionDisplay + +**Scenario**: FileAssert self-validation exercises the `--version` flag. + +**Expected**: Exits 0 and displays a version string matching the `N.N.N` semantic version format. + +#### FileAssert_HelpDisplay + +**Scenario**: FileAssert self-validation exercises the `--help` flag. + +**Expected**: Exits 0 and displays usage information containing `Usage:` and `Options:` headings. + +#### FileAssert_Results + +**Scenario**: FileAssert self-validation exercises the `--results` flag by running a configuration +with one passing test and one deliberately failing test, then verifying that a TRX results file +is written. + +**Expected**: Exits non-zero (due to the failing test) and creates a TRX results file at the +specified path. + +#### FileAssert_File + +**Scenario**: FileAssert self-validation exercises file glob matching and count assertions by +matching a glob pattern against a temporary directory containing a known set of files. + +**Expected**: Exits 0, confirming that glob-based file existence and count assertions pass. + +#### FileAssert_Text + +**Scenario**: FileAssert self-validation exercises text content assertions by asserting that a +temporary file contains a known string and does not contain an absent string. + +**Expected**: Exits 0, confirming that the `contains` and `does-not-contain` text assertions pass. + +#### FileAssert_Html + +**Scenario**: FileAssert self-validation exercises HTML structure assertions by parsing a +temporary HTML document with `html:` XPath queries and verifying node counts. + +**Expected**: Exits 0, confirming that the HTML document parses successfully and the structured +XPath queries evaluate against the parsed model. + +#### FileAssert_Pdf + +**Scenario**: FileAssert self-validation exercises PDF assertions by reading metadata fields +(Title, Author), checking page count constraints, and asserting PDF body text content against +a temporary PDF document. + +**Expected**: Exits 0, confirming that metadata assertions, page count constraints, and body +text assertions all resolve and pass. + +### Acceptance Criteria + +N/A - Acceptance criteria are managed at the system integration level. This shared package is +considered verified when the self-validation test scenarios that exercise its functionality +pass in the CI pipeline. diff --git a/requirements.yaml b/requirements.yaml index 0f53f4f..47e32de 100644 --- a/requirements.yaml +++ b/requirements.yaml @@ -38,7 +38,7 @@ includes: - docs/reqstream/ots/reviewmark.yaml - docs/reqstream/ots/pandoc.yaml - docs/reqstream/ots/weasyprint.yaml - - docs/reqstream/ots/fileassert.yaml + - docs/reqstream/shared/fileassert.yaml # cspell:ignore yamldotnet pdfpig htmlagilitypack filesystemglobbing - docs/reqstream/ots/yamldotnet.yaml - docs/reqstream/ots/pdfpig.yaml From 068ed71a9025ec3d4c3aa4683c131897ebd6151e Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Mon, 22 Jun 2026 09:24:08 -0400 Subject: [PATCH 2/4] chore: update demaconsulting.fileassert to 0.5.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .config/dotnet-tools.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.config/dotnet-tools.json b/.config/dotnet-tools.json index 53a729d..effe8af 100644 --- a/.config/dotnet-tools.json +++ b/.config/dotnet-tools.json @@ -57,7 +57,7 @@ ] }, "demaconsulting.fileassert": { - "version": "0.4.0", + "version": "0.5.1", "commands": [ "fileassert" ] From 111f7dbfd3f0b572ffe31aa890942766b8928095 Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Mon, 22 Jun 2026 11:50:19 -0400 Subject: [PATCH 3/4] chore: apply lint auto-fixes to reqstream and verification docs --- docs/reqstream/shared/fileassert.yaml | 59 +++++++++++++++++++------- docs/verification/shared/fileassert.md | 48 +++++++++------------ 2 files changed, 65 insertions(+), 42 deletions(-) diff --git a/docs/reqstream/shared/fileassert.yaml b/docs/reqstream/shared/fileassert.yaml index ada2e05..9fb450e 100644 --- a/docs/reqstream/shared/fileassert.yaml +++ b/docs/reqstream/shared/fileassert.yaml @@ -10,26 +10,55 @@ sections: sections: - title: FileAssert Requirements requirements: - - id: FileAssert-Shared-FileAssert - title: FileAssert shall validate generated documents against acceptance criteria. + - id: FileAssert-Shared-FileAssert-Results + title: FileAssert shall write TRX test result files. justification: | - DemaConsulting.FileAssert validates HTML and PDF documents produced during the - build, asserting that each document exists, is structurally valid, has correct - metadata, and contains expected content. It provides verification evidence for - Pandoc and WeasyPrint and independently confirms file assertion is functioning. - Self-validation proves the tool itself is operational before ReqStream consumes - the results. + The CI pipeline passes --results to FileAssert on every invocation; + the produced TRX file is consumed by ReqStream to verify requirements + traceability. Without TRX output, no test evidence is recorded and + ReqStream enforcement fails the build. tags: [shared] - # Test IDs below are sourced from the tool's `--validate` self-validation TRX output - # (artifacts/-self-validation*.trx in CI). Each ID matches a TestCase Name in - # the TRX; ReqStream consumes the artifacts/**/*.trx glob. - # Tests are selected to cover the specific assertion types used by this project: - # file existence (count), text content, HTML structure, and PDF metadata/pages/text. tests: - - FileAssert_VersionDisplay - - FileAssert_HelpDisplay - FileAssert_Results + + - id: FileAssert-Shared-FileAssert-File + title: FileAssert shall assert file count via glob patterns. + justification: | + Every test in .fileassert.yaml uses count: 1 to confirm that exactly one + generated file matches the expected path. This proves that Pandoc and + WeasyPrint each produced the expected output file. + tags: [shared] + tests: - FileAssert_File + + - id: FileAssert-Shared-FileAssert-Text + title: FileAssert shall assert text content of files. + justification: | + HTML document tests use text: contains: to confirm that generated HTML + files contain expected document headings and content keywords, providing + a basic content sanity check independent of HTML structure parsing. + tags: [shared] + tests: - FileAssert_Text + + - id: FileAssert-Shared-FileAssert-Html + title: FileAssert shall assert HTML document structure. + justification: | + Pandoc HTML output is validated using html: query: XPath expressions to + confirm each document has a valid head/title element, proving the HTML is + structurally well-formed. + tags: [shared] + tests: - FileAssert_Html + + - id: FileAssert-Shared-FileAssert-Pdf + title: FileAssert shall assert PDF document contents. + justification: | + WeasyPrint PDF output is validated using pdf: assertions. The justification + covers metadata fields (Title, Author, Subject confirming correct document + identity), page count constraints (confirming the document rendered with + substantive content), and body text content (confirming the expected + document text was included in the rendered output). + tags: [shared] + tests: - FileAssert_Pdf diff --git a/docs/verification/shared/fileassert.md b/docs/verification/shared/fileassert.md index 188f08d..1bddc52 100644 --- a/docs/verification/shared/fileassert.md +++ b/docs/verification/shared/fileassert.md @@ -5,13 +5,9 @@ for this shared package are defined in the FileAssert Shared Package Software Re ### Required Functionality -DemaConsulting.FileAssert validates HTML and PDF documents produced during the build, asserting that -each document exists, has a non-trivial size, is structurally valid, and contains expected content. -Self-validation proves the tool itself is operational before ReqStream consumes the results. - -The assertion types used by this project are: file existence and count (`FileAssert_File`), text -content assertions (`FileAssert_Text`), HTML structure assertions (`FileAssert_Html`), and PDF -metadata, page count, and body text assertions (`FileAssert_Pdf`). +DemaConsulting.FileAssert validates HTML and PDF documents produced during the build. The +specific capabilities required are: TRX result file output, file count via glob patterns, text +content assertions, HTML structure assertions, and PDF document content assertions. ### Verification Approach @@ -28,20 +24,10 @@ pipeline. ### Test Scenarios -#### FileAssert_VersionDisplay - -**Scenario**: FileAssert self-validation exercises the `--version` flag. - -**Expected**: Exits 0 and displays a version string matching the `N.N.N` semantic version format. - -#### FileAssert_HelpDisplay - -**Scenario**: FileAssert self-validation exercises the `--help` flag. - -**Expected**: Exits 0 and displays usage information containing `Usage:` and `Options:` headings. - #### FileAssert_Results +**Covers**: `FileAssert-Shared-FileAssert-Results` + **Scenario**: FileAssert self-validation exercises the `--results` flag by running a configuration with one passing test and one deliberately failing test, then verifying that a TRX results file is written. @@ -51,13 +37,17 @@ specified path. #### FileAssert_File +**Covers**: `FileAssert-Shared-FileAssert-File` + **Scenario**: FileAssert self-validation exercises file glob matching and count assertions by matching a glob pattern against a temporary directory containing a known set of files. -**Expected**: Exits 0, confirming that glob-based file existence and count assertions pass. +**Expected**: Exits 0, confirming that glob-based file count assertions pass. #### FileAssert_Text +**Covers**: `FileAssert-Shared-FileAssert-Text` + **Scenario**: FileAssert self-validation exercises text content assertions by asserting that a temporary file contains a known string and does not contain an absent string. @@ -65,20 +55,24 @@ temporary file contains a known string and does not contain an absent string. #### FileAssert_Html +**Covers**: `FileAssert-Shared-FileAssert-Html` + **Scenario**: FileAssert self-validation exercises HTML structure assertions by parsing a temporary HTML document with `html:` XPath queries and verifying node counts. -**Expected**: Exits 0, confirming that the HTML document parses successfully and the structured -XPath queries evaluate against the parsed model. +**Expected**: Exits 0, confirming that the HTML document parses successfully and the XPath +queries evaluate against the parsed document model. #### FileAssert_Pdf -**Scenario**: FileAssert self-validation exercises PDF assertions by reading metadata fields -(Title, Author), checking page count constraints, and asserting PDF body text content against -a temporary PDF document. +**Covers**: `FileAssert-Shared-FileAssert-Pdf` + +**Scenario**: FileAssert self-validation exercises PDF content assertions by reading metadata +fields (Title, Author), checking page count constraints, and asserting PDF body text content +against a temporary PDF document. -**Expected**: Exits 0, confirming that metadata assertions, page count constraints, and body -text assertions all resolve and pass. +**Expected**: Exits 0, confirming that metadata field assertions, page count constraints, and +body text assertions all resolve and pass. ### Acceptance Criteria From 47528f96d94789bd4c9899c6d8c9c75742b81aa6 Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Mon, 22 Jun 2026 11:59:44 -0400 Subject: [PATCH 4/4] fix: align table columns in fileassert shared package design doc --- docs/design/shared/fileassert.md | 69 ++++++++++++++------------------ 1 file changed, 29 insertions(+), 40 deletions(-) diff --git a/docs/design/shared/fileassert.md b/docs/design/shared/fileassert.md index bb3f23a..c9e6c38 100644 --- a/docs/design/shared/fileassert.md +++ b/docs/design/shared/fileassert.md @@ -1,53 +1,42 @@ ## FileAssert Shared Package Design DemaConsulting.FileAssert is the tool developed by this project. An earlier released version of it -is also consumed within this project's own CI pipeline to validate generated documents and to -produce self-validation evidence. Because this repository uses its own published package, it is -classified as a **Shared Package** rather than OTS. +is consumed within this project's own CI pipeline to validate generated documents and to produce +self-validation evidence. Because this repository uses its own published package, it is classified +as a **Shared Package** rather than OTS. -### Purpose +### Advertised Features Consumed -FileAssert validates HTML and PDF documents produced during the build pipeline, asserting that -each document exists, has a non-trivial size, is structurally valid, and contains expected -content. Its built-in self-validation suite (`--validate`) is run as a CI step to produce TRX -test evidence satisfying the `FileAssert-Shared-FileAssert` requirement. +The following advertised features of the released FileAssert package are consumed by this project, +each corresponding to a requirement in the FileAssert Shared Package Software Requirements: -FileAssert is chosen because it directly implements the project's own document-assertion -capability, making its CI use a natural dogfooding exercise that simultaneously validates the -tool and provides compliance evidence for the document pipeline. +| Requirement | Feature Consumed | +| :------------------------------------- | :------------------------------------------------------------- | +| `FileAssert-Shared-FileAssert-Results` | `--results ` writes TRX test result files for ReqStream | +| `FileAssert-Shared-FileAssert-File` | `count:` assertion confirms file existence via glob patterns | +| `FileAssert-Shared-FileAssert-Text` | `text: contains:` asserts text content of generated HTML files | +| `FileAssert-Shared-FileAssert-Html` | `html: query:` asserts HTML document structure via XPath | +| `FileAssert-Shared-FileAssert-Pdf` | `pdf:` asserts PDF metadata fields, page count, and body text | -### Integration +### Integration Pattern FileAssert is installed as a .NET local tool defined in `.config/dotnet-tools.json` under the -package name `demaconsulting.fileassert` and restored with `dotnet tool restore`. The CI -pipeline uses it in two ways: +package name `demaconsulting.fileassert` and restored with `dotnet tool restore` at the start of +each CI job. No wrapper code is written; the tool is invoked directly via two command patterns: -- **Self-validation**: `fileassert --validate --results artifacts/fileassert-self-validation.trx` - runs the built-in self-validation suite and writes TRX output for ReqStream consumption. -- **Document assertion**: `fileassert --config --results ` validates specific - generated HTML and PDF files throughout the pipeline using YAML configuration files. +- `dotnet fileassert --validate --results ` — runs the built-in self-validation suite + and writes TRX output consumed by ReqStream to satisfy the shared-package requirements. +- `dotnet fileassert --config --results ` — runs document assertions defined in + `.fileassert.yaml` YAML configuration files checked in alongside the generated documents. -### Configuration +A non-zero exit code from either invocation causes the CI job to fail immediately. -When used for document validation, FileAssert reads `.fileassert.yaml` configuration files that -define the glob patterns and acceptance criteria for each document type. These YAML files are -checked in alongside the documents they validate. When run with `--validate`, no configuration -file is required; the built-in test suite is used. +### Assumptions -### Interfaces - -The project uses the following FileAssert command-line interfaces: - -| Invocation | Effect | -| :----------------------------------------------------------- | :------------------------------------------------- | -| `dotnet fileassert --validate --results ` | Runs self-validation and writes TRX results | -| `dotnet fileassert --config --results ` | Runs document assertions from a YAML configuration | - -The TRX output from `--results` is consumed by ReqStream to satisfy `FileAssert-Shared-FileAssert`. - -### Dependencies - -FileAssert operates as an isolated tool process. Its NuGet dependencies — PdfPig, -HtmlAgilityPack, YamlDotNet, Microsoft.Extensions.FileSystemGlobbing, and -DemaConsulting.TestResults — are bundled within the tool and do not affect the main project's -dependency graph or the published NuGet package. +- The released package's self-validation test names (`FileAssert_Results`, `FileAssert_File`, + `FileAssert_Text`, `FileAssert_Html`, `FileAssert_Pdf`) remain stable across patch versions and + are present in the TRX output produced by `--validate`. +- The `--results` flag produces a valid TRX file parseable by ReqStream. +- The `count:`, `text:`, `html:`, and `pdf:` YAML assertion keys behave as documented in the + FileAssert User Guide for the pinned version. +- The tool exits non-zero when any assertion fails, causing CI to fail immediately.