Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .config/dotnet-tools.json
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@
]
},
"demaconsulting.fileassert": {
"version": "0.4.0",
"version": "0.5.1",
"commands": [
"fileassert"
]
Expand Down
12 changes: 6 additions & 6 deletions .reviewmark.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -335,13 +335,13 @@ reviews:
- "docs/design/ots/buildmark.md" # OTS design
- "docs/verification/ots/buildmark.md" # OTS verification

# FileAssert-OTS-FileAssert Review
- id: FileAssert-OTS-FileAssert
title: Review FileAssert OTS FileAssert Requirements and Verification
# FileAssert-Shared-FileAssert Review
- id: FileAssert-Shared-FileAssert
title: Review FileAssert Shared Package FileAssert Requirements and Verification
paths:
- "docs/reqstream/ots/fileassert.yaml" # OTS requirements
- "docs/design/ots/fileassert.md" # OTS design
- "docs/verification/ots/fileassert.md" # OTS verification
- "docs/reqstream/shared/fileassert.yaml" # Shared Package requirements
- "docs/design/shared/fileassert.md" # Shared Package design
- "docs/verification/shared/fileassert.md" # Shared Package verification

# FileAssert-OTS-Pandoc Review
- id: FileAssert-OTS-Pandoc
Expand Down
3 changes: 2 additions & 1 deletion docs/design/definition.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ input-files:
- docs/design/file-assert/ots-dependencies.md
- docs/design/ots.md
- docs/design/ots/buildmark.md
- docs/design/ots/fileassert.md
- docs/design/ots/pandoc.md
- docs/design/ots/reqstream.md
- docs/design/ots/reviewmark.md
Expand All @@ -38,6 +37,8 @@ input-files:
- docs/design/ots/versionmark.md
- docs/design/ots/weasyprint.md
- docs/design/ots/xunit.md
- docs/design/shared.md
- docs/design/shared/fileassert.md
template: template.html
table-of-contents: true
number-sections: true
8 changes: 8 additions & 0 deletions docs/design/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,14 @@ FileAssert integrates the third-party library; their artifacts sit parallel to s
- Design: `docs/design/ots/{ots-name}.md`
- Verification: `docs/verification/ots/{ots-name}.md`

Shared packages (earlier releases of the same program's own packages consumed in CI) have
integration/usage design docs at `docs/design/shared/{package-name}.md`; their artifacts also
sit parallel to system and OTS folders:

- Requirements: `docs/reqstream/shared/{package-name}.yaml`
- Design: `docs/design/shared/{package-name}.md`
- Verification: `docs/verification/shared/{package-name}.md`

Review-sets: defined in `.reviewmark.yaml`

## References
Expand Down
8 changes: 5 additions & 3 deletions docs/design/ots.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ builds and provides an audit record of which tool version produced each release
| OTS Item | Role |
| :---------- | :--------------------------------------------------------------------------- |
| BuildMark | Generates build-notes documentation from GitHub Actions metadata |
| FileAssert | Validates generated HTML and PDF documents against acceptance criteria |
| Pandoc | Converts Markdown source documents to HTML for each document collection |
| ReqStream | Enforces requirements traceability against TRX test-result files |
| ReviewMark | Generates review plan and review report from the review configuration |
Expand All @@ -35,12 +34,15 @@ builds and provides an audit record of which tool version produced each release
| WeasyPrint | Converts HTML documents to PDF for release artifact archiving |
| xUnit | Discovers, executes, and reports unit tests; produces TRX output |

> **Note**: FileAssert is not an OTS item for this project. Because this project consumes an
> earlier released version of its own package in CI, it is classified as a **Shared Package**.
> See the _Shared Package Integration Design_ section for details.

## Per-Item Design

Detailed design for each OTS item is provided in the following sections of this document:

- See _BuildMark OTS Design_ for the build-notes generation tool.
- See _FileAssert OTS Design_ for the document assertion tool.
- See _Pandoc OTS Design_ for the Markdown-to-HTML conversion tool.
- See _ReqStream OTS Design_ for the requirements traceability enforcement tool.
- See _ReviewMark OTS Design_ for the review plan and report generation tool.
Expand Down Expand Up @@ -83,7 +85,7 @@ manifest outside of the design artifact set.

OTS items in this project fall into two integration categories:

- **dotnet global tools** (BuildMark, FileAssert, Pandoc, ReqStream, ReviewMark, SarifMark,
- **dotnet global tools** (BuildMark, Pandoc, ReqStream, ReviewMark, SarifMark,
SonarMark, VersionMark, WeasyPrint) — installed globally in the CI environment via
`dotnet tool restore` from `.config/dotnet-tools.json` and invoked as command-line executables
within GitHub Actions workflow steps. No wrapper code is written; tools are invoked directly
Expand Down
52 changes: 0 additions & 52 deletions docs/design/ots/fileassert.md

This file was deleted.

28 changes: 28 additions & 0 deletions docs/design/shared.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Shared Package Integration Design

FileAssert uses one shared package from within the same program: an earlier released version of
itself. This section describes the integration strategy and usage design for this package.

## Integration Strategy

A **Shared Package** is a software package produced by the same program (repository), consumed as
a pinned released version rather than built from source. Unlike an OTS item, a shared package is
developed in-house; unlike in-scope source code, the consuming repository references its advertised
features rather than its internal design.

FileAssert consumes a previous release of itself as a dotnet global tool installed via
`.config/dotnet-tools.json`. This is a deliberate dogfooding practice: the CI pipeline uses a
stable released build of the tool to validate the in-development build's documentation outputs,
while the in-development build is what produces the next release.

## Shared Package Summary

| Package | Role |
| :---------- | :--------------------------------------------------------------------------- |
| FileAssert | Validates generated HTML and PDF documents against acceptance criteria |

## Per-Package Design

Detailed integration design is provided in the following section of this document:

- See _FileAssert Shared Package Design_ for the document assertion tool.
42 changes: 42 additions & 0 deletions docs/design/shared/fileassert.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
## FileAssert Shared Package Design

DemaConsulting.FileAssert is the tool developed by this project. An earlier released version of it
is consumed within this project's own CI pipeline to validate generated documents and to produce
self-validation evidence. Because this repository uses its own published package, it is classified
as a **Shared Package** rather than OTS.

### Advertised Features Consumed

The following advertised features of the released FileAssert package are consumed by this project,
each corresponding to a requirement in the FileAssert Shared Package Software Requirements:

| Requirement | Feature Consumed |
| :------------------------------------- | :------------------------------------------------------------- |
| `FileAssert-Shared-FileAssert-Results` | `--results <file>` writes TRX test result files for ReqStream |
| `FileAssert-Shared-FileAssert-File` | `count:` assertion confirms file existence via glob patterns |
| `FileAssert-Shared-FileAssert-Text` | `text: contains:` asserts text content of generated HTML files |
| `FileAssert-Shared-FileAssert-Html` | `html: query:` asserts HTML document structure via XPath |
| `FileAssert-Shared-FileAssert-Pdf` | `pdf:` asserts PDF metadata fields, page count, and body text |

### Integration Pattern

FileAssert is installed as a .NET local tool defined in `.config/dotnet-tools.json` under the
package name `demaconsulting.fileassert` and restored with `dotnet tool restore` at the start of
each CI job. No wrapper code is written; the tool is invoked directly via two command patterns:

- `dotnet fileassert --validate --results <trx-file>` — runs the built-in self-validation suite
and writes TRX output consumed by ReqStream to satisfy the shared-package requirements.
- `dotnet fileassert --config <file> --results <trx-file>` — runs document assertions defined in
`.fileassert.yaml` YAML configuration files checked in alongside the generated documents.

A non-zero exit code from either invocation causes the CI job to fail immediately.

### Assumptions

- The released package's self-validation test names (`FileAssert_Results`, `FileAssert_File`,
`FileAssert_Text`, `FileAssert_Html`, `FileAssert_Pdf`) remain stable across patch versions and
are present in the TRX output produced by `--validate`.
- The `--results` flag produces a valid TRX file parseable by ReqStream.
- The `count:`, `text:`, `html:`, and `pdf:` YAML assertion keys behave as documented in the
FileAssert User Guide for the pinned version.
- The tool exits non-zero when any assertion fails, causing CI to fail immediately.
28 changes: 0 additions & 28 deletions docs/reqstream/ots/fileassert.yaml

This file was deleted.

64 changes: 64 additions & 0 deletions docs/reqstream/shared/fileassert.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
# FileAssert Shared Package Software Requirements
#
# Requirements for the FileAssert document assertion tool functionality.
# FileAssert is classified as a Shared Package because this repository uses an earlier
# released version of itself in CI - a project cannot treat its own package as OTS.

sections:
- title: Shared Package Software Requirements
sections:
- title: FileAssert Requirements
requirements:
- id: FileAssert-Shared-FileAssert-Results
title: FileAssert shall write TRX test result files.
justification: |
The CI pipeline passes --results <file> to FileAssert on every invocation;
the produced TRX file is consumed by ReqStream to verify requirements
traceability. Without TRX output, no test evidence is recorded and
ReqStream enforcement fails the build.
tags: [shared]
tests:
- FileAssert_Results

- id: FileAssert-Shared-FileAssert-File
title: FileAssert shall assert file count via glob patterns.
justification: |
Every test in .fileassert.yaml uses count: 1 to confirm that exactly one
generated file matches the expected path. This proves that Pandoc and
WeasyPrint each produced the expected output file.
tags: [shared]
tests:
- FileAssert_File

- id: FileAssert-Shared-FileAssert-Text
title: FileAssert shall assert text content of files.
justification: |
HTML document tests use text: contains: to confirm that generated HTML
files contain expected document headings and content keywords, providing
a basic content sanity check independent of HTML structure parsing.
tags: [shared]
tests:
- FileAssert_Text

- id: FileAssert-Shared-FileAssert-Html
title: FileAssert shall assert HTML document structure.
justification: |
Pandoc HTML output is validated using html: query: XPath expressions to
confirm each document has a valid head/title element, proving the HTML is
structurally well-formed.
tags: [shared]
tests:
- FileAssert_Html

- id: FileAssert-Shared-FileAssert-Pdf
title: FileAssert shall assert PDF document contents.
justification: |
WeasyPrint PDF output is validated using pdf: assertions. The justification
covers metadata fields (Title, Author, Subject confirming correct document
identity), page count constraints (confirming the document rendered with
substantive content), and body text content (confirming the expected
document text was included in the rendered output).
tags: [shared]
tests:
- FileAssert_Pdf
3 changes: 2 additions & 1 deletion docs/verification/definition.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ input-files:
- docs/verification/file-assert/selftest/validation.md
- docs/verification/ots.md
- docs/verification/ots/buildmark.md
- docs/verification/ots/fileassert.md
- docs/verification/ots/pandoc.md
- docs/verification/ots/reqstream.md
- docs/verification/ots/reviewmark.md
Expand All @@ -38,6 +37,8 @@ input-files:
- docs/verification/ots/versionmark.md
- docs/verification/ots/weasyprint.md
- docs/verification/ots/xunit.md
- docs/verification/shared.md
- docs/verification/shared/fileassert.md
template: template.html
table-of-contents: true
number-sections: true
11 changes: 2 additions & 9 deletions docs/verification/ots.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ OTS software items used by FileAssert are verified by one of two complementary a
depending on the nature of the item:

- **Authored integration tests**: Items whose correct behaviour cannot be inferred purely from
pipeline success (xUnit, FileAssert) are verified by named test scenarios that exercise the
specific features required by this project. Each scenario identifies a test method, the expected
pipeline success (xUnit) are verified by named test scenarios that exercise the specific
features required by this project. Each scenario identifies a test method, the expected
outcome, and the requirement it covers.

- **CI pipeline evidence**: Items whose primary function is to produce artefacts consumed by
Expand All @@ -32,10 +32,6 @@ The following evidence is collected and retained for each OTS item:
- **xUnit**: TRX result files generated by `dotnet test` during CI. Each TRX file is consumed by
`reqstream --enforce` to confirm that all named test scenarios have a recorded passing result.

- **FileAssert**: The self-validation TRX file (`artifacts/fileassert-self-validation.trx`)
produced by `fileassert --validate --results` during CI. This file is also consumed by
`reqstream --enforce`.

- **All other OTS items**: A passing CI pipeline run, including Pandoc HTML generation, WeasyPrint
PDF rendering, FileAssert assertions on every generated document, and `reqstream --enforce` with
no unmet requirements. The combined CI log and artefacts constitute the qualification record.
Expand Down Expand Up @@ -63,9 +59,6 @@ The following OTS software items are used by FileAssert and are verified in this
- **BuildMark** — generates a markdown build-notes document from GitHub Actions workflow metadata,
verified by CI pipeline success and downstream document validation.

- **FileAssert** — validates generated HTML and PDF documents for existence, size, structure, and
content; verified by self-validation tests and transitive pipeline evidence.

- **Pandoc** — converts Markdown source documents to HTML as part of the documentation build;
verified by FileAssert assertions on each generated HTML document.

Expand Down
Loading
Loading