Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ All notable changes to this project are documented here. The format is based on

## [Unreleased]

### Added
- JSON report output via `--format json` (Markdown remains the default). The JSON includes
`metadata` (tool, version, UTC timestamp, format), `inputs`, `verdict`, `score`, a `summary`
(counts by severity and by finding type) and the full `findings` list. Validation, scoring and
matching behaviour are unchanged. ([#3](https://github.com/migmcc/quality-docs-validator/issues/3))

## [0.1.0] - 2026-06-20

First public release — the PFMEA ↔ Control Plan consistency checker MVP.
Expand Down
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,21 @@ The full Markdown report is written to `report.md`; a committed sample lives at
[examples/report.md](examples/report.md). Finding types and scoring are documented in
[docs/FINDINGS.md](docs/FINDINGS.md).

### JSON output

For automation, emit a machine-readable report with `--format json`:

```bash
qdv pfmea-control-plan \
--pfmea examples/pfmea.xlsx \
--control-plan examples/control-plan.xlsx \
--format json --out report.json
```

Markdown remains the default. The JSON contains `metadata` (tool, version, UTC timestamp),
`verdict`, `score`, a `summary` (counts by severity and by finding type) and the full `findings`
list.

## Architecture

See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the module layout and data flow.
Expand Down
39 changes: 19 additions & 20 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,27 @@
# Roadmap

## v0.1.0 — MVP (PFMEA ↔ Control Plan checker)
The single, narrow vertical slice. `.xlsx` only, recommended template + aliases, ≥5 finding types,
## v0.1.0 — MVP (PFMEA ↔ Control Plan checker) ✅ released
The single, narrow vertical slice. `.xlsx` only, recommended template + aliases, six finding types,
severity-weighted score, Markdown report + terminal summary, synthetic examples, tests, docs.
See [MVP_SCOPE.md](MVP_SCOPE.md).
See [MVP_SCOPE.md](MVP_SCOPE.md). All work packages (repo skeleton, data models, Excel parser,
matching, rules, scoring, report, CLI, examples, tests + CI, docs, release) are complete.

**Build sequence (work packages):**
- WP1 ✅ Repo skeleton & packaging *(this scaffold)*
- WP2 ⏳ Data models (`PFMEARow`, `ControlPlanRow`, `Finding`) + column-alias map
- WP3 ⏳ Excel parser (`parsers/excel.py`)
- WP4 ⏳ Matching engine (`core/matching.py`)
- WP5 ⏳ Validation rules (`modules/pfmea_control_plan.py` + `rules/*.yaml`)
- WP6 ⏳ Scoring (`core/scoring.py`)
- WP7 ⏳ Report generator (`core/report.py`)
- WP8 ⏳ CLI command body (`cli.py`)
- WP9 ⏳ Synthetic examples with a seeded gap
- WP10 ⏳ Tests + Windows/Linux CI
- WP11 ⏳ Docs & ≥3 good-first-issues
- WP12 ⏳ Release v0.1.0
## v0.2 — planned
Improve interoperability and real-world workbook compatibility while keeping the tool focused on
PFMEA ↔ Control Plan validation. Tracked under the
[v0.2 milestone](https://github.com/migmcc/quality-docs-validator/milestone/1). Recommended order:

## v0.2 — Flexibility
- **CSV input** (in addition to `.xlsx`).
- **Configurable column mapping** (beyond recommended template + aliases).
- **JSON / HTML report** output; GitHub Action summary; status badges.
1. **JSON output** ([#3](https://github.com/migmcc/quality-docs-validator/issues/3)) — first PR; a
machine-readable report alongside Markdown (Markdown stays the default).
2. **More PFMEA / Control Plan column aliases**
([#2](https://github.com/migmcc/quality-docs-validator/issues/2)) — broaden real-template coverage.
3. **Multi-sheet workbooks** ([#4](https://github.com/migmcc/quality-docs-validator/issues/4)) —
optional sheet selection.
4. **YAML-driven rules** ([#1](https://github.com/migmcc/quality-docs-validator/issues/1)) —
*stretch / optional*; a rule-engine refactor, deferred to v0.3 if scope grows.

Still out of scope for v0.2: CSV input, configurable column mapping, HTML output, UI, AI, and any
new document pairs. No dates promised.

## v0.3+ — More modules (each independent of the core)
- Process Flow ↔ PFMEA consistency.
Expand Down
24 changes: 20 additions & 4 deletions src/quality_docs_validator/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,24 @@

from __future__ import annotations

from enum import Enum
from pathlib import Path

import typer
from rich.console import Console

from . import __version__
from .core.report import print_terminal_summary, write_markdown
from .core.report import print_terminal_summary, write_json, write_markdown
from .modules.pfmea_control_plan import check_files
from .parsers.excel import ParseError


class ReportFormat(str, Enum):
"""Output format for the generated report file."""

md = "md"
json = "json"

app = typer.Typer(
add_completion=False,
help="Detect potential inconsistencies between PFMEA and Control Plan documents.",
Expand Down Expand Up @@ -53,7 +61,12 @@ def pfmea_control_plan(
Path("report.md"),
"--out",
"--output",
help="Path for the generated Markdown report.",
help="Path for the generated report.",
),
report_format: ReportFormat = typer.Option(
ReportFormat.md,
"--format",
help="Report file format: 'md' (default) or 'json'. Markdown behaviour is unchanged.",
),
) -> None:
"""Check a PFMEA against a Control Plan for potential inconsistencies."""
Expand All @@ -63,9 +76,12 @@ def pfmea_control_plan(
console.print(f"[red]Error:[/red] {exc}")
raise typer.Exit(code=2) from exc

write_markdown(result, pfmea.name, control_plan.name, out)
if report_format is ReportFormat.json:
write_json(result, pfmea.name, control_plan.name, out)
else:
write_markdown(result, pfmea.name, control_plan.name, out)
print_terminal_summary(result, pfmea.name, control_plan.name)
console.print(f"[dim]Report written to[/dim] {out}")
console.print(f"[dim]Report ({report_format.value}) written to[/dim] {out}")


if __name__ == "__main__": # pragma: no cover
Expand Down
67 changes: 67 additions & 0 deletions src/quality_docs_validator/core/report.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,20 @@

from __future__ import annotations

import json
from collections import Counter
from datetime import datetime, timezone
from pathlib import Path

from rich.console import Console
from rich.panel import Panel
from rich.table import Table

from .. import __version__
from ..models import ValidationResult

TOOL_NAME = "quality-docs-validator"

HUMAN_REVIEW_FOOTER = (
"> ⚠️ These are **potential** findings to support human review. "
"quality-docs-validator is **not a substitute for human technical judgement** and makes no "
Expand Down Expand Up @@ -75,6 +80,68 @@ def write_markdown(
return out_path


def build_report_data(
result: ValidationResult, pfmea_name: str, control_plan_name: str
) -> dict:
"""Build a stable, serialisable dict describing a validation run (used for JSON output).

Field names are chosen for automation consumers: findings expose `severity` (the model's
internal `level`). Scoring/matching/validation logic is untouched — this only reshapes output.
"""
by_type: dict[str, int] = dict(Counter(f.finding_type for f in result.findings))
return {
"metadata": {
"tool": TOOL_NAME,
"version": __version__,
"generated_at": datetime.now(timezone.utc).isoformat(),
"format": "json",
},
"inputs": {
"pfmea": {"file": pfmea_name, "rows": result.pfmea_rows},
"control_plan": {"file": control_plan_name, "rows": result.control_plan_rows},
},
"verdict": result.verdict,
"score": result.score,
"summary": {
"total": len(result.findings),
"by_severity": {
"critical": result.critical_count,
"warning": result.warning_count,
},
"by_type": by_type,
},
"findings": [
{
"type": f.finding_type,
"severity": f.level,
"operation_id": f.operation_id,
"message": f.message,
}
for f in result.findings
],
}


def render_json(result: ValidationResult, pfmea_name: str, control_plan_name: str) -> str:
return json.dumps(
build_report_data(result, pfmea_name, control_plan_name),
indent=2,
ensure_ascii=False,
)


def write_json(
result: ValidationResult, pfmea_name: str, control_plan_name: str, out_path: str | Path
) -> Path:
out_path = Path(out_path)
if out_path.parent and not out_path.parent.exists():
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_text(
render_json(result, pfmea_name, control_plan_name) + "\n", encoding="utf-8"
)
return out_path


def print_terminal_summary(
result: ValidationResult, pfmea_name: str, control_plan_name: str, console: Console | None = None
) -> None:
Expand Down
115 changes: 115 additions & 0 deletions tests/test_json_output.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
"""Tests for JSON report output (issue #3)."""

from __future__ import annotations

import json
from pathlib import Path

from typer.testing import CliRunner

from quality_docs_validator import __version__
from quality_docs_validator.cli import app
from quality_docs_validator.core.report import build_report_data
from quality_docs_validator.modules.pfmea_control_plan import check_files

runner = CliRunner()

EXPECTED_TYPES = {
"UNMATCHED_PROCESS_STEP",
"MISSING_CONTROL",
"SPECIAL_CHARACTERISTIC_NOT_CONTROLLED",
"MISSING_REACTION_PLAN",
"WEAK_DETECTION_METHOD",
"HIGH_SEVERITY_WEAK_CONTROL",
}


def test_markdown_is_default(example_files, tmp_path: Path) -> None:
pfmea, control_plan = example_files
out = tmp_path / "report.md"
result = runner.invoke(
app,
["pfmea-control-plan", "--pfmea", str(pfmea), "--control-plan", str(control_plan), "--out", str(out)],
)
assert result.exit_code == 0
text = out.read_text(encoding="utf-8")
# Still Markdown, not JSON.
assert "# PFMEA" in text
assert not text.lstrip().startswith("{")


def test_json_generated_with_flag(example_files, tmp_path: Path) -> None:
pfmea, control_plan = example_files
out = tmp_path / "report.json"
result = runner.invoke(
app,
[
"pfmea-control-plan",
"--pfmea", str(pfmea),
"--control-plan", str(control_plan),
"--format", "json",
"--out", str(out),
],
)
assert result.exit_code == 0
assert out.exists()
# Parseable JSON.
data = json.loads(out.read_text(encoding="utf-8"))
assert set(["metadata", "verdict", "score", "summary", "findings"]) <= set(data)


def test_json_metadata_shape(example_files, tmp_path: Path) -> None:
pfmea, control_plan = example_files
out = tmp_path / "report.json"
runner.invoke(
app,
["pfmea-control-plan", "--pfmea", str(pfmea), "--control-plan", str(control_plan), "--format", "json", "--out", str(out)],
)
data = json.loads(out.read_text(encoding="utf-8"))
md = data["metadata"]
assert md["tool"] == "quality-docs-validator"
assert md["version"] == __version__
assert md["format"] == "json"
assert md["generated_at"].endswith("+00:00") # ISO-8601 UTC


def test_json_contains_expected_finding_types(example_files) -> None:
pfmea, control_plan = example_files
result = check_files(pfmea, control_plan)
data = build_report_data(result, "pfmea.xlsx", "control-plan.xlsx")
types = {f["type"] for f in data["findings"]}
assert EXPECTED_TYPES <= types
# Each finding exposes the documented keys.
for f in data["findings"]:
assert set(f) == {"type", "severity", "operation_id", "message"}
assert f["severity"] in {"critical", "warning"}
# Summary is internally consistent.
assert data["summary"]["total"] == len(data["findings"])
assert sum(data["summary"]["by_type"].values()) == data["summary"]["total"]
assert (
data["summary"]["by_severity"]["critical"] + data["summary"]["by_severity"]["warning"]
== data["summary"]["total"]
)


def test_json_clean_case_is_empty(make_xlsx, tmp_path: Path) -> None:
pfmea = make_xlsx(
tmp_path / "pf.xlsx",
["Operation ID", "Failure Mode", "Severity", "Special Characteristic"],
[["10", "Leak", 6, "No"]],
)
cp = make_xlsx(
tmp_path / "cp.xlsx",
["Operation ID", "Control Method", "Reaction Plan", "Special Characteristic"],
[["10", "Pressure gauge test", "Stop and rework", "No"]],
)
out = tmp_path / "clean.json"
runner.invoke(
app,
["pfmea-control-plan", "--pfmea", str(pfmea), "--control-plan", str(cp), "--format", "json", "--out", str(out)],
)
data = json.loads(out.read_text(encoding="utf-8"))
assert data["score"] == 100
assert data["verdict"] == "PASS"
assert data["findings"] == []
assert data["summary"]["total"] == 0
Loading