What's wrong
The header table at the top of every feature-smoke HTML report (and the CROSS-HOST TEST preflight rows) renders:
Advertised IP unknown
ATM version unknown
Doctor NOT RUN
even when the run's own doctor-equivalent case genuinely ran and passed (e.g. AD11-CMD-DOCTOR-001 visible further down the same report as PASS).
Root cause
scripts/smoke/feature_smoke_report.py::host_preflight_facts (lines 52-73) only recognizes a case whose name is exactly "doctor" or ends with "doctor/version":
doctor_cases = [case for case in self_cases if case["name"] == "doctor" or case["name"].endswith("doctor/version")]
and only recognizes an IP-advertisement case whose name ends with "advertised host" (line 61). The thorough fixture's cases are named after their test IDs (e.g. AD11-CMD-DOCTOR-001), which match neither pattern, so host_preflight_facts silently falls through to "unknown" / "NOT RUN" even though the check ran and passed.
Repro
Open site/reports/smoke/macos/<host>/<timestamp>-thorough/thorough.html (or index.html) for any thorough smoke run and look at the top header table plus the "CROSS-HOST TEST" preflight row — both show unknown/NOT RUN despite AD11-CMD-DOCTOR-001 passing in the case ladder immediately below.
Impact
Cosmetic/report-accuracy only — the underlying smoke suite genuinely validated the daemon. But the report is misleading evidence: a reviewer skimming just the header table would wrongly conclude doctor/version was never checked.
Suggested fix
Widen the matcher in host_preflight_facts to also recognize the suite's actual doctor/version case-naming convention (e.g. match on case name containing DOCTOR or a dedicated kind/category tag on the case dict), rather than requiring the literal string "doctor".
What's wrong
The header table at the top of every feature-smoke HTML report (and the CROSS-HOST TEST preflight rows) renders:
even when the run's own doctor-equivalent case genuinely ran and passed (e.g.
AD11-CMD-DOCTOR-001visible further down the same report as PASS).Root cause
scripts/smoke/feature_smoke_report.py::host_preflight_facts(lines 52-73) only recognizes a case whosenameis exactly"doctor"or ends with"doctor/version":and only recognizes an IP-advertisement case whose name ends with
"advertised host"(line 61). Thethoroughfixture's cases are named after their test IDs (e.g.AD11-CMD-DOCTOR-001), which match neither pattern, sohost_preflight_factssilently falls through to"unknown"/"NOT RUN"even though the check ran and passed.Repro
Open
site/reports/smoke/macos/<host>/<timestamp>-thorough/thorough.html(orindex.html) for any thorough smoke run and look at the top header table plus the "CROSS-HOST TEST" preflight row — both showunknown/NOT RUNdespiteAD11-CMD-DOCTOR-001passing in the case ladder immediately below.Impact
Cosmetic/report-accuracy only — the underlying smoke suite genuinely validated the daemon. But the report is misleading evidence: a reviewer skimming just the header table would wrongly conclude doctor/version was never checked.
Suggested fix
Widen the matcher in
host_preflight_factsto also recognize the suite's actual doctor/version case-naming convention (e.g. match on case name containingDOCTORor a dedicatedkind/categorytag on the case dict), rather than requiring the literal string"doctor".