diff --git a/docs/explanation/validation-coverage.md b/docs/explanation/validation-coverage.md index a4c29f2e..7a60721e 100644 --- a/docs/explanation/validation-coverage.md +++ b/docs/explanation/validation-coverage.md @@ -1,18 +1,17 @@ # Validation coverage and write scope -Validation responses today are lists of -[`ValidationIssue`](../../proto/trogonatlas/api/eventmodel/v1alpha1/validation.proto) -findings. They do not say which check suites ran, against what scope, or -whether the underlying snapshot was truncated. Callers infer coverage only -from which RPC they invoked. That is not enough for agents, CI, or -dry-run/apply consumers that must refuse to treat an empty issue list as -"everything was checked." - -This page records what each entry point actually runs on current `main`, -and the reporting contract coverage and truncation work must share. It is -the inventory and design target for the Open Ontologies adoption Tier 1 -packages (coverage, then bounded results). It does not expand enforcement -by itself: truthful reporting can land before any route runs more checks. +Validation findings are lists of +[`ValidationIssue`](../../proto/trogonatlas/api/eventmodel/v1alpha1/validation.proto). +Coverage says which suites ran, against what scope, and whether the +snapshot was truncated. `ValidateEventModel` and `ValidateProject` return +[`ValidationCoverage`](../../proto/trogonatlas/api/eventmodel/v1alpha1/validation.proto) +alongside issues. Write and merge envelopes do not yet; callers must not +invent coverage client-side when it is absent. + +This page records what each entry point actually runs, and the shared +reporting contract for coverage and truncation. It does not expand +enforcement by itself: truthful reporting can land before any route runs +more checks. ## Check layers @@ -39,12 +38,12 @@ Inventoried at `23ade8f` (post [#263](https://github.com/TrogonStack/atlas/pull/ | Entry point | Suites that run | Scope | Response today | | --- | --- | --- | --- | -| `ValidateEventModel` | `model` always; strategic chain only when `EventModel.id` is set (ID-less inline bodies skip the chain) | Branch snapshot + visibility; in-body model or by-id | `issues[]` only | -| `ValidateProject` | `store_hygiene` + architecture resource diagnostics once (including data-product Event confirmation); `model` per in-scope EventModel; strategic once | Project / domain / namespace | `reports[]` + severity totals | -| `PutEntity` (apply and `validate_only`) | `request_shape`; architecture admission on arch kinds; `scenarios` Errors on applicable slices; advisory `scenarios`/`strategic` | Slice/strategic use the branch snapshot | `validation[]`; hard refusals as Status | -| `BatchMutate` (apply and `validate_only`) | Same as Put for ops; strategic and scenario on overlay | Post-lock snapshot + overlay | Per-op `validation[]` / batch failure | -| `MergeBranch` (apply and dry-run) | Architecture landing on landed arch resources; blocking `store_hygiene` Errors; advisory `strategic` | Baseline visibility + projected post-merge world | `validation[]` + merge status | -| MCP `apply_manifests` | Client plan → `BatchMutate` | Client branch/scope | Tool JSON from batch results | +| `ValidateEventModel` | `model` always; strategic chain only when `EventModel.id` is set (ID-less inline bodies skip the chain) | Branch snapshot + visibility; in-body model or by-id | `issues[]` + `coverage` (`model` / `strategic`, `snapshot_truncated`) | +| `ValidateProject` | `store_hygiene` + architecture resource diagnostics once (including data-product Event confirmation); `model` per in-scope EventModel; strategic once | Project / domain / namespace | `reports[]` + severity totals + `coverage` | +| `PutEntity` (apply and `validate_only`) | `request_shape`; architecture admission on arch kinds; `scenarios` Errors on applicable slices; advisory `scenarios`/`strategic` | Slice/strategic use the branch snapshot | `validation[]`; hard refusals as Status; **no coverage yet** | +| `BatchMutate` (apply and `validate_only`) | Same as Put for ops; strategic and scenario on overlay | Post-lock snapshot + overlay | Per-op `validation[]` / batch failure; **no coverage yet** | +| `MergeBranch` (apply and dry-run) | Architecture landing on landed arch resources; blocking `store_hygiene` Errors; advisory `strategic` | Baseline visibility + projected post-merge world | `validation[]` + merge status; **no coverage yet** | +| MCP `apply_manifests` | Client plan → `BatchMutate` | Client branch/scope | Tool JSON from batch results; **no coverage yet** | ### Explicit non-runs (do not overclaim) @@ -59,35 +58,40 @@ Inventoried at `23ade8f` (post [#263](https://github.com/TrogonStack/atlas/pull/ - **ValidateEventModel does not run** store hygiene (that is ValidateProject) and does not re-scan every member through full `SCENARIO_*` emission the way write-path `validate_scenarios` does. -- **No validation RPC** reports suite coverage or snapshot truncation. - `ValidateProject` can still surface an incomplete Event confirmation - scan as the finding `DATA_PRODUCT_CONFIRMATION_INCOMPLETE` (budget or - discovery exhaustion), which is an issue code rather than coverage - metadata. Unrelated truncation exists elsewhere (projections, source - coverage, delete `referrers_incomplete`). +- **Write and merge RPCs** do not yet report suite coverage or snapshot + truncation on their envelopes. `ValidateProject` can still surface an + incomplete Event confirmation scan as the finding + `DATA_PRODUCT_CONFIRMATION_INCOMPLETE` (budget or discovery exhaustion), + which is an issue code rather than coverage metadata. Unrelated + truncation exists elsewhere (projections, source coverage, delete + `referrers_incomplete`). -## Proposed coverage schema +## Coverage schema -Coverage must stay separate from findings: +Coverage stays separate from findings: - A **completed** suite can still emit Error findings. - A **skipped** suite must never be presented as a pass. - A missing or default field must not imply `complete`. +- Suites that ran against a truncated snapshot are reported as `PARTIAL` + with reason `snapshot_truncated`, and `ValidationCoverage.snapshot_truncated` + is true. -Proposed fields (names are design targets until the proto lands): +Wire fields on `ValidationCoverage` / `ValidationCheckResult` / +`ValidationScope`: | Field | Meaning | | --- | --- | -| `check_id` | Stable suite or pass identifier (`store_hygiene`, `model`, …) | +| `check_id` | Stable suite id (`store_hygiene`, `model`, `strategic`, `architecture`, …) | | `state` | `COMPLETED` / `SKIPPED` / `PARTIAL` / `ERROR` | -| `reason` | Why skipped/partial/error (unsupported on this route, truncated snapshot, execution failure, narrower requested set) | -| `subjects_examined` / `subjects_skipped` | Counts or refs when meaningful; zero-subject completed checks are allowed and must say so | -| `requested_scope` / `effective_scope` | Branch, lens/visibility, project/domain/namespace/model ids | -| `snapshot_truncated` / related bounds | Shared with [snapshot identity](snapshot-identity.md) and the truncation package | +| `reason` | Why skipped/partial/error (for example `event_model_id_absent`, `snapshot_truncated`) | +| `subjects_examined` | Count when meaningful; zero is a valid completed empty scan | +| `effective_scope` | What store-wide suites actually examined. On ValidateProject this is the full visible snapshot (empty namespace/project/domain), because hygiene and architecture are not narrowed to the request filter | +| `requested_scope` | What the caller asked to validate (may be narrower than `effective_scope`) | +| `snapshot_truncated` | Shared with [snapshot identity](snapshot-identity.md) and the truncation package | Write and dry-run envelopes (`PutEntity`, `BatchMutate`, `MergeBranch`) -should declare the suites they actually ran. Validate* responses should -list suite results alongside issues. Clients (Rust, CLI, MCP) must +should gain the same message next. Clients (Rust, CLI, MCP) must propagate server fields and must not invent coverage client-side. ### Incomplete validation and CLI exits diff --git a/proto/trogonatlas/api/eventmodel/v1alpha1/validation.proto b/proto/trogonatlas/api/eventmodel/v1alpha1/validation.proto index a61ff99f..7d2d184a 100644 --- a/proto/trogonatlas/api/eventmodel/v1alpha1/validation.proto +++ b/proto/trogonatlas/api/eventmodel/v1alpha1/validation.proto @@ -46,6 +46,9 @@ message ValidateEventModelRequest { message ValidateEventModelResponse { repeated ValidationIssue issues = 1; + // Suites that ran for this call, plus whether the snapshot was truncated. + // Absent or empty coverage must not be treated as "everything was checked". + ValidationCoverage coverage = 2; } // Static description of a single validation rule. Surfaces the catalog @@ -98,4 +101,63 @@ message ValidateProjectResponse { int32 total_errors = 2; int32 total_warnings = 3; int32 total_info = 4; + // Suites that ran for this call, plus whether the snapshot was truncated. + ValidationCoverage coverage = 5; +} + +// ---------- Coverage --------------------------------------------------------- +// +// Findings (`ValidationIssue`) say what was wrong. Coverage says what was +// checked. A completed suite can still emit Errors; a skipped suite must +// never be read as a pass. See docs/explanation/validation-coverage.md. + +message ValidationCheckResult { + enum State { + STATE_UNSPECIFIED = 0; + // Suite finished for the declared scope. Findings may still be Errors. + STATE_COMPLETED = 1; + // Suite was not applicable on this route or for this request shape. + STATE_SKIPPED = 2; + // Suite ran but could not cover the full declared scope (for example a + // truncated snapshot). Do not treat an empty issue list as complete. + STATE_PARTIAL = 3; + // Suite failed to execute. Distinct from finding Errors inside a + // completed suite. + STATE_ERROR = 4; + } + // Stable suite id: "model", "strategic", "store_hygiene", "architecture", + // "scenarios", "request_shape". + string check_id = 1; + State state = 2; + // Why skipped / partial / error. Empty when completed without caveat. + string reason = 3; + // Subjects examined when meaningful (models, namespaces, …). Zero is a + // valid completed empty scan and must not be rewritten by clients. + int32 subjects_examined = 4; +} + +message ValidationScope { + // Empty means baseline (no branch header). + string branch = 1; + // Effective project/domain/namespace/model when the route has one. + trogonatlas.type.v1alpha1.Id project_id = 2; + trogonatlas.type.v1alpha1.Id domain_id = 3; + string namespace = 4; + trogonatlas.type.v1alpha1.Id event_model_id = 5; +} + +message ValidationCoverage { + repeated ValidationCheckResult checks = 1; + // What store-wide suites actually examined. For ValidateProject this is the + // full visible snapshot under the branch (namespace / project / domain + // empty): hygiene and architecture are not narrowed to the request filter. + // Model and strategic suite subject counts still follow the request. + ValidationScope effective_scope = 2; + // True when the entity snapshot hit the configured cap. Shared with the + // truncation package; an empty issue list on a truncated snapshot is not + // a clean bill of health. + bool snapshot_truncated = 3; + // What the caller asked to validate (project / domain / namespace / model). + // May be narrower than `effective_scope` for store-wide checks. + ValidationScope requested_scope = 4; } diff --git a/rsworkspace/crates/trogon-atlas-proto/src/gen/trogonatlas.api.eventmodel.v1alpha1.rs b/rsworkspace/crates/trogon-atlas-proto/src/gen/trogonatlas.api.eventmodel.v1alpha1.rs index 98976425..13edafff 100644 --- a/rsworkspace/crates/trogon-atlas-proto/src/gen/trogonatlas.api.eventmodel.v1alpha1.rs +++ b/rsworkspace/crates/trogon-atlas-proto/src/gen/trogonatlas.api.eventmodel.v1alpha1.rs @@ -1174,6 +1174,10 @@ pub struct ValidateEventModelRequest { pub struct ValidateEventModelResponse { #[prost(message, repeated, tag="1")] pub issues: ::prost::alloc::vec::Vec, + /// Suites that ran for this call, plus whether the snapshot was truncated. + /// Absent or empty coverage must not be treated as "everything was checked". + #[prost(message, optional, tag="2")] + pub coverage: ::core::option::Option, } /// Static description of a single validation rule. Surfaces the catalog /// the validator dispatches on: stable machine `code`, canonical @@ -1253,6 +1257,9 @@ pub struct ValidateProjectResponse { pub total_warnings: i32, #[prost(int32, tag="4")] pub total_info: i32, + /// Suites that ran for this call, plus whether the snapshot was truncated. + #[prost(message, optional, tag="5")] + pub coverage: ::core::option::Option, } /// Nested message and enum types in `ValidateProjectResponse`. pub mod validate_project_response { @@ -1264,6 +1271,107 @@ pub mod validate_project_response { pub issues: ::prost::alloc::vec::Vec, } } +// ---------- Coverage --------------------------------------------------------- +// +// Findings (`ValidationIssue`) say what was wrong. Coverage says what was +// checked. A completed suite can still emit Errors; a skipped suite must +// never be read as a pass. See docs/explanation/validation-coverage.md. + +#[derive(Clone, PartialEq, Eq, Hash, ::prost::Message)] +pub struct ValidationCheckResult { + /// Stable suite id: "model", "strategic", "store_hygiene", "architecture", + /// "scenarios", "request_shape". + #[prost(string, tag="1")] + pub check_id: ::prost::alloc::string::String, + #[prost(enumeration="validation_check_result::State", tag="2")] + pub state: i32, + /// Why skipped / partial / error. Empty when completed without caveat. + #[prost(string, tag="3")] + pub reason: ::prost::alloc::string::String, + /// Subjects examined when meaningful (models, namespaces, …). Zero is a + /// valid completed empty scan and must not be rewritten by clients. + #[prost(int32, tag="4")] + pub subjects_examined: i32, +} +/// Nested message and enum types in `ValidationCheckResult`. +pub mod validation_check_result { + #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, ::prost::Enumeration)] + #[repr(i32)] + pub enum State { + Unspecified = 0, + /// Suite finished for the declared scope. Findings may still be Errors. + Completed = 1, + /// Suite was not applicable on this route or for this request shape. + Skipped = 2, + /// Suite ran but could not cover the full declared scope (for example a + /// truncated snapshot). Do not treat an empty issue list as complete. + Partial = 3, + /// Suite failed to execute. Distinct from finding Errors inside a + /// completed suite. + Error = 4, + } + impl State { + /// String value of the enum field names used in the ProtoBuf definition. + /// + /// The values are not transformed in any way and thus are considered stable + /// (if the ProtoBuf definition does not change) and safe for programmatic use. + pub fn as_str_name(&self) -> &'static str { + match self { + Self::Unspecified => "STATE_UNSPECIFIED", + Self::Completed => "STATE_COMPLETED", + Self::Skipped => "STATE_SKIPPED", + Self::Partial => "STATE_PARTIAL", + Self::Error => "STATE_ERROR", + } + } + /// Creates an enum from field names used in the ProtoBuf definition. + pub fn from_str_name(value: &str) -> ::core::option::Option { + match value { + "STATE_UNSPECIFIED" => Some(Self::Unspecified), + "STATE_COMPLETED" => Some(Self::Completed), + "STATE_SKIPPED" => Some(Self::Skipped), + "STATE_PARTIAL" => Some(Self::Partial), + "STATE_ERROR" => Some(Self::Error), + _ => None, + } + } + } +} +#[derive(Clone, PartialEq, Eq, Hash, ::prost::Message)] +pub struct ValidationScope { + /// Empty means baseline (no branch header). + #[prost(string, tag="1")] + pub branch: ::prost::alloc::string::String, + /// Effective project/domain/namespace/model when the route has one. + #[prost(message, optional, tag="2")] + pub project_id: ::core::option::Option, + #[prost(message, optional, tag="3")] + pub domain_id: ::core::option::Option, + #[prost(string, tag="4")] + pub namespace: ::prost::alloc::string::String, + #[prost(message, optional, tag="5")] + pub event_model_id: ::core::option::Option, +} +#[derive(Clone, PartialEq, ::prost::Message)] +pub struct ValidationCoverage { + #[prost(message, repeated, tag="1")] + pub checks: ::prost::alloc::vec::Vec, + /// What store-wide suites actually examined. For ValidateProject this is the + /// full visible snapshot under the branch (namespace / project / domain + /// empty): hygiene and architecture are not narrowed to the request filter. + /// Model and strategic suite subject counts still follow the request. + #[prost(message, optional, tag="2")] + pub effective_scope: ::core::option::Option, + /// True when the entity snapshot hit the configured cap. Shared with the + /// truncation package; an empty issue list on a truncated snapshot is not + /// a clean bill of health. + #[prost(bool, tag="3")] + pub snapshot_truncated: bool, + /// What the caller asked to validate (project / domain / namespace / model). + /// May be narrower than `effective_scope` for store-wide checks. + #[prost(message, optional, tag="4")] + pub requested_scope: ::core::option::Option, +} // Tenant type libraries are stored as proto source text and compiled on // demand. These messages let a client check a library and look up the // compiled types before it writes anything that depends on them. Every RPC diff --git a/rsworkspace/crates/trogon-atlas-server/src/service.rs b/rsworkspace/crates/trogon-atlas-server/src/service.rs index 798c5e11..0c650c38 100644 --- a/rsworkspace/crates/trogon-atlas-server/src/service.rs +++ b/rsworkspace/crates/trogon-atlas-server/src/service.rs @@ -7141,9 +7141,49 @@ impl pb::event_model_service_server::EventModelService for EventModelServiceImpl )); }; - let all = self.snapshot_for(branch.as_deref(), &vis).await?; + let (all, snapshot_truncated) = self + .snapshot_with_truncation(branch.as_deref(), &vis) + .await?; let issues = validation::validate_model(&model, &all); - Ok(Response::new(pb::ValidateEventModelResponse { issues })) + let mut checks = vec![validation::ran_against_snapshot( + validation::CHECK_MODEL, + 1, + snapshot_truncated, + )]; + match model.id.as_ref() { + Some(_) => checks.push(validation::ran_against_snapshot( + validation::CHECK_STRATEGIC, + 1, + snapshot_truncated, + )), + None => checks.push(validation::coverage_check( + validation::CHECK_STRATEGIC, + pb::validation_check_result::State::Skipped, + "event_model_id_absent", + 0, + )), + } + let model_scope = pb::ValidationScope { + branch: branch.clone().unwrap_or_default(), + project_id: None, + domain_id: None, + namespace: model + .id + .as_ref() + .map(|id| id.namespace.clone()) + .unwrap_or_default(), + event_model_id: model.id.clone(), + }; + let coverage = validation::validation_coverage( + checks, + model_scope.clone(), + model_scope, + snapshot_truncated, + ); + Ok(Response::new(pb::ValidateEventModelResponse { + issues, + coverage: Some(coverage), + })) } async fn list_validation_rules( @@ -7172,7 +7212,9 @@ impl pb::event_model_service_server::EventModelService for EventModelServiceImpl return Err(Status::invalid_argument("namespace must not be empty")); } - let all = self.snapshot_for(branch.as_deref(), &vis).await?; + let (all, snapshot_truncated) = self + .snapshot_with_truncation(branch.as_deref(), &vis) + .await?; let architecture_context = self .architecture_validation_context(&all, branch.as_deref(), &vis) @@ -7294,11 +7336,75 @@ impl pb::event_model_service_server::EventModelService for EventModelServiceImpl reports.insert(0, synthetic); } + let models_examined = i32::try_from( + reports + .iter() + .filter(|r| r.event_model_id.is_some()) + .count(), + ) + .unwrap_or(i32::MAX); + // Store hygiene / architecture walk the full visible snapshot. + // Request filters only select which EventModels and strategic + // namespaces are reported; they must not be written into + // `effective_scope` or clients will attribute store findings wrongly. + let branch_name = branch.clone().unwrap_or_default(); + let requested_scope = pb::ValidationScope { + branch: branch_name.clone(), + project_id: match &scope { + Scope::ProjectId(id) => Some(id.clone()), + _ => None, + }, + domain_id: match &scope { + Scope::DomainId(id) => Some(id.clone()), + _ => None, + }, + namespace: match &scope { + Scope::Namespace(ns) => ns.clone(), + _ => String::new(), + }, + event_model_id: None, + }; + let store_wide_scope = pb::ValidationScope { + branch: branch_name, + project_id: None, + domain_id: None, + namespace: String::new(), + event_model_id: None, + }; + let coverage = validation::validation_coverage( + vec![ + validation::ran_against_snapshot( + validation::CHECK_STORE_HYGIENE, + 1, + snapshot_truncated, + ), + validation::ran_against_snapshot( + validation::CHECK_ARCHITECTURE, + 1, + snapshot_truncated, + ), + validation::ran_against_snapshot( + validation::CHECK_MODEL, + models_examined, + snapshot_truncated, + ), + validation::ran_against_snapshot( + validation::CHECK_STRATEGIC, + i32::try_from(scoped_namespaces.len()).unwrap_or(i32::MAX), + snapshot_truncated, + ), + ], + store_wide_scope, + requested_scope, + snapshot_truncated, + ); + Ok(Response::new(pb::ValidateProjectResponse { reports, total_errors, total_warnings, total_info, + coverage: Some(coverage), })) } diff --git a/rsworkspace/crates/trogon-atlas-server/src/validation.rs b/rsworkspace/crates/trogon-atlas-server/src/validation.rs index 40cb4d6c..f59a6198 100644 --- a/rsworkspace/crates/trogon-atlas-server/src/validation.rs +++ b/rsworkspace/crates/trogon-atlas-server/src/validation.rs @@ -3001,6 +3001,68 @@ pub fn validate_model(model: &pb::EventModel, all: &[StoredEntity]) -> Vec, + state: pb::validation_check_result::State, + reason: impl Into, + subjects_examined: i32, +) -> pb::ValidationCheckResult { + pb::ValidationCheckResult { + check_id: check_id.into(), + state: state as i32, + reason: reason.into(), + subjects_examined, + } +} + +/// Mark a suite that ran against the loaded snapshot. Truncation makes the +/// result `PARTIAL` so an empty issue list cannot look complete. +#[must_use] +pub fn ran_against_snapshot( + check_id: impl Into, + subjects_examined: i32, + snapshot_truncated: bool, +) -> pb::ValidationCheckResult { + if snapshot_truncated { + coverage_check( + check_id, + pb::validation_check_result::State::Partial, + "snapshot_truncated", + subjects_examined, + ) + } else { + coverage_check( + check_id, + pb::validation_check_result::State::Completed, + "", + subjects_examined, + ) + } +} + +#[must_use] +pub fn validation_coverage( + checks: Vec, + effective_scope: pb::ValidationScope, + requested_scope: pb::ValidationScope, + snapshot_truncated: bool, +) -> pb::ValidationCoverage { + pb::ValidationCoverage { + checks, + effective_scope: Some(effective_scope), + snapshot_truncated, + requested_scope: Some(requested_scope), + } +} + #[must_use] pub fn validate_model_content( model: &pb::EventModel, diff --git a/rsworkspace/crates/trogon-atlas-server/tests/handler_tests.rs b/rsworkspace/crates/trogon-atlas-server/tests/handler_tests.rs index 7953ca2f..708e4182 100644 --- a/rsworkspace/crates/trogon-atlas-server/tests/handler_tests.rs +++ b/rsworkspace/crates/trogon-atlas-server/tests/handler_tests.rs @@ -359,19 +359,72 @@ async fn validate_event_model_reports_the_strategic_chain() { seed_confused_context(&svc, "vem-chain").await; put_entity(&svc, make_event_model("vem-chain", "checkout")).await; - let issues = svc + let resp = svc .validate_event_model(Request::new(pb::ValidateEventModelRequest { event_model: None, event_model_id: Some(id("vem-chain", "checkout", 1)), })) .await .unwrap() - .into_inner() - .issues; + .into_inner(); + let issues = resp.issues; for code in STRATEGIC_FINDINGS { assert_eq!(codes_of(&issues, code), 1, "{code}: {issues:?}"); } assert_eq!(codes_of(&issues, "REALIZES_MULTIPLE"), 0, "{issues:?}"); + let coverage = resp + .coverage + .expect("ValidateEventModel must report coverage"); + assert!(!coverage.snapshot_truncated); + let checks: Vec<_> = coverage + .checks + .iter() + .map(|c| (c.check_id.as_str(), c.state, c.reason.as_str())) + .collect(); + assert!( + checks.contains(&( + "model", + pb::validation_check_result::State::Completed as i32, + "" + )), + "{checks:?}" + ); + assert!( + checks.contains(&( + "strategic", + pb::validation_check_result::State::Completed as i32, + "" + )), + "{checks:?}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn validate_event_model_skips_strategic_without_an_id() { + let svc = svc().await; + let resp = svc + .validate_event_model(Request::new(pb::ValidateEventModelRequest { + event_model: Some(pb::EventModel { + id: None, + title: "inline".into(), + ..Default::default() + }), + event_model_id: None, + })) + .await + .unwrap() + .into_inner(); + let coverage = resp.coverage.expect("coverage required"); + let strategic = coverage + .checks + .iter() + .find(|c| c.check_id == "strategic") + .expect("strategic suite"); + assert_eq!( + strategic.state, + pb::validation_check_result::State::Skipped as i32 + ); + assert_eq!(strategic.reason, "event_model_id_absent"); } #[tokio::test(flavor = "multi_thread")] @@ -406,6 +459,33 @@ async fn validate_project_reports_the_strategic_chain_once() { for code in STRATEGIC_FINDINGS { assert_eq!(codes_of(&store_report.issues, code), 1, "{code}"); } + let coverage = resp.coverage.expect("ValidateProject must report coverage"); + assert!(!coverage.snapshot_truncated); + let check_ids: Vec<_> = coverage + .checks + .iter() + .map(|c| c.check_id.as_str()) + .collect(); + for id in ["store_hygiene", "architecture", "model", "strategic"] { + assert!(check_ids.contains(&id), "missing {id} in {check_ids:?}"); + } + let model = coverage + .checks + .iter() + .find(|c| c.check_id == "model") + .expect("model suite"); + assert_eq!(model.subjects_examined, 2); + assert_eq!( + model.state, + pb::validation_check_result::State::Completed as i32 + ); + let requested = coverage.requested_scope.expect("requested_scope required"); + assert_eq!(requested.namespace, "vp-chain"); + let effective = coverage.effective_scope.expect("effective_scope required"); + assert!( + effective.namespace.is_empty(), + "store-wide suites examine the full visible snapshot, not the request namespace: {effective:?}" + ); } #[tokio::test(flavor = "multi_thread")]