Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,15 @@ configs to pytest format, runs native pytest, and returns rich in-memory results
Validation classes live in `isvtest/src/isvtest/validations/` grouped by domain
(`generic.py`, `cluster.py`, `instance.py`, `network.py`, `iam.py`, `security.py`,
`host.py`, `k8s_*.py`, `slurm_*.py`, `bm_*.py`). Each subclass is auto-discovered.
Filtering labels live on the YAML wiring (`labels: [...]` per check in the suite/
provider configs), not on the class; the catalog, pytest marks, `isvctl docs`,
and the orchestrator's include/exclude-label filtering all read them from there.
Filtering labels live on the YAML wiring (`labels: [...]` per check), not on the
class; the catalog, pytest marks, `isvctl docs`, and the orchestrator's
include/exclude-label filtering all read them from there. Declare labels ONLY in
`isvctl/configs/suites/*.yaml` - never add `labels:` to per-check wiring under
`isvctl/configs/providers/**`; provider configs inherit labels from the suites
they import (top-level `exclude.labels:` filtering blocks are fine). Sole
exception: the single-node local providers
`isvctl/configs/providers/{k3s,microk8s,minikube}.yaml`, which wire host-level
checks that exist in no suite.

Workloads (`isvtest/src/isvtest/workloads/`) are long-running tests (NIM, NCCL,
stress) labelled `("workload", "slow", ...)` with manifests and helper scripts
Expand Down
2 changes: 0 additions & 2 deletions isvctl/configs/providers/nico/config/control-plane.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,9 @@ tests:
checks:
FieldExistsCheck:
test_id: "N/A"
labels: ["control_plane"]
fields: ["account_id", "tests"]
FieldValueCheck:
test_id: "CP03-01"
labels: ["control_plane"]
field: "success"
expected: true

Expand Down
2 changes: 0 additions & 2 deletions isvctl/configs/providers/nico/config/iam.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,9 @@ tests:
checks:
FieldExistsCheck:
test_id: "N/A"
labels: ["iam"]
fields: ["account_id", "authenticated", "tests"]
StepSuccessCheck:
test_id: "N/A"
labels: ["iam"]

exclude:
labels: []
8 changes: 1 addition & 7 deletions isvctl/configs/providers/nico/config/network.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -109,24 +109,20 @@ tests:
checks:
TenantListedCheck:
test_id: "N/A"
labels: ["network"]

vpc_info:
step: get_vpc
checks:
TenantInfoCheck:
test_id: "SDN01-02"
labels: ["min_req", "network"]
test_id: "N/A"

network_connectivity:
step: network_connectivity
checks:
StepSuccessCheck:
test_id: "N/A"
labels: ["network"]
FieldValueCheck:
test_id: "N/A"
labels: ["network"]
field: "tests.network_assigned.passed"
expected: true

Expand All @@ -135,10 +131,8 @@ tests:
checks:
StepSuccessCheck:
test_id: "N/A"
labels: ["network"]
FieldValueCheck:
test_id: "N/A"
labels: ["network"]
field: "tests.network_setup.passed"
expected: true

Expand Down