Skip to content

fix(labels): declare check labels only in suites#526

Open
mresvanis wants to merge 1 commit into
NVIDIA:mainfrom
mresvanis:labels-only-in-suites
Open

fix(labels): declare check labels only in suites#526
mresvanis wants to merge 1 commit into
NVIDIA:mainfrom
mresvanis:labels-only-in-suites

Conversation

@mresvanis

@mresvanis mresvanis commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Remove per-check labels: from the Nico provider configs (control-plane.yaml, iam.yaml, network.yaml) so labels are declared only in isvctl/configs/suites/*.yaml; isvctl catalog labels --files now attributes every label to suite files, with the single-node local providers (k3s/microk8s/minikube) as the only allowed exception.
  • Change Nico's TenantInfoCheck test_id SDN01-02 to the "N/A" sentinel: SDN01-02 is owned by VpcCrudCheck-read in suites/network.yaml, and the provider-local attribution trips the plan-coverage domain guardrail once the network label is gone.
  • Document the rule in AGENTS.md so agents never reintroduce provider labels (top-level exclude.labels: filtering blocks remain fine).

Accepted behavior change: isvctl test run --provider nico --label <x> no longer discovers these standalone configs (they carry no labels and import no suite); direct -f <file> runs are unaffected.

Test plan

  • uv run isvctl catalog labels --files lists only suites/*.yaml (plus the exempted providers/{k3s,microk8s,minikube}.yaml) for every label
  • make test (all packages + scripts)
  • make demo-test
  • uvx pre-commit run -a

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation

    • Clarified how label-based filtering works, including where labels should be defined and when provider-level exclusions are allowed.
  • Bug Fixes

    • Cleaned up several validation configurations so checks use a more consistent label setup.
    • Kept existing validation behavior intact while simplifying how checks are classified.

Per-check `labels:` wiring belongs in isvctl/configs/suites/*.yaml; the
nico provider configs redeclared control_plane/iam/network/min_req labels
in their standalone wiring, so `isvctl catalog labels --files` attributed
labels to provider files.

- Drop `labels:` from nico/config/{control-plane,iam,network}.yaml. Also
  change nico's TenantInfoCheck test_id SDN01-02 to the "N/A" sentinel:
  SDN01-02 is owned by VpcCrudCheck-read in suites/network.yaml, and the
  provider-local attribution now trips the plan-coverage domain guardrail
  (SDN implies label 'network', which TenantInfoCheck no longer carries).
- Document the rule in AGENTS.md so agents never reintroduce provider
  labels. Exception: the single-node local providers
  (k3s/microk8s/minikube.yaml), which wire host-level Bm* checks that
  exist in no suite.

Accepted behavior change: `isvctl test run --provider nico --label <x>`
no longer discovers these standalone configs (they carry no labels and
import no suite); direct `-f <file>` runs are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Michail Resvanis <mresvanis@nvidia.com>
@mresvanis mresvanis requested a review from a team as a code owner July 3, 2026 09:25
@coderabbitai

coderabbitai Bot commented Jul 3, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f48a5dee-b11f-4b2a-bbf1-3734757e6954

📥 Commits

Reviewing files that changed from the base of the PR and between 33a60bd and 106c643.

📒 Files selected for processing (4)
  • AGENTS.md
  • isvctl/configs/providers/nico/config/control-plane.yaml
  • isvctl/configs/providers/nico/config/iam.yaml
  • isvctl/configs/providers/nico/config/network.yaml
 ______________________________
< Bugs bunny, at your service. >
 ------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

🔐 TruffleHog Secret Scan

No secrets or credentials found!

Your code has been scanned for 700+ types of secrets and credentials. All clear! 🎉

🔗 View scan details

🕐 Last updated: 2026-07-03 09:27:10 UTC | Commit: 106c643

@coderabbitai

coderabbitai Bot commented Jul 3, 2026

Copy link
Copy Markdown

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details
{}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant