feat(scripts): kyverno-demo cluster for policy UI testing - #1393
feat(scripts): kyverno-demo cluster for policy UI testing#1393nadaverell wants to merge 1 commit into
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| --set cleanupController.replicas=1 \ | ||
| --set features.policyExceptions.enabled=true \ | ||
| --set features.policyExceptions.namespace="" \ | ||
| --wait --timeout 10m >/dev/null |
There was a problem hiding this comment.
Helm targets wrong kube context
High Severity
All helm upgrade calls omit --kube-context, so Helm uses whatever the current kubecontext is, while kubectl is pinned to the kind demo cluster via k. On cluster reuse, openreports, or modern-only, that can install or mutate Kyverno on the wrong cluster — including removing legacy CRDs.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 184574c. Configure here.
| "kind": dyn("NetworkPolicy"), | ||
| "metadata": dyn({"name": dyn("default-deny"), "namespace": dyn(string(object.metadata.name))}), | ||
| "spec": dyn({"podSelector": dyn({}), "policyTypes": dyn(["Ingress"])}) | ||
| } |
There was a problem hiding this comment.
Wrong GeneratingPolicy Apply namespace
Medium Severity
generator.Apply takes the target namespace as its first argument, but the fixture passes the NetworkPolicy name "default-deny" instead of the triggering namespace. New namespaces would generate into a non-existent default-deny namespace rather than receiving the default-deny NetworkPolicy.
Reviewed by Cursor Bugbot for commit 184574c. Configure here.
Bootstraps a kind cluster with Kyverno 1.18.2 and curated policy fixtures,
following the gitops-demo pattern. Without it, anyone testing policy UI is
testing against whatever cluster they happen to be pointed at, and several
non-obvious states are not reproducible by guesswork.
The fixtures encode the states that are hard to get right rather than a
minimal smoke set:
- The four enforcement-posture cases. A policy declaring Deny with
admission evaluation disabled (blocks nothing), and one with NO
validationActions at all (which Kyverno treats as Deny, undocumented
upstream). Both are verifiable by hand and the README shows how.
- A MutatingPolicy with mutateExisting and admission off — the case that
used to render "Inactive" while rewriting the cluster. Note its default
is false, unlike admission/background, so it has to be set explicitly.
- Both API families installed together, since family selection, the
sidebar grouping and the legacy-vs-modern renderer split only have
meaning during migration. PolicyException exists in both, same Kind and
plural, different specs.
- Two DeletingPolicies on six-hourly and every-minute crons, so the table
shows "Never run" beside a recent timestamp within two minutes of
bootstrap rather than needing a hand-patched status.
- Real workloads, one passing and one failing, so the background scanner
produces ~40 reports across four distinct results[].source values —
which is the engine-taxonomy case made visible.
Two subcommands cover states that need a chart reconfiguration:
`openreports` leaves wgpolicyk8s.io served but empty while the data lands in
openreports.io (verified: 0 vs 37), and `modern-only` reproduces the 1.20 API
surface.
`modern-only` prints a warning before it runs, and the README leads with it:
Kyverno 1.18.2's admission controller crashloops without the legacy CRDs
because it sanity-checks for them at startup. That is upstream behaviour, and
anyone who hits it without warning will think they broke the cluster.
The webhook readiness gate is not incidental. `helm --wait` returns before the
admission webhook accepts connections, and applying a policy in that window
fails with "connection refused" — which is how the first run of this script
failed. The gate waits for endpoints and then round-trips a real policy
through a server-side dry-run, since endpoint readiness alone still races the
TLS cert.
184574c to
ff0835d
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 3 total unresolved issues (including 2 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ff0835d. Configure here.
| --version "${KYVERNO_CHART_VERSION}" -n kyverno --reuse-values \ | ||
| --set crds.groups.kyverno.clusterpolicies=false \ | ||
| --set crds.groups.kyverno.policies=false \ | ||
| --no-hooks --wait --timeout 8m >/dev/null 2>&1 || true |
There was a problem hiding this comment.
modern-only wait always times out
Medium Severity
modern-only runs helm upgrade with --wait --timeout 8m while intentionally putting the admission controller into a crashloop by removing the legacy CRDs it sanity-checks. Helm waits for every release Deployment to become ready, so this path always burns the full eight-minute timeout before || true continues.
Reviewed by Cursor Bugbot for commit ff0835d. Configure here.
|
Superseded by #1402, which carries the Kyverno demo plus the context-safety, scenario assertions, review fixes, and end-to-end verification. Keeping this branch intact for history. |
1 participant
Scripts the Kyverno fixture set from #1371 so it survives the cluster it was built on. Follows
scripts/gitops-demo.sh— same subcommand shape, same fixture-numbering convention, same status output.make kyverno-demo·make kyverno-demo-down·make kyverno-demo-statusWhy script this one
Several of #1371's most load-bearing verifications came from cluster states that aren't reproducible by guesswork — a policy that declares
Denyand blocks nothing, one with novalidationActionsat all, a report family that's served but empty. Without a script, testing policy UI means testing against whatever cluster you happen to be pointed at.It's also the shape RAD-322 needs: admission-denial attribution requires live admission and real policies, which is what this bootstraps.
What's in the fixtures
The set encodes the states that are hard to get right, not a minimal smoke test.
The four enforcement-posture cases, which are why Radar computes an effective posture instead of echoing
spec.validationActions:require-run-as-nonrootDenyaudit-require-labelsAudit + Warnbackground-only-no-latest-tagBackground onlyimplicit-deny-configmap-labelsDenyvalidationActionsat all — Kyverno treats this as Deny, undocumented upstreamBoth surprising ones are verifiable by hand, and the README shows the commands. Confirmed on a fresh cluster from this script:
Everything else worth a fixture: a
MutatingPolicywithmutateExistingand admission off (the case that used to render "Inactive" while rewriting the cluster — note its default isfalse, unlike admission/background); anImageValidatingPolicywith cosign keyless attestors and SBOM/SLSA attestations; aGeneratingPolicywhose evaluation block is a different shape again; and namespaced twins.Both API families, because family selection, the sidebar grouping and the legacy-vs-modern renderer split only have meaning during migration.
PolicyExceptionis the sharp case — same Kind, same plural, two groups, different specs — so there's one of each.Two DeletingPolicies on six-hourly and every-minute crons, so within two minutes of bootstrap the table shows
Never runbeside a recent timestamp. That pairing is what the Last Run column exists to surface, and it happens on its own rather than needing a hand-patched status:Real workloads, one passing and one failing, so the background scanner produces ~40 reports across four distinct
results[].sourcevalues — the engine-taxonomy case made visible:One engine, four producer strings.
Two scenario subcommands
openreportsreconfigures the chart so reports land inopenreports.iowhile thewgpolicyk8s.ioCRDs stay served but empty — the selection case where taking the first served group shows zero findings on a cluster full of them. Verified on this script's cluster:wgpolicyk8s.io: 0,openreports.io: 37.modern-onlyremoves the legacy CRDs to reproduce the Kyverno 1.20 API surface.The gotcha, stated up front
modern-onlyprints a warning before it runs and the README leads with it:That's upstream behaviour, not a broken cluster. The reports controller stays healthy and the reports survive, which is what the detection gate needs. Anyone who hits this without warning will think they broke something.
Testing
Ran
upon a clean machine and it failed — which is why it's worth saying.helm --waitreturns once the Deployments report ready, but the admission webhook isn't accepting connections yet, so the first policy apply died withfailed calling webhook "validate-policy.kyverno.svc": connection refused.Fixed with a readiness gate that waits for the Service endpoints and then round-trips a real policy through a server-side dry-run, because endpoint readiness alone still races the TLS cert. Fixture applies also retry, since the webhook can blip under load and a half-applied fixture set is worse than a slow one.
After the fix, verified end-to-end on a fresh cluster:
upfrom nothing → all four controllers healthy, all fixtures applied, 40 reportsupagain on the existing cluster → idempotent, webhook ready first attemptstatus→ full inventory including the source breakdown aboveopenreports→ 0 vs 37 as describedbash -nclean;helprenders; unknown subcommand exits 1Not tested end-to-end:
modern-only. It intentionally breaks the cluster and costs a full reset to recover, and its behaviour was already established empirically during #1371 — that's where the crashloop and the surviving-reports observations come from. Flagging rather than implying full coverage.Note
Low Risk
Changes are limited to local dev scripts, fixtures, and documentation; no application runtime or auth paths are modified.
Overview
Adds a Kyverno demo bootstrap (same shape as
gitops-demo) so policy UI work can run against a predictable kind cluster instead of whatever kube context is active.scripts/kyverno-demo.shprovisions Kyverno 1.18.2 (pinned chart), applies numbered fixtures underscripts/kyverno-demo/, waits for PolicyReports, and prints inventory viastatus. It includes a webhook readiness gate (endpoints + server-side dry-run policy) and retried fixture applies soupdoes not fail when Helm reports ready before admission is actually accepting traffic.Makefile exposes
kyverno-demo,kyverno-demo-down, andkyverno-demo-status. The fixture set targets Radar policy rendering: the four effective enforcement-posture cases, modern policy kinds (image/mutate/generate/delete + namespaced twins), legacy + modern APIs including dual-shape PolicyException, workloads that seed ~40 reports, plus scenario togglesopenreports(served-but-emptywgpolicyk8s.iovs populatedopenreports.io) andmodern-only(legacy CRD removal / 1.20 surface simulation, with documented admission-controller crashloop caveat).scripts/kyverno-demo/README.mddocuments the coverage matrix and hand-verification commands.Reviewed by Cursor Bugbot for commit ff0835d. Bugbot is set up for automated code reviews on this repo. Configure here.