docs: k8s no-clone OCI parity + consistent one---set-per-axis#28
Merged
Conversation
The compose mode is presented as pull-from-oci (no clone); the k8s section still showed only the local `benchmarks/_chart` path — even though making the chart self-contained (it bundles each benchmark's preset) is exactly what lets it be pulled from a registry. Lead the k8s section with the published `oci://.../charts/eval` form (no clone), and keep the local-chart render as the clone/CLI path. Accurate to current behavior: the CLI's job mode still renders the local chart (unlike compose, which already defaults to oci).
helm's `--set a=x,y=z` splits on commas, so a value containing a comma breaks; the CLI deliberately emits one `--set` per axis for that reason. Make the docs match — including the example that purported to show the CLI's own output.
elronbandel
changed the title
--set-per-axis
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
docs: k8s no-clone OCI parity + consistent one-`--set`-per-axis
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
docs: k8s no-clone OCI parity + consistent one-`--set`-per-axis
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
docs: k8s no-clone OCI parity + consistent one-`--set`-per-axis
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Two doc consistency fixes around the k8s/
jobsurface:oci://quay.io/eval-containers/evaluate, but the k8s section still showed onlyhelm template benchmarks/_chart …, a local path requiring a clone. Making the chart self-contained (refactor(chart): bundle benchmark presets in the chart (self-contained, OCI-ready) #26) is what lets it be pulled from a registry; the docs never got that parity.--setper axis. Several examples used--set agent=claude-code,task=0. helm's--set a=x,y=zsplits on commas, so any value containing a comma breaks — which is exactly why the CLI deliberately emits one--setper axis. One example even claimed to show the CLI's output in the comma-joined form the CLI never produces.What
helm template aime oci://quay.io/eval-containers/charts/eval --version 0.1.0 --set benchmark=aime …(no-clone, under the existing Pre-release caveat); local./benchmarks/_chartkept as the clone/CLI path — accurate, since the CLI's--mode jobstill renders the local chart.--set→ separate--setper axis, across README,docs/guides/deploy-on-kubernetes.md,docs/concepts/the-helm-chart.md, and thedeploy/openshift-service-account.yamlcomment.Docs-only.
Follow-ups this surfaced
helm package/pushtooci://…/charts) — makes theoci://form resolve.run --mode jobis still repo-local (run_jobhardcodes./benchmarks/_chart;run_composealready defaults tooci://). Tracked separately.