ci(release): publish the crate to crates.io from release.yml (rule 2)#109
Merged
Conversation
release.yml built the CLI binaries but never published the crate (0.1.0 was manual). Adds a dist custom publish-job (publish-crate.yml) wired via publish-jobs; regenerates release.yml so a vX.Y.Z tag publishes the crate as part of the CLI release flow. The job guards tag==Cargo==Chart before publishing (doctrine/delivery/RULES.md:6). Requires a CARGO_REGISTRY_TOKEN repo secret. doctrine/delivery/RULES.md:2 (crates.io publish produced by release.yml). Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 11, 2026
release-images.yml (#95) and publish-crate.yml (#109) arrived via the rebase referencing the pre-reorg paths, and only run on v* tags — so PR CI never exercises them and the breakage would surface only at the next release: - version guard read `^version = ` from the root Cargo.toml, which is now the version-less workspace manifest → read cli/Cargo.toml instead - chart version + `helm package` read benchmarks/_chart → containers/benchmarks/_chart - the fleet bake glob (core/*/… agents/*/… …/docker-bake.hcl) → containers/… - `cargo run` / `cargo publish` at the virtual workspace root need a package selector → `-p eval-containers` Verified: the bake glob resolves 147 files; `cargo run -p eval-containers` runs. Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 14, 2026
release-images.yml (#95) and publish-crate.yml (#109) arrived via the rebase referencing the pre-reorg paths, and only run on v* tags — so PR CI never exercises them and the breakage would surface only at the next release: - version guard read `^version = ` from the root Cargo.toml, which is now the version-less workspace manifest → read cli/Cargo.toml instead - chart version + `helm package` read benchmarks/_chart → containers/benchmarks/_chart - the fleet bake glob (core/*/… agents/*/… …/docker-bake.hcl) → containers/… - `cargo run` / `cargo publish` at the virtual workspace root need a package selector → `-p eval-containers` Verified: the bake glob resolves 147 files; `cargo run -p eval-containers` runs. Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 14, 2026
…I (cli/) (#112) * refactor(layout): split the container fleet (containers/) from the CLI (cli/) Group the product and the tooling into top-level homes that match the manifesto. The container fleet — benchmarks, agents, models, core, gateways, compose, and the bake graph — moves under `containers/`. The Rust CLI becomes its own cargo-workspace member under `cli/`, and the fleet's integration-test suite becomes a sibling crate (`tests/`, depending on `cli`) so the quality process is first-class rather than buried inside the optional CLI. Platform- specific deployment (the `oc/` scripts) moves under `deploy/oc/`. The key invariant: only *filesystem* paths gain the `containers/` prefix. Registry image namespaces (`ghcr.io/exgentic/benchmarks/…`), bake target names, and OpenShift imagestream names are deliberately unchanged — the published-image contract is preserved. Layout: - containers/ agents benchmarks(+_chart) models core gateways compose, docker-bake.hcl, scripts - cli/ the `eval-containers` crate (src, Cargo.toml, README, LICENSE) - tests/ fleet integration tests as the `eval-containers-tests` workspace crate - deploy/oc/ OpenShift sweep scripts (REPO_DIR now resolves two levels up) Mechanics: - cargo workspace: virtual root Cargo.toml; cli + tests members; [profile.dist] at root - 147 docker-bake.hcl: build `context` → `containers/…`; `contexts{}` keys + `tags` unchanged - cli/src: filesystem reads (build.sh, compose.yaml, _chart, services.yaml, Dockerfile) → containers/ - tests resolve fleet files from the repo root (CARGO_MANIFEST_DIR/..), since cargo runs each crate's tests with the crate dir as cwd, not the repo root - CI (pre-commit hooks, oc-connectivity), README, and the oc scripts updated for the new paths Verified locally: - cargo build + clippy -D warnings + unit tests (cli crate) - daemon-free gate: check, dockerfile_inspection, task_inspection, cli_conformance - helm render gate: every benchmark renders through containers/benchmarks/_chart + kubeconform - docker buildx bake --print resolves agent-codex and the eval combination - docker compose config resolves the relative include to containers/compose/services.yaml Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * chore(layout): tidy the project root Follow-on cleanup so the root holds only source dirs + standard entry docs and tooling files: - Drop the root .dockerignore. Its only consumer was `run --mode container --local`, which did `docker build … .` (whole-repo context) for a FROM-only container.Dockerfile that reads nothing from the context. Scope that build's context to the benchmark's own dir instead — small and stable, no ignore file. - examples/ → deploy/examples/ (the OpenShift example sits with deploy/oc). - AUDIT.md → containers/AUDIT.md — the generated fleet rollup lives with the fleet; the audit rules + rollup skill are reworded from "repository root". - Delete RELEASE.md — a delegating stub since #104; its references now point at doctrine/delivery/ (the canonical release policy). README, LICENSE, CHANGELOG.md, AGENTS.md, CONTRIBUTING.md stay at root. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * refactor(layout): rename doctrine/ -> .agents/ Move the governance tree (rules + skills) from the visible doctrine/ into a hidden .agents/, pairing it with the root AGENTS.md entry and dropping it from the visible root listing. Path references are updated repo-wide (352 occurrences across 55 files); the prose concept word "doctrine" stays — the doctrine simply lives in .agents/ now, as docs live in docs/. Nothing in code or tests reads the path, so this is non-functional (build + clippy + the daemon-free gate stay green). Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(ci): repoint the tag-time release workflows to the new layout release-images.yml (#95) and publish-crate.yml (#109) arrived via the rebase referencing the pre-reorg paths, and only run on v* tags — so PR CI never exercises them and the breakage would surface only at the next release: - version guard read `^version = ` from the root Cargo.toml, which is now the version-less workspace manifest → read cli/Cargo.toml instead - chart version + `helm package` read benchmarks/_chart → containers/benchmarks/_chart - the fleet bake glob (core/*/… agents/*/… …/docker-bake.hcl) → containers/… - `cargo run` / `cargo publish` at the virtual workspace root need a package selector → `-p eval-containers` Verified: the bake glob resolves 147 files; `cargo run -p eval-containers` runs. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(tests): repoint docker-suite test paths to the new layout The scan (reorg-audit workflow) found real regressions the daemon-free PR gate never exercises — the container/cluster test suites read fleet files by bare, cwd-relative pre-reorg paths: - oracle/live/fleet/upstream: read_dir("benchmarks"|"agents") -> repo_root().join("containers/…") - gateways: read_to_string("gateways/portkey/…"), the per-flavor Dockerfile -> containers/gateways/… - build: tc_build context Path::new("models/replay") -> containers/models/replay - sanity/compose: read_dir("benchmarks") + the base compose/ templates -> containers/… - replay: 242 "benchmarks/<x>/compose.yaml" -> containers/…, and enter_repo_root() at the top of replay_compose (it captures cwd before building) - deploy/examples/openshift/build-eval.sh: `cd ../..` -> `../../..` (the script moved a level deeper, so the repo-root cd was one short) All resolved via the tests crate's repo_root() helper, so they work regardless of cwd. Verified: whole workspace compiles; daemon-free gate still green. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * docs: repoint remaining doc/template/comment paths to the new layout Follow-up from the reorg-audit scan (+ a deterministic grep pass that caught files the scan's sampling missed). Filesystem-path references updated to the post-reorg locations; registry namespaces, bake targets, imagestream names, and the prose concept-word "doctrine" left unchanged: - docs/ (concepts, guides, reference): benchmarks/_chart, benchmarks/<x>/compose.yaml, build contexts (benchmarks/aime, core/test-exact-match, benchmarks/gsm8k), src/, and the Oracle / chart-values / litellm links -> containers/ , cli/src , .agents/ - .github issue + PR templates: ../../<topic>/RULES.md doctrine links -> ../../.agents/... - .agents/delivery/build/SKILL.md: bake-graph + gen-bake examples -> containers/ - containers/core/oracle/README.md, deploy/oc/README.md, tests/{LOCAL.md, cli/RULES.md, containers/RULES.md} - cli/src/run.rs: doc-comment filesystem paths -> containers/ (doctrine shorthand + all code left untouched) Verified: workspace builds; deterministic grep finds no standalone stale fleet paths or dangling links; no double-prefix; registry refs intact. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * style: rustfmt the scan-fix test edits The perl-applied repo_root().join("containers/…") test-path fixes overran the line width; cargo fmt wraps them. Behavior-neutral; pre-commit fmt hook now passes. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(tests): adopt the rebased upstream_pins lint into the workspace The rebase onto current main pulled in #110's upstream_pins lint (and main's skills-bench shared-base rework). Adapt it to the new layout: - register it in tests/Cargo.toml (the version target lives in the tests crate now) - anchor it: it scans via bake::artifact_dirs_with_dockerfile (which reads containers/<category> relative to cwd), so enter_repo_root() first — otherwise the sweep silently finds zero Dockerfiles and false-passes - allowlist skills-bench's `skills-bench-base:latest` — a locally-built shared base (built once, reused across 86 tasks), like mle-bench's mlebench-env:latest. This was already failing the (ungated) lint on main; the test's own mechanism is the fix. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> --------- Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
release-images.yml (#95) and publish-crate.yml (#109) arrived via the rebase referencing the pre-reorg paths, and only run on v* tags — so PR CI never exercises them and the breakage would surface only at the next release: - version guard read `^version = ` from the root Cargo.toml, which is now the version-less workspace manifest → read cli/Cargo.toml instead - chart version + `helm package` read benchmarks/_chart → containers/benchmarks/_chart - the fleet bake glob (core/*/… agents/*/… …/docker-bake.hcl) → containers/… - `cargo run` / `cargo publish` at the virtual workspace root need a package selector → `-p eval-containers` Verified: the bake glob resolves 147 files; `cargo run -p eval-containers` runs. Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
…#109) release.yml built the CLI binaries but never published the crate (0.1.0 was manual). Adds a dist custom publish-job (publish-crate.yml) wired via publish-jobs; regenerates release.yml so a vX.Y.Z tag publishes the crate as part of the CLI release flow. The job guards tag==Cargo==Chart before publishing (doctrine/delivery/RULES.md:6). Requires a CARGO_REGISTRY_TOKEN repo secret. doctrine/delivery/RULES.md:2 (crates.io publish produced by release.yml). Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
…I (cli/) (#112) * refactor(layout): split the container fleet (containers/) from the CLI (cli/) Group the product and the tooling into top-level homes that match the manifesto. The container fleet — benchmarks, agents, models, core, gateways, compose, and the bake graph — moves under `containers/`. The Rust CLI becomes its own cargo-workspace member under `cli/`, and the fleet's integration-test suite becomes a sibling crate (`tests/`, depending on `cli`) so the quality process is first-class rather than buried inside the optional CLI. Platform- specific deployment (the `oc/` scripts) moves under `deploy/oc/`. The key invariant: only *filesystem* paths gain the `containers/` prefix. Registry image namespaces (`ghcr.io/exgentic/benchmarks/…`), bake target names, and OpenShift imagestream names are deliberately unchanged — the published-image contract is preserved. Layout: - containers/ agents benchmarks(+_chart) models core gateways compose, docker-bake.hcl, scripts - cli/ the `eval-containers` crate (src, Cargo.toml, README, LICENSE) - tests/ fleet integration tests as the `eval-containers-tests` workspace crate - deploy/oc/ OpenShift sweep scripts (REPO_DIR now resolves two levels up) Mechanics: - cargo workspace: virtual root Cargo.toml; cli + tests members; [profile.dist] at root - 147 docker-bake.hcl: build `context` → `containers/…`; `contexts{}` keys + `tags` unchanged - cli/src: filesystem reads (build.sh, compose.yaml, _chart, services.yaml, Dockerfile) → containers/ - tests resolve fleet files from the repo root (CARGO_MANIFEST_DIR/..), since cargo runs each crate's tests with the crate dir as cwd, not the repo root - CI (pre-commit hooks, oc-connectivity), README, and the oc scripts updated for the new paths Verified locally: - cargo build + clippy -D warnings + unit tests (cli crate) - daemon-free gate: check, dockerfile_inspection, task_inspection, cli_conformance - helm render gate: every benchmark renders through containers/benchmarks/_chart + kubeconform - docker buildx bake --print resolves agent-codex and the eval combination - docker compose config resolves the relative include to containers/compose/services.yaml Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * chore(layout): tidy the project root Follow-on cleanup so the root holds only source dirs + standard entry docs and tooling files: - Drop the root .dockerignore. Its only consumer was `run --mode container --local`, which did `docker build … .` (whole-repo context) for a FROM-only container.Dockerfile that reads nothing from the context. Scope that build's context to the benchmark's own dir instead — small and stable, no ignore file. - examples/ → deploy/examples/ (the OpenShift example sits with deploy/oc). - AUDIT.md → containers/AUDIT.md — the generated fleet rollup lives with the fleet; the audit rules + rollup skill are reworded from "repository root". - Delete RELEASE.md — a delegating stub since #104; its references now point at doctrine/delivery/ (the canonical release policy). README, LICENSE, CHANGELOG.md, AGENTS.md, CONTRIBUTING.md stay at root. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * refactor(layout): rename doctrine/ -> .agents/ Move the governance tree (rules + skills) from the visible doctrine/ into a hidden .agents/, pairing it with the root AGENTS.md entry and dropping it from the visible root listing. Path references are updated repo-wide (352 occurrences across 55 files); the prose concept word "doctrine" stays — the doctrine simply lives in .agents/ now, as docs live in docs/. Nothing in code or tests reads the path, so this is non-functional (build + clippy + the daemon-free gate stay green). Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(ci): repoint the tag-time release workflows to the new layout release-images.yml (#95) and publish-crate.yml (#109) arrived via the rebase referencing the pre-reorg paths, and only run on v* tags — so PR CI never exercises them and the breakage would surface only at the next release: - version guard read `^version = ` from the root Cargo.toml, which is now the version-less workspace manifest → read cli/Cargo.toml instead - chart version + `helm package` read benchmarks/_chart → containers/benchmarks/_chart - the fleet bake glob (core/*/… agents/*/… …/docker-bake.hcl) → containers/… - `cargo run` / `cargo publish` at the virtual workspace root need a package selector → `-p eval-containers` Verified: the bake glob resolves 147 files; `cargo run -p eval-containers` runs. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(tests): repoint docker-suite test paths to the new layout The scan (reorg-audit workflow) found real regressions the daemon-free PR gate never exercises — the container/cluster test suites read fleet files by bare, cwd-relative pre-reorg paths: - oracle/live/fleet/upstream: read_dir("benchmarks"|"agents") -> repo_root().join("containers/…") - gateways: read_to_string("gateways/portkey/…"), the per-flavor Dockerfile -> containers/gateways/… - build: tc_build context Path::new("models/replay") -> containers/models/replay - sanity/compose: read_dir("benchmarks") + the base compose/ templates -> containers/… - replay: 242 "benchmarks/<x>/compose.yaml" -> containers/…, and enter_repo_root() at the top of replay_compose (it captures cwd before building) - deploy/examples/openshift/build-eval.sh: `cd ../..` -> `../../..` (the script moved a level deeper, so the repo-root cd was one short) All resolved via the tests crate's repo_root() helper, so they work regardless of cwd. Verified: whole workspace compiles; daemon-free gate still green. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * docs: repoint remaining doc/template/comment paths to the new layout Follow-up from the reorg-audit scan (+ a deterministic grep pass that caught files the scan's sampling missed). Filesystem-path references updated to the post-reorg locations; registry namespaces, bake targets, imagestream names, and the prose concept-word "doctrine" left unchanged: - docs/ (concepts, guides, reference): benchmarks/_chart, benchmarks/<x>/compose.yaml, build contexts (benchmarks/aime, core/test-exact-match, benchmarks/gsm8k), src/, and the Oracle / chart-values / litellm links -> containers/ , cli/src , .agents/ - .github issue + PR templates: ../../<topic>/RULES.md doctrine links -> ../../.agents/... - .agents/delivery/build/SKILL.md: bake-graph + gen-bake examples -> containers/ - containers/core/oracle/README.md, deploy/oc/README.md, tests/{LOCAL.md, cli/RULES.md, containers/RULES.md} - cli/src/run.rs: doc-comment filesystem paths -> containers/ (doctrine shorthand + all code left untouched) Verified: workspace builds; deterministic grep finds no standalone stale fleet paths or dangling links; no double-prefix; registry refs intact. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * style: rustfmt the scan-fix test edits The perl-applied repo_root().join("containers/…") test-path fixes overran the line width; cargo fmt wraps them. Behavior-neutral; pre-commit fmt hook now passes. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(tests): adopt the rebased upstream_pins lint into the workspace The rebase onto current main pulled in #110's upstream_pins lint (and main's skills-bench shared-base rework). Adapt it to the new layout: - register it in tests/Cargo.toml (the version target lives in the tests crate now) - anchor it: it scans via bake::artifact_dirs_with_dockerfile (which reads containers/<category> relative to cwd), so enter_repo_root() first — otherwise the sweep silently finds zero Dockerfiles and false-passes - allowlist skills-bench's `skills-bench-base:latest` — a locally-built shared base (built once, reused across 86 tasks), like mle-bench's mlebench-env:latest. This was already failing the (ungated) lint on main; the test's own mechanism is the fix. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> --------- Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
release-images.yml (#95) and publish-crate.yml (#109) arrived via the rebase referencing the pre-reorg paths, and only run on v* tags — so PR CI never exercises them and the breakage would surface only at the next release: - version guard read `^version = ` from the root Cargo.toml, which is now the version-less workspace manifest → read cli/Cargo.toml instead - chart version + `helm package` read benchmarks/_chart → containers/benchmarks/_chart - the fleet bake glob (core/*/… agents/*/… …/docker-bake.hcl) → containers/… - `cargo run` / `cargo publish` at the virtual workspace root need a package selector → `-p eval-containers` Verified: the bake glob resolves 147 files; `cargo run -p eval-containers` runs. Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
…#109) release.yml built the CLI binaries but never published the crate (0.1.0 was manual). Adds a dist custom publish-job (publish-crate.yml) wired via publish-jobs; regenerates release.yml so a vX.Y.Z tag publishes the crate as part of the CLI release flow. The job guards tag==Cargo==Chart before publishing (doctrine/delivery/RULES.md:6). Requires a CARGO_REGISTRY_TOKEN repo secret. doctrine/delivery/RULES.md:2 (crates.io publish produced by release.yml). Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
…I (cli/) (#112) * refactor(layout): split the container fleet (containers/) from the CLI (cli/) Group the product and the tooling into top-level homes that match the manifesto. The container fleet — benchmarks, agents, models, core, gateways, compose, and the bake graph — moves under `containers/`. The Rust CLI becomes its own cargo-workspace member under `cli/`, and the fleet's integration-test suite becomes a sibling crate (`tests/`, depending on `cli`) so the quality process is first-class rather than buried inside the optional CLI. Platform- specific deployment (the `oc/` scripts) moves under `deploy/oc/`. The key invariant: only *filesystem* paths gain the `containers/` prefix. Registry image namespaces (`ghcr.io/exgentic/benchmarks/…`), bake target names, and OpenShift imagestream names are deliberately unchanged — the published-image contract is preserved. Layout: - containers/ agents benchmarks(+_chart) models core gateways compose, docker-bake.hcl, scripts - cli/ the `eval-containers` crate (src, Cargo.toml, README, LICENSE) - tests/ fleet integration tests as the `eval-containers-tests` workspace crate - deploy/oc/ OpenShift sweep scripts (REPO_DIR now resolves two levels up) Mechanics: - cargo workspace: virtual root Cargo.toml; cli + tests members; [profile.dist] at root - 147 docker-bake.hcl: build `context` → `containers/…`; `contexts{}` keys + `tags` unchanged - cli/src: filesystem reads (build.sh, compose.yaml, _chart, services.yaml, Dockerfile) → containers/ - tests resolve fleet files from the repo root (CARGO_MANIFEST_DIR/..), since cargo runs each crate's tests with the crate dir as cwd, not the repo root - CI (pre-commit hooks, oc-connectivity), README, and the oc scripts updated for the new paths Verified locally: - cargo build + clippy -D warnings + unit tests (cli crate) - daemon-free gate: check, dockerfile_inspection, task_inspection, cli_conformance - helm render gate: every benchmark renders through containers/benchmarks/_chart + kubeconform - docker buildx bake --print resolves agent-codex and the eval combination - docker compose config resolves the relative include to containers/compose/services.yaml Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * chore(layout): tidy the project root Follow-on cleanup so the root holds only source dirs + standard entry docs and tooling files: - Drop the root .dockerignore. Its only consumer was `run --mode container --local`, which did `docker build … .` (whole-repo context) for a FROM-only container.Dockerfile that reads nothing from the context. Scope that build's context to the benchmark's own dir instead — small and stable, no ignore file. - examples/ → deploy/examples/ (the OpenShift example sits with deploy/oc). - AUDIT.md → containers/AUDIT.md — the generated fleet rollup lives with the fleet; the audit rules + rollup skill are reworded from "repository root". - Delete RELEASE.md — a delegating stub since #104; its references now point at doctrine/delivery/ (the canonical release policy). README, LICENSE, CHANGELOG.md, AGENTS.md, CONTRIBUTING.md stay at root. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * refactor(layout): rename doctrine/ -> .agents/ Move the governance tree (rules + skills) from the visible doctrine/ into a hidden .agents/, pairing it with the root AGENTS.md entry and dropping it from the visible root listing. Path references are updated repo-wide (352 occurrences across 55 files); the prose concept word "doctrine" stays — the doctrine simply lives in .agents/ now, as docs live in docs/. Nothing in code or tests reads the path, so this is non-functional (build + clippy + the daemon-free gate stay green). Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(ci): repoint the tag-time release workflows to the new layout release-images.yml (#95) and publish-crate.yml (#109) arrived via the rebase referencing the pre-reorg paths, and only run on v* tags — so PR CI never exercises them and the breakage would surface only at the next release: - version guard read `^version = ` from the root Cargo.toml, which is now the version-less workspace manifest → read cli/Cargo.toml instead - chart version + `helm package` read benchmarks/_chart → containers/benchmarks/_chart - the fleet bake glob (core/*/… agents/*/… …/docker-bake.hcl) → containers/… - `cargo run` / `cargo publish` at the virtual workspace root need a package selector → `-p eval-containers` Verified: the bake glob resolves 147 files; `cargo run -p eval-containers` runs. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(tests): repoint docker-suite test paths to the new layout The scan (reorg-audit workflow) found real regressions the daemon-free PR gate never exercises — the container/cluster test suites read fleet files by bare, cwd-relative pre-reorg paths: - oracle/live/fleet/upstream: read_dir("benchmarks"|"agents") -> repo_root().join("containers/…") - gateways: read_to_string("gateways/portkey/…"), the per-flavor Dockerfile -> containers/gateways/… - build: tc_build context Path::new("models/replay") -> containers/models/replay - sanity/compose: read_dir("benchmarks") + the base compose/ templates -> containers/… - replay: 242 "benchmarks/<x>/compose.yaml" -> containers/…, and enter_repo_root() at the top of replay_compose (it captures cwd before building) - deploy/examples/openshift/build-eval.sh: `cd ../..` -> `../../..` (the script moved a level deeper, so the repo-root cd was one short) All resolved via the tests crate's repo_root() helper, so they work regardless of cwd. Verified: whole workspace compiles; daemon-free gate still green. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * docs: repoint remaining doc/template/comment paths to the new layout Follow-up from the reorg-audit scan (+ a deterministic grep pass that caught files the scan's sampling missed). Filesystem-path references updated to the post-reorg locations; registry namespaces, bake targets, imagestream names, and the prose concept-word "doctrine" left unchanged: - docs/ (concepts, guides, reference): benchmarks/_chart, benchmarks/<x>/compose.yaml, build contexts (benchmarks/aime, core/test-exact-match, benchmarks/gsm8k), src/, and the Oracle / chart-values / litellm links -> containers/ , cli/src , .agents/ - .github issue + PR templates: ../../<topic>/RULES.md doctrine links -> ../../.agents/... - .agents/delivery/build/SKILL.md: bake-graph + gen-bake examples -> containers/ - containers/core/oracle/README.md, deploy/oc/README.md, tests/{LOCAL.md, cli/RULES.md, containers/RULES.md} - cli/src/run.rs: doc-comment filesystem paths -> containers/ (doctrine shorthand + all code left untouched) Verified: workspace builds; deterministic grep finds no standalone stale fleet paths or dangling links; no double-prefix; registry refs intact. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * style: rustfmt the scan-fix test edits The perl-applied repo_root().join("containers/…") test-path fixes overran the line width; cargo fmt wraps them. Behavior-neutral; pre-commit fmt hook now passes. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(tests): adopt the rebased upstream_pins lint into the workspace The rebase onto current main pulled in #110's upstream_pins lint (and main's skills-bench shared-base rework). Adapt it to the new layout: - register it in tests/Cargo.toml (the version target lives in the tests crate now) - anchor it: it scans via bake::artifact_dirs_with_dockerfile (which reads containers/<category> relative to cwd), so enter_repo_root() first — otherwise the sweep silently finds zero Dockerfiles and false-passes - allowlist skills-bench's `skills-bench-base:latest` — a locally-built shared base (built once, reused across 86 tasks), like mle-bench's mlebench-env:latest. This was already failing the (ungated) lint on main; the test's own mechanism is the fix. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> --------- Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
…#109) release.yml built the CLI binaries but never published the crate (0.1.0 was manual). Adds a dist custom publish-job (publish-crate.yml) wired via publish-jobs; regenerates release.yml so a vX.Y.Z tag publishes the crate as part of the CLI release flow. The job guards tag==Cargo==Chart before publishing (doctrine/delivery/RULES.md:6). Requires a CARGO_REGISTRY_TOKEN repo secret. doctrine/delivery/RULES.md:2 (crates.io publish produced by release.yml). Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
…I (cli/) (#112) * refactor(layout): split the container fleet (containers/) from the CLI (cli/) Group the product and the tooling into top-level homes that match the manifesto. The container fleet — benchmarks, agents, models, core, gateways, compose, and the bake graph — moves under `containers/`. The Rust CLI becomes its own cargo-workspace member under `cli/`, and the fleet's integration-test suite becomes a sibling crate (`tests/`, depending on `cli`) so the quality process is first-class rather than buried inside the optional CLI. Platform- specific deployment (the `oc/` scripts) moves under `deploy/oc/`. The key invariant: only *filesystem* paths gain the `containers/` prefix. Registry image namespaces (`ghcr.io/exgentic/benchmarks/…`), bake target names, and OpenShift imagestream names are deliberately unchanged — the published-image contract is preserved. Layout: - containers/ agents benchmarks(+_chart) models core gateways compose, docker-bake.hcl, scripts - cli/ the `eval-containers` crate (src, Cargo.toml, README, LICENSE) - tests/ fleet integration tests as the `eval-containers-tests` workspace crate - deploy/oc/ OpenShift sweep scripts (REPO_DIR now resolves two levels up) Mechanics: - cargo workspace: virtual root Cargo.toml; cli + tests members; [profile.dist] at root - 147 docker-bake.hcl: build `context` → `containers/…`; `contexts{}` keys + `tags` unchanged - cli/src: filesystem reads (build.sh, compose.yaml, _chart, services.yaml, Dockerfile) → containers/ - tests resolve fleet files from the repo root (CARGO_MANIFEST_DIR/..), since cargo runs each crate's tests with the crate dir as cwd, not the repo root - CI (pre-commit hooks, oc-connectivity), README, and the oc scripts updated for the new paths Verified locally: - cargo build + clippy -D warnings + unit tests (cli crate) - daemon-free gate: check, dockerfile_inspection, task_inspection, cli_conformance - helm render gate: every benchmark renders through containers/benchmarks/_chart + kubeconform - docker buildx bake --print resolves agent-codex and the eval combination - docker compose config resolves the relative include to containers/compose/services.yaml Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * chore(layout): tidy the project root Follow-on cleanup so the root holds only source dirs + standard entry docs and tooling files: - Drop the root .dockerignore. Its only consumer was `run --mode container --local`, which did `docker build … .` (whole-repo context) for a FROM-only container.Dockerfile that reads nothing from the context. Scope that build's context to the benchmark's own dir instead — small and stable, no ignore file. - examples/ → deploy/examples/ (the OpenShift example sits with deploy/oc). - AUDIT.md → containers/AUDIT.md — the generated fleet rollup lives with the fleet; the audit rules + rollup skill are reworded from "repository root". - Delete RELEASE.md — a delegating stub since #104; its references now point at doctrine/delivery/ (the canonical release policy). README, LICENSE, CHANGELOG.md, AGENTS.md, CONTRIBUTING.md stay at root. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * refactor(layout): rename doctrine/ -> .agents/ Move the governance tree (rules + skills) from the visible doctrine/ into a hidden .agents/, pairing it with the root AGENTS.md entry and dropping it from the visible root listing. Path references are updated repo-wide (352 occurrences across 55 files); the prose concept word "doctrine" stays — the doctrine simply lives in .agents/ now, as docs live in docs/. Nothing in code or tests reads the path, so this is non-functional (build + clippy + the daemon-free gate stay green). Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(ci): repoint the tag-time release workflows to the new layout release-images.yml (#95) and publish-crate.yml (#109) arrived via the rebase referencing the pre-reorg paths, and only run on v* tags — so PR CI never exercises them and the breakage would surface only at the next release: - version guard read `^version = ` from the root Cargo.toml, which is now the version-less workspace manifest → read cli/Cargo.toml instead - chart version + `helm package` read benchmarks/_chart → containers/benchmarks/_chart - the fleet bake glob (core/*/… agents/*/… …/docker-bake.hcl) → containers/… - `cargo run` / `cargo publish` at the virtual workspace root need a package selector → `-p eval-containers` Verified: the bake glob resolves 147 files; `cargo run -p eval-containers` runs. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(tests): repoint docker-suite test paths to the new layout The scan (reorg-audit workflow) found real regressions the daemon-free PR gate never exercises — the container/cluster test suites read fleet files by bare, cwd-relative pre-reorg paths: - oracle/live/fleet/upstream: read_dir("benchmarks"|"agents") -> repo_root().join("containers/…") - gateways: read_to_string("gateways/portkey/…"), the per-flavor Dockerfile -> containers/gateways/… - build: tc_build context Path::new("models/replay") -> containers/models/replay - sanity/compose: read_dir("benchmarks") + the base compose/ templates -> containers/… - replay: 242 "benchmarks/<x>/compose.yaml" -> containers/…, and enter_repo_root() at the top of replay_compose (it captures cwd before building) - deploy/examples/openshift/build-eval.sh: `cd ../..` -> `../../..` (the script moved a level deeper, so the repo-root cd was one short) All resolved via the tests crate's repo_root() helper, so they work regardless of cwd. Verified: whole workspace compiles; daemon-free gate still green. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * docs: repoint remaining doc/template/comment paths to the new layout Follow-up from the reorg-audit scan (+ a deterministic grep pass that caught files the scan's sampling missed). Filesystem-path references updated to the post-reorg locations; registry namespaces, bake targets, imagestream names, and the prose concept-word "doctrine" left unchanged: - docs/ (concepts, guides, reference): benchmarks/_chart, benchmarks/<x>/compose.yaml, build contexts (benchmarks/aime, core/test-exact-match, benchmarks/gsm8k), src/, and the Oracle / chart-values / litellm links -> containers/ , cli/src , .agents/ - .github issue + PR templates: ../../<topic>/RULES.md doctrine links -> ../../.agents/... - .agents/delivery/build/SKILL.md: bake-graph + gen-bake examples -> containers/ - containers/core/oracle/README.md, deploy/oc/README.md, tests/{LOCAL.md, cli/RULES.md, containers/RULES.md} - cli/src/run.rs: doc-comment filesystem paths -> containers/ (doctrine shorthand + all code left untouched) Verified: workspace builds; deterministic grep finds no standalone stale fleet paths or dangling links; no double-prefix; registry refs intact. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * style: rustfmt the scan-fix test edits The perl-applied repo_root().join("containers/…") test-path fixes overran the line width; cargo fmt wraps them. Behavior-neutral; pre-commit fmt hook now passes. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> * fix(tests): adopt the rebased upstream_pins lint into the workspace The rebase onto current main pulled in #110's upstream_pins lint (and main's skills-bench shared-base rework). Adapt it to the new layout: - register it in tests/Cargo.toml (the version target lives in the tests crate now) - anchor it: it scans via bake::artifact_dirs_with_dockerfile (which reads containers/<category> relative to cwd), so enter_repo_root() first — otherwise the sweep silently finds zero Dockerfiles and false-passes - allowlist skills-bench's `skills-bench-base:latest` — a locally-built shared base (built once, reused across 86 tasks), like mle-bench's mlebench-env:latest. This was already failing the (ungated) lint on main; the test's own mechanism is the fix. Signed-off-by: Elron Bandel <elron.bandel@ibm.com> --------- Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
added a commit
that referenced
this pull request
Jun 15, 2026
release-images.yml (#95) and publish-crate.yml (#109) arrived via the rebase referencing the pre-reorg paths, and only run on v* tags — so PR CI never exercises them and the breakage would surface only at the next release: - version guard read `^version = ` from the root Cargo.toml, which is now the version-less workspace manifest → read cli/Cargo.toml instead - chart version + `helm package` read benchmarks/_chart → containers/benchmarks/_chart - the fleet bake glob (core/*/… agents/*/… …/docker-bake.hcl) → containers/… - `cargo run` / `cargo publish` at the virtual workspace root need a package selector → `-p eval-containers` Verified: the bake glob resolves 147 files; `cargo run -p eval-containers` runs. Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the one remaining CLI-side wiring gap from the release-process review:
release.ymlbuilt the binaries but never published the crate (I published 0.1.0 by hand).doctrine/delivery/RULES.md:2says the crates.io publish MUST be produced byrelease.yml.This adds a dist custom publish-job (
.github/workflows/publish-crate.yml) wired viapublish-jobsindist-workspace.toml, then regeneratesrelease.ymlso it invokes it (custom-publish-crate→uses: ./.github/workflows/publish-crate.yml,secrets: inherit, runs afterhost). AvX.Y.Ztag now publishes the crate as part of the CLI release flow — no manualcargo publish.tag == Cargo.toml == Chart.yamlbefore publishing (doctrine/delivery/RULES.md:6, the release-time half — the PR-time half ischeck.rs).Add a repository secret
CARGO_REGISTRY_TOKEN(a crates.io API token, publish scope) under Settings → Secrets and variables → Actions. Without it the publish step fails.The rest of the punch-list — not here, by design
main(repo_version_aligns_across_cargo_and_chartincheck.rs); nothing to add.mainalready migrated toghcr.io/exgentic; resolved.release-images.yml); repoint it separately.