Skip to content
BranchesTags

Repository files navigation

opcli

A local-first CLI tool for Canonical operator developers to build charms, rocks, and snaps; manage test environments; and run integration tests — identically on a developer laptop and inside a CI job.

opcli replaces the monolithic operator-workflows approach with a modular pipeline based on explicit build plans (artifacts.yaml), stable build output (artifacts.build.yaml), and spread-based test execution.

If your charm repo already builds with charmcraft/rockcraft, provisions Juju with concierge, and runs pytest-jubilant through tox or hand-written spread tasks, opcli doesn't replace any of that — it wires those same tools together behind one build manifest and one test config, so the exact same commands run locally and in CI without duplicating charm-file paths, image refs, or provisioning steps by hand.

Contents

Documentation

This README is the primary documentation for using opcli. Additional resources:

Document Purpose
examples/ Minimal example project layout with artifacts.yaml, spread.yaml, and concierge.yaml
canonical/haproxy-operator Real-world production charm wired end-to-end with opcli (artifacts.yaml, spread.yaml, integration/doc-test/publish workflows)
CHANGELOG.md Notable changes per release
AGENTS.md Contributor/AI-agent guide for working on opcli itself (not needed to use it)

Installation

Prerequisites

  • Linux with snapd (opcli install all installs most tools via snap; other platforms aren't supported for the local-build/spread workflow)
  • Python 3.12+
  • uv (sudo snap install astral-uv --classic)

Install opcli

sudo snap install astral-uv --classic
uv tool install "opcli[cli] @ git+https://github.com/canonical/charm-ci.git@v1.0.0"
export PATH="$HOME/.local/bin:$PATH"  # or: uv tool update-shell && exec $SHELL
opcli --help

Pin to a released tag (@v1.0.0, or later) for reproducible builds. Omitting the @ ref tracks main, which is convenient for trying opcli out but not recommended for CI or a real project — main can change under you.

Note: The [cli] extra is required for the CLI. The bare opcli package (without [cli]) installs only the pytest plugin — useful when your project already has a conflicting typer pin.

Install all local dev tools

After opcli is installed, one command installs everything else (gh, spread, uv, concierge, tox, LXD, charmcraft, rockcraft, snapcraft):

opcli install all

Idempotent — skips any tool already present. Requires passwordless sudo for snap-based installs when not running as root. To verify your environment afterwards:

opcli install doctor

Troubleshooting installation

  • opcli install all fails with a permission/sudo error — it needs passwordless sudo for snap installs (LXD, charmcraft, etc.). Run sudo -n true to check; if that fails, configure passwordless sudo or run the failing step manually.
  • opcli spread run/opcli env provision can't find/use LXD right after install — joining the lxd group requires a new login session. Log out and back in (or newgrp lxd), then verify with lxc list (should run without sudo/permission errors) — opcli install doctor only confirms the lxd binary is present, not that group membership has taken effect.
  • opcli env provision fails looking for concierge.yaml — it isn't generated by opcli artifacts init/opcli spread init. Download examples/concierge.yaml (or examples/concierge-k8s.yaml for a k8s charm) into your repo first — see concierge.yaml contract. For example: 
    curl -o concierge.yaml https://raw.githubusercontent.com/canonical/charm-ci/v1.0.0/examples/concierge.yaml
  • A specific spread test selector fails to match — selectors are backend/system/suite-specific. Run opcli spread run -- -list to print the exact selectors available in your repo, then copy one.

Quick start

Both paths below assume opcli install all has already been run (see Installation), that your repo has a concierge.yaml (copy examples/concierge.yaml as a starting point if you don't have one yet — see Troubleshooting installation for a download command, or the concierge.yaml contract for the full schema), and that opcli is listed as a test dependency in your tox/uv integration environment (see pytest-opcli plugin) so the fixtures are available when tests run.

Local testing with spread

opcli artifacts init     # discover charms/rocks/snaps → artifacts.yaml

# Review artifacts.yaml before building: check each charm's OCI resources
# map to the intended rock, and add per-charm channels/platforms if needed
# — see the artifacts.yaml schema section below.

opcli artifacts build    # build all → build/artifacts.build.yaml
opcli spread init        # generate spread.yaml with integration-suites
opcli spread expand      # preview expanded spread config
opcli spread run         # run integration tests (LXD backend)

# List available test selectors (backend:system:suite/task:variant):
opcli spread run -- -list

# Target one of the selectors printed above:
opcli spread run -- integration-test-local:ubuntu-24.04:tests/integration/run:test_charm

# Run a specific suite without spread (monorepo — requires a matching entry
# in spread.yaml's integration-suites, see the integration-suites section):
opcli pytest run --suite k8s-charm/tests/integration/

Local testing without spread

opcli artifacts init
opcli artifacts build
opcli env provision                                    # concierge (auto-elevates with sudo)
opcli artifacts push-images --missing-registry deploy  # push rocks to local registry (k8s only)
opcli pytest run                                       # run all integration tests via tox

Publishing to CharmHub

# After a successful build (local or CI):
opcli artifacts publish --channel latest/edge

# If each charm declares its own channel in artifacts.yaml, the flag can be omitted:
opcli artifacts publish

# Dry-run to preview what would be uploaded:
opcli artifacts publish --channel latest/edge --dry-run

# Publish only specific charms:
opcli artifacts publish --channel latest/stable --charm my-charm

Requires charmcraft credentials: run charmcraft login interactively or set CHARMCRAFT_AUTH in CI. Publish channel resolution is:

  1. --channel
  2. per-charm channel in artifacts.yaml
  3. error if neither is set for a charm

The command reads artifacts.build.yaml to resolve charm files and resource→rock mappings, then:

  1. Uploads OCI-image resources (rocks from registry or local file, external images from upstream-source)
  2. Uploads each .charm file and releases it to the channel with bound resource revisions

How it works

opcli chains four file-based contracts into one pipeline that runs identically on a laptop and in CI. Each stage reads/writes a file, so any stage can be inspected, re-run, or swapped independently:

  1. artifacts.yaml declares what to build — charms, rocks, snaps, and which rocks back which charm resources. Generated once with opcli artifacts init, then edited as your project's build plan changes.
  2. opcli artifacts build builds everything the plan declares and writes artifacts.build.yaml — the same build plan, extended with the paths (local) or registry refs (CI) of what was actually built. This file is the stable interface the rest of the pipeline (and the pytest-opcli plugin) consumes — no other step needs to know how or where the build happened.
  3. concierge.yaml declares the test environment (LXD, MicroK8s, Juju controller, etc.), consumed by concierge via opcli env provision. The same file works locally and in CI; CI-only differences (like an image-registry mirror) are patched in, not hardcoded separately.
  4. spread.yaml orchestrates provisioning + test execution through spread. You declare one logical integration-test backend; opcli spread expand resolves it to a concrete local (LXD VM) or CI (current runner) backend depending on where it runs, so the same spread.yaml drives both.

In familiar terms: artifacts.build.yaml is the recorded equivalent of the --charm-file/OCI-image refs you'd otherwise track down and pass into pytest-jubilant by hand (or wire up manually between CI jobs). The integration-suites block in spread.yaml generates the spread task.yaml boilerplate that would otherwise provision concierge and invoke tox by hand for each suite.

Artifacts must be built (stage 2) before running spread — opcli spread run only drives stages 3–4 (provision + test) using whatever artifacts.build.yaml already exists; it doesn't build for you. Two paths cover the full pipeline: opcli artifacts build → opcli spread run (closest to what CI does — CI runs the two as separate parallel/sequential jobs), or opcli artifacts build → opcli env provision → opcli pytest run for faster local iteration without spread's VM overhead (see Quick start).

Commands

Every command accepts a global --verbose/-v flag (before the subcommand name, e.g. opcli -v artifacts fetch ...) that surfaces INFO-level detail from opcli's internal operations (per-artifact download/localize/publish results). Warnings and errors are always shown regardless of this flag.

Run opcli --version to print the installed opcli version and exit.

opcli artifacts

Command Description
init Discover charms/rocks/snaps and generate artifacts.yaml. --force to overwrite.
build Build artifacts → build/artifacts.build.yaml. Filter: --charm, --rock, --snap. --build-timeout <seconds> (default: 3600).
matrix Print JSON build matrix for GitHub Actions.
collect <partial>... Merge partial artifacts.build.yaml from parallel jobs.
fetch Download CI artifacts and rewrite to local paths. --run-id (required), --repo, --wait, --wait-timeout <seconds> (default: 1800; implies --wait).
localize Rewrite CI artifact refs to local paths (after manual download).
push-images Load rock OCI images into a local registry. -r for registry (default: localhost:32000). --missing-registry: skip (default), deploy (auto-provision), or fail.
publish Upload charms and OCI resources to CharmHub. --channel (optional; per-charm channels supported), --charm (filter), --dry-run.
path Print absolute path(s) to built artifacts. Optional NAME arg, --type, --arch.

publish retries known-transient CharmHub failures automatically (upload-status polling timeouts, RemoteDisconnected, connection resets/aborts): up to 3 total attempts, with 5s then 15s backoff between attempts. Permanent errors (e.g. missing publisher permission) fail immediately without retrying.

opcli install

Command Description
all Install all local dev tools in one shot: gh, spread (built from source), uv, concierge, tox, LXD, charmcraft, rockcraft, snapcraft.
doctor Print a ✓/✗ status table for all required tools with versions. Exits 1 if any are missing.

Individual per-tool commands (gh, spread, tox, concierge, lxd, uv, charmcraft, rockcraft, snapcraft) are visible in a grouped panel in opcli install --help for use in CI prepare scripts.

opcli env

Command Description
provision Run concierge prepare to provision the test environment. -c for concierge path.
deploy-registry Deploy local OCI registry at localhost:32000 (auto-detects k8s provider).

opcli spread

Command Description
init Generate spread.yaml with integration-suites. --force to overwrite.
expand Print fully expanded spread.yaml to stdout.
run Expand virtual backend and run spread. Args after -- forwarded verbatim.
jobs Print CI test matrix JSON (one entry per spread task/variant). --include <pattern> filters by raw spread -list selector (fnmatch glob, e.g. --include 'my-docs-ci:*').

opcli pytest

Command Description
run Assemble and execute the tox integration test command. -e for env, --suite for suite, -- forwards args.
expand Print full tox -e integration -- <flags> command. -e for env, --suite for suite, -- forwards args.

By default, opcli pytest run/expand invokes tox -e integration with no extra flags. Artifact fixtures are injected automatically by the pytest-opcli plugin — no CLI flag plumbing needed.

To pass Juju-specific options or other pytest flags, use pytest-arguments-template on the suite entry in spread.yaml:

integration-suites:
  tests/integration/:
    pytest-arguments-template: |
      --model testing
      --keep-models

env is available in both templates. Use it in pytest-arguments-template to inject dynamic values — for example reading the Juju model name from an environment variable:

integration-suites:
  tests/integration/:
    pytest-arguments-template: |
      --model={{ env.get("JUJU_MODEL", "testing") }}
      --keep-models

To pass artifacts as environment variables instead of fixtures:

integration-suites:
  tests/integration/:
    pytest-environment-template: |
      {% for build in artifacts.charms[0].builds if build.arch == arch %}
      CHARM_PATH={{ build.path }}
      {% endfor %}

To forward environment variables from the root spread shell into tox/pytest (e.g. CI tokens or job metadata), reference env in the template. Spread sets SPREAD_JOB automatically for every task:

integration-suites:
  tests/integration/:
    pytest-environment-template: |
      SPREAD_JOB={{ env.get("SPREAD_JOB", "") }}
      GITHUB_TOKEN={{ env.get("GITHUB_TOKEN", "") }}

Important: variables forwarded via the template are only available to pytest if tox also passes them through. Add them to passenv in your tox.ini:

[testenv:integration]
passenv =
    SPREAD_JOB
    GITHUB_TOKEN

env access patterns:

  • env.get("VAR", "") — safe, returns the default if the variable is absent (recommended for optional vars)
  • env.VAR or env["VAR"] — strict, raises ConfigurationError if the variable is not set (useful as a self-documenting "this must be set" assertion)

The --suite flag selects a specific integration suite (useful in monorepos with multiple test directories):

opcli pytest run --suite k8s-charm/tests/integration/
opcli pytest expand --suite machine-charm/tests/integration/

When a single integration-suites entry exists, --suite is auto-detected. With multiple suites, it's required.

opcli tutorial

Command Description
expand <file> Extract shell commands from a Markdown (.md) or RST (.rst) tutorial file and print them to stdout as a shell script.

Typical usage in a spread task.yaml backed by the opcli-minimal backend:

runuser -l ubuntu -s /bin/bash -c 'set -ex; . <(opcli tutorial expand -- "$1")' _ "${SPREAD_PATH}${TUTORIAL}"

What gets extracted:

File type Included Excluded
.md 3-backtick code fences (all languages except {…} tags like {terminal}) 4+-backtick fences; <!-- SPREAD SKIP --> … <!-- SPREAD SKIP END --> ranges
.md <!-- SPREAD … --> HTML comment blocks (always) —
.rst .. code-block:: directives (directive options like :caption: are skipped) .. SPREAD SKIP … .. SPREAD SKIP END ranges
.rst .. SPREAD … .. SPREAD END blocks (always) —

artifacts.yaml schema

version: 1
rocks:
  - name: my-rock
    rockcraft-yaml: rocks/my-rock/rockcraft.yaml
    platforms:
      - arch: amd64
      - arch: arm64
        runner: [self-hosted, arm64]
charms:
  - name: my-charm
    charmcraft-yaml: charmcraft.yaml
    channel: latest/edge
    resources:
      my-rock-image:
        type: oci-image
        rock: my-rock
snaps:
  - name: my-snap
    snapcraft-yaml: snap/snapcraft.yaml
    pack-dir: .

Key fields:

  • *-yaml: explicit path to the craft YAML file (not a directory).
  • pack-dir: working directory for the build tool (defaults to the YAML's parent dir).
  • platforms[].runner: GitHub Actions runner labels (used by opcli artifacts matrix; defaults to ["ubuntu-latest"] at matrix generation time when omitted).
  • channel: optional CharmHub channel for opcli artifacts publish; used when --channel is not passed.

Validation notes: name must be unique within each of rocks/charms/snaps (duplicates fail validation); platforms[].arch must be a hyphen-free lowercase-alphanumeric token (e.g. amd64, arm64, s390x — no ubuntu-24.04-style values); platforms defaults to a single amd64 entry when omitted entirely.

artifacts.build.yaml schema

Written by opcli artifacts build/collect to build/artifacts.build.yaml (build/ should be in .gitignore). Same charms/rocks/snaps shape as artifacts.yaml, but each builds[] entry records where the built artifact actually is, in a form that differs between local and CI runs:

# Locally — plain file paths:
version: 1
rocks:
  - name: my-rock
    rockcraft-yaml: rocks/my-rock/rockcraft.yaml
    builds:
      - arch: amd64
        file: ./rocks/my-rock/my-rock_1.0_amd64.rock
charms:
  - name: my-charm
    charmcraft-yaml: charmcraft.yaml
    builds:
      - arch: amd64
        base: ubuntu-24.04
        path: ./my-charm_ubuntu-24.04-amd64.charm
    resources:
      my-rock-image:
        type: oci-image
        rock: my-rock
# In CI — OCI images pushed to GHCR, charms as downloadable GH artifacts:
version: 1
rocks:
  - name: my-rock
    rockcraft-yaml: rocks/my-rock/rockcraft.yaml
    builds:
      - arch: amd64
        image: ghcr.io/canonical/my-rock:abc1234-amd64
charms:
  - name: my-charm
    charmcraft-yaml: charmcraft.yaml
    builds:
      - arch: amd64
        artifact: charm-my-charm
        run-id: "1234567890"
    resources:
      my-rock-image:
        type: oci-image
        rock: my-rock
  • These local/CI examples show the two common shapes, but they're not strictly mutually exclusive: rocks built on a fork PR combine file + artifact + run-id (uploaded as a GH artifact instead of pushed to GHCR — see Fork PR support). opcli resolves which fields are populated based on GITHUB_ACTIONS/OPCLI_ROCK_UPLOAD (see CI vs local).
  • Consumed by opcli artifacts publish, the pytest-opcli plugin (see pytest-opcli plugin), and opcli artifacts fetch (which downloads the CI-form artifacts back to local paths).

concierge.yaml contract

Standard concierge configuration file — see examples/concierge.yaml/examples/concierge-k8s.yaml. opcli env provision runs concierge prepare against it (auto-elevating with sudo as needed).

When opcli env provision is passed a non-empty --image-registry <url>, it patches the concierge file in place first, injecting image-registry: {url: <url>} into each enabled microk8s/k8s provider section (other provider types are untouched). This is how the CI integration-test backend routes registry pulls through a mirror (via DOCKERHUB_MIRROR) to avoid rate limits — no patching happens if --image-registry isn't passed. See canonical/concierge#181 for the underlying upstream limitation this works around.

spread.yaml virtual backends

opcli recognises two virtual backend types and expands them into concrete spread backends at runtime.

integration-test — full integration backend

For standard charm integration tests with Juju and concierge provisioning:

backends:
  integration-test:
    type: integration-test
    systems:
      - ubuntu-24.04:
          runner: [self-hosted, noble]   # CI runner labels
          cpu: 4                         # local LXD VM vCPUs
          memory: 8                      # local LXD VM RAM (GiB)
          disk: 20                       # local LXD VM disk (GiB)
  • Locally (CI unset): expands to integration-test-local with an LXD backend; installs concierge, Juju, and opcli, then runs opcli env provision.
  • In CI (CI=true): expands to integration-test-ci with an adhoc backend targeting the current runner.

opcli-minimal — lightweight backend

For tests that only need opcli itself (e.g. tutorial runs, doc validation), with no Juju or concierge:

backends:
  my-docs:
    type: opcli-minimal
    systems:
      - ubuntu-24.04:
          runner: [self-hosted, noble]
  • Locally: expands to my-docs-local; installs uv and opcli only.
  • In CI: expands to my-docs-ci; prepare installs uv and opcli only (no concierge, no Juju, no provisioning).

Users write their own task.yaml for this backend (or use opcli tutorial expand — see Tutorial testing).

The runner, cpu, memory, and disk fields are opcli-only metadata — they are stripped before spread sees the YAML.

integration-suites in spread.yaml

Instead of committing boilerplate task.yaml files, declare test suites declaratively:

integration-suites:
  tests/integration/:
    working-dir: ./
    summary: top-level integration tests
    backends:
      - integration-test
    environment:
      CONCIERGE/test_k8s_charm: concierge-k8s.yaml

  # Monorepo pattern — sub-charm with its own tests
  k8s-charm/tests/integration/:
    working-dir: k8s-charm/
    discover-path: k8s-charm/tests/integration/
    summary: k8s-charm sub-charm tests
    backends:
      - integration-test

  # Explicit variants (no auto-discovery)
  machine-charm/tests/integration/:
    working-dir: machine-charm/
    auto-discover: false
    summary: machine-charm tests
    backends:
      - integration-test
    environment:
      MODULE/test_charm: test_charm

At expand time, integration-suites entries are converted into native spread suites: entries with:

  • Auto-discovery (default): scans the suite directory for test_*.py files and generates MODULE/<name> spread variants.
  • working-dir: tells opcli pytest which directory to run pytest from. Defaults to ./ (project root).
  • discover-path: overrides the directory scanned for auto-discovery. When omitted, opcli uses the suite key path.
  • task.yaml generation: written into the build/ directory at runtime (e.g. build/tests/integration/run/task.yaml). Files persist for inspection and are overwritten on next run. Add build/ to your .gitignore.
  • discover-pattern: customize the glob for auto-discovery (e.g., discover-pattern: "test_*.py" is the default; use "*_test.py" if your project uses that convention).

Migrating from native suites: Replace your suites: block and committed task.yaml with an integration-suites: entry. Delete the task.yaml file — opcli generates it at runtime. Existing native suites: entries coexist and are passed through unchanged.

Note: reroot in spread.yaml is incompatible with opcli. opcli manages reroot internally during expansion (to resolve paths from the build/ directory back to the project root).

Suite-specific keys

Key Default Description
working-dir ./ Working directory for pytest invocation (opcli-only, stripped from spread output)
discover-path (suite key) Directory scanned for auto-discovery; overrides the suite key path when set
auto-discover true Scan for test_*.py and generate MODULE/ variants
discover-pattern test_*.py Glob pattern for auto-discovery
pytest-arguments-template — Jinja2 template for pytest CLI args (opcli-only, stripped)
pytest-environment-template — Jinja2 template for env vars (opcli-only, stripped)
backends (required) Which virtual backends to run this suite on
summary — Spread suite summary
environment — Additional environment variables (merged with auto-discovered modules)

Overriding the tox environment with TOX_ENV

The generated task.yaml runs opcli pytest expand -e "${TOX_ENV:-integration}", so it defaults to the integration tox environment. If your project already has a differently-named tox environment (e.g. charms-integration, k8s-integration), set TOX_ENV in the suite's environment: block instead of renaming your tox env or adding a redundant [testenv:integration] alias:

integration-suites:
  tests/integration/:
    working-dir: ./
    backends:
      - integration-test
    environment:
      TOX_ENV: charms-integration

This also applies to opcli pytest run/expand directly — pass -e explicitly (e.g. opcli pytest expand -e "$TOX_ENV") since only the generated task.yaml shell script reads TOX_ENV automatically.

Pytest invocation templates

Controls how opcli pytest run and opcli pytest expand pass extra flags to the test framework. These keys live per-suite inside integration-suites — they are opcli-only and stripped from the spread output:

Key Effect
pytest-arguments-template Jinja2 template rendered into CLI args passed to tox/pytest
pytest-environment-template Jinja2 template rendered into KEY=VALUE env vars

When no template is specified, opcli pytest runs bare tox -e integration with no extra flags. Artifact fixtures are provided by the pytest-opcli plugin automatically. Use pytest-arguments-template to pass additional options (Juju model name, test selection flags, etc.):

Template context: artifacts (full ArtifactsGenerated model), arch (current architecture string), and env (snapshot of the current process environment). Because opcli pytest expand runs as root inside the spread task, env captures root's environment. Use env.get("VAR", "") for optional variables.

For projects with multiple suites, use a YAML anchor to avoid repetition:

x-pytest-args: &pytest-args
  pytest-arguments-template: |
    --model testing
    --keep-models

integration-suites:
  tests/integration/:
    <<: *pytest-args
    working-dir: ./
    backends:
      - integration-test
  k8s-charm/tests/integration/:
    <<: *pytest-args
    working-dir: k8s-charm/
    backends:
      - integration-test
    pytest-arguments-template: |   # override for this suite
      --model testing-k8s
      --keep-models

The x-pytest-args key at the top level is ignored by both spread and opcli — it exists only for the YAML anchor.

pytest-opcli plugin

opcli ships a pytest plugin that auto-discovers artifacts.build.yaml and injects built artifacts as session-scoped fixtures. Integration tests stop needing manual --charm-file / --resource-image CLI flags in conftest.py.

Installation

The plugin is bundled inside opcli and activates automatically whenever opcli is installed in the same Python environment as pytest. Add it as a test dependency alongside pytest-jubilant and your other test packages.

The bare opcli package (without [cli]) installs only pydantic and ruamel.yaml — no typer dependency. This makes it safe to add as a test dependency even when your project pins an older typer version.

With uv — add to your project (recommended — Renovate can track this):

uv's [tool.uv.sources] table lets you pin the Git tag separately from the plain dependency name, which is what makes the pin visible to Renovate's pep621 manager (the inline name @ git+URL@tag form below is not tracked by any Renovate manager). canonical/charm-ubuntu uses this pattern in production:

uv add --group integration "opcli @ git+https://github.com/canonical/charm-ci.git" --tag v1.0.0

This writes both blocks below automatically:

[dependency-groups]
integration = [
    "opcli",
    "pytest-jubilant",
]

[tool.uv.sources]
opcli = { git = "https://github.com/canonical/charm-ci.git", tag = "v1.0.0" }

Quick/manual alternative (not Renovate-trackable):

[dependency-groups]
integration = [
    "opcli @ git+https://github.com/canonical/charm-ci.git@v1.0.0",
    "pytest-jubilant",
]

With tox — add to the integration test env:

tox.ini has no equivalent to [tool.uv.sources], so a plain deps: pin is never Renovate-trackable out of the box. Either move the pytest-plugin dependency into pyproject.toml ([dependency-groups], as above) and drive tox from it with uv sync --group integration, or add a Renovate customManagers regex manager targeting this line:

[testenv:integration]
deps =
    opcli @ git+https://github.com/canonical/charm-ci.git@v1.0.0
    pytest-jubilant

No further configuration is required. The pytest11 entry point registers the plugin as soon as the package is installed.

Fixtures

All fixtures are session-scoped and architecture-aware (they filter builds to the machine's current CPU architecture).

Fixture / Helper Return type Description
opcli_build_yaml_path Path Resolved path to artifacts.build.yaml. Use as a dependency in custom conftest fixtures.
opcli_artifacts ArtifactsGenerated Full model parsed from artifacts.build.yaml. Always requires yaml; not available in CLI-flag mode.
charm_path str Path to the single built .charm. Fails if the repo contains more than one charm, or if the single charm has more than one build for the current arch (use charm_paths instead).
charm_paths dict[str, CharmPathList] All .charm paths per charm name. Use .path for the single-base shortcut, or ['ubuntu@X'] for base-keyed access.
resource_images dict[str, str] {resource_name: image_ref}. In yaml mode: resolves each OCI-image resource to its rock image for the single charm. In CLI-flag mode: uses --resource-image values directly. Fails if the repo contains zero or more than one charm (yaml mode only).
charm_resource_images dict[str, dict[str, str]] {charm_name: {resource_name: image_ref}}. yaml mode only. Resolves OCI-image resources for every charm. Use this instead of resource_images in multi-charm repos (e.g. 12-factor charms).
build_rock_images(artifacts, root) dict[str, str] Helper function (not a fixture) — returns {rock_name: image_ref} for the current arch. Use in a conftest rock_images fixture for multi-charm repos.

Usage examples

Single charm with OCI resources (most common):

def test_deploy(juju, charm_path, resource_images):
    juju.deploy(charm_path, resources=resource_images)
    juju.wait(jubilant.all_active)

Single charm built for multiple bases:

def test_deploy(juju, charm_paths):
    # single base — use .path shortcut
    juju.deploy(charm_paths["my-charm"].path)
    juju.wait(jubilant.all_active)

Or to target a specific base explicitly:

def test_deploy(juju, charm_paths):
    juju.deploy(charm_paths["my-charm"]["ubuntu@24.04"])
    juju.wait(jubilant.all_active)

Multi-charm repo (e.g. 12-factor charms):

def test_deploy(juju, charm_paths, charm_resource_images):
    juju.deploy(charm_paths["operator"].path,
                resources=charm_resource_images["operator"])
    juju.deploy(charm_paths["agent"].path,
                resources=charm_resource_images["agent"])
    juju.wait(jubilant.all_active)

Multi-charm repo (custom rock_images in conftest.py, advanced):

# conftest.py
from pathlib import Path
import pytest
from opcli.models.artifacts_build import ArtifactsGenerated
from opcli.pytest_plugin import artifacts_root_from_yaml_path, build_rock_images

@pytest.fixture(scope="session")
def rock_images(opcli_artifacts: ArtifactsGenerated, opcli_build_yaml_path: Path) -> dict[str, str]:
    return build_rock_images(opcli_artifacts, artifacts_root_from_yaml_path(opcli_build_yaml_path))
# test_deploy.py
def test_deploy(juju, charm_paths, rock_images):
    juju.deploy(charm_paths["operator"].path, resources={"backend": rock_images["backend-rock"]})
    juju.deploy(charm_paths["agent"].path)
    juju.wait(jubilant.all_active)

Discovery

The plugin locates artifacts.build.yaml in this order:

  1. --artifacts-build-yaml pytest CLI option.
  2. OPCLI_ARTIFACTS_BUILD_YAML environment variable (path; relative paths are resolved from the current working directory, so an absolute path is recommended for CI).
  3. Walk up from pytest's rootdir until build/artifacts.build.yaml is found (stops at git root).
  4. pytest.UsageError if none of the above succeed — run opcli artifacts build first.

CLI-flag mode (no build step)

If you already have charm files and OCI image references (for example, from a previous CI stage), you can pass them directly as pytest CLI flags without needing artifacts.build.yaml:

pytest \
  --charm-file my-charm=./my-charm.charm \
  --resource-image oci-image=ghcr.io/org/rock:sha256-abc

Both flags are repeatable. Use NAME=VALUE format where NAME is the charm name (for --charm-file) or the Juju resource name (for --resource-image).

Flag Format Maps to
--charm-file NAME=PATH charm_path / charm_paths
--resource-image NAME=REF resource_images

Each fixture independently checks its own CLI flags first, then falls back to yaml discovery. Mixing modes per-fixture is supported (for example, pass --charm-file but let resource_images come from yaml).

Tutorial testing

opcli tutorial expand extracts shell commands from a Markdown or RST tutorial file and prints them as a shell script. Combined with the opcli-minimal backend, this lets you gate docs PRs by running the tutorial in CI.

Example setup

  1. Declare the backend and suite in spread.yaml:
backends:
  docs:
    type: opcli-minimal
    systems:
      - ubuntu-24.04:
          runner: [ubuntu-latest]

suites:
  docs/tutorial/:
    summary: Tutorial smoke test
    systems:
      - ubuntu-24.04

environment:
  TUTORIAL: docs/tutorial/getting-started.md
  1. Write docs/tutorial/run/task.yaml:
summary: Run getting-started tutorial

execute: |
  runuser -l ubuntu -s /bin/bash -c \
    'set -ex; . <(opcli tutorial expand -- "$1")' \
    _ "${SPREAD_PATH}${TUTORIAL}"
  1. Select only the integration-test jobs for the main matrix (the tutorial docs run in a separate job):
opcli spread jobs --include 'integration-test-ci:*'

Marker syntax

Inline SPREAD markers let you include commands that are not in code fences, or skip blocks that shouldn't run:

<!-- SPREAD
sudo snap install my-charm --classic
-->

Normal prose here. The next shell block is skipped:

<!-- SPREAD SKIP -->
```bash
$ interactive-command  # won't be extracted

RST equivalent uses `.. SPREAD`, `.. SPREAD END`, `.. SPREAD SKIP`, and `.. SPREAD SKIP END` directives.

## CI vs local

| Env var | Controls | Local | CI |
|---|---|---|---|
| `CI` | Spread backend expansion | `*-local` (LXD VM) | `*-ci` (current runner) |
| `GITHUB_ACTIONS` | Artifact output format | Local file paths | GHCR images + artifact refs |
| `OPCLI_ROCK_UPLOAD` | Rock build output mode | — (not set) | `registry` (push to GHCR) or `artifact` (upload `.rock` as GH artifact, for fork PRs) |
| `OPCLI_GIT_REF` | opcli version inside spread VM | defaults to `main` | set by workflow |

### Scope

**What `opcli` owns:** file-based contracts (`artifacts.yaml`/`artifacts.build.yaml`), artifact discovery/download, subprocess execution (charmcraft/rockcraft/snapcraft/spread/concierge), YAML transforms, and publishing to CharmHub.
**What `opcli` does NOT own:** the GitHub Actions workflow *composition* — triggers, permissions, artifact-upload policy, and runner selection stay in each consumer repo's own `.github/workflows/` files. `opcli` provides [reusable workflows](#github-actions-reusable-workflows) below that those files call, but doesn't orchestrate them itself.

## GitHub Actions reusable workflows

Four reusable workflows are available for operator repositories:

| Workflow | Purpose |
|---|---|
| `build-artifacts.yml` | Build matrix generation, parallel artifact builds, merged `artifacts.build.yaml`; debug builds open a detached tmate session, using the self-hosted or GitHub-hosted action as appropriate |
| `integration-test.yml` | Download artifacts, generate spread task matrix, run integration tests |
| `publish-artifacts.yml` | Publish validated artifacts to CharmHub; `channel` is optional and falls back to per-charm channels in `artifacts.yaml` |
| `doc-test.yml` | Generate spread task matrix, run documentation/tutorial tests (no artifact build) |

Example usage for integration tests — `integration-test.yml` already calls `build-artifacts.yml` internally, so a single job is enough (no separate `build` job needed):

```yaml
jobs:
  test:
    uses: canonical/charm-ci/.github/workflows/integration-test.yml@v1.0.0
    permissions:
      contents: read
      packages: write
      actions: read
    secrets: inherit
    with:
      working-directory: .
      # upload-image: artifact       # uncomment for fork PRs (no GHCR push)
      # pre-build-rock-script: ./download_netbox.sh  # forwarded to build-artifacts; runs before each rock build (skipped on cache hits)
      # build-timeout-minutes: 60    # forwarded to build-artifacts (default: 60)
      # test-timeout-minutes: 120    # max minutes per spread test job (default: 120)
      # charmcraft-channel: latest/stable  # forwarded to build-artifacts (snap channel for charmcraft)
      # rockcraft-channel: latest/stable   # forwarded to build-artifacts (snap channel for rockcraft)
      # environment: integration-tests  # optional: scope secrets to a GitHub Environment

Example usage for documentation tests:

jobs:
  doc-test:
    uses: canonical/charm-ci/.github/workflows/doc-test.yml@v1.0.0
    permissions:
      contents: read
      actions: read
    with:
      working-directory: .
      # spread-jobs-include: "docs-ci:*"  # optional: restrict to matching jobs

Example usage for publishing:

jobs:
  publish:
    uses: canonical/charm-ci/.github/workflows/publish-artifacts.yml@v1.0.0
    permissions:
      contents: write
      actions: read
    secrets:
      CHARMHUB_TOKEN: ${{ secrets.CHARMHUB_TOKEN }}
    with:
      # channel: latest/edge  # optional
      # inject-version: true   # optional; set false to publish charms unchanged
      # create-tags: true      # optional; set false to skip per-revision git tags
      # create-release: true   # optional; set false to skip the combined GitHub Release
      # environment: charmhub-publish  # optional: scope CHARMHUB_TOKEN to a GitHub Environment
      working-directory: .

The publish workflow can also create a git tag ({charm-name}-rev{revision}) for every published charm revision, and at most one combined GitHub Release per publish workflow run summarizing everything published (charms, revisions, bases, architectures, and resource revisions), with links to each per-revision tag. Both are enabled by default (create-tags: true, create-release: true) and can be independently disabled — e.g. to defer visible releases to a separate "promote" workflow, or to reduce the number of per-revision tags in repositories that publish many bases/architectures per run. Disabling create-tags does not eliminate tags entirely: a GitHub Release cannot exist without a tag, so when create-release: true one publish-<run-id> tag is still created per publish run (in addition to, or instead of, the per-revision tags depending on create-tags). When enabled, the combined release includes GitHub-generated release notes such as the "What's Changed" and "New Contributors" sections. Before uploading to CharmHub, the workflow injects a version file containing the first 8 characters of the publish commit SHA into each fetched .charm archive, unless that archive already contains a version file. Set inject-version: false to skip this reusable-workflow-only behavior. Local opcli artifacts publish never modifies charm archives.

Behavior change: previously this workflow created one GitHub Release per published charm revision ({charm-name}-rev{revision}). It now creates at most one combined Release per publish run instead — per-revision git tags still exist (when create-tags: true), but per-revision Release objects no longer do. If you have automation that queries gh release view {charm-name}-rev{revision} directly, switch it to read the combined release instead, or to resolve the tag directly rather than via a Release lookup.

Pinning to a SHA or tag automatically installs the matching opcli version via canonical/get-workflow-version-action.

Fork PR support

When a pull request comes from a fork, the GITHUB_TOKEN is read-only and cannot push OCI images to GHCR. The build-artifacts.yml workflow handles this automatically:

  1. Fork detection — checks github.event.pull_request.head.repo.fork and sets OPCLI_ROCK_UPLOAD=artifact.
  2. Artifact mode — the .rock file is uploaded as a GitHub Actions artifact instead of being pushed to GHCR.
  3. Test phase — opcli artifacts fetch downloads the .rock artifact, opcli artifacts localize rewrites paths, and opcli artifacts push-images --missing-registry deploy provisions a local registry and pushes the rock there.
  4. Debugging — when GitHub Actions debug logging is enabled, the build job opens a detached tmate session; self-hosted runners use canonical/action-tmate, GitHub-hosted runners use mxschmitt/action-tmate.

To manually test the fork path, pass upload-image: artifact to build-artifacts.yml (or use workflow_dispatch if configured).

Secrets for integration tests

Integration tests often need secrets (cloud credentials, API tokens, etc.). opcli supports this identically locally and in CI.

Locally: .secrets.env

Create a .secrets.env file in your repo root (gitignored) with plain KEY=VALUE pairs:

# .secrets.env — never commit this file
S3_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE
DATABASE_URL=postgres://user:pass@host/db

opcli auto-loads this file before running spread (local mode only), so no manual export is needed.

In spread.yaml

Declare the secrets as spread environment variables using the $(HOST: echo ...) pattern:

environment:
  S3_ACCESS_KEY: '$(HOST: echo "${S3_ACCESS_KEY:-}")'
  DATABASE_URL: '$(HOST: echo "${DATABASE_URL:-}")'

This self-documents what secrets your test suite requires.

In CI: workflow inputs

Pass secret names to the reusable workflow via test-secret-{1..5}-name inputs:

jobs:
  integration-test:
    uses: canonical/charm-ci/.github/workflows/integration-test.yml@v1.0.0
    secrets: inherit
    with:
      test-secret-1-name: S3_ACCESS_KEY
      test-secret-2-name: DATABASE_URL

The workflow resolves values from your repository's GitHub Secrets, masks them with ::add-mask::, and exports them to the environment before spread runs. It also generates a random Spread SSH password, masks it, and passes it to Spread with -pass so verbose allocation traces cannot expose the credential.

Note: Running opcli spread run -- -vv locally will print secret values to the terminal (spread's verbose mode). The reusable workflows mask known secrets before invoking Spread. Custom CI using -vv must similarly register a generated password with its log-masking mechanism and pass that password to Spread with -pass.

Scoping secrets to a GitHub Environment

By default, named secrets referenced by integration-test.yml and publish-artifacts.yml (test-secret-{1..5}-name, CHARMHUB_TOKEN) come from repository-level secrets, which are visible to every workflow in the repo. To scope them to a dedicated GitHub Environment instead, pass the optional environment input. Note: this only affects named secrets — it has no effect on the auto-generated GITHUB_TOKEN, whose scope is controlled solely by the job's permissions: block and repo/org policy, not by GitHub Environments.

jobs:
  integration-test:
    uses: canonical/charm-ci/.github/workflows/integration-test.yml@v1.0.0
    secrets: inherit
    with:
      environment: integration-tests
      test-secret-1-name: S3_ACCESS_KEY

  publish:
    uses: canonical/charm-ci/.github/workflows/publish-artifacts.yml@v1.0.0
    secrets:
      CHARMHUB_TOKEN: ${{ secrets.CHARMHUB_TOKEN }}
    with:
      environment: charmhub-publish

This sets environment: on the job that consumes secrets (test / publish), so it only has access to the named environment's secrets and (if configured) must satisfy that environment's protection rules — e.g. required reviewers — before running. It's purely additive: if a secret isn't defined in the named environment, GitHub falls back to the repository secret of the same name, so existing repo-secret-based setups keep working unchanged. The named environment must exist in the calling repository; on public repos (or private repos on a GitHub Team/Enterprise/Pro plan) it's auto-created with no protection rules the first time it's referenced if it doesn't already exist. On GitHub Free, environments cannot be configured for private repositories at all — see GitHub's docs for plan restrictions. Leaving environment unset (the default) is fully backward compatible — the job has no environment: key at all.

Development

Requires Python 3.12+ and uv.

uv sync                                    # install deps
uv run opcli --help                        # run the tool
uv run ruff check src/ tests/              # lint
uv run ruff format --check src/ tests/     # format check
uv run mypy src/                           # type check
uv run pytest tests/unit/                  # unit tests

Project structure

src/opcli/
  commands/    # CLI layer (Typer) — parses args, delegates to core/
  core/        # All business logic
  models/      # Pydantic V2 models (artifacts.yaml, artifacts.build.yaml)
  data/        # Bundled static files (e.g. registry.yaml manifest)
tests/
  unit/        # Fast tests — mock external processes
  integration/ # Requires LXD/spread — skip-guarded
examples/      # Example project layout

Versioning policy

opcli follows Semantic Versioning as of v1.0.0. This covers the documented public surface: CLI commands/flags and their documented behavior, the artifacts.yaml/artifacts.build.yaml schemas, the spread.yaml virtual-backend keys, and reusable-workflow inputs. Breaking changes to any of these require a major version bump and are documented in CHANGELOG.md with a migration note.

Internal Python modules/functions (anything under src/opcli/ not exposed as a CLI command, schema field, or workflow input), undocumented output formatting, and unspecified behavior are not covered by this guarantee and may change in any release.

Releases are tagged (vX.Y.Z) and published as GitHub Releases. opcli is installed via git+https://github.com/canonical/charm-ci.git pins (a tag, or via [tool.uv.sources] for Renovate-trackable updates) — there is no PyPI package.

Prior to v1.0.0 (the 0.0.1-alpha.N releases), no backward-compatibility guarantee applied; see CHANGELOG.md for what changed at the v1.0.0 boundary.

License

Apache License 2.0 — see LICENSE.

About

Tooling and CI workflows for building, testing, and integrating Canonical charms

Topics

platform-engineeringproduct-charm-cisquad-emea

Resources

Readme
Apache-2.0 license
Activity
Custom properties

Stars

4 stars

Watchers

0 watching

Forks

3 forks
Report repository

Releases

Packages

Contributors

Languages