Skip to content
BranchesTags

Repository files navigation

wads

wads creates, configures and publishes Python packages: a new project gets a Hatchling pyproject.toml and a five-line GitHub Actions stub that calls one shared reusable workflow, and every CI setting lives in [tool.wads.ci]. It also migrates legacy setup.cfg projects, wires CI secrets, and diagnoses CI failures.

Still typing code with your own fingers? Skip to For carbon-based contributors.

PyPI version Python versions

For AI agents

Skills. Twelve agent skills ship inside the package, in wads/data/skills/: setup-py-project, wads-migrate, wads-repo-doctor, wads-ci-health, wads-changelog, wads-docs-coverage, wads-docstring-render, wads-import-time, wads-pypi-polish, wads-skillify, wads-test-coverage and wads-type-coverage. Enable them either way:

pip install "wads[create]" && wads-install-skills   # symlinks them into ~/.claude/skills/
gh skill install i2mint/wads wads-repo-doctor       # one skill, from GitHub

wads-install-skills --list prints the available names. In a clone of this repo, .claude/skills/ links every skill for Claude Code, plus the maintainer skill wads-dev-workflow (in skills/), which covers changing wads itself.

Project instructions. .claude/CLAUDE.md explains the architecture (pyproject as the single source of truth, the reusable workflow, the two-layer secrets model), the config sections, and the conventions.

What an agent can do with wads:

  • Scaffold a package with populate, or a whole repo with the setup-py-project skill.
  • Move a legacy repo to pyproject.toml and the CI stub with wads-migrate.
  • Declare CI secrets and env vars with wads-secrets add NAME.
  • Read or render a repo's CI configuration from Python, as below.
  • Run the CI plan locally with wads ci-local, and audit dependency licences with wads-licence-check.
  • Diagnose a failed run with wads-ci-debug owner/repo.

A minimal example, runnable with only the light core (pip install wads):

from wads.ci_config import CIConfig
from wads.migration import migrate_ci_to_stub

config = CIConfig(
    {
        "project": {"name": "mypkg", "optional-dependencies": {"dev": ["httpx"]}},
        "tool": {"wads": {"ci": {"testing": {"python_versions": ["3.12"]}}}},
    }
)
assert config.python_versions == ["3.12"]
assert config.project_name == "mypkg"
# A dev extra CI would never install (no [tool.wads.ci.install].extras):
assert config.uninstalled_test_extras == {"dev": ["httpx"]}

stub = migrate_ci_to_stub()  # the ci.yml a new repo gets
assert "uses: i2mint/wads/.github/workflows/uv-ci.yml@master" in stub
assert "PYPI_PASSWORD: ${{ secrets.PYPI_PASSWORD }}" in stub

What is Wads?

Wads helps you:

  • Create new Python projects with modern tooling (pyproject.toml, GitHub Actions)
  • Manage CI/CD workflows with configuration-driven GitHub Actions
  • Handle system dependencies declaratively in pyproject.toml
  • Migrate legacy projects from setup.cfg to modern formats
  • Debug CI failures with automated diagnostics

Installation

pip install wads          # light core: config reading + templating engine
pip install wads[create]  # full project-creation / publishing toolchain
pip install wads[all]     # create + docs

wads ships a light core (just enough to read [tool.wads.ci] / package.json config and run the templating engine — handy in CI) plus a heavier create extra (requests, build, wheel, ruamel.yaml) for scaffolding and publishing. Use wads[create] (or wads[all]) when running populate/pack to create or publish projects.

Quick Start

Create a New Project

populate my-project --root-url https://github.com/user/my-project
cd my-project

This creates a complete project structure with:

  • pyproject.toml (modern build configuration)
  • README.md, LICENSE, .gitignore
  • Package directory with __init__.py
  • GitHub Actions CI/CD workflow (optional)

Add a frontend component for JS/TS parts (optional)

Python projects often ship a frontend component (a widget, a browser UI, a TypeScript library). populate --frontend <profile> adds a parametrized NPM CI alongside the Python one, following the same "config-file-driven, fixed-workflow" model. Pick one or more profiles:

Profile Adds Subdir CI
js package.json (npm) js/ single-package npm-ci.yml
ts package.json + tsconfig.json + src/index.ts (tsup build, vitest) ts/ single-package npm-ci.yml
ts-monorepo pnpm workspace root + turbo.json + an example packages/core ts/ matrixed npm-ci-monorepo.yml 
# A single TypeScript component:
populate my-project --root-url https://github.com/user/my-project --frontend ts

# Several components at once — each in its own subdir, no workflow collision:
populate my-project --root-url https://github.com/user/my-project --frontend js,ts

--with-npm is kept as a back-compat alias for --frontend js.

Each component gets:

  • a package.json with a namespaced "wads" config block (wads.ci.*) controlling node versions, lint/test/build commands, and publishing — analogous to [tool.wads.ci] in pyproject.toml;
  • a path-filtered .github/workflows/npm-ci[-<subdir>].yml stub calling wads's reusable NPM workflow (the js component keeps the bare npm-ci.yml; every other component gets npm-ci-<subdir>.yml, so multiple components never collide).

Validation runs on every push/PR; publishing is opt-in. It publishes only when wads.ci.publish.enabled is true and the commit message contains the marker [publish-npm] (deliberately distinct from the Python side). Publishing uses npm OIDC trusted publishing + provenance by default (no long-lived token). For a single component, customize the subdirectory and package name with --npm-subdir / --npm-package-name.

Package manager: npm or pnpm. The single-package reusable workflow drives npm by default and pnpm when selected — either explicitly via wads.ci.packageManager (or populate --npm-package-manager pnpm) or auto-detected from a pnpm-lock.yaml in the package directory. pnpm consumers should declare a "packageManager": "pnpm@x.y.z" field in their package.json (pnpm's own convention); the CI reads the pnpm version from there. Existing npm consumers are unaffected (no pnpm-lock.yaml → npm). The ts-monorepo profile is pnpm-based by design.

The profile set is extensible: register your own with wads.profiles.register_frontend_profile(...).

Configure CI in pyproject.toml

Edit your pyproject.toml to configure CI behavior:

[tool.wads.ci.testing]
python_versions = ["3.10", "3.12"]
pytest_args = ["-v", "--tb=short"]
coverage_enabled = true
test_on_windows = true

[tool.wads.ci.quality.ruff]
enabled = true

[tool.wads.ci.build]
sdist = true
wheel = true

# Opt-in licence gate over the installed dependency closure (default: off).
# See "Licence Perimeter" under CI Configuration Reference.
[tool.wads.licence]
enabled = false

The default ci.yml is a small stub that calls wads's reusable workflow (i2mint/wads/.github/workflows/uv-ci.yml@master); all behavior is driven by [tool.wads.ci.*] above. Publishing to PyPI happens automatically on your repo's default branch — but only when the Linux test matrix passes.

Configure Secrets (CI environment variables)

If your tests need API keys or other secrets, declare them once and let wads wire up both the GitHub Actions transport and the job environment:

wads-secrets add OPENAI_API_KEY            # env var == GitHub secret name
wads-secrets add HF_TOKEN HF_WRITE_TOKEN    # env var <- differently-named secret
wads-secrets add DATABASE_URL --kind required   # fail CI if the secret is unset
wads-secrets add TEST_LEVEL --variable      # non-sensitive value -> repo variable
wads-secrets list                           # show what's configured

wads-secrets add (a) records the variable in [tool.wads.ci.env], (b) adds its secret to the stub's secrets: list, and (c) runs gh secret set (or gh variable set with --variable) if gh is installed (value taken from $VAR_NAME or --value). Under the hood there are two layers: a transport, where the stub passes secrets to the reusable workflow by name (PYPI_PASSWORD plus each declared one), and an env policy ([tool.wads.ci.env]: required_envvars / test_envvars / extra_envvars / defaults / secret_aliases) that decides which values become job env vars. Each declared name resolves against secrets first, then repository variables (the right home for non-sensitive values); committed constants can go straight into [tool.wads.ci.env].defaults. A required name that resolves to nothing fails the build; an undeclared secret is never written to the environment.

Named secrets must be in the frozen superset in wads.ci_secrets.DEFAULT_CI_SECRETS, or GitHub rejects the workflow at parse time; wads-secrets and wads-migrate warn about such names. The opt-in alternative, wads-migrate ci-to-stub --transport json, passes the repo's whole secrets context as one WADS_CI_SECRETS_JSON secret, so any name works. It is not the default because GitHub's malicious-workflow scanner holds its runs on new repositories: every run ends action_required with zero jobs and no log (#74). Re-rendering an existing stub keeps whichever transport it already uses.

Declare System Dependencies

Need ffmpeg, ODBC drivers, or other system packages in CI? Declare them in pyproject.toml:

[tool.wads.ops.ffmpeg]
description = "Multimedia framework for video/audio processing"
url = "https://ffmpeg.org/"

check.linux = "which ffmpeg"
check.macos = "which ffmpeg"

install.linux = "sudo apt-get install -y ffmpeg"
install.macos = "brew install ffmpeg"
install.windows = "choco install ffmpeg -y"

note = "Required for audio processing features"

The install-system-deps action in your CI workflow will automatically install these.

Core Features

1. Project Setup (populate)

Create new Python projects with modern best practices:

# Basic usage
populate my-project

# With custom settings
populate my-project \
  --root-url https://github.com/myorg/my-project \
  --description "My awesome project" \
  --author "Your Name" \
  --license apache

Options:

  • --root-url: GitHub repository URL
  • --description: Project description
  • --author: Author name
  • --license: License type (mit, apache, bsd, etc.)
  • --keywords: Comma-separated keywords
  • --overwrite: Files to overwrite if they exist

Tip: Configure defaults in wads_configs.json or use WADS_CONFIGS_FILE environment variable to point to your custom config.

2. Package and Publish (pack)

Build and publish packages to PyPI:

# See current configuration
pack current-configs

# Increment version and publish
pack go .

# Or step-by-step
pack increment-configs-version
pack run-setup
pack twine-upload-dist

3. Migration Tools (wads-migrate)

Migrate legacy projects to modern format:

# Migrate setup.cfg to pyproject.toml
wads-migrate setup-to-pyproject setup.cfg -o pyproject.toml

# Migrate old CI workflow to new format
wads-migrate ci-old-to-new .github/workflows/old-ci.yml -o .github/workflows/ci.yml

Python API:

from wads.migration import migrate_setuptools_to_hatching, migrate_github_ci_old_to_new

# From setup.cfg file
pyproject_content = migrate_setuptools_to_hatching("setup.cfg")

# From setup.cfg dict
config = {"metadata": {"name": "myproject", "version": "1.0.0"}}
pyproject_content = migrate_setuptools_to_hatching(config)

# Migrate CI workflow
new_ci = migrate_github_ci_old_to_new(".github/workflows/ci.yml")

4. CI Debugging (wads-ci-debug)

Diagnose and fix GitHub Actions CI failures:

# Analyze latest failure
wads-ci-debug myorg/myrepo

# Analyze specific run
wads-ci-debug myorg/myrepo --run-id 1234567890

# Generate fix instructions
wads-ci-debug myorg/myrepo --fix --local-repo .

The tool will:

  • Fetch CI logs from GitHub
  • Parse test failures and errors
  • Identify root causes
  • Generate fix instructions with file locations and suggested changes

On-demand CI

For repos that should not run CI on every push (Actions minutes are free on public repos but metered on private ones), set:

[tool.wads.ci.trigger]
mode = "on-demand"          # default "auto": every push and PR, as before
run_ci_marker = "[run ci]"  # the default marker

Nothing runs unless asked. CI runs only when the commit subject (its first line) contains the marker, or when someone starts it by hand (gh workflow run ci.yml --ref <branch>, or the Actions tab), which is always allowed. Tests, publishing and GitHub Pages all obey the same gate.

git commit -m "Fix the parser [run ci]"   # runs CI
git commit -m "Fix the parser"            # runs nothing

A marker quoted only in a squash-merged PR body does not count. The stub's zero-cost pre-filter lets that run start (one short setup job), and the reusable workflow checks the extracted subject before running anything else.

Worth knowing before you flip:

  • A manual run on the default branch releases when publishing is enabled, exactly like a [run ci] push. To test without releasing, run it on another branch, or use wads ci-local.
  • The [publish] marker needs [run ci] in the same subject. This applies to repos with publishing disabled, where [publish] forces a release.
  • Only the pushed head commit's subject counts, not earlier commits in the same push.
  • On-demand stubs drop the pull_request trigger. A branch-protection rule that requires CI status checks will block PRs until those checks are removed from the rule.

Flip a repo in one idempotent command. It sets on-demand, python_versions = ["3.12"] and test_on_windows = false, and re-renders the stub, keeping its pin and secrets transport:

wads-migrate ci-on-demand --dry-run   # show the diff, write nothing
wads-migrate ci-on-demand --commit    # apply and commit; then `git push`

Do locally what CI would have done, from the same [tool.wads.ci] config:

wads ci-local             # lint (ruff), tests (a fresh uv venv per python_versions entry), build
wads ci-local --dry-run   # print the plan, run nothing
wads ci-local --publish   # instead of the plain build: format, bump version (isee), build, PyPI upload, commit, tag, push

--publish refuses up front on a dirty tree, off the default branch, behind origin, or without a PyPI token ($PYPI_PASSWORD, then $UV_PUBLISH_TOKEN, then ~/.pypirc [pypi] with username = __token__). wads ci-local needs uv and git on PATH, and does not install [tool.wads.ops.*] system packages.

To go back, set mode = "auto" and run wads-migrate ci-to-stub. Workflow files other than ci.yml (cron jobs, Pages, npm) are not governed by the trigger; ci-on-demand lists any that still run unasked.

CI Configuration Reference

Wads uses pyproject.toml as a single source of truth for CI configuration. Here's what you can configure:

Install Extras

New projects default to installing the dev extra in CI (.[dev]), matching the dev extra the template declares under [project.optional-dependencies] -- so test-time tooling (pytest, ruff, ...) actually gets installed instead of silently never running. Override or opt out explicitly:

[tool.wads.ci.install]
extras = "dev"          # or a list, e.g. ["dev", "test"]; "" installs core deps only

Python Versions and Testing

[tool.wads.ci.testing]
python_versions = ["3.10", "3.11", "3.12"]  # Test matrix
pytest_args = ["-v", "--tb=short"]           # Pytest arguments
coverage_enabled = true                      # Enable coverage
coverage_threshold = 80                      # Minimum coverage %
exclude_paths = ["examples", "scrap"]        # Paths to exclude
test_on_windows = true                       # Run Windows tests
windows_blocking = false                     # true = a failing Windows leg reddens the run

windows_blocking defaults to false: the Windows job is informational (continue-on-error), which is how Windows-only defects — backslash path separators, locale-decoded read_text() — merge behind a green tick. Set it to true to make the run red. It reddens the run, not the release: publish does not depend on the Windows job.

This knob only reaches the uv-based CI (github_ci_uv.yml / the reusable uv-ci.yml workflow, the default for new projects). The legacy github_ci_publish_2025.yml template and its actions/windows-tests action still hardcode continue-on-error: true unconditionally — a failing Windows leg there stays informational regardless of this setting. Repos on the legacy template should migrate to the uv stub (wads-migrate ci-to-stub) to pick it up (see #90).

Code Quality Tools

[tool.wads.ci.quality.ruff]
enabled = true
# line_length = 88

[tool.wads.ci.quality.mypy]
enabled = false
# strict = true

Custom Commands

[tool.wads.ci.commands]
pre_test = [
    "python scripts/setup_test_data.py",
]
post_test = [
    "python scripts/cleanup.py",
]

Build and Publish

[tool.wads.ci.build]
sdist = true
wheel = true

[tool.wads.ci.publish]
enabled = true  # Publish to PyPI on main/master

Licence Perimeter (wads-licence-check)

Fails the build when the installed dependency closure carries a licence the project's policy forbids — copyleft (GPL / AGPL / LGPL), or source-available / non-commercial (SSPL, BUSL, Elastic-2.0, RAIL, CC-BY-NC).

It walks the transitive closure, not just the declared list, because that is where the exposures actually hide, and it reads a distribution's declaration through a precision ladder — PEP 639 License-Expression, then the License :: trove classifiers, then the first line of the free-text License field. No single field is enough: click declares an expression and no classifiers, i2 declares neither and only a free-text field, and a whole-field substring scan flags BSD-3-Clause numpy as copyleft because its field carries an LGPL URL for a vendored notice.

Run it anywhere:

wads-licence-check                                    # this project
wads-licence-check path/to/project --json             # for a fleet sweep
wads-licence-check . --python .venv/bin/python        # read another env

The CI gate is opt-in. A repo that declares nothing sees no change:

[tool.wads.licence]
enabled = true                    # default false: opt in per repo
include-extras = []               # [] = hard deps only; ["*"] = every extra
unknown-is-failure = true         # a blank licence field is *unaudited*, not fine
unclassified-is-failure = false   # e.g. MPL-2.0: reported, does not fail

[tool.wads.licence.exceptions]
certifi = "MPL-2.0 — weak, file-level, over an unmodified CA bundle. Audited 2026-08."

That is the whole configuration most repos need. The allowed / forbidden pattern lists are deliberately absent from it: they REPLACE the defaults, they do not extend them, so writing them out by hand is a narrowing unless the list is a superset of what ships. If you do set them, write them as TOML literal strings — single quotes — because a basic string processes escapes and turns "\bGPL" into a backspace character followed by GPL, which matches nothing:

[tool.wads.licence]
# Start from wads.licence_check.DFLT_FORBIDDEN and ADD, rather than replacing:
forbidden = [
  '\bAGPL',
  '\bAffero\b',
  '\bGPL(?![\w.+-]*\s+with\b)',
  '\bGNU General Public\b',
  '\bLGPL',
  '\bLesser General Public\b',
  '\bLibrary General Public\b',
  '\bNethack General Public\b',
  '\bEUPL\b',
  '\bBusiness Source\b',
  '\bBUSL\b',
  '\bSSPL\b',
  '\bElastic[- ]?(2\.0|License|v2)\b',
  '(?:\b|-)(?:open)?rail(?:-m)?\b',
  '\bCC[- ]BY[- ]NC\b',
  '\bNon[- ]?Commercial\b',
  '\bProprietary\b',
  '\bYourOwnAddition\b',   # the point: ADD to the defaults, never restate a subset
]

Exceptions take either the terse map above or an array-of-tables, which is the shape to reach for when the decision needs a record behind it:

[[tool.wads.licence.exceptions]]
dependency = "PyGithub"
licence = "LGPL (classifier only; no SPDX expression published)"
scope = "core"
decided = "2026-08-30"
decided_in = "https://github.com/thorwhalen/hubcap/issues/10"
reason = """
Accepted, not removable: PyGithub's objects are this package's values, so there
is no honest "core without it" to install. Consumed by ordinary import, neither
vendored nor patched, so the LGPL relink freedom is intact.
"""

Note the table lives under [tool.wads], not [tool.wads.ci]: the policy is a fact about the package, and the tool is useful outside CI. Only enabled is a CI concern.

Exit codes are 0 (holds), 1 (breached) and 2 (the tool could not run — bad config, unreadable environment, a policy that cannot detect).

The self-check

Every run first proves the live policy still catches known-copyleft declarations and still clears known-permissive ones, and refuses to report at all if it cannot — a detector nobody has demonstrated is a detector nobody has checked.

The bar is per licence FAMILY, caught whole or not at all. Permitting a family outright is a coherent stance (LGPL for dynamically linked libraries is the usual one) and stays expressible; it is then named in every run's output, so "PERIMETER HOLDS" is never read as "there is no LGPL in here". Catching part of a family is refused, because it is never a stance — it is the bug. A gate whose LGPL pattern ended in \b caught the legacy GNU Library or Lesser classifier and missed LGPLv2, LGPLv2+, LGPLv3 and LGPLv3+, i.e. every modern spelling, while reporting itself as working.

What counts as a failure

forbidden and a blank declaration (unknown-is-failure, default on) fail, and so does a declared dependency that is not installed in the environment being read: that is a piece of the perimeter nobody looked at, and it is the same confident-green failure as reading the wrong environment. The one exception is a requirement gated on an environment marker (tomli; python_version < "3.11"), which is reported as NOT APPLICABLE and does not fail.

For the same reason the tool refuses to run at all on a project whose [project].dependencies is absent or listed in dynamic — an empty closure it could not read is not an empty closure. Write dependencies = [] if a project genuinely has none.

System Dependencies

System dependencies are declared using the [tool.wads.ops.*] format and automatically installed in CI via the install-system-deps action.

Format:

[tool.wads.ops.{package-name}]
description = "Description of the package"
url = "https://package-homepage.com"

# Check if already installed (exit code 0 = present)
check.linux = "which package-name"
check.macos = "brew list package-name"
check.windows = "where package-name"

# Install commands (string or list of strings)
install.linux = "sudo apt-get install -y package-name"
install.macos = "brew install package-name"
install.windows = "choco install package-name -y"

# Optional metadata
note = "Additional installation notes"
alternatives = ["alternative-package"]

Real-world example (ODBC drivers):

[tool.wads.ops.unixodbc]
description = "ODBC driver interface for database connectivity"
url = "https://www.unixodbc.org/"

check.linux = "dpkg -s unixodbc || rpm -q unixODBC"
check.macos = "brew list unixodbc"

install.linux = [
    "sudo apt-get update",
    "sudo apt-get install -y unixodbc unixodbc-dev"
]
install.macos = "brew install unixodbc"

note = "On Alpine: apk add unixodbc unixodbc-dev"
alternatives = ["iodbc"]

See misc/docs/SYSTEM_DEPENDENCIES.md for comprehensive examples.

Documentation

Troubleshooting

Version Tag Misalignment

If PyPI publishing fails with "appears to already exist":

WARNING  Skipping mypackage-0.1.4-py3-none-any.whl because it appears to already exist

This means your git tags are misaligned with the version in pyproject.toml.

Fix:

  1. Check the current PyPI version: https://pypi.org/project/your-package/
  2. Update version in pyproject.toml to a higher number
  3. Create and push git tag: 
    git tag 0.1.5
git push origin 0.1.5

CI Failures

Use wads-ci-debug to analyze failures:

wads-ci-debug myorg/myrepo --fix

Common issues:

  • Missing system dependencies → Add to [tool.wads.ops.*]
  • Python version incompatibilities → Check python_versions in [tool.wads.ci.testing]
  • Test failures → Review generated fix instructions

For carbon-based contributors

Everything above is for users of wads, human or not. This part is for working on wads itself.

Dev setup. The test suite scaffolds and builds packages, so it needs the create extra:

uv venv && . .venv/bin/activate
uv pip install -e ".[create,docs,skills,test]"

Run the tests the way CI does. CI calls pytest with no path, so testpaths = ["wads"] collects both wads/tests and every doctest in the package:

python -m pytest --doctest-modules -o doctest_optionflags='ELLIPSIS IGNORE_EXCEPTION_DETAIL' --ignore=examples --ignore=scrap

CI tests Python 3.10 and 3.12; run 3.11 as well before merging, because a dataclass default once broke there and nowhere else (#100).

Build the docs with pip install -e ".[docs]" and then epythet build.

Why it is built this way. wads is a foundation package. Its reusable workflow and actions run from @master in every wads-managed repo, and every merge to master publishes a release to PyPI. So changes to .github/workflows/uv-ci.yml, actions/* or the stub template are fleet-wide changes: they come with tests (the push-back script, for example, is exercised against real throwaway git repos), and the populate output is pinned by golden files. .claude/CLAUDE.md has the design rationale, and the wads-dev-workflow skill has the maintainer checklist.

Contributing and questions. Open an issue or a pull request at github.com/i2mint/wads.

License

Apache Software License 2.0

Links

About

Populating, packaging and pip-publishing projects.

Resources

Readme
Apache-2.0 license
Activity
Custom properties

Stars

0 stars

Watchers

3 watching

Forks

1 fork
Report repository

Releases

Packages

Contributors

Languages