Skip to content

Build targets: script and native binaries next to the zipapp, on pybash 0.4 - #58

Merged
jaykv merged 7 commits into
mainfrom
feat/build-targets
Oct 12, 2026
Merged

jaykv merged 7 commits into
mainfrom
feat/build-targets

Conversation

@jaykv

@jaykv jaykv commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Stack 7 (one more to follow). Base: docs-overhaul. Merge in order; each PR shows only its own changes.

Summary

cli build gains two targets next to the shiv zipapp, picked with a flag, and cliffy moves to pybash 0.4 for $ / > lines in run blocks.

cliffy now requires pybash>=0.4.0, which is on PyPI (jaykv/pybash#11).

Build targets

cli build <manifest-or-loaded-cli> --target zipapp|script|native (default zipapp). The manifest does not change between targets. Blueprint-spec YAMLs work with all three.

Target Produces The other machine needs The build machine needs
zipapp dist/<name>, a shiv zipapp a compatible Python nothing extra
script dist/<name>.py, executable, uv run --script shebang and a PEP 723 dependency block uv, or Python 3.11+ with the pinned framework installed nothing extra
native dist/<name>, one compiled executable built with PyApp nothing cargo, uv, network, about 700 MB of temp space

Measured with examples/hello.yaml on Apple Silicon:

Target Size Build time First run Later runs
zipapp 1.8 MB 1.7 s 0.19 s 0.05 s
script 1-12 kB 0.4 s 0.24 s (warm uv cache) 0.05 s
native about 30 MB 54-60 s 1.5 s (unpacks about 110 MB) 0.05 s

How the native target works: it installs a relocatable CPython with uv, builds a wheel of the generated CLI, installs the wheel and its dependencies into that Python, archives it, and compiles PyApp 0.30.0 with the archive embedded (PYAPP_DISTRIBUTION_PATH, PYAPP_SKIP_INSTALL=1, PYAPP_FULL_ISOLATION=1). The binary therefore needs no network and no Python on first run; it only unpacks itself.

Built CLIs depend on the framework and the manifest's requires only. When a run block uses shell operators, the small pybash helper is pasted into the built file instead of imported, for every target.

In builder.py the targets are small classes behind one interface (BuildTarget, registered in TARGETS), so a fourth target is one class.

pybash 0.4

  • Run blocks can use &&, ||, ;, |, |&, redirects (2>, &>, 2>&1), $(command) capture, any expression in {{ }}, and injection-safe interpolation in > shell lines.

  • import pybash.runtime as _pybash is added to a generated CLI only when its code uses an operator. Plain $ ls stays a bare subprocess call.

  • A bad $ line, or a Python syntax error in a run block, is reported against the manifest with no traceback:

    ~ error in deploy.yaml: command `ship`, run, line 2, column 11: running in the background with & is not supported in a $ command; use a > shell command
        $ sleep 5 &
                  ^
    

Changes in behaviour

  • Run blocks are now checked for Python syntax at load time. Three existing test inputs used run blocks that were not valid Python (run: echo "Hello"); they now use $ echo .... No assertion was changed.
  • --python has a per-target default, and is rejected for --target native.
  • The build output line now states what the artifact needs and how to run it.
  • pybash 0.4 passes arguments exactly as a shell would; see the migration notes in the docs for the quoting differences from 0.3.
  • SOUL.md: the status line about unshipped targets is removed.

How verified

  • Python 3.11 and 3.15 on macOS arm64: ruff format --check, ruff check, mypy cliffy --strict, mypy tests clean; pytest -n auto 548 passed, 1 skipped (the opt-in native test). make docs-check and mkdocs build --strict pass.
  • script: an end-to-end test builds an example and runs it through uv run, through its shebang and through the current interpreter, and checks that neither cliffy nor pybash is in its environment.
  • native: real builds of examples/hello.yaml and the blueprint spec examples/blueprints/text-tools.yaml. The binary was run with env -i PATH=/bin HOME=<empty dir> inside a sandbox that denies network access, and printed its output. The opt-in test test_native_build_runs_in_an_empty_environment (CLIFFY_TEST_NATIVE=1, needs cargo and uv) passed on 3.11 and 3.15.
  • Not verified: the native target on Linux (install paths in the guide come from PyApp's docs). It assumes bin/python3, so Windows is not supported.

Open items

  • examples/generated/*.py are not regenerated and still show the old pybash formatting.
  • Every native build is a full compile of about a minute: cargo does not notice a change in PYAPP_* variables, so each build uses its own target directory.
  • The native binary has no self remove; users delete PyApp's data directory to uninstall.
  • The embedded Python is not trimmed (pip is still inside), and the PyApp version is a constant.
  • The script end-to-end test needs uv on PATH and network or a warm uv cache.

🤖 Generated with Claude Code

Summary by Sourcery

Expand cli build beyond zipapps and adopt pybash 0.4 for richer, safer shell-aware run blocks.

New Features:

  • Add zipapp, script, and native build targets selected through cli build --target.
  • Support expanded shell scripting with pybash 0.4, including command chaining, pipelines, redirects, captures, interpolation, and shell-line syntax validation.

Bug Fixes:

  • Report invalid shell commands and Python syntax errors with manifest locations instead of unhandled tracebacks.
  • Embed the pybash runtime in built CLIs when required so artifacts do not depend on cliffy or pybash at runtime.

Enhancements:

  • Refactor builds around a common target interface and improve build output with artifact requirements, size, and execution guidance.
  • Apply per-target interpreter defaults and reject interpreter overrides for native builds.

Build:

  • Implement native artifact creation using standalone Python, wheel packaging, uv, and PyApp.

Documentation:

  • Document build target capabilities, runtime requirements, native build prerequisites, and pybash 0.4 shell behavior and migration notes.

Tests:

  • Add comprehensive coverage for all build targets, standalone artifacts, native build orchestration, shell operators, syntax errors, and isolated end-to-end execution.

Chores:

  • Upgrade the pybash dependency to version 0.4.0 and update the lockfile.

jaykv and others added 5 commits October 12, 2026 11:59
pybash 0.4.0 is not on PyPI yet, so uv resolves it from GitHub. The source
override has to go before cliffy is released. The warning filter for the old
pybash's re.sub() call is no longer needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Shell lines are translated with runtime=pybash.RUNTIME_NAME. The generated
code imports pybash.runtime as _pybash only when a line needs it (operators,
values in > lines); plain commands stay bare subprocess calls.

A shell line pybash cannot translate, or a Python syntax error in a run
block, callback or function, now stops with the manifest file, the script,
the line and the column instead of a traceback or a broken CLI. Three tests
loaded run blocks that were not Python (echo without $); their inputs now
say what they meant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Operators, redirects, $( ) capture, f{ } formatting, quoting of values in >
lines, {{! }}, continuation lines and comments, each with a tested snippet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A build target is a small class behind one interface; zipapp (shiv) stays
the default. script writes one .py file with a uv shebang and PEP 723 inline
metadata. native installs the CLI as a wheel into a standalone Python,
embeds it in PyApp and compiles one executable that needs nothing installed.

All targets paste the pybash helper into the code when the CLI uses shell
operators, so a built CLI depends on its framework and its requires only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The building guide is organized around zipapp, script and native with a
comparison table of measured sizes and start times. Adds the changelog and
migration entries for the targets and for pybash 0.4, and removes the status
line from SOUL.md now that all three targets exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 12, 2026

Copy link
Copy Markdown

Sorry @jaykv, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 7 days by commenting @sourcery-ai review. Upgrade to get a review now.

@coderabbitai

coderabbitai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2149b043-1130-490b-838f-194f6169c0b3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Oct 12, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR introduces pluggable zipapp, script, and native build targets, embeds the pybash 0.4 runtime when generated CLIs need advanced shell features, validates run-block syntax early with manifest-specific diagnostics, and updates tests, dependency resolution, and documentation for the new distribution and shell semantics.

Sequence diagram for native CLI build

sequenceDiagram
    participant User
    participant Cliffy
    participant UV
    participant Cargo
    participant PyApp
    User->>Cliffy: cli build --target native
    Cliffy->>UV: uv python install
    Cliffy->>UV: uv build --wheel
    Cliffy->>UV: uv pip install wheel and requirements
    Cliffy->>Cliffy: archive Python distribution
    Cliffy->>Cargo: cargo install pyapp
    Cargo->>PyApp: compile with embedded archive
    PyApp-->>Cliffy: native executable
    Cliffy-->>User: artifact path and run notes
Loading

Sequence diagram for embedding the pybash runtime in built CLIs

sequenceDiagram
    participant Manifest
    participant Transformer
    participant PyBash
    participant Builder
    participant Target
    Manifest->>Transformer: load and generate CLI
    Transformer->>PyBash: transform_script(run_block)
    PyBash-->>Transformer: generated Python and runtime references
    Transformer->>Builder: generated CLI code
    Builder->>PyBash: runtime_source()
    PyBash-->>Builder: inline runtime source
    Builder->>Target: build standalone code and requirements
    Target-->>Manifest: packaged CLI without cliffy or pybash dependency
Loading

Flow diagram for early run-block validation

flowchart TD
    Load[Load manifest] --> Transform[transform_script]
    Transform --> Parse[ast.parse generated Python]
    Parse --> Valid[Generate CLI]
    Parse --> Invalid[ScriptError with manifest, line, column, and caret]
    Invalid --> Stop[Report error and stop loading]
Loading

File-Level Changes

Change Details Files
Adds a target abstraction allowing cli build to produce zipapps, dependency-declaring scripts, or self-contained native executables.
  • Introduces BuildJob, BuildTarget, target registry, and target-specific result metadata.
  • Adds PEP 723 script generation with executable shebangs and per-target interpreter defaults.
  • Builds native artifacts by preparing a standalone Python, installing a wheel, and embedding it in a PyApp launcher.
  • Adds target selection, native-tool validation, --python restrictions, and richer build output.
  • Covers target behavior, reproducibility, failure reporting, blueprint inputs, and script/native end-to-end execution.
cliffy/builder.py
cliffy/cli.py
tests/test_build_targets.py
README.md
docs/blueprints.md
docs/guides/building.md
docs/quickstart.md
docs/reference/cli.md
scripts/docs.py
Upgrades shell-script translation to pybash 0.4 and makes generated CLIs self-contained when advanced shell operators are used.
  • Translates shell lines through the new pybash runtime and supports operators, capture, interpolation, redirects, chaining, and safer shell-line quoting.
  • Imports the runtime only when generated code needs it, then pastes its source into built artifacts instead of requiring pybash or cliffy.
  • Validates translated run blocks as Python during manifest processing and reports manifest-oriented line/column errors without tracebacks.
  • Updates shell test execution, callback/function handling, migration documentation, examples, and shell coverage.
cliffy/parser.py
cliffy/commander.py
cliffy/commanders/click.py
cliffy/commanders/typer.py
cliffy/transformer.py
cliffy/tester.py
cliffy/helper.py
tests/test_shell.py
tests/test_cli.py
tests/test_commander.py
docs/guides/shell-commands.md
docs/project/migration.md
docs/project/changelog.md
docs/snippets/shell.yaml
Moves the project dependency and lock configuration to the unreleased pybash 0.4 revision.
  • Raises the pybash requirement to 0.4.0 and pins dependency resolution to the specified Git revision through uv sources.
  • Removes the obsolete warning suppression associated with pybash 0.3.5.
pyproject.toml
uv.lock
Updates product and project documentation to describe the new build targets and shipped shell-runtime behavior.
  • Reframes build output as a choice of artifact rather than a zipapp-only workflow.
  • Documents native platform/tooling constraints, runtime extraction, script metadata, and shell quoting migration behavior.
  • Removes the outdated unshipped-target status from project guidance.
README.md
SOUL.md
docs/index.md
docs/installation.md
docs/project/changelog.md
docs/guides/building.md
docs/guides/shell-commands.md
docs/project/migration.md
scripts/docs.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

jaykv and others added 2 commits October 12, 2026 12:28
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pybash 0.4.0 is published, so the temporary git source override is no
longer needed and the lockfile resolves it from the index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jaykv
jaykv changed the base branch from docs-overhaul to main October 12, 2026 21:23
@jaykv
jaykv merged commit ded5e78 into main Oct 12, 2026
7 checks passed
@jaykv
jaykv deleted the feat/build-targets branch October 12, 2026 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant