Skip to content

Latest commit

 

History

History
217 lines (181 loc) · 21.4 KB

roadmap.md

File metadata and controls

217 lines (181 loc) · 21.4 KB

Roadmap

Agents: start with "Next up". It's the ordered work list. Take the first unfinished item, check its ADR, and mark it done here (with a link to the findings or commit) when it's finished. Everything below "Next up" is a possibility, not a commitment: each direction needs its own ADR before work starts, and everything has to respect the accepted ADRs (in particular ADR-0002 on scope and ADR-0006 on staying out of uv's territory).

Next up

In order. Take the first open item that isn't marked "(the owner)". Decisions need an ADR; one the owner hasn't confirmed is marked Proposed.

Focus (the owner, 2026-10-09): more kinds of projects in, more kinds of outputs out. Recipes for particular hosts (MCP servers and the like) are niche; don't add them.

  1. Skill bundles under GitHub's 100 MB per file done 2026-10-09 (ADR-0045): --split 100MB writes a bundle that doesn't fit as a small .pyz and a folder of parts; CI runs a split skill on three OSes.
  2. Standalone executables done 2026-10-09 (ADR-0047, findings): --format exe, scie-jump + a python-build-standalone interpreter + the .pyz, built for any platform from any machine; pex --scie measured. Unsigned (macOS can't sign this format); a signed launcher of our own is the next step only if someone needs notarized binaries.
  3. One literal .py file output done 2026-10-09 (ADR-0046): --format py, a readable # /// bundleup header, the loader, the bundle as base64 comment lines; the gauntlet runs every project as one (--tool bundleup-py).
  4. Pilot the founding skill with colleagues (the owner): two or three people on macOS (with and without the developer tools), Windows and Linux install the skill and use it; their problems become the next items. The fifth review's top recommendation: five reviews are no substitute for users.
  5. Owner review of the Proposed ADRs done 2026-10-08: 0023-0026, 0033-0038 and 0040 accepted; 0035 with downloads announced.
  6. --smoke done 2026-10-08 (ADR-0042): runs the finished bundle once in a fresh home folder, offline on macOS and Linux.
  7. Decide how child processes see the bundle decided 2026-10-08 (the owner, on the agent's recommendation): keep the sitecustomize approach (ADR-0027, 0037, 0040, all passing); revisit a venv in the cache (as pex --venv does) if real users hit child-process trouble.
  8. Decide whether a build may download an interpreter by default decided 2026-10-08: yes, when --python 3.X asks for an uninstalled version, announced and off with UV_PYTHON_DOWNLOADS=never (ADR-0035); older syntax in a range is checked in-process with ast.parse(..., feature_version=...) (ADR-0050).
  9. Decide CI's shape decided 2026-10-08: the full matrix stays on every push (it's free for a public repository and caught real bugs); agents push in batches instead.
  10. A noexec cache done 2026-10-09: native bundles skip noexec cache folders; Linux CI's gauntlet mounts one (noexec-cache).
  11. check --audit removed 2026-10-14 (ADR-0050, superseding ADR-0043): live PyPI auditing belongs to dedicated tools (pip-audit).
  12. Describe a target environment as data removed 2026-10-14 (ADR-0050, superseding ADR-0044): [tool.bundleup] and recipes cover destination settings directly.
  13. MCP servers dropped 2026-10-09 (the owner): too niche. The recipe, the CI jobs and the test server were removed; the focus is taking in more kinds of projects and producing more kinds of outputs.
  14. One environment, each platform's format done 2026-10-09: recipe for Docker (tested in CI, read-only and noexec included), E2B, Modal and setup scripts.
  15. Release 0.1.0 done 2026-10-13 (docs/releasing.md, CHANGELOG, release.yml): published 0.1.0 to PyPI via Trusted Publishing and tagged v0.1.0.

Frozen until 0.1 has users: named targets and nightly automation; new formats only when the owner asks (they asked for --split, py and exe on 2026-10-09). Deferred: faster warm rebuilds (cache compressed zip members per wheel); packages a target already provides (would break "the lock decides"); agent triage of corpus-failure issues (ADR-0017, by the owner); Lambda layers; escape hatches (extra files, external dependencies).

Done

Details are in the linked ADRs and findings, and in git history.

# Item Done Where
1 Engineering tooling: Ruff, pyright, hooks 2026-10-05 ADR-0015
2 CI on 4 OSes: checks, tests, gauntlet 2026-10-05 ci.yml, ADR-0017
3 Owner review of ADRs 0010, 0011, 0016 (verbs: bundleup build) 2026-10-05 ADR-0016
4 Correctness: lock and RECORD checks, manifest + verify, matches-venv, nightly smoke test, real projects' test suites 2026-10-06 ADR-0019, findings
5 CLI follows the style guide; typed library API 2026-10-05 cli-style-guide.md, ADR-0018
6 Nightly corpus with corpus-failure issues (agent triage deferred) 2026-10-05 corpus.yml, ADR-0017
7 Cross-target builds 2026-10-05 ADR-0014
8 Runtime hardening: unpack lock, isolation, cache list/clean 2026-10-05 ADR-0021, ADR-0022
9 Faster builds: parallel zip, bytecode cache 2026-10-05 ADR-0020
10 bundleup check; wheel executables and .pth files 2026-10-05 ADR-0023, ADR-0024
11 --format dir/lambda, presets, bundleup targets (presets replaced by recipes and --max-size, ADR-0039) 2026-10-06 ADR-0025
12 pylock.toml input 2026-10-06 ADR-0026
13 The independent review's bugs: child-process leak, lockfiles, libc/macOS checks, claude-api preset, RECORD paths, and more 2026-10-07 ADR-0027, ADR-0028, ADR-0029, review
14 Pure-Python bundles run on a range of Python versions 2026-10-07 ADR-0030
15 Wheel coverage from the lock, for any number of targets 2026-10-07 ADR-0031
16 _build.py split by step; shared helpers 2026-10-07 src/bundleup/
18 Re-benchmark against pex's fastest configuration: builds 1.1-2.4× faster, start-up far ahead 2026-10-07 findings
19a [tool.bundleup] configuration 2026-10-07 ADR-0032
20 The second review's fixes: --locked by default, cache-root trust, runtime variables, range check, any-OS pure bundles, Skills size limit 2026-10-07 findings, ADR-0033, ADR-0034
22 The third review's fixes: children activate only for bundle code, exact range check, re-run with a matching Python, macOS 26 / Windows CPU, in-use copies kept, check --matrix 2026-10-07 findings, ADR-0036, ADR-0037
21 One .pyz for several platforms and Python versions 2026-10-07 ADR-0038
17 Docs match the code: MISSION, vision, README, roadmap, CLAUDE.md 2026-10-07 the review's section 1

Before writing code in an unfamiliar area, look at references.md for projects that solved similar problems.

The common thread: every direction is a form of bundling something for a destination. That's why the name is bundleup and not something tied to zip files or Python (ADR-0009).

What it does, and what people use it for

What it does: bundleup makes self-contained Python files. Your code and all its dependencies go into one .pyz that runs with plain python: no install step, no uv, no network. Lead with that, literally (ADR-0012).

What people use it for (examples, never the definition):

  • Simplicity: one file to hand over or keep, instead of a project plus setup instructions.
  • Easier distribution: internal tools and CLIs for colleagues or customers, scripts shared between machines.
  • Places you can't set things up:
    • AI agents and sandboxes: skills and scripts in sandboxes with no network, such as Claude API Skills (see "Agents" below);
    • serverless: AWS Lambda and similar, where you upload an artifact and can't run installers;
    • CI runners and build machines you don't own;
    • locked-down or air-gapped servers where outbound network or pip install isn't allowed.

Nobody should read the description and conclude "this isn't for me". The engine stays general; destinations are outputs and examples.

Gaps in today's tools, and what answers them

Measured in the baseline against pex, shiv, zipapps and a naive zipapp. Each gap maps to work in this roadmap.

Gap today Evidence Answered by
pex adds ~210 ms to every start unless you know two flags (which then break with the wrong python3 on PATH) baseline: 273 ms vs 59 ms installed venv Fast-by-default loader (Now)
Builds take 1.8–12 s (pex drives an old pip on 3.9) baseline build times uv-driven builds (Now), per-wheel analysis cache (Near)
No tool warns before shipping; every failure appeared at run time baseline: zero build-time warnings bundleup check (Near)
Errors are pip logs or "Pip install failed!" baseline: refusal messages for project 17 Plain-language errors (Now), machine-readable output (Near)
Child sys.executable processes can't see the bundle's packages (pex, shiv) gauntlet 19 Loader sets up child processes (Now)
shiv fails 100% with an unwritable HOME; zipapps writes into the working directory and races on simultaneous first runs baseline hostile conditions Cache fallback chain + atomic extraction (Now, ADR-0005)
A naive zipapp silently drops compiled speedups gauntlet 10 check warnings (Near)
Every tool needed requirements exported and wheels built by hand baseline setup Reads uv.lock / pylock.toml / PEP 723 directly (Now)
No tool targets other machines from one machine with a check that it will work baseline: no cross-platform coverage Multi-platform bundles (Near)
Skill scripts with dependencies can't run in no-network sandboxes: Claude API Skills have "no network access and no runtime package installation", yet the Agent Skills guide recommends uv run round 3 synthesis Target profiles + skill output (Near)

Now: the first version

What MISSION.md defines as done:

  • One command from pyproject.toml + uv.lock / pylock.toml / PEP 723 to a checked .pyz.
  • Runs on the user's Python with no install and no network, native extensions included (extracted to a cache, ADR-0005).
  • Version and platform check at start-up with a plain-language error.
  • Matches pex on the gauntlet; beats it on default start-up time, build time, warnings and error messages (baseline).

Near: make the core deeper

Idea What it is Why
bundleup check done (ADR-0024) The pre-ship analyzer as its own command, runnable in CI on any project: "will this survive bundling?" Useful even to people who bundle with something else. Small today (~440 lines); what's left if uv ships bundling, so worth deepening
Multi-platform bundles done (one per target: ADR-0014; one for several: ADR-0038) One .pyz that runs on several OS/CPU/Python combinations, or one per target from a single machine (--python, --platform) Build once on a Mac, ship to Linux servers. Round 4's #1 priority: it unlocks most strong-fit use cases (ADR-0014, accepted)
Cache override and runtime hardening done (item 8) BUNDLEUP_CACHE, cache order (env → user cache → temp), per-build locks, stale-cache cleanup, isolated sys.path Lambda's read-only filesystem, read-only roots, HPC node-local scratch, 1,000 jobs starting at once
--format dir done (ADR-0025) A vendored directory built for a host application's Python and platform Splunk, QGIS, Maya/Houdini, Azure Functions' .python_packages all hand-roll pip install --target --platform today (ADR-0014, accepted)
Target profiles (built, then replaced by recipes and --max-size: ADR-0039) Named targets for environments with a fixed, known Python and platform, starting with claude-api (CPython 3.11, manylinux x86_64, no network) Known targets make compiled wheels (pydantic, numpy) shippable; the most agent-specific feature (ADR-0013, Proposed)
Skill output (a recipe for now) scripts/<tool>.pyz plus a ready SKILL.md stanza and an honest compatibility line No platform offers skill scripts with dependencies that run offline (ADR-0013, Proposed)
Size and contents report (per package: bundleup check -v) What's in the bundle, what's heavy, why (like webpack-bundle-analyzer / esbuild's metafile) Native wheels dominate size; people need to see it
Python API done (ADR-0018) Call bundleup as a library from uv, Hatch, Pants, CI scripts Be the component others call, the way Vite calls esbuild
Machine-readable output done (--json, schemas) --json for build results and check findings CI systems and agents can act on results without parsing prose
Bundle manifest done (ADR-0019) A list of exactly what's inside each bundle, with hashes and versions Trust and supply-chain review: the reviewed artifact is the executed artifact. Secondary: doesn't address malicious skill instructions
Tested offline guarantee done (every gauntlet run blocks the network on macOS and Linux) State, and test on every gauntlet run, that a bundle never touches the network Makes "runs offline" a promise rather than a hope (the harness already blocks network)
Opt-in pruning Drop whole distributions that are provably unreachable Smaller bundles without the risk of function-level tree-shaking

Middle: the same bundle, different destinations

Destination What we'd produce Why it's a real gap
AWS Lambda / serverless zip done (--format zip, ADR-0049; layers not yet) --format zip: a flat .zip of code, packages and bytecode (not a .pyz: Lambda already unzips, and only /tmp is writable) Most common serverless request; hand-built packages often ship Mac wheels (ADR-0014, accepted)
Container images (recipe and CI test done 2026-10-09; revisit) The 3-line Dockerfile that copies app.pyz onto python:3.x-slim (recipe); later perhaps a daemonless image writer (base image + one layer, as ko and jib do) Round 4 recommends against building images ourselves; the owner wants it revisited (2026-10-08 note)
Skill bundles under GitHub's 100 MB per file done 2026-10-09 (--split, ADR-0045) Layers as separate files beside a small .pyz (or one .pyz per platform and a tiny launcher), so no file is over 100 MB and no Git LFS is needed The founding skill with PyMuPDF is 144 MiB in one file; delivery through git (agent package managers) can't take that without LFS (review)
One literal .py file done 2026-10-09 (--format py, ADR-0046) --format py: a readable header (contents, pinned versions as a PEP 723-style block) and the bundle as base64 with a ~30-line unpacker; python tool.py Goes where only .py files go (gists, agent tools, uploads); +33% size, so for small tools. The owner's idea; note
Standalone executables done 2026-10-09 (--format exe, ADR-0047) One file per OS/CPU that needs nothing installed: scie-jump + python-build-standalone + the .pyz Desktop users without Python, the one big audience a .pyz can't reach (round 4). Unsigned: a launcher of our own (payload inside the executable format, as Node SEA does) is what signing and notarization would need
"No Python installed" answered by --format exe (ADR-0047), which carries the interpreter instead of downloading it A tiny launcher that downloads a Python on first run, then runs the bundle The "user has no usable Python" problem (primer §3); downloading at run time would need the network bundleup promises not to
MCP servers dropped 2026-10-09 (the owner: too niche) A bundled MCP server that starts with python server.pyz (a recipe and CI test existed briefly, commit e1bb71b) MCPB moved Python to a uv server type because vendored compiled dependencies broke per version; a multi-platform .pyz covers that offline (rounds 6 and 7)
Agents as operators (hypothesis) A bundleup skill so an agent can bundle a script it wrote (build where there's network, run in an offline sandbox) Plausible and unserved, but no evidence of demand found yet; validate with users first
Notebooks (docs only) Document the recipe: nbconvert → PEP 723 script → bundleup Magics and display calls break automatic conversion; a recipe is enough

Far: bigger bets

  • Run bundles from a URL: bundleup run https://…/tool.pyz, cached and verified, like npx or uvx but for bundles.
  • Bundles for the browser: packaging Python for Pyodide/WebAssembly, which is painful today.
  • Vendoring for libraries: bundle a library's own dependencies under renamed imports (like Java's Shade) so they can't conflict with the user's versions. Also for --format dir: two plugins in one host application with different versions of a package (note).
  • --strip of known-dead files (C headers, Cython sources, type stubs, test folders): ~1 MB of a 40 MB payload measured; small, so only if users ask.
  • Other languages: skills ship Python and Node scripts. One tool that bundles either (driving esbuild for Node) is plausible; nothing in the name says Python.

Agents (answered by round 3 research)

Agents are a use case, not the category (round 3 synthesis). The one concrete, documented gap is no-network sandboxes, above all Claude API Skills (Python 3.11, Linux x86_64, no network, no installs), where the officially recommended uv run can't work. Elsewhere (cloud coding agents, desktop MCP) the ecosystem already installs dependencies at setup time with uv.

The answer to "what do you mean it's for agents?":

Agent sandboxes are the most common place you can't install anything, so bundleup turns a skill's script and its locked dependencies into one file that runs there with plain python.

Features: target profiles and skill output (Near), compatibility pre-check (part of check), agent-operable CLI (machine-readable output). See ADR-0013 (Proposed).

Not our territory

Package management, task runners, an upgrade command, a watch-mode dev runner: these belong to uv (or are absorbed by it quickly). Competing there contradicts ADR-0006.