Skip to content

verification-notes.md

Latest commit

 

History

History
140 lines (113 loc) · 10.5 KB

verification-notes.md

File metadata and controls

140 lines (113 loc) · 10.5 KB
Raw

Verification notes

Spot-checks run against primary sources while reviewing the four research reports (2026-10-03). The reports were generated by two different deep-research tools; one of them (the "gemini" files) is known to hallucinate, so claims that drive decisions were checked by hand.

Claims checked

Claim Source of claim Result Evidence
uv has an open request for a uv zipapp command (#7419, since Sep 2024) round 1 (both) ✅ Open, no maintainer commitment visible astral-sh/uv#7419
An Astral maintainer said bundling is something they're "interested in doing someday" round-1-compass ❓ Could not find the quote. Treat as unverified same issue
uv's uv bundle / PyInstaller-style request (#5802) is labelled "wish – Not on the immediate roadmap" round-2-compass ✅ Open, labelled wish, no maintainer comments astral-sh/uv#5802
uv's watch-mode request (#9652) is open and "help wanted" both rounds ✅ Open, labels enhancement + help wanted astral-sh/uv#9652
PR #12847 (uv run --watch) is "still unmerged" round-1-compass ⚠️ Inaccurate: the author closed it on 2025-06-22 for lack of review. Nobody is working on it astral-sh/uv#12847
OpenAI announced an agreement to acquire Astral on 2026-03-19 both rounds ✅ Announced; closing (regulatory approval) not confirmed in sources found https://blog.jetbrains.com/pycharm/2026/03/openai-acquires-astral-what-it-means-for-pycharm-users/
Bun was acquired by Anthropic (2025-12-02) and stays MIT-licensed round-2-compass ✅ Confirmed by Bun's own post https://bun.com/blog/bun-joins-anthropic
pex is actively maintained and supports multi-platform PEX files round 1 (both) ✅ 2.103.4 released 2026-10-03; multi-platform documented https://pypi.org/project/pex/
Claude API code-execution containers have no network access early discussion ✅ Per docs: pre-installed packages only, Python 3.11, x86_64 Linux https://platform.claude.com/docs/en/agents-and-tools/tool-use/code-execution-tool
tinyBundle (a single-file Python bundler) was archived early discussion ✅ Archived Oct 2024 https://github.com/jakewdr/tinyBundle

Report quality notes

round-1-landscape-gemini.md

  • Top recommendation (a single .py inliner that never extracts to disk) repeats stickytape's failed approach and cannot work with native extensions, which the same report says must be extracted.
  • Invented history: says tinyBundle "struggled to keep pace with changing wheel metadata specifications". No source.
  • Invented audience sizes ("2.5 million", "4 million", "500,000 library authors"). No sources.
  • Factual errors: cites PEP 515 for entry points (PEP 515 is underscores in numeric literals); PEP 665 rejected "2021" (it was Jan 2022); PEP 723 "ratified in 2023" (accepted Jan 2024); Nuitka "complex commercial licensing" (Apache-2.0 with an optional paid add-on); pex "tightly coupled to Pants" (standalone tool).
  • Hypothesis 5 verdict "confirmed" is wrong: minimum-version checking exists (vermin).
  • Several citations don't support the claims attached to them.

round-2-evolution-gemini.md

  • Its most-cited source, [4], is round-1-landscape-compass.md itself (it was attached as context). Its Python sections therefore restate round 1 rather than independently confirm it.
  • Citation [3] is a 2013 Mozilla post about TowTruck, cited for claims about uv.
  • Says Bun uses a binary bun.lockb; Bun moved to a text bun.lock in 2025.
  • Repeats "PEP 723 (2023)".
  • The ty vs. Pyrefly comparison table (conformance 76% vs 96%, re-check 4.7ms vs 2.38s) has no checkable source. Do not repeat these numbers.

round-1-landscape-compass.md

  • Mostly well-sourced. Errors found: PR #12847 status (above); the "someday" maintainer quote is unverified. Several toolchain rows are explicitly marked "not re-verified".
  • Its central claim — that __file__/metadata assumptions break more bundles than native code does — is argued, not measured. The gauntlet exists partly to test it.

round-2-evolution-compass.md

  • Could not access the attached round-1 report, so it researched from scratch. Its agreement with round 1 is therefore more meaningful than the gemini report's.
  • Node type-stripping "stable" version differs between reports (v24.12 vs v25.2); irrelevant here.
  • Recommends a self-contained-executable output mode. That reintroduces code signing and notarization, the problem that started this project. Treat as out of scope for the core tool.

Round 1b (a second run of the round 1 brief, 2026-10-04)

Claims checked

Claim Source Result Evidence
Astral said bundling is something they're "interested in doing someday" and "not sure zipapp is the ideal format" round-1b-compass ✅ Zanie Blue (Astral member), 2024-10-08; issue still open, labelled enhancement, wish. This resolves the "unverified quote" from round 1 astral-sh/uv#7419

round-1b-landscape-gemini.md

  • Calls shiv "Active"; its last release was Nov 2024 (maintenance mode, as Compass says).
  • Invented audience sizes again: "3 to 4 million", "6 to 8 million", "approximately 2 million", "roughly 1.5 million", "approximately 3 million". No sources.
  • Hypothesis 5 "confirmed" again while ignoring vermin, which checks minimum versions.
  • Says PyOxidizer went on hiatus "in late 2023"; Szorc wrote it "fell into a state of neglect" by early 2023 (status post March 2024).
  • Recommends building in Rust and a downleveler as opportunity #2; both conflict with measured evidence (see the synthesis).
  • First citation is an unrelated agency blog (wajusoft.com).

round-1b-landscape-compass.md

  • pex version slightly stale (2.103.2 vs 2.103.4 on 2026-10-03). Otherwise consistent with earlier verification.
  • Lists "python-build-standalone moved to Astral" as unverified; it was verified in round 1 (Astral blog, Dec 2024).

Round 3 (agents) and round 2b (unanchored evolution), 2026-10-04

Claims checked

Claim Source Result Evidence
Claude API Skills run "in a sandboxed container with no network access and no runtime package installation"; claude.ai network varies by settings; Claude Code has full network round-3-compass ✅ quoted from the docs https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview
The Agent Skills guide recommends PEP 723 + uv run for scripts with dependencies round-3-compass ✅ "Run with uv (recommended) … uv run creates an isolated environment, installs the declared dependencies". No mention of offline environments https://agentskills.io/skill-creation/using-scripts
MCPB README: Python bundles "Cannot portably bundle compiled dependencies (e.g., pydantic…)"; recommends Node; newer uv server type has the host install deps round-3-compass ✅ https://github.com/modelcontextprotocol/mcpb
goodreads-mcp #89: a .mcpb that only runs on Linux x86_64 / CPython 3.11 due to vendored compiled wheels round-3-compass ✅ issue exists (closed) Danathar/goodreads-mcp#89
uv #13503 (PyInstaller output) closed as not planned round-2b-compass ✅ state_reason: not_planned astral-sh/uv#13503
uv #12035 (single-file Lambda bundling) round-2b-compass ⚠️ Closed 2025-05-20 by its own author as "completed", with no maintainer comments and no implementation. Not evidence that uv shipped it astral-sh/uv#12035

round-3-agents-gemini.md

  • Truncated: only the executive verdict and the beginning of Part 1 survived. Its verdict ("agent-washing" if positioned as exclusively agent tech; position as a general bundler with agents among primary targets) agrees with Compass.

round-3-agents-compass.md

  • Marks Devin, Windsurf, Daytona and Modal as unverified, and some details (claude.ai on gVisor, Claude Code cloud GA date) as third-party. Don't quote those externally without checking.

round-2b-evolution-compass-unanchored.md

  • Run without round 1 attached; conclusions match round 2. Several JS/other-ecosystem details are explicitly from background knowledge, as the report itself says.

Round 5 (CLI and library design), 2026-10-04

Claim Source Result Evidence
argparse gained suggest_on_error and color in Python 3.14; allow_abbrev exists round-5-compass ✅ suggest_on_error default False, color default True (3.14); allow_abbrev since 3.5 https://docs.python.org/3/library/argparse.html
  • round-5-gemini: startup numbers not credible ("argparse 0.00 ms"; Cyclopts 12–16 ms, faster than Click). Don't cite.
  • round-5-compass: did not receive the attachments; its bundleup-specific examples (preset names, platform strings, versions) are placeholders.

Round 4 (use cases and output formats), 2026-10-04

Claims checked

Claim Source Result Evidence
Lambda: 50 MB zipped (direct upload), 250 MB unzipped incl. layers, 5 layers, /tmp 512–10,240 MB, container images 10 GB both ✅ AWS quotas page https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html
Lambda Python runtimes both ✅ python3.10–3.14 supported (3.10 deprecates 2026-10-31), python3.15 in public preview, python3.9 deprecated 2025-12-15; all runtimes support x86_64 and arm64. Gemini's "3.10–3.13" is out of date https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
Splunk: install deps into bin/lib/ with --platform manylinux2014_aarch64 --only-binary=:all:, matching "the platform Splunk is built and ran on, NOT the one you're writing your App on" round-4-compass ✅ quoted from the SDK README https://github.com/splunk/splunk-sdk-python

round-4-use-cases-gemini.md

  • Platform Python ranges are stale (Lambda "3.10–3.13", Cloud Run "3.10–3.12", Azure "3.10–3.12"; Compass, citing official docs, lists 3.14 on all three).
  • Over-rates fit: Cloud Run, Ray and PySpark "strong" and a flat zip "for Spark", although --py-files can't load compiled code and Cloud Run builds from source with uv.
  • Unverified specifics: Azure "30s init timeout, 0.8 GB /tmp", Blender "embedded 3.11".

round-4-use-cases-compass.md

  • Lists its own unverified areas (HPC centre docs, GitHub Actions/pre-commit docs, Ray, Airflow, Maya/Houdini/KNIME/ArcGIS, CLI behaviours of other tools). Check before quoting in docs.