Skip to content

Build-time auto-bundling of pip-installed pure Python packages #31

@deucalioncodes

Description

@deucalioncodes

Summary

When a user's main.py imports a pip-installed package (e.g. ic-python-db, ic-python-logging, or any pure Python package from PyPI), basilisk build should automatically detect and bundle those packages into the Wasm.

Motivation

Currently, Basilisk bundles only the user's source code into the canister Wasm. Any additional packages (ORM, logging, etc.) must be manually included. This friction prevents the natural Python workflow:

pip install ic-python-db
pip install ic-python-logging
from ic_python_db import Entity, String
from ic_python_logging import get_logger

class User(Entity):
    name = String()

Developers expect pip install + import to just work.

Package architecture

Basilisk ships as a minimal CDK. Application framework features are optional pip packages:

Package What it provides Depends on
ic-basilisk CDK core: @query, @update, ic.*, Candid types, memfs
ic-python-db Entity ORM, StableBTreeMap
ic-python-logging Structured logging
ic-basilisk-os Tasks, Wallet, Crypto (Codex, TaskSchedule, KeyEnvelope, ICRC-1, etc.) ic-python-db

Usage:

# Minimal canister
pip install ic-basilisk

# With ORM
pip install ic-python-db

# Full application framework (tasks, wallet, encryption)
pip install ic-basilisk-os

Proposed behavior

  1. basilisk build scans the user's Python source for import statements
  2. Resolves imports against the local venv (standard pip-installed packages)
  3. Rejects packages with C extensions: ERROR: numpy contains native code, not supported on IC
  4. Bundles pure Python .py files into the Wasm data segment alongside user code
  5. At canister init, extracted into memfs under /lib/site-packages/ and added to sys.path

Design decisions

  • Use pip + PyPI — no custom package manager or registry
  • Build-time bundling only — no runtime pip install inside the canister (that's a separate feature)
  • Pure Python filter — any package with native extensions is rejected with a clear error message
  • Transitive deps — follow the dependency chain (pip already resolved this in the venv)

Scope

  • ~100-200 lines of build logic
  • No changes to the Wasm template
  • No changes to the canister runtime (memfs + sys.path already exist)

Related

  • Future: runtime %pip install in the shell (separate issue)
  • Future: on-chain package hash verification for governance (Realms)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions