feat(python): expose factory-backed SandboxPool with safe lease lifecycle

[sd2k]

Problem

Rust exposes SandboxPool, PoolConfig, PooledSandbox, and pool statistics, but pyeryx does not. Python applications that need bounded concurrent execution, acquisition timeouts, and reusable sandbox configuration must implement their own pooling and lease lifecycle.

Expose the Rust pool through pyeryx, with a factory-aware construction path and explicit Python lease semantics. Reuse the existing Rust implementation rather than maintaining a second pool in Python.

Related: #453 and #454. Factory-local Tokio runtime sharing addresses per-wrapper runtime creation directly. Pool bindings should build on that work, not create another runtime per acquisition. Benchmark against fresh wrappers using the shared factory runtime; pooling's incremental performance benefit is not established by comparisons with the old per-wrapper runtime implementation.

Proposed API direction

Illustrative names/signatures, not a fixed API contract:

factory = eryx.SandboxFactory(imports=["json"], cache=True)
pool = factory.create_pool(
    max_size=4,
    min_idle=1,
    acquire_timeout_ms=1000,
    idle_timeout_ms=300_000,
)

with pool.acquire(
    resource_limits=eryx.ResourceLimits(execution_timeout_ms=1000),
    on_stderr=handle_stderr,
) as sandbox:
    result = sandbox.execute(code)

stats = pool.stats()
pool.close()

Start with a synchronous context-manager API consistent with current Python Sandbox.execute(). Define acquisition timeout/exhaustion/closed errors separately from guest execution errors. Consider try_acquire() and a pool context manager. Async bindings can be a separate scope.

Required adaptations and decisions

1. Construct from the actual Python factory baseline

SandboxPool::new() currently calls SandboxBuilder::into_factory(), which captures only the runtime source and stdlib path. Its supported source variants exclude the in-memory precompiled artifact used by Python SandboxFactory; it also drops other builder configuration.

Add a factory-aware path, potentially using SandboxPool::with_builder(), that preserves the cached artifact, package mounts/lifetimes, trusted preinitialization, Python's tracing-disabled setting, and intended configuration defaults. Retain the factory's shared runtime and required backing resources while a pool or lease can still use them. Avoid re-deserializing the component on each cache miss when factory caching is enabled.

2. Separate pool baseline from request-specific capabilities

Each acquisition needs fresh request callbacks, output handlers, resource limits, and any explicitly supported request VFS configuration. Release must remove references to the previous borrower's Python callables and collectors before publishing the wrapper for reuse.

clear_per_request_state() currently clears callbacks, output/trace handlers and the VFS override, but resets resource limits to library defaults, not custom pool defaults. Choose and test a consistent baseline/reapplication model, including factory callback declarations and their host implementations.

Secrets, networking policy, volumes, packages and preamble are not all reset by this method. Treat them as immutable, explicitly shared pool configuration unless complete per-lease reset/revocation semantics are implemented. A pooled wrapper must not silently retain tenant-specific capabilities supplied by a prior borrower.

3. Make Python leases deterministic and exclusive

A context manager must release on success and exceptions, reject use after release, and make repeated release safe. Do not expose an independently usable Sandbox alias that survives return to the pool. Prevent concurrent execution/release of the same lease.

Release the GIL while waiting for capacity and executing guest code so another Python thread can finish work and return its lease. Define abandoned-lease/GC behavior without relying on GC for normal release, and ensure interrupted acquisition cannot leak a permit. Define reuse versus discard after execution failures/timeouts; any discard path must maintain pool accounting.

4. Fix/document relevant Rust lifecycle behavior before exposing it

Source observations at 5f5a20977eab2f8a91874905895fb580911fcecd:

• reset_on_release claims to invoke session reset, but is not consulted by the release path. Actual release clears host request configuration. Correct/deprecate the misleading option; do not imply that clearing a persistent Session is a tenant-isolation mechanism.
• PooledSandbox::detach() takes the sandbox without decrementing current_size; its later drop skips the return path. Fix accounting before using this as a discard mechanism.
• PoolStats.available is populated from semaphore permits, although documented as available sandboxes. Distinguish immediately reusable idle objects from unused concurrency capacity.
• close() only sets a flag. Define and implement prompt behavior for blocked acquisitions, idle-resource cleanup, and outstanding leases (including acquisitions racing with close).
• Idle eviction is an explicit evict_idle() call. Expose an explicit maintenance operation or arrange maintenance, and document what min_idle/idle timeout actually guarantee.

Relevant implementation: pool.rs, sandbox.rs, Python factory.

5. Specify process and isolation boundaries

Initialize the factory/pool and its retained Tokio runtime after worker fork, or explicitly detect and reject inherited objects. Document shutdown/lifetime behavior when factory, pool, and leases are dropped in different orders.

Pooling reuses host sandbox configuration/executors. Each stateless execute() must continue to create a fresh Store/guest instance and default VFS. This is neither Session reuse nor the background fresh-instance pool proposed in #411. Shared host capabilities still require correct ownership; bounded active leases are not a full tenant quota or fairness policy.

Acceptance criteria

• Python bindings, type stubs, documentation and a factory-based example.
• Tests for exclusive concurrent leases, capacity/exhaustion/timeouts, GIL progress, close with waiters, repeated release, use after release, discard accounting, and backing-resource lifetimes.
• Repeated-borrower tests for stderr/callback separation and configured limits, including failures/timeouts.
• Fresh guest-state tests covering globals, builtins/modules, environment and VFS files across executions of a reused wrapper.
• A documented and tested worker-fork policy.
• Benchmarks of pool versus fresh wrappers with perf(python): reuse a Tokio runtime within SandboxFactory #454's shared runtime: process CPU per operation, latency, throughput, native thread count and retained memory. Separate wrapper savings from guest instantiation and workload execution.