Skip to content

Repository files navigation

Lambda MicroVM Notebook

A Python & SQL notebook running on AWS Lambda MicroVMs — each session gets its own Firecracker VM with persistent state, VM-level isolation, and automatic suspend/resume.

Proof-of-concept demonstrating Lambda MicroVMs as stateful code execution sandboxes. Extensible to other runtimes (R, Node.js, Julia) by swapping the executor and image.

Contents: Demo Videos · Quick Start · Why MicroVMs? · Architecture · Features · Data Sources · Configuration · Network Egress Control · Testing · Project Structure · Technical Details · Prerequisites · Cost

Demo Videos


How to create a Docker image for a data science environment and deploy it as Lambda MicroVM images

Running Notebooks on Lambda MicroVMs

AI Assistant with Full MicroVM Context

Quick Start

./local_install.sh        # builds images, starts proxy + AI agent + UI (SQLite, no Aurora)

Opens at http://localhost:5173. Each notebook tab auto-launches a MicroVM. Closing a tab suspends its VM (reopening the notebook resumes it); deleting a notebook terminates its VM.

Requirements: AWS CLI 2.35.10+, Python 3.11+, Node.js 18+, configured AWS credentials. See Prerequisites for details.

Aurora is optional (opt-in)

The stack runs fully on local SQLite by default. Aurora PostgreSQL — and the SQLite→Aurora sync worker that feeds its curated v_nb_* views — is entirely opt-in via a flag. Pick the command that matches what you want:

You want… Command What runs Aurora
No Aurora (fully local) ./local_install.sh proxy + AI agent + UI (SQLite) untouched; sync worker not started
Aurora + sync ./local_install.sh --AuroraSetup proxy + AI agent + UI + sync worker ensured (created if missing, else reused)
Reset Aurora schema ./local_install.sh --AuroraReset same as --AuroraSetup drops + recreates the schema (destructive)

First-time user:

  • Without Aurora — just run ./local_install.sh. No Aurora resources are created or referenced; the sync worker is skipped. This is the simplest path.
  • With Aurora — run ./local_install.sh --AuroraSetup. On the first run this provisions the Aurora Serverless v2 cluster (control plane) and creates the roles, tables, indexes, and curated v_nb_* views over the RDS Data API (works from a laptop — no VPC/psycopg2 needed). The resolved ARNs are written to scripts/.aurora_env and wired into the proxy (VM read path) and the sync worker.

Subsequent runs:

  • ./local_install.sh — same fully-local run every time.
  • ./local_install.sh --AuroraSetup — detects the existing cluster and reuses it as-is: it only re-resolves the ARNs from your account (no schema DDL, no data loss) and starts all four services. This is the everyday Aurora command.
  • ./local_install.sh --AuroraReset — only when you deliberately want to wipe and rebuild the Aurora schema (also resets the local SQLite DB).

Aurora ARNs are not hardcoded in scripts/config.sh — they are discovered from your own AWS account at launch and cached in scripts/.aurora_env (gitignored). A blank Aurora config simply means "no Aurora data source", which is the correct default for the no-flag run.

Teardown: bash scripts/teardown.sh (terminates all VMs, deletes images)


Why Lambda MicroVMs for Notebooks?

Traditional notebook platforms run kernels as containers on shared Kubernetes nodes. Lambda MicroVMs offer a fundamentally better primitive for this workload:

EKS Pods (containers) Lambda MicroVMs
Isolation Shared host kernel (namespaces + cgroups) Dedicated Firecracker guest kernel per session — hardware VM boundary
Idle cost 5-min eviction tail + 24/7 EBS volumes Suspends to ~$0 instantly, snapshot only
Resume Kill pod → recreate → reattach EBS (~10-30s) Snapshot restore with memory + disk intact (~1-2s)
Infra overhead Cluster autoscaler, node pools, PDBs, etcd tuning None — fully managed, no cluster to operate
Blast radius Runaway kernel affects co-located pods on same node Confined to its own VM, terminated cleanly

Key benefits for notebook use cases:

  • VM-level tenant isolation — Customer-supplied Python runs in its own Firecracker VM with a dedicated guest kernel. Container escapes, fork bombs, and malicious dependencies cannot cross the boundary. Materially stronger for SOC 2 and enterprise security reviews.

  • Instant suspend, zero idle cost — No eviction timers, no EBS volumes running 24/7. The VM freezes the moment the user stops typing and resumes in 1-2 seconds when they return. You pay only for active compute.

  • Eliminates control-plane churn — Notebook kernels stop being Kubernetes pods, so they generate zero scheduling, eviction, and etcd write load. The control-plane stability problem disappears for this workload.

  • Simpler operations — No cluster autoscaler, node drain logic, or EBS reattach choreography. Lifecycle is a single API: run → suspend → resume → terminate.

Cost comparison (directional, per user/month, 2 vCPU / 4 GB kernel, ~2.5 hr/day session of which ~30 min is active cell execution):

Scale Lambda MicroVMs EKS + EBS (evict-on-idle) Saving
Per user ~$3.25 ~$5.71 ~43%
1,000 users ~$3,254 ~$5,781 ~$30K/yr
2,000 users ~$6,508 ~$11,490 ~$60K/yr

Architecture

Session-Based Routing

All callers interact with the proxy using only an X-Session-Id header. The proxy hides all VM internals — endpoints, auth tokens, VM IDs, recovery state:

┌────────────────────┐         ┌─────────────────────────────────────┐
│  Browser / Client  │  HTTP   │         Smart Proxy (:8081)         │
│                    ├────────►│                                     │
│  X-Session-Id: uuid│         │  • Session registry (sid → VM)      │
│                    │         │  • Auth token injection (JWE)       │
│                    │  WS     │  • Reactive recovery (eternal mode) │
│  Terminal (xterm)  ├────────►│  • Terminal relay (WS → SHELL_INGRESS)│
│                    │         │  • Checkpoint orchestration         │
│                    │         │  • AI agent (Strands/Bedrock)       │
└────────────────────┘         └──────────────┬──────────────────────┘
                                              │ HTTPS + auth (HTTP_INGRESS)
                                              │ WSS + subprotocols (SHELL_INGRESS)
                                              ▼
                               ┌────────────────────────-──┐
                               │  Lambda MicroVM           │
                               │  (Firecracker, ARM64)     │
                               │  FastAPI + SandboxExecutor│
                               │  Platform Shell (bash PTY)│
                               └─────────────────────-─────┘

Why this design:

  • Transparent recovery — same session ID, different VM underneath (eternal mode relinks after a VM dies)
  • No credential leakage — caller never handles auth tokens
  • Mode-agnostic — same API works in eternal and checkpoint mode
  • Simplified clients — just track session IDs, nothing else

Two Persistence Modes

Lambda MicroVMs have an 8-hour max lifetime (AWS limit), and a VM can also be reclaimed when it hits its idle/suspend timeout. Both modes keep session state safe in S3; they differ only in when a new VM is spun up and re-linked.

Eternal (default) Checkpoint
How a new VM is provisioned Reactively — on the next request after the VM is found dead On the user's next launch/open
What happens when the VM dies Proxy launches a fresh VM, restores from S3, relinks the session, and serves the request — one-time cold start (~9-12s) State stays in S3; VM is gone until the user returns
User experience Seamless — same session ID, no manual step Must click "Restore" next time
Scheduler / timers None — no in-process rotation timer or polling loop None
Best for Always-on notebooks Intermittent, cost-sensitive use

How state is persisted (both modes): state is saved to S3 on /suspend and on /terminate, gated by a dirty-flag so unchanged state is never re-saved. This is what makes recovery safe regardless of how the VM died (suspended-then- reclaimed VMs never fire /terminate, but the /suspend save already covered them).

What survives a recovery/restore: variables, DataFrames, /tmp/ files, pip packages, variable provenance, and imports — the executor records every successful import and replays them on restore, so import pandas as pd etc. keep working on the new VM. What's excluded: non-serializable objects such as matplotlib figures/axes and open handles (they can't be pickled). Their module bindings (e.g. plt) are replayed via the import ledger; only the transient objects are dropped.

Eternal mode is reactive, not pre-emptive. Earlier versions rotated the VM ~60s before max lifetime using an in-process timer. That was removed: a timer dies with the proxy instance and doesn't survive a stateless / horizontally scaled proxy, and AWS emits no VM-termination event to trigger it. Recovering on the next request needs no scheduler and works across proxy restarts.

PFR filed: Request submitted to increase max lifetime from 8h to 2 weeks. Once approved, recovery/checkpoint becomes unnecessary for most use cases.

Configure via environment: SESSION_PERSISTENCE_MODE=eternal (or checkpoint)


Features

Notebook

  • Three cell types — Python, SQL (DuckDB/Athena), Markdown
  • Sequential executionShift+Enter runs cells in order, no race conditions
  • Rich output — DataFrames as styled HTML tables, matplotlib inline, syntax highlighting
  • File upload — CSV, Excel, Parquet, JSON → auto-loaded as pandas DataFrames
  • Multi-tab — each tab = separate notebook + separate MicroVM
  • Save/Open.notebook.json preserves code, output, charts, AI explanations

SQL Engine (DuckDB + Athena + DynamoDB)

Native SQL cells with intelligent auto-routing — write standard SQL, engine chosen transparently:

Source Syntax Engine
DataFrame in memory SELECT * FROM df_name DuckDB
Local file SELECT * FROM '/tmp/file.csv' DuckDB
S3 file SELECT * FROM read_csv('s3://...') DuckDB + httpfs
DynamoDB table SELECT * FROM dynamodb."table" PartiQL (or scan → DuckDB)
Athena table SELECT * FROM db.table Athena
Mixed JOIN Any combination above Materialize remote → DuckDB

AI Assistant (Strands Agents + Bedrock)

  • Chat panel — conversational agent with notebook context awareness
  • Explain — one-click plain-English explanation of any cell
  • Fix — AI-suggested fixes for error cells with one-click apply
  • NLP-to-Code — type natural language, get Python
  • Auto-Annotate — document all cells with AI in one click
  • Agent runs in the proxy (not the MicroVM) — no image bloat, instant iteration
  • Auto-detects Bedrock credentials; hides AI buttons if not configured
  • Model: Claude Sonnet 4.6 (via Amazon Bedrock)

Workbook Intelligence (AI-Powered Data Analysis)

Automatic data profiling and intelligence generation when data sources are connected:

  • Two-phase generation — structured tab cards appear fast (~25-30s), full prose report generates in background (~35-45s)
  • 4 insight tabs — Suggested Analyses, Visualizations, Investigations, Alerts — each card is clickable (sends prompt to AI chat)
  • Entity discovery — automatic schema profiling for all connected S3, DynamoDB, Athena, and local files
  • Cross-source verification — agent validates referential integrity (FK overlaps) and flags issues
  • Incremental updates — uploading a new file triggers a fast delta (only new findings, ~16s)
  • Deletion pruning — removing a file auto-prunes related insights (~6s)
  • Full report modal — comprehensive markdown report with relationships, join paths, and data quality analysis
  • Data quality alerts — PII detection, null rates, type mismatches, cardinality anomalies
  • Model: Claude Haiku 4.5 (optimized for speed; configurable via INTEL_MODEL_ID)
  • Prompt caching enabled for multi-turn agent loop (reduces latency on subsequent tool calls)

Logs Panel

  • Real-time VM logs — CloudWatch log stream from the MicroVM, live-updating
  • Execution traces — stdout/stderr from code execution, agent tool calls
  • Intel tracing — per-tool-call trace of the Workbook Intel agent (which tools called, inputs, results, timing)
  • Session-aware — automatically follows the active notebook tab's VM

MicroVM Management

  • 4 memory tiers: 1 GB (0.5 vCPU) through 8 GB (4 vCPU), burst to 4×
  • Configurable idle suspend (1 min – 2 hr), auto-resume on traffic (~1s)
  • Configurable max duration (30 min – 8 hr) — controls absolute VM lifetime
  • Real-time cost tracking (running + suspended + burst)
  • Connection status pill: 🟢 Running, 🟠 Suspended, 🔴 Terminated
  • Instance panel: specs, lifecycle, resources, cost breakdown per VM

Sidebar (VS Code-style)

Notebooks, Outline, Data Sources, Variables, Packages, Terminal, Logs, Intel, Snippets, Samples — resizable, collapsible, grouped with visual dividers:

  • Group 1: Notebooks
  • Group 2: Outline, Data Sources, Variables, Packages
  • Group 3: Terminal, Logs, Intel (Workbook Intelligence)
  • Group 4: Snippets, Sample Notebooks

Snippets Library

Pre-loaded helper functions available in every cell — no imports needed:

  • Data Loadingread_s3_csv(), read_dynamodb(), read_athena(), read_url(), sample_data()
  • Data Exportto_s3_csv(), to_s3_parquet(), to_local()
  • Visualizationplot_line(), plot_bar(), plot_scatter(), plot_histogram(), plot_heatmap()
  • Utilitiesprofile(), whoami(), compare_df(), list_s3(), head_s3(), timer

Click any snippet in the sidebar panel to insert an example with comments into the current cell.

Secrets & Environment Variables

Inject secrets and config into MicroVMs at launch time:

  • AWS Secrets Manager — Browse available secrets from your account, select which to inject. Values are fetched inside the VM (proxy never sees them).
  • Direct env vars — Key-value pairs injected via runHookPayload. Values masked in the UI.
  • Access in codeimport os; api_key = os.environ['MY_SECRET']
  • Security — Secrets are fetched by the VM's execution role at boot. The proxy only lists secret names (not values). Same trust boundary as Lambda + Secrets Manager.

Interactive Terminal

  • Full shell access — bash terminal inside the MicroVM via AWS SHELL_INGRESS connector
  • Platform-managed PTY — no custom shell server needed, uses Lambda's built-in shell endpoint
  • Resizable bottom panel — drag to resize, appears below the notebook
  • Idle auto-disconnect — WebSocket closes after 30s of no input, allowing VM to suspend
  • Auto-reconnect on type — typing in a disconnected terminal reconnects transparently (resumes VM if suspended)
  • Session-aware — switches to the correct VM when you switch notebook tabs
  • Pre-installed toolspython3, pip, git, tar, gzip available out of the box
  • Package access — all packages from requirements.txt on PATH (pandas, numpy, boto3, etc.)

Data Source Connectivity

┌─────────────────────────────────────────────────────────────────────────────────┐
│  Lambda MicroVM (Firecracker)                                                   │
│                                                                                 │
│  ┌───────────────────────────────────────────────────────────────────────────┐  │
│  │  Notebook Code (Python / SQL)                                             │  │
│  │                                                                           │  │
│  │  • pandas, numpy, polars, matplotlib, scipy                               │  │
│  │  • DuckDB (in-process SQL engine)                                         │  │
│  │  • boto3 (AWS SDK)                                                        │  │
│  └───┬───────────┬──────────────┬────────────────┬───────────────────────────┘  │
│      │           │              │                │                              │
│      ▼           │              │                │                              │
│  ┌────────┐      │              │                │                              │
│  │ /tmp/  │      │              │                │                              │
│  │ Local  │      │              │                │                              │
│  │ Files  │      │              │                │                              │
│  └────────┘      │              │                │                              │
│                  │              │                │                              │
└──────────────────┼──────────────┼────────────────┼──────────────────────────────┘
                   │              │                │
    ┌──────────────┼──────────────┼────────────────┼───────────────────────────┐
    │              ▼              ▼                ▼                           │
    │  ┌─────────────────────────────────────────────────────────────────────┐ │
    │  │              IAM Execution Role (auto-injected credentials)         │ │
    │  └──┬──────────┬───────────┬──────────────┬───────────────┬──────────-─┘ │
    │     │          │           │              │               │              │
    │     ▼          ▼           ▼              ▼               ▼              │
    │  ┌──────┐  ┌────────┐  ┌────────┐  ┌──────────┐  ┌───────────┐           │
    │  │  S3  │  │DynamoDB│  │ Athena │  │   Glue   │  │    STS    │           │
    │  │      │  │        │  │        │  │ (Catalog)│  │           │           │
    │  │Bucket│  │ Tables │  │Workgrp │  │  Tables  │  │  Assume   │           │
    │  └──────┘  └────────┘  └────────┘  └──────────┘  └───────────┘           │
    │                                                                          │
    │                 AWS Account (IAM-based access)                           │
    └──────────────────────────────────────────────────────────────────────────┘

                   │
    ┌──────────────┼───────────────────────────────────────────────────────────┐
    │              ▼                                                           │
    │  ┌─────────────────────────────────────────────────────────────────────┐ │
    │  │           VPC Egress Connector (ENI in customer subnets)            │ │
    │  └──┬──────────┬───────────┬──────────────┬──────────────────────────-─┘ │
    │     │          │           │              │                              │
    │     ▼          ▼           ▼              ▼                              │
    │  ┌──────┐  ┌────────┐  ┌────────-──┐  ┌───────────────┐                  │
    │  │ RDS  │  │Redshift│  │ElastiCache│  │  On-premises  │                  │
    │  │      │  │        │  │           │  │  (Direct      │                  │
    │  │Postgres││  DWH   │  │  Redis    │  │   Connect)    │                  │
    │  │MySQL │  │        │  │           │  │               │                  │
    │  └──────┘  └────────┘  └────────-──┘  └───────────────┘                  │
    │                                                                          │
    │                 Customer VPC (private subnet access)                     │
    └──────────────────────────────────────────────────────────────────────────┘

                   │
    ┌──────────────┼───────────────────────────────────────────────────────────┐
    │              ▼                                                           │
    │  ┌─────────────────────────────────────────────────────────────────────┐ │
    │  │              Internet Egress (default, no VPC needed)               │ │
    │  └──┬──────────┬───────────┬───────────────────────────────────-───────┘ │
    │     │          │           │                                             │
    │     ▼          ▼           ▼                                             │
    │  ┌──────┐  ┌────────┐  ┌────────────┐                                    │
    │  │Public│  │  pip   │  │ SaaS APIs  │                                    │
    │  │ APIs │  │install │  │ (Snowflake,│                                    │
    │  │      │  │        │  │  Databricks│                                    │
    │  └──────┘  └────────┘  │  etc.)     │                                    │
    │                        └────────────┘                                    │
    │                 Public Internet                                          │
    └──────────────────────────────────────────────────────────────────────────┘
Access Pattern Mechanism Data Sources
Local In-VM filesystem /tmp/*.csv, .parquet, .json, .xlsx
IAM Role Auto-injected credentials S3, DynamoDB, Athena, Glue, STS
VPC Connector ENI in private subnets RDS, Redshift, ElastiCache, OpenSearch, on-prem (DX)
Internet Default egress Public APIs, pip packages, SaaS (Snowflake, Databricks)

Sample data auto-provisioned: DynamoDB table, 4 S3 CSVs, Athena database with 4 tables.


Configuration

Key settings in scripts/config.sh:

SESSION_PERSISTENCE_MODE="eternal"       # "eternal" (reactive recovery) or "checkpoint"
MAX_LIFETIME_SECONDS="28800"             # 8h (AWS max)
AWS_REGION="us-west-2"
IMAGE_ARCHES="arm64"                     # Arches to build+offer (e.g. "arm64 x86_64")
DEFAULT_IMAGE_ARCH="arm64"               # Arch pre-selected in the UI
IMAGE_SIZES_ARM64="1024 2048 4096 8192"  # arm64 memory tiers (MiB)
IMAGE_SIZES_X86_64="1024 2048 4096 8192" # x86_64 memory tiers (MiB) — set independently

Memory tiers are per-architecture (arm64 and x86_64 need not match). Images are named notebook-microvm-{arch}-{mem}; the UI shows an arch picker only when more than one arch is built.

Override for testing: SESSION_PERSISTENCE_MODE=checkpoint MAX_LIFETIME_SECONDS=180 ./local_install.sh

AI config in proxy/notebook/ai/constants.py:

  • DEFAULT_MODEL_ID — Claude Sonnet 4.6 for the AI Assistant (chat, explain, fix)
  • INTEL_MODEL_ID — Claude Haiku 4.5 for Workbook Intelligence (faster, 2x speedup with near-equal quality)
  • Temperature, token limits, truncation constants

Testing

Tests auto-detect the proxy's persistence mode and run the appropriate suite:

bash tests/run_tests.sh
tests/
├── run_tests.sh              # Auto-detect mode, run common + mode-specific
├── common/                   # Both modes
│   ├── test_burst_behavior.py
│   ├── test_interrupt_execution.py
│   ├── test_microvm_lifecycle.py
│   └── test_sql_engine.py
├── eternal/
│   └── test_reactive_rotation.py  # Reactive recovery: kill VM → recover on next
│                                  # request (FIFO ordering, AWS max-lifetime kill,
│                                  # 2-suspend latest-state, mixed install, timings)
└── checkpoint/
    ├── test_auto_checkpoint.py         # /terminate-hook save + full restore
    ├── test_s3_restore.py              # S3 checkpoint round-trip
    └── test_suspend_checkpoint_timing.py  # save-on-suspend (dirty-flag) timing

All tests use X-Session-Id only — no VM internals referenced.


Project Structure

local_install.sh             # One-click launcher: AWS setup + starts proxy, AI chat agent, UI.
                             #   No flag = fully local (SQLite, no sync worker).
                             #   --AuroraSetup = also ensure Aurora + start sync worker (non-destructive,
                             #                   provisions if missing; schema DDL via the RDS Data API).
                             #   --AuroraReset = --AuroraSetup + drop/recreate Aurora schema & reset SQLite.
dev_run.sh                   # Lightweight dev launcher (backend + UI only)

app/                          # Runs INSIDE the MicroVM
├── server.py                 # FastAPI entrypoint, pre-loaded libs
├── platform/
│   ├── hooks.py              # Lifecycle: /run, /suspend, /resume, /terminate, /checkpoint-save, /restore-state
│   └── checkpoint.py         # DELTA checkpoint: per-var + per-file objects, manifests, change detection, import-ledger replay, module/IO exclusion
└── notebook/
    ├── executor.py           # SandboxExecutor (stateful Python engine, import ledger, interrupt)
    ├── code_engine.py        # /execute endpoint
    ├── sql_engine.py         # /execute-sql with auto-routing
    ├── data_catalog.py       # /data-catalog endpoint (S3, DynamoDB, Athena discovery)
    ├── dtypes.py             # DataFrame type detection utilities
    └── routes.py             # /install, /variables, /health, /metrics, /upload

batch/                        # Background jobs (entity discovery)
├── __init__.py
└── entity_discovery.py       # Auto-profile all data sources (S3, DynamoDB, Athena, local files)

proxy/                        # Runs on your machine (hides all VM internals)
├── server.py                 # FastAPI entrypoint, WebSocket terminal relay, health
├── agent_runtime.py          # AI CHAT AGENT as its own local service (BedrockAgentCoreApp on $AI_PORT,
│                             #   POST /invocations + GET /ping). Same artifact deployable to AWS
│                             #   Bedrock AgentCore Runtime. Proxy /ai/chat forwards here.
├── platform/
│   ├── microvm_manager.py    # Session registry, tokens (HTTP + shell), reactive recovery, AWS client
│   ├── cost_tracker.py       # Burst + baseline cost tracking
│   ├── package_classifier.py # PyPI-based package category detection
│   ├── token_cache.py        # Auth-token cache (in-process L1; Redis L2 future)
│   ├── errors.py             # Centralized error handling + metrics
│   ├── datasources/          # Schema discovery providers + registry
│   │   ├── registry.py       # Single source of truth for source types (order/labels/docs)
│   │   ├── interface.py      # DataSourceProvider abstraction
│   │   └── s3.py, dynamodb.py, athena.py, aurora.py, local.py
│   └── routes/
│       ├── microvm.py        # /launch, /terminate (session-optional), /suspend, /resume, /proxy/{path}, /instances
│       ├── sessions.py       # /sessions, /datasources (+ /catalog, /schema), /secrets
│       ├── metrics.py        # /instances/metrics, /instances/metrics/history
│       └── terminal.py       # WebSocket terminal relay → VM shell
├── notebook/
│   ├── ai/
│   │   ├── constants.py      # Model IDs, token limits, CHAT_AGENT_ID, SOURCE_TYPE_LABELS
│   │   ├── prompts.py        # All LLM prompts (agent, intel Phase 1/2, incremental, deletion) — self-contained, no proxy.* import
│   │   ├── notebook_agent.py # Strands chat Agent (chat/chat_stream, fix, annotate, suggest) + prompt caching
│   │   ├── memory.py         # SessionManager seam: MEMORY_BACKEND factory (sqlite DB ↔ AgentCore Memory)
│   │   ├── sessions.py       # In-process agent cache (keyed by session_id)
│   │   ├── workbook_intel.py # Workbook Intel entrypoint (delegates to intel/)
│   │   ├── intel/            # Workbook Intel pipeline (Class C — stays in proxy)
│   │   │   ├── generate.py   # Phase 1 (Strands agent) + Phase 2 (direct converse) full generation
│   │   │   ├── delta.py      # Incremental (file upload) + deletion pruning
│   │   │   ├── context.py    # Entity-doc assembly, source counts
│   │   │   ├── parsing.py    # Intel JSON/delimited response parsing + salvage
│   │   │   └── store.py      # Per-session FIFO scheduler + S3 read/write
│   │   └── tools/            # Agent tools (execute_code, get_variables, install_package, ...) — all HTTP to proxy
│   └── routes/
│       ├── ai.py             # /ai/chat (forwards to agent), /ai/chat/sync, /ai/annotate, /ai/fix, /ai/suggest-tag,
│       │                     #   GET /ai/chat/{id}/messages (transcript), /terminal/suggest
│       ├── intel.py          # /workbook-intel (GET/POST) endpoints
│       ├── notebooks.py      # Notebook CRUD + versions
│       └── logs.py           # CloudWatch log streaming
├── storage/                  # Abstract storage seam (proxy code never touches a DB driver directly)
│   ├── interface.py          # Abstract StorageBackend (notebooks, vm_sessions, agent_sessions, ...)
│   ├── sqlite_db.py          # SQLite implementation (local default)
│   ├── postgres_db.py        # Postgres/Aurora implementation (cloud)
│   ├── async_storage.py      # Async wrapper (asyncio.to_thread) over the sync backend
│   ├── versioning.py         # Notebook version/document helpers
│   └── _transcript.py        # Agent-message → UI-bubble transcript translation (shared)
└── data/                     # Local SQLite database (microvm.db)

sync/                         # LOCAL-DEV-ONLY SQLite → Aurora bridge (throwaway)
└── sync_to_aurora.py         # Standalone worker: replays local SQLite writes/deletes into Aurora
                              #   over the RDS Data API (generic FK-topo introspection). Unneeded once
                              #   the proxy runs directly on Postgres (STORAGE_BACKEND=postgres).

web/src/                      # React UI (Vite)
├── App.jsx                   # Main app, tab management, intel watching
├── components/
│   ├── panels/
│   │   ├── TerminalPanel.jsx # xterm.js terminal (WebSocket -> proxy -> VM shell)
│   │   ├── DataSourcesPanel.jsx # Data source browser with schemas
│   │   └── OutlinePanel.jsx  # Cell navigator with Run All status icon
│   ├── Cell.jsx              # Code/SQL/Markdown cell with output display
│   ├── Notebook.jsx          # Cell list, execution, Run All
│   ├── Sidebar.jsx           # Activity bar with grouped icons
│   ├── IntelPanel.jsx        # Workbook Intelligence (4 tabs + full report modal)
│   ├── LogsPanel.jsx         # Real-time CloudWatch log viewer
│   ├── ConnectionPanel.jsx   # VM connection status + MicroVM management
│   ├── PackageManager.jsx    # pip package search + install
│   ├── Icons.jsx, Modal.jsx, TabBar.jsx
│   └── ErrorBoundary.jsx     # React error boundary
├── services/microvm.js       # API client (all calls use X-Session-Id)
├── utils/dragOverlay.js      # Plotly chart drag-resize handler
└── constants.js              # Frontend constants

scripts/                      # Setup and operations
├── config.sh                 # All configuration (region, bucket, IAM, ports, AI_PORT/AGENT_URL,
│                             #   BEDROCK_MODEL_ID, MEMORY_BACKEND, AGENTCORE_MEMORY_ID, Aurora, ...)
├── setup_iam.sh              # Create IAM roles
├── setup_sample_data.sh      # Seed e-commerce data (S3, DynamoDB, Athena)
├── setup_aurora_postgres.sh  # Provision Aurora PostgreSQL: cluster + roles + schema + v_nb_* views,
│                             #   all DDL over the RDS Data API (local-safe). Idempotent; drops only on
│                             #   AURORA_RESET=true. Writes resolved ARNs to scripts/.aurora_env.
├── setup_agentcore_memory.sh # STANDALONE: create-or-get an AgentCore Memory resource for the chat
│   + setup_agentcore_memory.py #   agent (short-term). NOT wired into local_install.sh. Prints memoryId.
├── build_all_images.sh       # Build MicroVM Docker images
├── teardown.sh               # Clean up all AWS resources
└── benchmark_intel_models.py # LLM model benchmark (Phase1/2, incremental, deletion, judge)

tests/                        # E2E tests + test fixtures
├── run_tests.sh              # Auto-detect mode, run common + mode-specific
├── product_returns.csv       # Test fixture for deletion-flow testing
├── test_vm_lifecycle_triggers.py  # Which lifecycle hooks fire on which kill path
├── test_resume_before_expire.py   # Resume-before-max-lifetime behavior
├── common/                   # Run in BOTH modes
│   ├── test_microvm_lifecycle.py  # launch → suspend → resume → terminate → restore
│   ├── test_sql_engine.py         # SQL auto-routing (DuckDB/Athena/DynamoDB)
│   ├── test_interrupt_execution.py# Stop button: interrupt runaway loops, sandbox stays healthy
│   └── test_burst_behavior.py     # 4× burst allocation + billing
├── eternal/                  # Eternal mode (reactive recovery)
│   └── test_reactive_rotation.py  # kill VM → recover on next request (FIFO, AWS max-life, timings)
├── checkpoint/               # Checkpoint mode + DELTA checkpoint
│   ├── test_auto_checkpoint.py         # /terminate-hook save + full restore
│   ├── test_s3_restore.py              # S3 checkpoint round-trip
│   ├── test_suspend_checkpoint_timing.py  # save-on-suspend timing + delta showcase (vars + files)
│   ├── test_delta_core.py              # (offline) per-variable delta logic
│   ├── test_delta_s3_roundtrip.py      # (offline) real delta save/restore vs fake S3
│   ├── test_file_delta_core.py         # (offline) per-file delta logic
│   └── test_dirty_flag_file_delete.py  # (offline) dirty-flag reacts to file add/change/delete
├── storage/                  # Storage-backend conformance (SQLite always; Postgres if TEST_PG_DSN set)
│   └── test_storage_conformance.py     # One suite, both backends: notebooks, vm_*, agent_sessions/messages, ...
└── sync/                     # SQLite → Aurora sync worker E2E
    └── test_sync_e2e.py                # Independent versions, restore, batch, cascade delete, PK-level parity

Offline tests (test_delta_core, test_delta_s3_roundtrip, test_file_delta_core, test_dirty_flag_file_delete) need no AWS/proxy — they exercise the delta save/restore logic directly and run in milliseconds.


Technical Details

Lifecycle Hooks

Hook When Purpose
/run VM starts Initialize session, optionally restore from S3 (namespace + files + packages + provenance + imports)
/suspend Going idle Save state to S3 if dirty (both modes) — primary save point, since a suspended VM reclaimed by AWS never fires /terminate
/resume Traffic arrives Validate state
/terminate Shutting down (VM RUNNING at kill time) Save state to S3 if dirty (both modes) — covers the RUNNING→terminate path

State is saved on both /suspend and /terminate, in both modes, gated by a dirty-flag (fingerprint of provenance clock + installed packages) so unchanged state is never re-serialized. Note: AWS does not fire /terminate for a VM that is SUSPENDED when reclaimed — the /suspend save is what makes that case safe.

Checkpoint Serialization

  • Save: exclude modules + non-picklable display objects → dill.dumps(namespace) → bulk serialize with per-var fallback → upload to S3: checkpoint.pkl, files.tar.gz, requirements.txt, metadata.json, data_catalog.json, provenance.json, imports.json
  • Restore: download from S3 → dill.loads()copy.deepcopy() mutable containers → pip install tracked packages → restore provenance → replay the import ledger
  • Import ledger: modules can't be pickled, so the executor records each successful import/from-import (keyed by bound name) and replays those lines on restore. This re-establishes pd, np, etc. so the user's existing cells don't break with NameError after a transparent recovery. Only the non-picklable objects (matplotlib figures/axes, open handles) are dropped.
  • Module exclusion prevents _csv.writer and similar C-extension crashes
  • Deepcopy breaks dill internal references that prevent list mutations from surviving re-serialization

Reactive Recovery (Eternal Mode)

Scheduler-free. When a proxied request finds the session's VM dead (AWS state is TERMINATED/FAILED/NOT_FOUND, or a forward fails), the proxy recovers on that request: launch a fresh VM → wait healthy → restore from S3 (with retry/backoff) → relink the session → replay the request. Concurrent requests during recovery are queued and replayed in FIFO arrival order (one recoverer drains the queue), and exactly one recovery runs per session. Per-phase timings are exposed at GET /recovery-timings. Typical end-to-end: ~9-12s (VM boot dominates). On restore failure it fails closed (never registers an empty VM over good S3 state).

Interrupt (Stop)

The /interrupt endpoint injects KeyboardInterrupt into the running cell's thread (PyThreadState_SetAsyncExc), reliably stopping runaway pure-Python loops; the executor drains any stray pending exception in its finally so it can't leak into the next cell. Cells blocked in a C-level call (time.sleep, socket recv, C extensions) can't be force-killed in-process — that would require per-cell process isolation — so the sandbox serializes one cell at a time and that case is a documented limitation.

Burst Model

VMs get 4× baseline resources pre-allocated from boot. Usage above baseline incurs burst billing at the same vCPU + memory rates. Exceeding 4× = OOM crash.


Prerequisites

AWS MicroVM Mode

Requirement Version
AWS CLI 2.35.10+ (for lambda-microvms subcommand)
Python 3.11+
Node.js 18+
AWS credentials Configured via ~/.aws/credentials
# Install/upgrade AWS CLI (macOS)
curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o /tmp/AWSCLIV2.pkg
sudo installer -pkg /tmp/AWSCLIV2.pkg -target /

Local Dev Mode (no AWS)

Just Python 3.11+ and Node.js 18+. Run ./dev_run.sh.

IAM Roles (auto-created)

  • MicroVMSandboxBuildRole — S3 read during image build
  • MicroVMSandboxExecRole — S3, DynamoDB, Athena, Glue, STS for running VMs

AI Features (optional)

Amazon Bedrock access with Claude Sonnet enabled. If not configured, AI buttons are hidden — everything else works.


Cost

Pricing rates (from AWS Lambda MicroVM pricing, Graviton/ARM64, us-east-1):

Component Rate
vCPU (compute) $0.0000276944 / vCPU-second
Memory $0.0000036667 / GB-second
Snapshot (suspended) ~$0.08 / GB-month

Note: CPU is allocated at 2 GB : 1 vCPU. A 4 GB kernel = 2 vCPU.

Example breakdown — 2 vCPU / 4 GB kernel, ~2.5 hr/day session of which ~30 min is active cell execution (~11 hr/month compute, 22 workdays):

Component Calculation Monthly Cost
vCPU 2 vCPU × 39,600s × $0.0000276944 $2.19
Memory 4 GB × 39,600s × $0.0000036667 $0.58
Snapshot storage ~6 GB × $0.08/GB-month $0.48
Total ~$3.25

See Why MicroVMs for comparison vs EKS (~43% savings).


References


Network Egress Control (Layer 7)

Note: This section provides architectural guidance and implementation patterns for production deployments. None of the egress control mechanisms below are implemented in this POC — MicroVMs have unrestricted internet access by default. These options are documented for teams planning production deployment with compliance requirements.

MicroVMs have full internet access by default via the INTERNET_EGRESS network connector. For production deployments where you need to control which domains user code can reach (e.g., block unauthorized data exfiltration, restrict to approved APIs only), you can implement Layer 7 egress filtering.

Option A: AWS Network Firewall

Route MicroVM traffic through a VPC with AWS Network Firewall for infrastructure-enforced domain filtering.

Architecture

┌─────────────┐     ┌──────────────────┐     ┌─────────────────────┐     ┌──────────┐
│  MicroVM    │────▶│  VPC NAT Gateway │────▶│  AWS Network        │────▶│ Internet │
│  (Lambda)   │     │  (private subnet)│     │  Firewall           │     │          │
└─────────────┘     └──────────────────┘     │  (L7 domain rules)  │     └──────────┘
                                             └─────────────────────┘

Instead of the default INTERNET_EGRESS connector, MicroVMs are launched into a VPC private subnet with a NAT Gateway. All outbound traffic passes through AWS Network Firewall, which inspects TLS SNI (Server Name Indication) to filter by domain.

Setup Steps

1. Create a VPC with Network Firewall
# Create VPC with public + private + firewall subnets
aws ec2 create-vpc --cidr-block 10.0.0.0/16

# Private subnet (MicroVMs egress here)
aws ec2 create-subnet --vpc-id vpc-xxx --cidr-block 10.0.1.0/24

# Firewall subnet (Network Firewall ENIs)
aws ec2 create-subnet --vpc-id vpc-xxx --cidr-block 10.0.2.0/24

# Public subnet (NAT Gateway → Internet Gateway)
aws ec2 create-subnet --vpc-id vpc-xxx --cidr-block 10.0.3.0/24
2. Create Network Firewall Domain Allowlist
# Create a stateful rule group with domain filtering
aws network-firewall create-rule-group \
  --rule-group-name "microvm-egress-allowlist" \
  --type STATEFUL \
  --capacity 100 \
  --rule-group '{
    "RulesSource": {
      "RulesSourceList": {
        "Targets": [
          ".amazonaws.com",
          ".aws.amazon.com",
          "pypi.org",
          "files.pythonhosted.org",
          "github.com",
          "raw.githubusercontent.com",
          "api.openai.com",
          "bedrock-runtime.us-west-2.amazonaws.com"
        ],
        "TargetTypes": ["TLS_SNI", "HTTP_HOST"],
        "GeneratedRulesType": "ALLOWLIST"
      }
    }
  }'

This allows MicroVMs to reach:

  • AWS services (S3, DynamoDB, Athena, Bedrock) — required for data access
  • PyPI — for pip install of user packages
  • GitHub — for package downloads that reference GitHub
  • Everything else is BLOCKED — no data exfiltration to unauthorized endpoints
3. Create Firewall Policy
aws network-firewall create-firewall-policy \
  --firewall-policy-name "microvm-egress-policy" \
  --firewall-policy '{
    "StatelessDefaultActions": ["aws:forward_to_sfe"],
    "StatelessFragmentDefaultActions": ["aws:forward_to_sfe"],
    "StatefulRuleGroupReferences": [
      {
        "ResourceArn": "arn:aws:network-firewall:us-west-2:ACCOUNT:stateful-rulegroup/microvm-egress-allowlist"
      }
    ]
  }'
4. Deploy the Firewall
aws network-firewall create-firewall \
  --firewall-name "microvm-egress-firewall" \
  --vpc-id vpc-xxx \
  --subnet-mappings SubnetId=subnet-firewall \
  --firewall-policy-arn "arn:aws:network-firewall:us-west-2:ACCOUNT:firewall-policy/microvm-egress-policy"
5. Route MicroVM Traffic Through Firewall

Update route tables so the private subnet (where MicroVMs run) sends 0.0.0.0/0 traffic to the Network Firewall endpoint, which then forwards allowed traffic to the NAT Gateway → Internet.

# Private subnet route table → Firewall endpoint
aws ec2 create-route \
  --route-table-id rtb-private \
  --destination-cidr-block 0.0.0.0/0 \
  --vpc-endpoint-id vpce-firewall-endpoint
6. Configure MicroVM Network Connector

Replace the INTERNET_EGRESS connector with a VPC connector pointing to the private subnet:

# In scripts/config.sh or environment:
export MICROVM_EGRESS_CONNECTOR="arn:aws:lambda:us-west-2:ACCOUNT:network-connector:vpc-connector-private-subnet"

Policy Examples

Minimal (data access only):

ALLOW: .amazonaws.com (S3, DynamoDB, Athena, Bedrock)
DENY: all others

Standard (data + packages):

ALLOW: .amazonaws.com, pypi.org, files.pythonhosted.org
DENY: all others

Permissive (data + packages + APIs):

ALLOW: .amazonaws.com, pypi.org, files.pythonhosted.org, api.github.com, *.openai.com
DENY: all others

Monitoring & Audit

Network Firewall logs all allowed/denied connections to CloudWatch Logs or S3:

aws network-firewall update-logging-configuration \
  --firewall-arn arn:aws:network-firewall:... \
  --logging-configuration '{
    "LogDestinationConfigs": [{
      "LogType": "ALERT",
      "LogDestinationType": "CloudWatchLogs",
      "LogDestination": {
        "logGroup": "/aws/network-firewall/microvm-egress"
      }
    }]
  }'

This gives you an audit trail of every domain a MicroVM tried to reach — useful for compliance and detecting unauthorized access patterns.

Cost Considerations

Component Cost
Network Firewall $0.395/hr per AZ ($285/mo)
Traffic processing $0.065/GB
NAT Gateway $0.045/hr + $0.045/GB

For development/demo, use the default INTERNET_EGRESS connector (no VPC needed). For production with compliance requirements, the Network Firewall adds ~$300/mo fixed cost plus per-GB processing.

Option B: In-VM Transparent Egress Proxy

An alternative to AWS Network Firewall — bake a lightweight policy-driven proxy inside the MicroVM image. Similar to how Cilium enforces L7 policies in Kubernetes pods via eBPF, but implemented as an application-level proxy since MicroVMs don't expose the host kernel.

Architecture

┌─────────────────────────────────────────────────────────┐
│  MicroVM                                                │
│                                                         │
│  ┌──────────┐     ┌──────────────────-─┐     ┌────────┐ │
│  │ User Code│────▶│ Egress Proxy       │────▶│Network │─┼──▶ Internet
│  │ (Python) │     │ (localhost:8888)   │     │        │ │
│  └──────────┘     │ • Domain allowlist │     └────────┘ │
│                   │ • Path rules       │                │
│  HTTP_PROXY=      │ • Rate limiting    │                │
│  localhost:8888   │ • Audit logging    │                │
│                   └────────┬──────-────┘                │
│                            │                            │
│                   ┌────────▼────────┐                   │
│                   │ Policy (from S3)│                   │
│                   └─────────────────┘                   │
└─────────────────────────────────────────────────────────┘

How it works

  1. A small proxy binary (~5MB) is baked into the Docker image
  2. At VM boot, the proxy starts and fetches policy from S3: s3://bucket/policies/egress-policy.yaml
  3. HTTP_PROXY / HTTPS_PROXY env vars route all Python HTTP traffic through it
  4. Every outbound request is checked against the policy — allowed requests pass, denied requests return 403

Policy Format (centrally managed via S3)

# s3://artifacts-bucket/policies/egress-policy.yaml
version: 1
default: deny

rules:
  - action: allow
    domains: ["*.amazonaws.com", "*.aws.amazon.com"]
    description: "AWS API access (S3, DynamoDB, Athena, Bedrock)"

  - action: allow
    domains: ["pypi.org", "files.pythonhosted.org"]
    description: "Python package installation"

  - action: allow
    domains: ["api.github.com"]
    paths: ["/repos/*"]
    description: "GitHub API (read-only)"

  - action: deny
    domains: ["*"]
    log: true
    description: "Default deny — block all other egress"

Update the YAML in S3 → all new MicroVMs pick up the policy on launch. For running VMs, the proxy can poll S3 periodically for hot-reload.

Implementation

Multiple open-source proxies can serve this role — no custom binary needed:

Proxy Size L7 Depth Best For
Squid ~20MB Domain-level Simple allowlists, proven in production
mitmproxy ~50MB Full L7 (path, headers, body) Dynamic policies, Python scripting
Tinyproxy ~100KB Domain-level Minimal footprint
Envoy ~30MB Full L7 + rate limiting Istio-like policies without K8s

Example with Squid (simplest):

# Dockerfile addition
RUN apt-get update && apt-get install -y squid && rm -rf /var/lib/apt/lists/*
COPY squid.conf /etc/squid/squid.conf
ENV HTTP_PROXY=http://localhost:3128
ENV HTTPS_PROXY=http://localhost:3128
ENV NO_PROXY=localhost,127.0.0.1,169.254.169.254
# squid.conf — domain allowlist
acl allowed_domains dstdomain .amazonaws.com .aws.amazon.com
acl allowed_domains dstdomain pypi.org files.pythonhosted.org
acl allowed_domains dstdomain api.github.com raw.githubusercontent.com

http_access allow allowed_domains
http_access deny all

http_port 3128

Example with mitmproxy (full L7 with path rules):

RUN pip install mitmproxy
COPY egress_policy.py /opt/egress_policy.py
ENV HTTP_PROXY=http://localhost:8888
ENV HTTPS_PROXY=http://localhost:8888
ENV NO_PROXY=localhost,127.0.0.1,169.254.169.254
# egress_policy.py — mitmproxy addon with S3-loaded policy
import json, boto3, mitmproxy.http

POLICY = None

def load_policy():
    global POLICY
    s3 = boto3.client('s3')
    obj = s3.get_object(Bucket='artifacts-bucket', Key='policies/egress-policy.json')
    POLICY = json.loads(obj['Body'].read())

def request(flow: mitmproxy.http.HTTPFlow):
    if not POLICY:
        load_policy()
    domain = flow.request.host
    path = flow.request.path
    
    for rule in POLICY.get('rules', []):
        if any(domain.endswith(d.lstrip('*')) for d in rule['domains']):
            if rule['action'] == 'allow':
                return  # Allow
    
    # Default deny
    flow.response = mitmproxy.http.Response.make(403, b"Blocked by egress policy")

Boot sequence (app/server.py):

# Start proxy before accepting requests
import subprocess
subprocess.Popen(["squid", "-N"])  # or: ["mitmdump", "-s", "/opt/egress_policy.py", "-p", "8888"]

Security Hardening

To prevent user code from bypassing the proxy:

  • iptables rules: Force all port 80/443 traffic through the proxy (requires CAP_NET_ADMIN)
  • Read-only env vars: Set HTTP_PROXY in the image (cannot be unset at runtime)
  • Binary integrity: Verify proxy binary hash at boot

Comparison: Network Firewall vs In-VM Proxy

Feature AWS Network Firewall In-VM Transparent Proxy
Monthly cost ~$300 fixed + per-GB $0 (runs inside VM)
Domain filtering ✅ (TLS SNI inspection) ✅ (CONNECT tunnel)
Path-level rules ✅ (/api/v1/* patterns)
Header inspection ✅ (inspect/inject headers)
Rate limiting ✅ (per-domain limits)
Request body inspection ✅ (block large uploads)
VPC required ✅ (subnets + NAT + routing) ❌ (works with default INTERNET_EGRESS)
Setup complexity High Low (binary + env var)
Enforcement level Network (cannot bypass) Application (env-var based)
Bypass risk None (infra-enforced) Low (mitigated with iptables)
Central policy AWS Console / API S3 YAML file
Audit logging CloudWatch (async) Inline stdout (real-time)
Hot policy reload Immediate (rule update) Poll-based (60s)

Recommendation:

  • Compliance/regulated workloads → AWS Network Firewall (cannot be bypassed)
  • Cost-sensitive / flexible policies → In-VM Proxy (zero infra cost, path-level rules)
  • Defense in depth → Both (Network Firewall as hard boundary + proxy for fine-grained L7 rules)

License

Apache License 2.0. See LICENSE.

About

Lambda MicroVM Notebook — Interactive Python/SQL notebook powered by AWS Lambda MicroVMs with AI assistant, @PARAM widgets, and Plotly charts.

Topics

Resources

Code of conduct

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages