Skip to content

feat(sandbox): Add kubernetes sandbox support - #126

Open
whatever wants to merge 3 commits into
mainfrom
feat/kubernetes-sandbox
Open

feat(sandbox): Add kubernetes sandbox support#126
whatever wants to merge 3 commits into
mainfrom
feat/kubernetes-sandbox

Conversation

@whatever

@whatever whatever commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds a sandbox backend factory (CLEARWING_SANDBOX_BACKEND=kubernetes) so cwpro deployments without Docker-in-Docker can run hunter agent containers as Kubernetes Pods
  • KubeSandboxContainer implements the same SandboxContainer interface (start/exec/write_file/read_file/copy_tree_into/stop) backed by K8s Pod lifecycle + exec
  • On-cluster image building via Kaniko Jobs with content-hash tags and registry push (CLEARWING_SANDBOX_REGISTRY)
  • HunterSandbox.build_image() short-circuits for k8s: resolves from CLEARWING_SANDBOX_IMAGE override or builds on-cluster
  • spawn() uses create_sandbox() factory dispatch instead of direct SandboxContainer instantiation

Environment variables

Var Purpose
CLEARWING_SANDBOX_BACKEND docker (default) or kubernetes
CLEARWING_SANDBOX_IMAGE Explicit image override (skips builds)
CLEARWING_SANDBOX_REGISTRY Registry prefix for Kaniko-built images
CLEARWING_SANDBOX_NAMESPACE K8s namespace for pods/jobs (default: pod namespace)
CLEARWING_KANIKO_IMAGE Kaniko executor image
CLEARWING_KANIKO_SERVICE_ACCOUNT SA for build jobs

Test plan

  • Unit tests for factory dispatch, backend detection, image tag computation, and k8s image resolution paths (10 tests passing)
  • Integration test with a real k8s cluster (cwpro staging)
  • Verify graceful degradation when neither Docker nor k8s is available (existing _effective_agent_mode fallback)

@whatever whatever changed the title Add Kubernetes sandbox backend for cwpro-managed deployments Add kubernetes sandbox support Aug 3, 2026
@whatever whatever changed the title Add kubernetes sandbox support feat(sandbox): kubernretAdd kubernetes sandbox support Aug 3, 2026
@whatever
whatever marked this pull request as draft August 3, 2026 23:46
@whatever whatever changed the title feat(sandbox): kubernretAdd kubernetes sandbox support feat(sandbox): Add kubernetes sandbox support Aug 4, 2026
@whatever
whatever force-pushed the feat/kubernetes-sandbox branch from 3240829 to f187ec8 Compare August 5, 2026 19:17
@whatever
whatever changed the base branch from main to context-shortener-node August 5, 2026 19:17
This was referenced Aug 5, 2026
Merged
Merged
@whatever
whatever force-pushed the feat/kubernetes-sandbox branch 2 times, most recently from 24824d2 to 1e9e6f9 Compare August 5, 2026 20:44
@whatever
whatever force-pushed the context-shortener-node branch from 41a012b to 94a629f Compare August 5, 2026 20:51
@whatever
whatever force-pushed the feat/kubernetes-sandbox branch from 1e9e6f9 to 3a7d243 Compare August 5, 2026 20:51
@whatever
whatever force-pushed the context-shortener-node branch from 94a629f to cac23a9 Compare August 5, 2026 20:56
@whatever
whatever force-pushed the feat/kubernetes-sandbox branch from 3a7d243 to b29c350 Compare August 5, 2026 20:58
@whatever
whatever force-pushed the context-shortener-node branch from cac23a9 to d09e9fe Compare August 5, 2026 21:03
@whatever
whatever force-pushed the feat/kubernetes-sandbox branch from b29c350 to 34e02ca Compare August 5, 2026 21:04
jorge-garcia-le
jorge-garcia-le approved these changes Aug 5, 2026 •
edited by whatever
Loading
View reviewed changes
@whatever
whatever marked this pull request as ready for review August 5, 2026 23:20
@whatever
whatever force-pushed the feat/kubernetes-sandbox branch from 34e02ca to 6140030 Compare August 6, 2026 00:54
@whatever
whatever changed the base branch from context-shortener-node to main August 6, 2026 00:54
@whatever
whatever force-pushed the feat/kubernetes-sandbox branch from 6140030 to 8970347 Compare August 6, 2026 01:30
whatever added 3 commits August 5, 2026 22:35
@whatever
Add Kubernetes sandbox backend for cwpro-managed deployments
b251791 
Introduces a sandbox backend factory that dispatches between Docker
(default) and Kubernetes, so cwpro deployments without Docker-in-Docker
can run hunter agents as K8s Pods.

New modules:
- sandbox/factory.py: backend dispatcher (CLEARWING_SANDBOX_BACKEND env)
- sandbox/kube_sandbox.py: KubeSandboxContainer — same interface as
  SandboxContainer but backed by K8s Pods with exec-based I/O
- sandbox/kube_builder.py: on-cluster image building via Kaniko Jobs
  with content-hash tagging and registry push

HunterSandbox changes:
- build_image() short-circuits for k8s: resolves images from
  CLEARWING_SANDBOX_IMAGE override or builds on-cluster via Kaniko
- spawn() uses create_sandbox() factory instead of direct SandboxContainer

Also updates the sandbox-unavailable warning to reference k8s as an
alternative to Docker.
@whatever
Harden k8s sandbox: proper exec exit codes, stdin file transfer, GC s…
525d865 
…afety

Fixes critical issues identified during code review:

- exec() now uses WSClient with separate channels + parses the real exit
  code from channel 3 status JSON (was silently returning 0 for everything)
- write_file/copy_tree_into pipe tar directly via stdin instead of
  base64-in-shell (no arg size limits, no injection risk, single round-trip)
- Sandbox pods get activeDeadlineSeconds (3x timeout, floor 15min) so
  zombies self-terminate if our process crashes
- ownerReference set to parent pod when running in-cluster — k8s GC
  cascades deletion on parent crash
- Extracted shared kube_client.py (cached ApiClient, namespace(), owner
  ref helper) — eliminates duplicated config loading and repeated token
  file reads
@whatever
Add debug logging to kube sandbox exec
b86e547
@whatever
whatever force-pushed the feat/kubernetes-sandbox branch from 8970347 to b86e547 Compare August 6, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jorge-garcia-le jorge-garcia-le jorge-garcia-le approved these changes
@ropoctl ropoctl Awaiting requested review from ropoctl ropoctl is a code owner
@ehartford ehartford Awaiting requested review from ehartford ehartford is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@whatever @jorge-garcia-le