TwinOps

GitOps control plane for industrial digital twins (OpenUSD + Omniverse).

Evidence: docs/EVIDENCE.md

TwinOps lifecycle — Desired PLM/Git → Rendered OpenUSD → Observed telemetry → Drift → Reconcile → SYNCED
(MP4 for smoother local playback)

Case study: TwinOps Control Plane · Andrey Lesnikov

What it does

• Reconciles PLM metadata, OpenUSD scene composition, and live telemetry into a versioned digital-twin runtime
• Runs a Kubernetes operator over DigitalTwin CRs with immutable output revisions (ConfigMap / OCI / S3)
• Detects three-way drift (desired vs rendered vs observed) and drives reconcile toward SYNCED

Why

• Digital-twin pipelines need GitOps-style desired state, not one-off scene exports
• Operators need observable drift and deterministic builds without treating Omniverse Kit as mandatory

Quickstart

make install
make live-demo
make live-demo-smoke

Architecture

Control plane first: materialize → compose (inline or Job) → three-way drift → immutable output revisions. Kit / WebRTC are optional runtimes.

flowchart LR
  subgraph Sources
    PLM[PLM / ERP]
    Git[Git twin.yaml + USD]
    MQTT[MQTT / IoT]
  end

  subgraph ControlPlane["Control plane (the product)"]
    CR[DigitalTwin CR]
    Mat[Materialize + digest]
    Build["Build: inline | Job"]
    USD[OpenUSD compose]
    Drift[Three-way drift]
    Out[Immutable publish]
  end

  subgraph Durable
    CM["ConfigMap rN"]
    OCI[OCI registry]
    S3[S3 / MinIO]
  end

  subgraph Optional["Optional runtimes"]
    Web[Web UI / live API]
    Kit[Omniverse Kit]
    RTC[WebRTC sidecar]
  end

  PLM --> Git
  Git --> CR
  CR --> Mat --> Build --> USD --> Drift --> Out
  MQTT --> Drift
  Out --> CM
  Out --> OCI
  Out --> S3
  Drift --> Web
  Out --> Kit
  Out --> RTC

Loading

Full one-pager: docs/architecture-one-pager.md · operator: docs/operator.md · release: docs/release-1.4.md

---

Why this exists

Most Omniverse demos show a beautiful 3D scene.
Most Kubernetes demos show Helm, Terraform, and autoscaling.

TwinOps is not “Omniverse in Kubernetes” and not “USD files + Grafana”. It is a control plane: reconcile desired PLM / rendered OpenUSD / observed telemetry into GitOps artifacts, then drive optional runtimes (Kit, web UI, lab WebRTC) through a stable highlight contract — including on a laptop without a GPU.

TwinOps connects both worlds:

DevOps	TwinOps
Kubernetes manifest	DigitalTwin manifest
container image	USD asset
configuration overlay	USD layer
environment	scene variant
GitOps reconciliation	stage composition
deployment rollout	twin revision rollout
runtime metrics	telemetry and scene state
drift detection	PLM / scene / IoT drift
rollback	previous USD composition

The distinctive feature is three-way drift detection (see diagram above):

Plane	Source	Role
Desired	Git + PLM mappings	Engineering intent
Rendered	Composed OpenUSD	What the twin scene encodes
Observed	MQTT / IoT	Live factory signal

Frozen contracts: docs/stability.md.

---

2-minute live demo

No GPU required.

git clone https://github.com/justrunme/twinops-control-plane.git
cd twinops-control-plane
make install
make live-demo

Open http://127.0.0.1:8080/

1. Trigger heat spike      → CRITICAL / DRIFT + scene highlight
2. Apply reconciliation    → USD overlay + healed telemetry
3. Twin returns to SYNCED  → timeline + mock Kit viewport calm

Smoke check without a browser:

make live-demo-smoke

Canonical productization scenario (persist + replay + artifacts):

make e2e-demo
make streaming-sidecar-smoke

Full walkthrough: docs/demo.md · docs/e2e-demo.md · docs/streaming-sidecar.md

---

Demo

Terminal capture of the CI end-to-end flow (scripts/e2e_demo.sh): heat spike → CRITICAL → reconcile → SYNCED → incident replay.

Offline self-healing compose demo:

make demo

Live UI demo: docs/demo.md · E2E walkthrough: docs/e2e-demo.md

Results

Measured numbers from CI (not estimates). Full table: docs/results.md · machine-readable: docs/results.json.

Metric	Value	Source run
Smoke: spike → SYNCED	0.43 s	38066458460 (Python 3.12)
E2E: spike → replay → persistence	1.90 s	same run
Reconcile changes to SYNCED	3	same run (reconcile.json)
Incident replay verified	true (stepsPlayed=6)	same run (replay-verify.json)

---

Quickstart extras

Offline CLI demo

make demo

Produces composed USDA layers, a drift HTML report, and a GitOps reconciliation proposal.

MQTT publish + ingest

make mqtt-smoke
twinopsctl mqtt topics

Starts Mosquitto, publishes simulator telemetry, injects an external PLC heat spike, and asserts CRITICAL drift. Topic catalog: examples/assembly-line/mqtt-topics.json / GET /api/mqtt/topics.

Mock PLM adapter

twinopsctl plm show
twinopsctl plm get 1004711 --catalog examples/assembly-line/plm-catalog.json
twinopsctl plm compare

See docs/plm-adapter.md.

Omniverse highlight contract (no GPU)

make serve
# other terminal:
make scene-live      # fetch + validate /api/scene
make scene-highlight # Kit stub client
make scene           # offline JSON + HTML highlight report

See docs/omniverse.md.

Local DX probes

make doctor
make live-status
make timeline
twinopsctl proposal
twinopsctl live spike
twinopsctl live reconcile
twinopsctl incident export --from-url http://127.0.0.1:8080 --out /tmp/incident.json
twinopsctl incident replay examples/assembly-line/incident-heat-spike.json \
  --desired examples/assembly-line/desired.yaml \
  --stage examples/assembly-line/generated/root.usda \
  --observed examples/assembly-line/telemetry.json --json
make scene-live
twinopsctl openapi --out /tmp/twinops-openapi.json
make verify-all

Docs index: docs/README.md. Security: SECURITY.md.

Dev UI (Vite hot reload)

make serve      # terminal 1 — API :8080
make web-dev    # terminal 2 — UI  :5173

Kubernetes operator

make operator-demo              # out-of-cluster manager (k3d/kind)
make operator-incluster-e2e     # docker build → kind load → Helm → restart recovery
make operator-demo-cleanup

In-cluster path publishes a durable bundle.tar.gz on the twin (status.output.uri). See docs/operator.md, docs/images.md.

Tests

make test
make go-test
make verify-all

See CONTRIBUTING.md.

---

Two documents, two roles

TwinOps uses the same apiVersion family for two different documents. Do not mix them up.

1) Twin manifest (compiler input — lives inside the artifact)

Stored as twin.yaml in a ConfigMap / tarball. Consumed by twinopsctl build.

# twin.yaml — what to compose (OpenUSD + PLM + telemetry mappings)
apiVersion: twinops.io/v1alpha1
kind: TwinManifest   # conceptual name; file may say DigitalTwin historically
metadata:
  name: assembly-line-a
spec:
  source:
    baseStage: assets/root.usda   # nested paths preserved in URL/tar artifacts
  configuration:
    variant: high-throughput
  telemetry:
    provider: mqtt
    endpoint: mqtt://factory-broker
    mappings:
      - topic: factory/robot-01/temperature
        prim: /World/Factory/LineA/Robot01
        attribute: twinops:temperature
  plm:
    provider: mock
    mappings:
      - itemId: "1004711"
        revision: "C"
        prim: /World/Factory/LineA/Robot01

2) DigitalTwin CR (Kubernetes — tells the operator where the artifact is)

# DigitalTwin CR — operator control loop, not the scene itself
apiVersion: twinops.io/v1alpha1
kind: DigitalTwin
metadata:
  name: assembly-line-a
  namespace: twinops-system
spec:
  artifactSource:
    configMapName: assembly-line-inputs   # or url: https://…/bundle.tar.gz
    # expectedDigest: sha256:…
  intervalSeconds: 30
  # outputPublish.enabled defaults true → status.output.uri = configmap://…/assembly-line-a-output

The compiler turns the manifest into OpenUSD overlay layers with twinops:* attributes.
The CR drives materialize → build → drift → durable bundle.tar.gz publish.

---

Repository layout

twinops-control-plane/
├── python/twinops/          # Compiler, drift, live API, PLM mock, CLI
├── examples/assembly-line/  # Demo factory line + sample USDA + PLM/MQTT catalogs
├── scripts/                 # live-demo / mqtt-smoke / operator-demo / sync helpers
├── docs/                    # Architecture, USD model, ADRs, roadmap
├── api/                     # DigitalTwin CRD types (Go)
├── controllers/             # Kubernetes operator controllers
├── cmd/manager/             # Operator manager entrypoint
├── deploy/helm/             # Helm chart for the operator
├── Dockerfile.live          # Demo live API + web UI image
├── Dockerfile.operator      # Operator manager image
├── extensions/              # Omniverse Kit highlight stub
└── web/                     # Live control-plane UI + mock Kit viewport

Compiler, drift engine, operator, GitOps, observability, and mock adapters run without an NVIDIA GPU. A GPU is required only for Kit rendering and the host NVENC streaming bridge.

---

Demo story: Self-Healing Production Line

Robot → Conveyor → Scanner → Packaging

1. Change desired robot revision in Git
2. TwinOps compiles a new USD overlay layer
3. Drift engine compares desired / rendered / observed state
4. Scene metadata highlights revision or telemetry drift
5. A reconciliation proposal restores the desired composition

Milestones 1–2 deliver composition, drift detection, HTML report, and a reconciliation proposal. The Kubernetes operator reconciles DigitalTwin CRs via twinopsctl.

---

Roadmap (summary)

Track	Focus	Status
0–5	Compiler, drift, live API, web demo	done
Operator	CRD, Helm, durable ConfigMap/OCI/S3 output, Job isolation, in-cluster E2E	done (1.4.3)
Kit / media	Highlight contract, lab WebRTC, single-session NVENC ingest	lab / optional
Next	Multi-site fleet, large non-ConfigMap Job inputs	not claimed

See full docs/roadmap.md and docs/architecture.md.

---

What we will not claim yet

Honest positioning: single-twin pilot / reference control plane, not a plant platform.

This project does not claim:

• multi-site / multi-tenant enterprise readiness
• NVCF or production Omniverse App Streaming
• vendor-specific PLM product SDKs (generic File/REST adapters only)
• multi-user GPU streaming farm

PLM integration ships as mock + File/REST adapters.

---

License

Apache License 2.0 — see LICENSE.