|
1 | | -# Running a chat-built workflow at the edge |
2 | | - |
3 | | -You author a workflow by chatting with the Builder. The result is a **cloud** |
4 | | -`uw_*` UserWorkflow spec that runs in the Convilyn cloud. On the edge, the |
5 | | -`convilyn-edge` SDK runs a workflow as a composition of **typed operators**, where |
6 | | -the model step is a [`ModelOperator`](https://pypi.org/project/convilyn-edge/) — |
7 | | -"run typed, schema-constrained inference at a chosen placement", returning a |
8 | | -validated `ModelResult`, **never a bare string**. |
9 | | - |
10 | | -Those are two different representations. This page documents the **first, low-cost |
11 | | -bridge between them** — Path A — which needs **zero platform change**. |
12 | | - |
13 | | -## Two paths |
14 | | - |
15 | | -| | Path A (this page) | Path B (follow-on) | |
16 | | -|---|---|---| |
17 | | -| **What runs the model** | The published cloud workflow, over the consumer SDK | A device-local SLM, via the on-device compute interface | |
18 | | -| **Placement** | `"cloud"` | `"edge"` | |
19 | | -| **Platform change** | **None** — the `uw_*` already runs | Compile `uw_*` → an edge-deployable bundle | |
20 | | -| **When** | Available today | Not yet generally available — see [Path B](#path-b--the-cloudevice-bundle-delivery-chain) | |
21 | | - |
22 | | -Path A is the recommended starting point: ship the workflow to a device today by |
23 | | -wrapping it, and move the model on-device later without changing the workflow. |
24 | | - |
25 | | -## The pattern |
26 | | - |
27 | | -A `ModelOperator`'s `placement="cloud"` implementation wraps |
28 | | -`client.goals.run(user_workflow_id="uw_…")`: |
29 | | - |
30 | | -```python |
31 | | -job = await client.goals.run(user_workflow_id="uw_acme.pos_error_explainer", slots={...}) |
32 | | -# terminal + message → ModelResult(status="success", output=..., evidence=(job cite,)) |
33 | | -# failed / timed out → ModelResult(status="unavailable") # offline-first fallback |
34 | | -``` |
35 | | - |
36 | | -> Discover / manage the `uw_` ids you own with the typed |
37 | | -> `client.user_workflows` namespace (`list` / `get` / `runs` / `export` |
38 | | -> / `delete`) — no raw endpoint calls needed. |
39 | | -
|
40 | | -- **Generic reference:** [`examples/10_uw_as_edge_operator.py`](../examples/10_uw_as_edge_operator.py) |
41 | | - — a scenario-free `UserWorkflowModelOperator` with a self-verifying offline demo. |
42 | | - |
43 | | -## Boundary — what the adapter must NOT do |
44 | | - |
45 | | -The bridge is deliberately thin. The **server holds every deterministic safety |
46 | | -gate** — redaction, budget, retry, cycle detection, tool permission — and |
47 | | -**re-grounds every value** the workflow returns. The adapter only *submits* a job |
48 | | -and *reads* the typed result; it re-implements none of those gates. |
49 | | - |
50 | | -## Offline-first |
51 | | - |
52 | | -Any failure / timeout / non-terminal outcome maps to `status="unavailable"` (or |
53 | | -`"uncertain"`) — **never raises past the model boundary** — so the workflow takes |
54 | | -its fixed fallback path and the device keeps working when the cloud is unreachable. |
55 | | -The finished cloud job is cited as `Evidence` (`convilyn://jobs/<id>`) so an auditor |
56 | | -can pull its full trace; the device is never a second source of truth. |
57 | | - |
58 | | -## Path B — the cloud→device bundle delivery chain |
59 | | - |
60 | | -Path B compiles the same chat-built `uw_*` into an **edge-deployable bundle** the |
61 | | -device pulls, verifies, and runs with a local model. The platform half is shipped |
62 | | -(dark — every step sits behind `edge_push_transport_enabled`, default OFF); the |
63 | | -device-run half is hardware-gated. The actual API sequence: |
64 | | - |
65 | | -1. **Author** — build the workflow by chat as usual. To be edge-compilable it must |
66 | | - declare an author intent, e.g. |
67 | | - `agent_config.provider_intent = {"reason": {"edge_eligible": true, "residency": "edge"}}` |
68 | | - (the intent says *what you need* — never a model id or silicon). |
69 | | -2. **Compile + persist** — an operator seeds the bundle: |
70 | | - `POST /api/v1/edge/bundles {"spec_id": "uw_…", "device_profile": "jetson_orin_8gb"}` |
71 | | - (optionally a `device_manifest` the device itself reported). Server-side this is |
72 | | - a **deterministic compiler** — the single provider resolver + an engine→format |
73 | | - registry; same spec + same device always compile a byte-identical |
74 | | - references-only manifest → `{"bundle_id": "bundle-…"}`. |
75 | | -3. **Stage assets** — the same call uploads the bundle's weight artifacts |
76 | | - (GGUF / ONNX / TensorRT) to the edge-asset bucket, keyed exactly as the push |
77 | | - endpoint signs, with a `content-sha256` integrity pin as object metadata. |
78 | | -4. **Device pull** — the device calls |
79 | | - `POST /api/v1/edge/push {"bundle_id": …, "device_id": …}` and receives a |
80 | | - `DeviceInstallablePayload`: every reference resolved to a short-TTL signed |
81 | | - URL, references only, never secrets or raw bytes. |
82 | | -5. **Verify + install + run offline** — the device fetches each URL, **re-hashes |
83 | | - against the manifest's digest** (the device is the integrity authority), |
84 | | - installs, and runs the workflow fully offline through the on-device |
85 | | - mini-orchestrator — the *same* graph and the same seven deterministic safety |
86 | | - gates the server runs, re-bound locally. Results queue and reconcile |
87 | | - exactly-once when connectivity returns. |
88 | | - |
89 | | -Real-hardware acceptance for steps 4-5 lives in `backend-api/tests/e2e/edge/` |
90 | | -(`jetson_hardware`-gated; arms with `RUN_JETSON_HARDWARE_E2E=true` + a plugged-in |
91 | | -device — the env contract is in that directory's `conftest.py`). |
| 1 | +# Running a chat-built workflow at the edge |
| 2 | + |
| 3 | +You author a workflow by chatting with the Builder. The result is a **cloud** |
| 4 | +`uw_*` UserWorkflow spec that runs in the Convilyn cloud. On the edge, the |
| 5 | +`convilyn-edge` SDK runs a workflow as a composition of **typed operators**, where |
| 6 | +the model step is a [`ModelOperator`](https://pypi.org/project/convilyn-edge/) — |
| 7 | +"run typed, schema-constrained inference at a chosen placement", returning a |
| 8 | +validated `ModelResult`, **never a bare string**. |
| 9 | + |
| 10 | +Those are two different representations. This page documents the **first, low-cost |
| 11 | +bridge between them** — Path A — which needs **zero platform change**. |
| 12 | + |
| 13 | +## Two paths |
| 14 | + |
| 15 | +| | Path A (this page) | Path B (follow-on) | |
| 16 | +|---|---|---| |
| 17 | +| **What runs the model** | The published cloud workflow, over the consumer SDK | A device-local SLM, via the on-device compute interface | |
| 18 | +| **Placement** | `"cloud"` | `"edge"` | |
| 19 | +| **Platform change** | **None** — the `uw_*` already runs | Compile `uw_*` → an edge-deployable bundle | |
| 20 | +| **When** | Available today | Not yet generally available — see [Path B](#path-b--the-cloudevice-bundle-delivery-chain) | |
| 21 | + |
| 22 | +Path A is the recommended starting point: ship the workflow to a device today by |
| 23 | +wrapping it, and move the model on-device later without changing the workflow. |
| 24 | + |
| 25 | +## The pattern |
| 26 | + |
| 27 | +A `ModelOperator`'s `placement="cloud"` implementation wraps |
| 28 | +`client.goals.run(user_workflow_id="uw_…")`: |
| 29 | + |
| 30 | +```python |
| 31 | +job = await client.goals.run(user_workflow_id="uw_acme.pos_error_explainer", slots={...}) |
| 32 | +# terminal + message → ModelResult(status="success", output=..., evidence=(job cite,)) |
| 33 | +# failed / timed out → ModelResult(status="unavailable") # offline-first fallback |
| 34 | +``` |
| 35 | + |
| 36 | +> Discover / manage the `uw_` ids you own with the typed |
| 37 | +> `client.user_workflows` namespace (`list` / `get` / `runs` / `export` |
| 38 | +> / `delete`) — no raw endpoint calls needed. |
| 39 | +
|
| 40 | +- **Generic reference:** [`examples/10_uw_as_edge_operator.py`](../examples/10_uw_as_edge_operator.py) |
| 41 | + — a scenario-free `UserWorkflowModelOperator` with a self-verifying offline demo. |
| 42 | + |
| 43 | +## Boundary — what the adapter must NOT do |
| 44 | + |
| 45 | +The bridge is deliberately thin. The **server holds every deterministic safety |
| 46 | +gate** — redaction, budget, retry, cycle detection, tool permission — and |
| 47 | +**re-grounds every value** the workflow returns. The adapter only *submits* a job |
| 48 | +and *reads* the typed result; it re-implements none of those gates. |
| 49 | + |
| 50 | +## Offline-first |
| 51 | + |
| 52 | +Any failure / timeout / non-terminal outcome maps to `status="unavailable"` (or |
| 53 | +`"uncertain"`) — **never raises past the model boundary** — so the workflow takes |
| 54 | +its fixed fallback path and the device keeps working when the cloud is unreachable. |
| 55 | +The finished cloud job is cited as `Evidence` (`convilyn://jobs/<id>`) so an auditor |
| 56 | +can pull its full trace; the device is never a second source of truth. |
| 57 | + |
| 58 | +## Path B — the cloud→device bundle delivery chain |
| 59 | + |
| 60 | +Path B compiles the same chat-built `uw_*` into an **edge-deployable bundle** the |
| 61 | +device pulls, verifies, and runs with a local model. The platform half is shipped |
| 62 | +(dark — every step sits behind `edge_push_transport_enabled`, default OFF); the |
| 63 | +device-run half is hardware-gated. The actual API sequence: |
| 64 | + |
| 65 | +1. **Author** — build the workflow by chat as usual. To be edge-compilable it must |
| 66 | + declare an author intent, e.g. |
| 67 | + `agent_config.provider_intent = {"reason": {"edge_eligible": true, "residency": "edge"}}` |
| 68 | + (the intent says *what you need* — never a model id or silicon). |
| 69 | +2. **Compile + persist** — an operator seeds the bundle: |
| 70 | + `POST /api/v1/edge/bundles {"spec_id": "uw_…", "device_profile": "jetson_orin_8gb"}` |
| 71 | + (optionally a `device_manifest` the device itself reported). Server-side this is |
| 72 | + a **deterministic compiler** — the single provider resolver + an engine→format |
| 73 | + registry; same spec + same device always compile a byte-identical |
| 74 | + references-only manifest → `{"bundle_id": "bundle-…"}`. |
| 75 | +3. **Stage assets** — the same call uploads the bundle's weight artifacts |
| 76 | + (GGUF / ONNX / TensorRT) to the edge-asset bucket, keyed exactly as the push |
| 77 | + endpoint signs, with a `content-sha256` integrity pin as object metadata. |
| 78 | +4. **Device pull** — the device calls |
| 79 | + `POST /api/v1/edge/push {"bundle_id": …, "device_id": …}` and receives a |
| 80 | + `DeviceInstallablePayload`: every reference resolved to a short-TTL signed |
| 81 | + URL, references only, never secrets or raw bytes. |
| 82 | +5. **Verify + install + run offline** — the device fetches each URL, **re-hashes |
| 83 | + against the manifest's digest** (the device is the integrity authority), |
| 84 | + installs, and runs the workflow fully offline through the on-device |
| 85 | + mini-orchestrator — the *same* graph and the same seven deterministic safety |
| 86 | + gates the server runs, re-bound locally. Results queue and reconcile |
| 87 | + exactly-once when connectivity returns. |
| 88 | + |
| 89 | +Real-hardware acceptance for steps 4-5 lives in `backend-api/tests/e2e/edge/` |
| 90 | +(`jetson_hardware`-gated; arms with `RUN_JETSON_HARDWARE_E2E=true` + a plugged-in |
| 91 | +device — the env contract is in that directory's `conftest.py`). |
0 commit comments