Skip to content

feat(canvas): build and securely deliver cloud artifacts - #73874

Merged
trunk-io[bot] merged 60 commits into
masterfrom
posthog-code/canvas-build-pipeline-phase-6
Aug 3, 2026
Merged

feat(canvas): build and securely deliver cloud artifacts#73874
trunk-io[bot] merged 60 commits into
masterfrom
posthog-code/canvas-build-pipeline-phase-6

Conversation

@k11kirky

@k11kirky k11kirky commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Problem

The canvas build pipeline shipped on top of the desktop "file system" — a generic path tree that had come to hold four different things (canvases, channels, task filings, sidebar stars), bridged to the real Channel model by name-matching. That framing made the product confusing to reason about, left the build queue able to wedge permanently, and put single-product code in posthog/api/ against the repo's own placement rules. Canvas is internal-only, so this remodel breaks compatibility deliberately instead of carrying the old surface.

Changes

  • products/canvas/: canvases become first-class rows (Canvas + CanvasSourceVersion + CanvasBuild) served at /api/projects/:id/canvases/ under a new canvas API scope, with source/versions/revert endpoints, guarded publishes, and validated capability declarations. The builder package (and its platform manifest — pinned deps, CSP, size limits, now a single source of truth) moves in too, along with the canvas agent skills.
  • Channels absorb what the tree carried: CONTEXT.md instructions, context-generation markers, and per-user stars live on the tasks Channel; task filings collapse into the existing Task.channel FK; loops attach to channels directly. One channel identity everywhere.
  • A data migration converts existing desktop trees in place (canvas UUIDs preserved; single-file source carried until the next publish), then the desktop file-system surface is deleted — viewsets, routes, folder-instruction models, MCP tools. The web-surface file system is untouched.
  • Pipeline hardening: a stuck-build sweeper (lease-expired and never-claimed builds no longer pin team capacity), the per-team capacity cap checked under an advisory lock, attribute-order-insensitive entry-HTML parsing, artifacts served with ETag/304 revalidation, bucket-stable signed URLs, and CORS (the sandboxed iframe's opaque origin fetches module scripts in CORS mode — verified by rendering real builder output in a sandboxed iframe in headless Chromium), boot-time artifact-origin config checks, and the build script baked into the sandbox image.
  • MCP tools regenerate as canvas-* and channel-*; the skills now require agents to wait out the queued build and iterate on failure diagnostics rather than exiting with a silently failed build. The building-canvases resolve step also covers composer-initiated tasks that arrive without a canvas id: build on a clearly-matching existing canvas (saying so in the reply), else create one with a descriptive name — never "Untitled canvas". Sandbox-created canvases are bound to their generating task via the stamped task-id header, and DEBUG environments post agent thread updates unconditionally (the fail-closed flag check silently dropped them in local dev).

Pairs with PostHog/code#3835 (the desktop client side); the two should land together — the old routes are gone, so a client on the old surface 404s until the client PR ships.

How did you test this code?

Automated (run locally against Postgres/Redis/ClickHouse): 235 backend tests across products/canvas/backend/tests/ (API CRUD/source/publish/edit/revert/builds/actions, build worker, sweeper, retention, artifact origin incl. ETag/expiry), products/tasks/backend/tests/ (channel instructions/stars/context-generation, loop runs and loops API retargeted to channels), and the trimmed posthog/api/file_system suites. The full migration chain (all posthog migrations + tasks + canvas + cleanup) was applied end-to-end on a database seeded with a realistic desktop tree (channels incl. personal, canvases with pins/home/context, instruction history, stars, task filings, a context-attached loop), and every migrated artifact was asserted. The builder was smoke-tested against reversed-attribute-order entry HTML. OAuth bearer-token access to the canvases API is covered too (wildcard, explicit canvas scope, and the 403 for enumerated grants predating the scope — the desktop's only auth path). Not done: no manual UI testing from this PR alone (the client half lives in the code repo).


Created with PostHog Code

@k11kirky k11kirky self-assigned this Jul 27, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Hey @k11kirky! 👋

It looks like your git author email on this PR isn't your @posthog.com address (k11kirky@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

  • Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
  • Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

Bundle size — no change

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 65.58 MiB · no change

No file changed by more than 1000 B.

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1.25 MiB · 22 files no change ███░░░░░░░ 27.7% of 4.51 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
8.14 MiB · 3,036 files no change ████████░░ 83.8% of 9.71 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB src/RootErrorBoundary.tsx
912 B ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
789 B src/scenes/ChunkLoadErrorBoundary.tsx
762 B src/index.tsx
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
285.5 KiB ../node_modules/.pnpm/posthog-js@1.410.1/node_modules/posthog-js/dist/rrweb.js
267.7 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
235.5 KiB src/taxonomy/core-filter-definitions-by-group.json
231.5 KiB ../node_modules/.pnpm/posthog-js@1.410.1/node_modules/posthog-js/dist/module.js
154.3 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.4 KiB src/lib/api.ts
94.7 KiB ../packages/quill/packages/quill/dist/index.js
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
90.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

Toolbar bundle — eager 2.19 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric Size Δ vs base Budget
Eager (shipped)
entry + static imports
2.19 MiB · 17 files no change ████░░░░░░ 38.3% of 5.72 MiB
Deferred (lazy) 2.08 MiB · 33 files no change n/a — loads on demand
Loader dist/toolbar.js 1.1 KiB no change █░░░░░░░░░ 5.8% of 19.5 KiB
Largest eagerly-shipped chunks
Size File
718.1 KiB dist/toolbar/toolbar-app-6ZY7LRCK.css
551.5 KiB dist/toolbar/chunk-chunk-WSFEVVLR.js
484.6 KiB dist/toolbar/chunk-chunk-CKRU2M7T.js
133.6 KiB dist/toolbar/chunk-chunk-2LRYFHTB.js
131.8 KiB dist/toolbar/chunk-chunk-T5KY5WYR.js
71.0 KiB dist/toolbar/toolbar-app-RJT4WE5W.js
69.0 KiB dist/toolbar/chunk-chunk-27JL52RE.js
35.6 KiB dist/toolbar/chunk-chunk-N3TQ3NGZ.js
20.9 KiB dist/toolbar/chunk-chunk-D5JG3COG.js
12.2 KiB dist/toolbar/chunk-chunk-PIK3PADE.js

Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile

Dist folder size — 🟢 -33.3 KiB (-0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1389.54 MiB · 🟢 -33.3 KiB (-0.0%)

ℹ️ MCP UI apps size — 32 app(s), 17070.3 KB JS

Built size of each MCP UI app (main.js + styles.css).

App JS CSS
debug 599.6 KB 187.7 KB
action 457.8 KB 187.7 KB
action-list 564.4 KB 187.7 KB
cohort 456.8 KB 187.7 KB
cohort-list 563.4 KB 187.7 KB
email-template 456.6 KB 187.7 KB
error-details 472.4 KB 187.7 KB
error-issue 457.5 KB 187.7 KB
error-issue-list 564.3 KB 187.7 KB
experiment 561.5 KB 187.7 KB
experiment-list 565.2 KB 187.7 KB
experiment-results 563.2 KB 187.7 KB
feature-flag 567.2 KB 187.7 KB
feature-flag-list 570.9 KB 187.7 KB
feature-flag-testing 461.0 KB 187.7 KB
insight-actors 562.2 KB 187.7 KB
invite-email-preview 456.0 KB 187.7 KB
llm-costs 559.5 KB 187.7 KB
session-recording 458.6 KB 187.7 KB
session-summary 463.9 KB 187.7 KB
survey 458.4 KB 187.7 KB
survey-global-stats 562.2 KB 187.7 KB
survey-list 565.1 KB 187.7 KB
survey-stats 562.2 KB 187.7 KB
trace-span 457.2 KB 187.7 KB
trace-span-list 564.3 KB 187.7 KB
workflow 457.1 KB 187.7 KB
workflow-list 563.7 KB 187.7 KB
loops-review 461.2 KB 187.7 KB
query-results 747.4 KB 187.7 KB
render-ui 828.0 KB 187.7 KB
visual-review-snapshots 461.6 KB 187.7 KB
Playwright — all passed

All tests passed.

View test results →

ℹ️ Backend snapshots — skipped stale run for 2f8aeeb

Skipped the snapshot commit because the branch advanced to f2d40ae while the workflow was testing 2f8aeeb.

The new commit will trigger its own snapshot update workflow. If a fresh run does not start, merge master or push an empty commit.

⚠️ Backend coverage — 91.0% of changed backend lines covered — 236 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ██████████████████░░ 91.0% (2,424 / 2,660)

File Patch Uncovered changed lines
products/canvas/backend/checks.py 23.8% 17–22, 24, 26–28, 35–36, 43–45, 52
products/tasks/backend/logic/services/modal_sandbox.py 37.5% 472–473, 494–495, 753
products/canvas/backend/tasks.py 37.9% 22, 24, 30, 32–38, 44, 46–52
products/canvas/backend/migrations/0003_migrate_desktop_tree.py 60.3% 42–43, 49, 89, 91–92, 98, 117–119, 139–140, 168–173, 184, 186–190, 197–201, 210–216, 220–227, 230–233, 237–239, 244
products/tasks/backend/facade/loops.py 69.2% 655–658
products/tasks/backend/presentation/views/channels_api.py 83.7% 128, 159, 183–184, 188–189, 194, 207, 218, 222, 232, 248, 264, 266
products/canvas/backend/build_service.py 84.5% 108, 117, 131, 134, 139, 145, 159–160, 163–165, 169–174, 180, 194, 199, 201, 207, 227, 232, 235, 238, 247, 295, 299, 348, 450–451, 510, 514, 536–537, 542, 551–552, 557, 561–562, 616, 619, 629–636, 711–717, 728–729, 732–733, 745, 899
products/tasks/backend/facade/api.py 87.8% 606, 5576, 5699, 5713, 5739, 5741, 5753, 5789, 5806, 5817, 5830, 6181
products/canvas/backend/source.py 89.7% 145, 165, 227, 241, 253–256, 266–267, 311, 343–344, 348, 358
products/canvas/backend/presentation/views.py 91.4% 134–135, 254–257, 281, 291–295, 366–367, 424–425, 462, 464, 516, 562, 576
products/canvas/backend/artifacts.py 92.4% 83, 107, 111–112, 134, 142–143
products/tasks/backend/logic/services/loop_runs.py 93.3% 691, 693
products/canvas/backend/presentation/serializers.py 97.2% 449–452
products/tasks/backend/models.py 97.4% 85

🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 30851781543 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)
Product Coverage Lines
platform_features ██░░░░░░░░░░░░░░░░░░ 12.1% 7 / 58
batch_exports ████████░░░░░░░░░░░░ 39.5% 8,808 / 22,287
demo ███████████░░░░░░░░░ 56.3% 1,497 / 2,661
warehouse_sources_queue ████████████░░░░░░░░ 59.2% 148 / 250
data_tools ██████████████░░░░░░ 70.0% 63 / 90
tasks ██████████████░░░░░░ 70.5% 33,843 / 48,021
ai_gateway ███████████████░░░░░ 75.0% 9 / 12
signals ████████████████░░░░ 81.5% 25,461 / 31,226
cdp ████████████████░░░░ 82.1% 3,320 / 4,042
data_modeling █████████████████░░░ 85.7% 7,756 / 9,054
notebooks █████████████████░░░ 86.0% 7,794 / 9,060
managed_warehouse █████████████████░░░ 86.5% 5,875 / 6,790
actions █████████████████░░░ 86.6% 717 / 828
wizard █████████████████░░░ 86.7% 1,090 / 1,257
data_warehouse █████████████████░░░ 87.5% 11,011 / 12,591
cohorts ██████████████████░░ 87.6% 6,484 / 7,398
exports ██████████████████░░ 87.9% 7,078 / 8,054
product_tours ██████████████████░░ 87.9% 1,303 / 1,482
business_knowledge ██████████████████░░ 89.0% 4,384 / 4,928
engineering_analytics ██████████████████░░ 89.4% 6,551 / 7,331
dashboards ██████████████████░░ 89.4% 5,983 / 6,693
visual_review ██████████████████░░ 89.5% 5,870 / 6,558
conversations ██████████████████░░ 90.0% 18,294 / 20,337
alerts ██████████████████░░ 90.3% 4,482 / 4,966
links ██████████████████░░ 90.6% 183 / 202
streamlit_apps ██████████████████░░ 90.7% 2,630 / 2,901
error_tracking ██████████████████░░ 91.0% 10,981 / 12,070
canvas ██████████████████░░ 91.1% 2,018 / 2,216
slack_app ██████████████████░░ 91.2% 9,802 / 10,751
stamphog ██████████████████░░ 91.3% 4,505 / 4,936
marketing_analytics ██████████████████░░ 91.5% 12,700 / 13,874
mcp_store ██████████████████░░ 92.2% 6,525 / 7,076
product_analytics ███████████████████░ 92.5% 5,849 / 6,321
managed_migrations ███████████████████░ 92.6% 1,556 / 1,681
early_access_features ███████████████████░ 92.6% 1,287 / 1,390
ai_observability ███████████████████░ 92.6% 16,036 / 17,319
mcp_analytics ███████████████████░ 92.6% 3,775 / 4,076
notifications ███████████████████░ 92.6% 1,017 / 1,098
surveys ███████████████████░ 93.2% 5,853 / 6,281
posthog_ai ███████████████████░ 93.2% 1,326 / 1,422
approvals ███████████████████░ 93.3% 3,437 / 3,682
reminders ███████████████████░ 93.4% 468 / 501
web_analytics ███████████████████░ 93.5% 15,393 / 16,470
legal_documents ███████████████████░ 93.8% 1,628 / 1,736
endpoints ███████████████████░ 94.2% 8,655 / 9,192
workflows ███████████████████░ 94.2% 7,559 / 8,021
tracing ███████████████████░ 94.5% 2,671 / 2,827
review_hog ███████████████████░ 94.6% 8,246 / 8,715
skills ███████████████████░ 94.6% 3,158 / 3,337
messaging ███████████████████░ 94.7% 2,885 / 3,048
logs ███████████████████░ 95.5% 10,491 / 10,990
experiments ███████████████████░ 95.6% 26,982 / 28,230
annotations ███████████████████░ 96.2% 732 / 761
revenue_analytics ███████████████████░ 96.3% 1,887 / 1,960
growth ███████████████████░ 96.3% 4,767 / 4,949
feature_flags ███████████████████░ 96.4% 17,488 / 18,144
replay_vision ███████████████████░ 96.4% 16,616 / 17,230
user_interviews ███████████████████░ 96.5% 2,638 / 2,734
access_control ███████████████████░ 96.9% 870 / 898
customer_analytics ███████████████████░ 97.1% 10,427 / 10,737
warehouse_sources ███████████████████░ 97.3% 360,837 / 370,830
data_catalog ████████████████████ 97.7% 2,605 / 2,665
analytics_platform ████████████████████ 98.0% 2,153 / 2,197
metrics ████████████████████ 98.2% 2,491 / 2,536
pulse ████████████████████ 98.4% 2,017 / 2,049
live_debugger ████████████████████ 99.2% 613 / 618
field_notes ████████████████████ 99.4% 158 / 159

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

⚠️ MCP snapshots — 1 updated (1 modified, 0 added, 0 deleted)

Snapshots: MCP unit test snapshots updated

Changes: 1 snapshots (1 modified, 0 added, 0 deleted)

What this means:

  • Snapshots have been automatically updated to match current output

Next steps:

  • Review the changes to ensure they're intentional
  • If unexpected, investigate what caused the output to change

Review snapshot changes →

⚠️ Django migration SQL — 8 new migrations to review

We've detected new migrations on this PR. Review the SQL output for each migration:

posthog/migrations/1285_drop_desktop_file_system.py

BEGIN;
--
-- Custom state/database change combination
--
-- (no-op)
COMMIT;

products/canvas/backend/migrations/0001_initial.py

BEGIN;
--
-- Create model Canvas
--
CREATE TABLE "posthog_canvas" ("id" uuid NOT NULL PRIMARY KEY, "name" varchar(400) NOT NULL, "template_id" varchar(64) NOT NULL, "context" text NOT NULL, "generation_task_id" uuid NULL, "pinned_at" timestamp with time zone NULL, "is_home" boolean NOT NULL, "legacy_code" text NULL, "created_at" timestamp with time zone NOT NULL, "updated_at" timestamp with time zone NOT NULL, "deleted" boolean NOT NULL, "channel_id" uuid NOT NULL, "created_by_id" integer NULL, "team_id" integer NOT NULL);
--
-- Create model CanvasBuild
--
CREATE TABLE "posthog_canvas_build" ("id" uuid NOT NULL PRIMARY KEY, "status" varchar(16) NOT NULL, "artifact_object_prefix" text NULL, "integrity" varchar(64) NULL, "diagnostics" jsonb NOT NULL, "manifest" jsonb NULL, "pinned" boolean NOT NULL, "attempt_count" integer NOT NULL CHECK ("attempt_count" >= 0), "lease_expires_at" timestamp with time zone NULL, "created_at" timestamp with time zone NOT NULL, "finished_at" timestamp with time zone NULL, "canvas_id" uuid NOT NULL, "team_id" integer NOT NULL);
--
-- Add field published_build to canvas
--
ALTER TABLE "posthog_canvas" ADD COLUMN "published_build_id" uuid NULL CONSTRAINT "posthog_canvas_published_build_id_b1d59cb6_fk_posthog_c" REFERENCES "posthog_canvas_build"("id") DEFERRABLE INITIALLY DEFERRED; SET CONSTRAINTS "posthog_canvas_published_build_id_b1d59cb6_fk_posthog_c" IMMEDIATE;
--
-- Create model CanvasSourceVersion
--
CREATE TABLE "posthog_canvas_source_version" ("id" uuid NOT NULL PRIMARY KEY, "source_hash" varchar(64) NOT NULL, "source_object_key" text NOT NULL, "source_size" integer NOT NULL CHECK ("source_size" >= 0), "task_id" uuid NULL, "task_run_id" uuid NULL, "prompt" text NULL, "created_at" timestamp with time zone NOT NULL, "canvas_id" uuid NOT NULL, "created_by_id" integer NULL, "parent_version_id" uuid NULL, "team_id" integer NOT NULL);
--
-- Add field source_version to canvasbuild
--
ALTER TABLE "posthog_canvas_build" ADD COLUMN "source_version_id" uuid NULL CONSTRAINT "posthog_canvas_build_source_version_id_10d990cb_fk_posthog_c" REFERENCES "posthog_canvas_source_version"("id") DEFERRABLE INITIALLY DEFERRED; SET CONSTRAINTS "posthog_canvas_build_source_version_id_10d990cb_fk_posthog_c" IMMEDIATE;
--
-- Add field current_source_version to canvas
--
ALTER TABLE "posthog_canvas" ADD COLUMN "current_source_version_id" uuid NULL CONSTRAINT "posthog_canvas_current_source_versi_1aac4f83_fk_posthog_c" REFERENCES "posthog_canvas_source_version"("id") DEFERRABLE INITIALLY DEFERRED; SET CONSTRAINTS "posthog_canvas_current_source_versi_1aac4f83_fk_posthog_c" IMMEDIATE;
--
-- Create index canvas_source_version_recency on field(s) canvas, -created_at of model canvassourceversion
--
CREATE INDEX "canvas_source_version_recency" ON "posthog_canvas_source_version" ("canvas_id", "created_at" DESC);
--
-- Create index canvas_build_recency on field(s) canvas, -created_at of model canvasbuild
--
CREATE INDEX "canvas_build_recency" ON "posthog_canvas_build" ("canvas_id", "created_at" DESC);
--
-- Create index canvas_build_active on field(s) team, status of model canvasbuild
--
CREATE INDEX "canvas_build_active" ON "posthog_canvas_build" ("team_id", "status") WHERE "status" IN ('queued', 'building');
--
-- Create index canvas_build_retention on field(s) finished_at of model canvasbuild
--
CREATE INDEX "canvas_build_retention" ON "posthog_canvas_build" ("finished_at") WHERE ("artifact_object_prefix" IS NOT NULL AND NOT "pinned");
--
-- Create index canvas_channel_recency on field(s) channel, -created_at of model canvas
--
CREATE INDEX "canvas_channel_recency" ON "posthog_canvas" ("channel_id", "created_at" DESC);
--
-- Create constraint unique_home_canvas_per_channel on model canvas
--
CREATE UNIQUE INDEX "unique_home_canvas_per_channel" ON "posthog_canvas" ("channel_id") WHERE (NOT "deleted" AND "is_home");
CREATE INDEX "posthog_canvas_channel_id_a6e6eb20" ON "posthog_canvas" ("channel_id");
CREATE INDEX "posthog_canvas_created_by_id_06376fca" ON "posthog_canvas" ("created_by_id");
CREATE INDEX "posthog_canvas_team_id_cea17ed5" ON "posthog_canvas" ("team_id");
ALTER TABLE "posthog_canvas_build" ADD CONSTRAINT "posthog_canvas_build_canvas_id_dbd1e20f_fk_posthog_canvas_id" FOREIGN KEY ("canvas_id") REFERENCES "posthog_canvas" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "posthog_canvas_build_canvas_id_dbd1e20f" ON "posthog_canvas_build" ("canvas_id");
CREATE INDEX "posthog_canvas_build_team_id_384b0ee7" ON "posthog_canvas_build" ("team_id");
CREATE INDEX "posthog_canvas_published_build_id_b1d59cb6" ON "posthog_canvas" ("published_build_id");
ALTER TABLE "posthog_canvas_source_version" ADD CONSTRAINT "posthog_canvas_sourc_canvas_id_796972a9_fk_posthog_c" FOREIGN KEY ("canvas_id") REFERENCES "posthog_canvas" ("id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "posthog_canvas_source_version" ADD CONSTRAINT "posthog_canvas_sourc_parent_version_id_5d0b9a7e_fk_posthog_c" FOREIGN KEY ("parent_version_id") REFERENCES "posthog_canvas_source_version" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "posthog_canvas_source_version_canvas_id_796972a9" ON "posthog_canvas_source_version" ("canvas_id");
CREATE INDEX "posthog_canvas_source_version_created_by_id_a76f1ad8" ON "posthog_canvas_source_version" ("created_by_id");
CREATE INDEX "posthog_canvas_source_version_parent_version_id_5d0b9a7e" ON "posthog_canvas_source_version" ("parent_version_id");
CREATE INDEX "posthog_canvas_source_version_team_id_b474f966" ON "posthog_canvas_source_version" ("team_id");
CREATE INDEX "posthog_canvas_build_source_version_id_10d990cb" ON "posthog_canvas_build" ("source_version_id");
CREATE INDEX "posthog_canvas_current_source_version_id_1aac4f83" ON "posthog_canvas" ("current_source_version_id");
COMMIT;

products/canvas/backend/migrations/0002_source_version_required.py

BEGIN;
--
-- Alter field source_version on canvasbuild
--
ALTER TABLE "posthog_canvas_build" ALTER COLUMN "source_version_id" SET NOT NULL;
COMMIT;

products/canvas/backend/migrations/0003_migrate_desktop_tree.py

BEGIN;
--
-- Raw Python operation
--
-- THIS OPERATION CANNOT BE WRITTEN AS SQL
COMMIT;

products/canvas/backend/migrations/0004_build_enqueued_at.py

BEGIN;
--
-- Add field enqueued_at to canvasbuild
--
ALTER TABLE "posthog_canvas_build" ADD COLUMN "enqueued_at" timestamp with time zone NULL;
COMMIT;

products/canvas/backend/migrations/0005_backfill_build_enqueued_at.py

BEGIN;
--
-- Raw Python operation
--
-- THIS OPERATION CANNOT BE WRITTEN AS SQL
COMMIT;

products/canvas/backend/migrations/0006_require_build_enqueued_at.py

BEGIN;
--
-- Alter field enqueued_at on canvasbuild
--
ALTER TABLE "posthog_canvas_build" ALTER COLUMN "enqueued_at" SET DEFAULT '2026-08-03 20:51:40.256309+00:00'::timestamptz;
UPDATE "posthog_canvas_build" SET "enqueued_at" = '2026-08-03 20:51:40.256293+00:00'::timestamptz WHERE "enqueued_at" IS NULL; SET CONSTRAINTS ALL IMMEDIATE;
ALTER TABLE "posthog_canvas_build" ALTER COLUMN "enqueued_at" SET NOT NULL;
ALTER TABLE "posthog_canvas_build" ALTER COLUMN "enqueued_at" DROP DEFAULT;
COMMIT;

products/tasks/backend/migrations/0081_channelcontextgeneration_channelinstructions_and_more.py

BEGIN;
--
-- Create model ChannelContextGeneration
--
CREATE TABLE "posthog_task_channel_context_generation" ("id" uuid NOT NULL PRIMARY KEY, "task_id" uuid NULL, "created_at" timestamp with time zone NOT NULL, "updated_at" timestamp with time zone NOT NULL, "channel_id" uuid NOT NULL UNIQUE, "team_id" integer NOT NULL);
--
-- Create model ChannelInstructions
--
CREATE TABLE "posthog_task_channel_instructions" ("id" uuid NOT NULL PRIMARY KEY, "content" text NOT NULL, "version" integer NOT NULL CHECK ("version" >= 0), "is_latest" boolean NOT NULL, "deleted" boolean NOT NULL, "created_at" timestamp with time zone NOT NULL, "updated_at" timestamp with time zone NOT NULL, "channel_id" uuid NOT NULL, "created_by_id" integer NULL, "team_id" integer NOT NULL);
--
-- Create model ChannelStar
--
CREATE TABLE "posthog_task_channel_star" ("id" uuid NOT NULL PRIMARY KEY, "created_at" timestamp with time zone NOT NULL, "channel_id" uuid NOT NULL, "team_id" integer NOT NULL, "user_id" integer NOT NULL, CONSTRAINT "unique_channel_star_per_user" UNIQUE ("channel_id", "user_id"));
ALTER TABLE "posthog_task_channel_context_generation" ADD CONSTRAINT "posthog_task_channel_channel_id_51828a73_fk_posthog_t" FOREIGN KEY ("channel_id") REFERENCES "posthog_task_channel" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "posthog_task_channel_context_generation_team_id_a905f6b5" ON "posthog_task_channel_context_generation" ("team_id");
ALTER TABLE "posthog_task_channel_instructions" ADD CONSTRAINT "posthog_task_channel_channel_id_d960fb21_fk_posthog_t" FOREIGN KEY ("channel_id") REFERENCES "posthog_task_channel" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE UNIQUE INDEX "unique_channel_instructions_version" ON "posthog_task_channel_instructions" ("channel_id", "version") WHERE NOT "deleted";
CREATE UNIQUE INDEX "unique_channel_instructions_latest" ON "posthog_task_channel_instructions" ("channel_id") WHERE (NOT "deleted" AND "is_latest");
CREATE INDEX "posthog_task_channel_instructions_channel_id_d960fb21" ON "posthog_task_channel_instructions" ("channel_id");
CREATE INDEX "posthog_task_channel_instructions_created_by_id_40bd8504" ON "posthog_task_channel_instructions" ("created_by_id");
CREATE INDEX "posthog_task_channel_instructions_team_id_71c9b2ca" ON "posthog_task_channel_instructions" ("team_id");
ALTER TABLE "posthog_task_channel_star" ADD CONSTRAINT "posthog_task_channel_channel_id_00ddc170_fk_posthog_t" FOREIGN KEY ("channel_id") REFERENCES "posthog_task_channel" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "posthog_task_channel_star_channel_id_00ddc170" ON "posthog_task_channel_star" ("channel_id");
CREATE INDEX "posthog_task_channel_star_team_id_57ea58cc" ON "posthog_task_channel_star" ("team_id");
CREATE INDEX "posthog_task_channel_star_user_id_314efb4c" ON "posthog_task_channel_star" ("user_id");
COMMIT;

Last updated: 2026-08-03 20:51 UTC (f7e0e03)

Django migration risk — migration analysis complete

We've analyzed your migrations for potential risks.

Summary: 4 Safe | 4 Needs Review | 0 Blocked

⚠️ Needs Review

May have performance impact

canvas.0002_source_version_required
  └─ #1 ⚠️ AlterField
     Field alteration may cause table locks or data loss (check if changing type or constraints)
     model: canvasbuild, field: source_version, field_type: ForeignKey
canvas.0003_migrate_desktop_tree
  └─ #1 ⚠️ RunPython: RunPython data migration needs review for performance
canvas.0005_backfill_build_enqueued_at
  └─ #1 ⚠️ RunPython: RunPython data migration needs review for performance
canvas.0006_require_build_enqueued_at
  └─ #1 ⚠️ AlterField
     Field alteration may cause table locks or data loss (check if changing type or constraints)
     model: canvasbuild, field: enqueued_at, field_type: DateTimeField

✅ Safe

Brief or no lock, backwards compatible

tasks.0081_channelcontextgeneration_channelinstructions_and_more
  └─ #1 ✅ CreateModel
     Creating new table is safe
     model: ChannelContextGeneration
  └─ #2 ✅ CreateModel
     Creating new table is safe
     model: ChannelInstructions
  └─ #3 ✅ CreateModel
     Creating new table is safe
     model: ChannelStar
  │
  └──> ℹ️  INFO:
       ℹ️  Skipped operations on newly created tables (empty tables
       don't cause lock contention).
canvas.0001_initial
  └─ #1 ✅ CreateModel
     Creating new table is safe
     model: Canvas
  └─ #2 ✅ CreateModel
     Creating new table is safe
     model: CanvasBuild
  └─ #3 ✅ AddField
     Adding nullable field requires brief lock
     model: canvas, field: published_build
  └─ #4 ✅ CreateModel
     Creating new table is safe
     model: CanvasSourceVersion
  └─ #5 ✅ AddField
     Adding nullable field requires brief lock
     model: canvasbuild, field: source_version
  └─ #6 ✅ AddField
     Adding nullable field requires brief lock
     model: canvas, field: current_source_version
  │
  └──> ℹ️  INFO:
       ℹ️  Skipped operations on newly created tables (empty tables
       don't cause lock contention).
canvas.0004_build_enqueued_at
  └─ #1 ✅ AddField
     Adding nullable field requires brief lock
     model: canvasbuild, field: enqueued_at
posthog.1285_drop_desktop_file_system
  └─ #1 ✅ SeparateDatabaseAndState: Only state operations (no database changes)

📚 How to Deploy These Changes Safely

AddField:

This operation acquires a brief lock but doesn't rewrite the table.

Deployment uses lock timeouts with automatic retries, so lock contention will cause retries rather than connection pile-up.

RunPython:

Use batching for large data migrations:

  • Use .iterator() to avoid loading all rows into memory
  • Use .bulk_update() instead of saving individual objects
  • Batch size: 1,000-10,000 rows per batch
  • Add pauses between batches
  • Consider background jobs for very large updates (millions of rows)

See the migration safety guide

Last updated: 2026-08-03 20:52 UTC (f7e0e03)

Hobby preview — Health check did not pass within 35 minutes

Failing fast because: Health check did not pass within 35 minutes


Run 30851781690 | Consecutive failures: 1

@k11kirky
k11kirky force-pushed the posthog-code/canvas-build-pipeline-phase-6 branch from b03ba2a to 8d37859 Compare July 28, 2026 11:25
@k11kirky k11kirky changed the title feat(canvas): securely deliver build artifacts feat(canvas): build and securely deliver cloud artifacts Jul 28, 2026
@k11kirky
k11kirky changed the base branch from posthog-code/canvas-build-pipeline-phase-5 to master July 28, 2026 11:26
@k11kirky
k11kirky marked this pull request as ready for review July 28, 2026 19:43
@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Too many files changed for review. (123 files found, 100 file limit)

Bypass the limit by tagging @greptile-apps to review.

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested review from a team July 28, 2026 19:43
Comment thread products/canvas/backend/presentation/views.py
Comment thread products/tasks/backend/facade/loops.py
Comment thread products/canvas/backend/presentation/views.py
@veria-ai

veria-ai Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

PR overview

This pull request adds canvas workflows for building cloud artifacts and delivering or publishing them through unattended loop runs. It also integrates task creation and canvas maintenance actions into the loop-run service.

Nine issues have been addressed, with one authorization concern still open. Loop agents retain project-wide write permissions, allowing a teammate who can edit shared channel instructions to steer an unattended run into creating tasks or modifying and publishing visible canvases as the loop owner. The impact is limited to enabled tools and accessible project resources, but resource-bound enforcement is still needed.

Open issues (1)

Fixed/addressed: 9 · PR risk: 6/10

@trunk-io

trunk-io Bot commented Jul 28, 2026

Copy link
Copy Markdown

😎 Merged successfully - details.

Comment thread products/canvas/packages/canvas_builder/manifest.json Outdated
Comment thread products/canvas/backend/migrations/0002_migrate_desktop_tree.py Outdated
Comment thread products/canvas/backend/presentation/views.py Outdated
@posthog
posthog Bot force-pushed the posthog-code/canvas-build-pipeline-phase-6 branch from f088d05 to 99a4561 Compare July 29, 2026 01:40
@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

🦔 Hogbox preview · ✅ ready

▶ Open the preview

🔑 Login test@posthog.com / 12345678 (demo data)
🧩 Running this PR's backend and frontend, on the PostHog :master base
🔗 Link stable across rebuilds — a re-push swaps the box underneath, the URL stays
🔒 Access tailnet only (PostHog VPN)
🛠️ Admin inspect & debug state in hogland
💤 Idle sleeps after ~30 min idle (snapshot to S3, zero node cost) and wakes on your next visit in ~30s, behind a brief "waking up" screen

commit f7e0e03 · box box-4a6bee579fda · ready in 912s (push → usable) · build log · rebuilds on every push, torn down on close

@posthog

posthog Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Automated PR babysitting pass (CI You Later) — rebased this branch onto master to clear the merge conflict. No functional change: the PR's diff against master is byte-identical to before the rebase apart from the two resolutions below.

Resolutions

  1. posthog/migrations/max_migration.txtmaster landed 1266_comment_convo_content_trgm, so this branch's migration was renumbered 1266_drop_desktop_file_system1267_drop_desktop_file_system and its posthog dependency repointed at 1266_comment_convo_content_trgm. The canvas.0002_migrate_desktop_tree dependency is unchanged.
  2. posthog/api/file_system/test/test_file_system.pyTestDesktopFileSystemSurface deleted, as this PR intends (the desktop_file_system route is gone). master had only added tests inside that class, so the whole class went.

One thing to check#74282 landed on master yesterday, exposing created_by on FileSystemSerializer, and its only two tests (test_desktop_list_returns_creator, test_desktop_list_returns_null_for_deleted_creator) were written against the desktop_file_system route, so deleting that class drops them. The serializer change itself merged cleanly and still applies to the web route, but nothing now covers created_by in the file-system list response. Worth porting those two assertions onto /api/projects/:id/file_system/. I did not add them myself: this sandbox has no Postgres, so I could not run the test before pushing and did not want to risk turning CI red with unverified tests.

Main CI had never run on the previous head (GitHub can't build a merge commit for a conflicting PR), so this push is the first real CI signal on these changes.

Comment thread products/tasks/backend/presentation/serializers_loops.py Outdated
Comment thread products/canvas/backend/build_service.py Outdated
@posthog

posthog Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Second pass. Pushed four commits of CI fixes; one failure is left that needs your call, and it is the one gating everything else.

Needs your decision — tach.toml blocks all of Backend CI

Repo checks fails with:

[[interfaces]] for canvas mixes facade/presentation exposes with internal exposes — split into separate blocks or remove the facade patterns from the legacy leak block

That failure sets deterministic_failure=true, and cancel-backend-on-repo-check-failure (.github/workflows/ci-backend.yml:1196) then cancels the whole Backend CI run. So Django tests, product tests, Validate migrations and Validate OpenAPI types are all cancelled before they run — this PR currently has zero backend test signal, and will keep having zero until this one check goes green.

I did not fix it because both plausible fixes change intent, and I verified each against the linter locally rather than guessing:

  • Split the blockbackend.tasks.* is the internal expose, so it moves to its own block. But then the remaining presentation + routes block becomes a canonical facade block, and validate_facade_alternation fails with canonical facade alternation lists 'canvas' but products/canvas/backend/facade/contracts.py is missing. Canvas has backend/contract.py (the builder manifest contract), which is unrelated. So splitting requires adopting the strict isolation regime — a real refactor.
  • Drop the [[interfaces]] block entirely — this is what every comparable product does (approvals, pulse, reminders, signals, feature_flags, conversations all import backend.tasks from posthog/tasks/scheduled.py and declare no interfaces block at all). Passes the linter, but throws away the narrowing you deliberately added.

A third option, if you want to keep the narrowing without a facade: export the artifact view through backend/routes.py so posthog/urls.py stops importing backend.presentation.artifacts, then a routes-only block plus a tasks-only block both pass (a routes-only block is explicitly not treated as canonical). Your call which of the three.

Fixed this pass

  • fix(canvas): point the image build and CI at the moved builder packageDockerfile, .dockerignore and both ci-backend.yml files still pointed at common/canvas-builder; the image build failed with "/common/canvas-builder": not found and took E2E Hobby CI with it.
  • fix(canvas): publish the canvas OAuth scopes and refresh MCP snapshotsoauth-scopes.generated.ts had never been regenerated, so OAUTH_SCOPES_SUPPORTED omitted canvas:read / canvas:write. Since the MCP server publishes that list as RFC 9728 scopes_supported, the canvas tools would not have been discoverable over OAuth. Regenerated with bin/build-mcp-oauth-scopes.py; tool-schema snapshots refreshed in the same commit (desktop-file-system-* out, canvas-* / channel-* in, loops context now keys on channel_id). All 2409 MCP unit tests pass locally.
  • chore(canvas): commit the lint and typegen artifacts CI asked for — ruff I001 on posthog/urls.py, oxfmt on the builder package.json, and the kea typegen output for the new canvas scope in three access-control logic files.
  • chore(canvas): apply ruff format to the files this PR touchedruff format --check --diff . was red on seven PR-touched files.

Not looked at: the six open review threads from the security bots. Two of them I did verify are real code-level findings rather than bot noise (the canvas queryset has no channel-visibility filter; channel_by_top_path in the migration is keyed on the path segment alone, so every member's me folder maps to the first-iterated member's personal channel). Both are security-semantics calls on your own design, so I left them for you.

k11kirky and others added 26 commits August 3, 2026 21:46
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Generated-By: PostHog Code
Task-Id: 45da9f78-5b15-4cfe-a8a5-9afcbd6b4755
Reuse: home the channel-visibility rule on Channel.visible_to_q and delegate
visible_channels_q to it; collapse the duplicated context-canvas visibility
check into one logic-layer helper; adopt the diagnostic() constructor at all
build_service call sites; share one relative-path validator between source
uploads and artifact paths; extract _requeue_or_fail for the object-storage
retry blocks and _claim_canvas_head for the publish/revert guard.

Efficiency: make the pre-upload publish check lock-free (the locked check in
the commit transaction stays authoritative); batch the retention sweep's S3
deletes and prefix-clearing updates; resolve a loop's feed channel once per
run; use a lightweight channel_exists facade helper instead of building a
ChannelDTO for the create-time visibility check.

Simplification: drop the dead structlog logger and the vestigial
_allow_delete_without_ref hook from the core file-system viewset; hoist the
duplicated publish-instructions extend_schema block; compute the sandbox task
id once in _publish; remove the unused externalImports plumbing in the
canvas builder.

Altitude: move the canvas artifact token tests from posthog/api/test into
products/canvas/backend/tests.

Generated-By: PostHog Code
Task-Id: aea6c5f4-02ae-448b-b4dc-f10f581ad597
2 updated
Run: c8a59153-f640-42fa-a523-84f91a17443d

Co-authored-by: charlesvien <5378415+charlesvien@users.noreply.github.com>
Generated-By: PostHog Code
Task-Id: 290107cf-56ba-45ae-a530-b60098d10c0d
Generated-By: PostHog Code
Task-Id: 483cc3d9-8406-4a59-85ca-5b7f4b0b2b45
@k11kirky

k11kirky commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

/trunk merge

@deployment-status-posthog

deployment-status-posthog Bot commented Aug 3, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-08-03 22:26 UTC Run
prod-us ✅ Deployed 2026-08-03 22:43 UTC Run
prod-eu ✅ Deployed 2026-08-03 22:46 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants