Skip to content

feat: script module mode with CLI sync, preview, and WAC UI improvements#8380

Draft
rubenfiszel wants to merge 20 commits intomainfrom
module-mode
Draft

feat: script module mode with CLI sync, preview, and WAC UI improvements#8380
rubenfiszel wants to merge 20 commits intomainfrom
module-mode

Conversation

@rubenfiszel
Copy link
Contributor

@rubenfiszel rubenfiszel commented Mar 15, 2026

Summary

This PR introduces script module mode — a foundational feature that allows Windmill scripts to have companion module files, enabling the Workflow-as-Code (WAC) pattern where scripts can reference task modules via taskScript("./helper.ts").

Script Module System

  • Scripts can have associated module files in a __mod/ folder, supporting two layouts:
    • Flat layout: my_script.ts + my_script__mod/helper.ts
    • Folder layout: my_script__mod/script.ts + my_script__mod/helper.ts
  • Modules are stored as a modules JSONB column on the script table
  • Backend executors (Bun, Python) write module files to the job directory for execution
  • Modules are included in script hashing, cloning, workspace export/import

CLI: Sync Pull/Push

  • readModulesFromDisk() / writeModulesToDisk() handle reading/writing module files from __mod/ folders
  • Fix pull→push idempotency: lock: lock ?? undefined preserves empty strings (API returns lock: "" for modules without deps)
  • Module files are pushed/pulled alongside their parent script
  • handleScriptMetadata() detects folder layout via isModuleEntryPoint()

CLI: Per-Module Hash Tracking

  • Each module gets its own hash entry in wmill-lock.yaml (following the flow inline script pattern)
  • SCRIPT_TOP_HASH meta-hash combines main script hash + sorted module hashes
  • computeModuleHashes() walks the __mod/ folder and hashes each module's content + workspace deps
  • updateModuleLocks() accepts optional changedModules parameter for selective lock regeneration
  • Stale detection output shows which specific modules changed: f/test/my_script (bun) [changed modules: helper.ts]

CLI: Preview with Modules

  • wmill script preview reads modules from __mod/ and passes them in the runScriptPreviewAndWaitResult API call
  • Supports both flat and folder layouts

CLI: Unified generate-metadata

  • Module helper files are filtered out (isScriptModulePath && !isModuleEntryPoint) so they're not treated as standalone scripts
  • Module changes are detected per-module, not all-or-nothing

Backend: auto_kind Refactor

  • Renamed no_main_func: boolauto_kind: VARCHAR(20) on the script table
  • Values: NULL (normal script), "lib" (library/no main), "wac" (workflow-as-code)
  • Parser detects WAC scripts by analyzing workflow() + task() patterns in TypeScript and Python
  • Migration converts existing no_main_func = true rows to auto_kind = 'lib'

Backend: WAC v2 Execution

  • Bun executor generates a WAC v2 wrapper that imports WorkflowCtx, StepSuspend, setWorkflowCtx from windmill-client
  • Wrapper handles checkpoint/replay semantics: complete, dispatch, sleep, approval modes
  • Python executor supports WAC via wmill.workflow() / wmill.task() decorators

Frontend: WAC UI

  • WAC TypeScript/Python templates with task(), taskScript(), step(), sleep(), waitForApproval()
  • TypeScript templates shown before Python in all template selectors
  • Module tab system in ScriptEditor: add, rename (pencil icon popover on hover), delete modules
  • Fixed-width icon container (w-[32px]) prevents layout shift on hover
  • Simplified +Script button (removed dropdown), consolidated WAC actions under +Flow dropdown
  • "Import Workflow-as-Code" drawer for importing WAC scripts from YAML/JSON
  • WAC export drawer for exporting scripts as YAML

Key Files Changed

Area Files Changes
Backend types windmill-types/src/scripts.rs auto_kind field, ScriptModule struct, hash impl
Backend API windmill-api-scripts/src/scripts.rs auto_kind in create/update
Backend workers bun_executor.rs, python_executor.rs, worker.rs Module file writing, WAC v2 wrapper
Backend parsers windmill-parser-ts, windmill-parser-py WAC detection
Backend migration 20260313000000_script_auto_kind.up.sql Column rename
CLI script cli/src/commands/script/script.ts readModulesFromDisk, preview, push/pull
CLI metadata cli/src/utils/metadata.ts Hash tracking, selective lock regen
CLI generate-metadata cli/src/commands/generate-metadata/ Module filtering
Frontend editor ScriptEditor.svelte Module tabs, rename popover
Frontend builder ScriptBuilder.svelte WAC templates, ordering
Frontend actions CreateActionsFlow.svelte, CreateActionsScript.svelte WAC import/dropdown

Test plan

  • bun test test/script_modules.test.ts — module sync unit tests
  • bun test test/resource_folders_unit.test.ts — path utility tests
  • bun test test/unified_generate_metadata.test.ts — metadata generation including module modification detection
  • bun test test/sync_pull_push.test.ts — sync round-trip tests
  • bun test test/preview.test.ts — preview with modules (flat + folder layout)
  • cargo check --features zip with -D warnings — clean compilation
  • svelte-check — 0 errors in changed files
  • Manual: wmill sync pullwmill sync push produces 0 changes for scripts with modules
  • Manual: Change one module → wmill generate-metadata only regenerates that module's lock
  • Manual: wmill script preview works with WAC scripts that have modules
  • Manual: Module tab rename popover in ScriptEditor

🤖 Generated with Claude Code

rubenfiszel and others added 12 commits March 8, 2026 17:12
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add prominent comments explaining that all computation must happen
inside task/step/taskScript or it will be replayed on resume/retry.
Clarify that waitForApproval does not hold a worker and that
approve/reject URLs are available in the timeline step details.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
# Conflicts:
#	frontend/package-lock.json
#	frontend/package.json
…nd preview support

- Fix pull→push idempotency: use `??` instead of `||` for module lock
  field so empty strings are preserved (matches API's `lock: ""`)
- Add per-module hash tracking in wmill-lock.yaml following the flow
  inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes)
- Selective module lock regeneration: only regenerate locks for modules
  whose content actually changed, not all modules
- Use unfiltered rawWorkspaceDependencies for module hashes to match
  what updateModuleLocks passes to fetchScriptLock
- Show changed module names in stale script output for clarity
- Add module support to `script preview` command: read modules from
  __mod/ folder and pass them in the preview API request
- Add preview tests for taskScript pattern (flat and folder layout)
- Update test assertion for module stale detection output

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ename, import consolidation

- Reorder WAC template buttons: TypeScript before Python in
  ScriptBuilder, CreateActionsScript, and CreateActionsFlow
- Remove dropdown items from +Script button (simplify to direct link)
- Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer
- Add module tab rename: pencil icon on hover opens popover with
  validation, fixed-width icon container prevents layout shift

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Resolve conflicts:
- cli/src/commands/script/script.ts: keep both isRawAppPath and module filters
- frontend/package.json: keep local parser wasm paths
- frontend/package-lock.json: regenerated from main + local deps
- Restore backend/.sqlx cache from main
- Add isScriptModulePath filter to unified generate-metadata command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Backend parser updates for WAC detection
- CLI sync/types updates for raw app path and module support
- Frontend UI polish (Dev.svelte, ScriptRow, script hash page)
- Test fixture updates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 15, 2026

Deploying windmill with  Cloudflare Pages  Cloudflare Pages

Latest commit: a4cb9c9
Status: ✅  Deploy successful!
Preview URL: https://824e3bce.windmill.pages.dev
Branch Preview URL: https://module-mode.windmill.pages.dev

View logs

rubenfiszel and others added 8 commits March 15, 2026 19:23
…adata

Verifies that modifying a single module file re-triggers stale
detection and only the changed module is listed, not all modules.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use
  "windmill-client" import instead of absolute filesystem path
- Fix missed no_main_func → auto_kind rename in parser TS test
- Add modules column to clone_script SQL (windmill-common and
  windmill-api-workspaces) so cloned scripts retain their modules
- Add modules: None to RawCode structs in worker tests
- Restore complete sqlx cache (merge main's cache + our new queries)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Change `.clone()` on double reference to `*k` dereference in
scripts.rs hash implementation. Update sqlx cache with new query
hashes from modified clone_script SQL.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The local file:// paths for windmill-parser-wasm-py and
windmill-parser-wasm-ts don't exist in the Cloudflare Pages build
environment. Revert to published npm versions (1.655.0).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py
v1.657.2 which include auto_kind/WAC detection changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ackages

Regenerating package-lock.json from scratch pulled different dependency
versions causing svelte-check type errors. Instead, start from main's
lockfile and only update the two changed packages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Script<SR> struct has a modules field (FromRow), but
fetch_script_for_update didn't SELECT modules, causing a runtime
error "no column found for name: modules" when the worker processed
dependency jobs. This was the root cause of the relock_skip test
timeout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant