Skip to content

test: make the twin host tests fast and deterministic - #472

Merged
kody-w merged 1 commit into
mainfrom
fix/twin-test-speed
Sep 27, 2026
Merged

kody-w merged 1 commit into
mainfrom
fix/twin-test-speed

Conversation

@kody-w

@kody-w kody-w commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Root cause

The Twin shard was running large conversation/approval integration scenarios as single Vitest tests while the host fixture used the production append-only filesystem durability path. On macOS CI that made individual tests do several minutes of repeated canonical store scans, approval/runtime setup, and fsync-heavy commits, so different tests crossed Vitest's 120s per-test timeout depending on runner speed.

Fix

  • Keep durable filesystem syncs as the production default, but add an explicit non-durable fixture mode for host integration tests; workspace-store durability/crash tests still exercise the real fsync path.
  • Split the broad Twin integration cases into smaller tests that assert the same provider/model/computer/agent/routine/approval rejection behavior and the same agent/routine/settings apply behavior without one test accumulating all work.
  • Replace the approval-recommendation Twin test's real guest-runtime setup with canonical pending approval/run evidence, then verify Twin apply still refuses to decide and the explicit Work approval decision path is required.

Timings

Before:

  • CI run 36328101083 attempt 2: test/twin.test.ts 827.25s total; failed tests at 120.073s and 136.889s; slow passing tests included 141.498s, 90.027s, 85.108s, 70.486s.
  • Local before: 372.51s total; slowest tests 45.921s, 42.181s, 42.146s, 42.036s.

After:

  • Local after: 160.41s total; slowest tests 18.433s, 17.333s, 16.230s.
  • CI green attempt: 480.90s total; slowest tests 54.646s, 53.537s, 53.248s, all below the 120s per-test timeout.

Validation

  • npm --workspace @rapp-work/host run build
  • npm --workspace @rapp-work/host test -- --run test/twin.test.ts --reporter=verbose
  • npm --workspace @rapp-work/host run typecheck
  • npm --workspace @rapp-work/workspace-store test -- --run
  • targeted production regression: npm --workspace @rapp-work/host test -- --run test/production.test.ts --reporter=verbose -t "approval can be cancelled"
  • privacy scan before push: denylist_scan.py diff <clone> origin/main => 0 findings

Green CI

@kody-w
kody-w force-pushed the fix/twin-test-speed branch 2 times, most recently from bc5ef06 to 6246990 Compare September 27, 2026 16:56
Use the real append-only store logic in host fixtures while disabling expensive POSIX fsync calls that are already covered by workspace-store durability tests. Production services keep durable syncs by default.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b6562323-1ad0-4e5e-9951-f2273c033d4a
@kody-w
kody-w force-pushed the fix/twin-test-speed branch from 6246990 to 5e65e50 Compare September 27, 2026 17:20
@kody-w
kody-w merged commit 538ef00 into main Sep 27, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant