Skip to content

feat: Wave 3 — community bug fixes & platform support (v0.11.6.0) #29

feat: Wave 3 — community bug fixes & platform support (v0.11.6.0)

feat: Wave 3 — community bug fixes & platform support (v0.11.6.0) #29

Triggered via pull request March 23, 2026 22:58
Status Cancelled
Total duration 3m 27s
Artifacts 7

evals.yml

on: pull_request
build-image
10s
build-image
Matrix: evals
Fit to window
Zoom out
Zoom in

Annotations

25 errors and 19 warnings
error: expect(received).toBeGreaterThanOrEqual(expected): test/skill-llm-eval.test.ts#L88
Expected: >= 4 Received: 3 at <anonymous> (/__w/gstack/gstack/test/skill-llm-eval.test.ts:88:33)
error: expect(received).toBeGreaterThanOrEqual(expected): test/skill-llm-eval.test.ts#L88
Expected: >= 4 Received: 3 at <anonymous> (/__w/gstack/gstack/test/skill-llm-eval.test.ts:88:33)
evals (e2e-browse, test/skill-e2e-browse.test.ts)
Process completed with exit code 143.
evals (e2e-qa-workflow, test/skill-e2e-qa-workflow.test.ts)
Canceling since a higher priority waiting request for evals-garrytan/wave3-bugfixes exists
evals (e2e-qa-workflow, test/skill-e2e-qa-workflow.test.ts)
The operation was canceled.
evals (e2e-review, test/skill-e2e-review.test.ts)
Canceling since a higher priority waiting request for evals-garrytan/wave3-bugfixes exists
evals (e2e-review, test/skill-e2e-review.test.ts)
The operation was canceled.
evals (e2e-plan, test/skill-e2e-plan.test.ts)
Canceling since a higher priority waiting request for evals-garrytan/wave3-bugfixes exists
evals (e2e-design, test/skill-e2e-design.test.ts)
Canceling since a higher priority waiting request for evals-garrytan/wave3-bugfixes exists
evals (e2e-design, test/skill-e2e-design.test.ts)
The operation was canceled.
evals (e2e-plan, test/skill-e2e-plan.test.ts)
The operation was canceled.
evals (e2e-workflow, test/skill-e2e-workflow.test.ts)
Canceling since a higher priority waiting request for evals-garrytan/wave3-bugfixes exists
evals (e2e-workflow, test/skill-e2e-workflow.test.ts)
The operation was canceled.
error: expect(received).toBeGreaterThan(expected): test/skill-e2e-workflow.test.ts#L215
Expected: > 1 Received: 1 at <anonymous> (/__w/gstack/gstack/test/skill-e2e-workflow.test.ts:215:27)
evals (e2e-qa-bugs, test/skill-e2e-qa-bugs.test.ts)
Canceling since a higher priority waiting request for evals-garrytan/wave3-bugfixes exists
evals (e2e-qa-bugs, test/skill-e2e-qa-bugs.test.ts)
The operation was canceled.
evals (e2e-deploy, test/skill-e2e-deploy.test.ts)
Canceling since a higher priority waiting request for evals-garrytan/wave3-bugfixes exists
evals (e2e-deploy, test/skill-e2e-deploy.test.ts)
The operation was canceled.
evals (e2e-routing, test/skill-routing-e2e.test.ts)
Canceling since a higher priority waiting request for evals-garrytan/wave3-bugfixes exists
evals (e2e-routing, test/skill-routing-e2e.test.ts)
The operation was canceled.
error: Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls:: test/skill-routing-e2e.test.ts#L172
Expected: > 0 Received: 0 at <anonymous> (/__w/gstack/gstack/test/skill-routing-e2e.test.ts:172:208)
error: Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: Agent: test/skill-routing-e2e.test.ts#L329
Expected: > 0 Received: 0 at <anonymous> (/__w/gstack/gstack/test/skill-routing-e2e.test.ts:329:208)
error: Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls:: test/skill-routing-e2e.test.ts#L172
Expected: > 0 Received: 0 at <anonymous> (/__w/gstack/gstack/test/skill-routing-e2e.test.ts:172:208)
report
Process completed with exit code 5.
E2E Evals
Canceling since a higher priority waiting request for evals-garrytan/wave3-bugfixes exists
build-image
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, docker/login-action@v3. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-gemini, test/gemini-e2e.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-gemini, test/gemini-e2e.test.ts)
No files were found with the provided path: ~/.gstack-dev/evals/*.json. No artifacts will be uploaded.
evals (e2e-codex, test/codex-e2e.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-codex, test/codex-e2e.test.ts)
No files were found with the provided path: ~/.gstack-dev/evals/*.json. No artifacts will be uploaded.
evals (llm-judge, test/skill-llm-eval.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-browse, test/skill-e2e-browse.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-qa-workflow, test/skill-e2e-qa-workflow.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-qa-workflow, test/skill-e2e-qa-workflow.test.ts)
No files were found with the provided path: ~/.gstack-dev/evals/*.json. No artifacts will be uploaded.
evals (e2e-review, test/skill-e2e-review.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-plan, test/skill-e2e-plan.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-design, test/skill-e2e-design.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-plan, test/skill-e2e-plan.test.ts)
No files were found with the provided path: ~/.gstack-dev/evals/*.json. No artifacts will be uploaded.
evals (e2e-workflow, test/skill-e2e-workflow.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-qa-bugs, test/skill-e2e-qa-bugs.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-qa-bugs, test/skill-e2e-qa-bugs.test.ts)
No files were found with the provided path: ~/.gstack-dev/evals/*.json. No artifacts will be uploaded.
evals (e2e-deploy, test/skill-e2e-deploy.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
evals (e2e-routing, test/skill-routing-e2e.test.ts)
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/upload-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
report
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4, actions/download-artifact@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/

Artifacts

Produced during runtime
Name Size Digest
eval-e2e-browse
8.59 KB
sha256:725d8c8c7ad58bbd5970bee90e30bfb5ede7a9a556f0ad91c620bf25d1f0b710
eval-e2e-deploy
44.8 KB
sha256:1191ff620c438eb67fcae9d7cf18684d49e6a864392bb79171ac84c4907d692b
eval-e2e-design
19.1 KB
sha256:54986ca41bef7b478390eb50718c6375fd297b1c99eca30bf7840e6c493e35f8
eval-e2e-review
40.8 KB
sha256:dba3b8206ea1c841f6614d46cda0cd657005a885df24c4543c7465afef343ddf
eval-e2e-routing
284 KB
sha256:8b46d8cdc79378d0a1c917d0fd6a7df2c501d081dffd412c2556f33013a8543f
eval-e2e-workflow
39.4 KB
sha256:ccee1f32d42f699f0c065bccb5df0432f184ce8a605ebd3f5a4b31c97008feac
eval-llm-judge
20.6 KB
sha256:480b9b5862897e5973a775046005ed7584a416bf0f6b7101e2281fbd228bccaa