Repository navigation
CI: check the generated docs, run e2e in the Playwright container, add a nightly job for the slow tests - #2
Merged
Merged
Conversation
…d add a nightly job for the slow tests Why: the full unit tests and the e2e tests move from the developer's machine to GitHub Actions. CI did not run the accuracy and AI guide checks or 8 of the 10 slow test files, and 3 of the last 9 runs failed in "Install Chromium" (apt, exit code 100), never in a test. ci.yml, check job (blocks a pull request): - `node scripts/accuracy.ts --check`: docs/accuracy.md is what the script generates. - `node scripts/ai-guide.ts --check`: the AI guide is up to date. ci.yml, e2e job (blocks a pull request): - Runs in mcr.microsoft.com/playwright:v1.61.0-noble, which has Chromium and its system libraries, so there is no apt step. A step fails the job when @playwright/test in package-lock.json is not the same version as the container. - `--retries=2 --trace=on-first-retry --reporter=list,github`: a test that passes only on a retry is reported as flaky, with a trace. playwright.config.ts is unchanged, so a local run is too. - The results are uploaded on every run that is not cancelled, not only on a failure. nightly.yml (new; does not block a pull request): - Every day at 19:00 UTC (03:00 in Taipei), and by hand for any branch. - Three groups, each on its own runner, so tests with a time budget do not compete for CPU: format; timescale + import/limit; learn + constraints + components + oncall + queueing. With golden and briefs, which the check job runs, this is every file of `npm run test:slow`. - `--retry=1`; a failure sends GitHub's default mail for a scheduled workflow. Checked locally: both files parse as YAML; every test file and script they name exists; the version check step passes with 1.61.0 and fails with 1.60.0; the container tag exists in the registry; the playwright and vitest flags exist. actionlint was not available. Not run on GitHub yet: the workflows run only after a push. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015uxaDVERbfy9cVbzQ8qB9F
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
🚀 Deploying Preview to Cloudflare 🚀Preview Deployments by commit
|
minchenlee
added a commit
that referenced
this pull request
Nov 18, 2026
* chore: add codemod for English source text * refactor(i18n): make English the source text Output of scripts/codemod-english-source.ts plus the hand-made parts: t()/tx() ids, KINDS, examples, scenarios and the 17 practice questions are English; src/i18n/zh-TW.ts (English id -> Traditional Chinese) replaces src/i18n/en.ts. Thirteen English texts with two Chinese texts carry a context mark (Off@@not-used) that t() strips. Old Chinese ids in saved designs still translate in English mode. One English text changes: the saas-instagram PostgreSQL label was 61 characters, over the 60-character label limit, and is now "PostgreSQL (12 shards, replicas in another AZ)" (lead decision). Golden hashes do not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(i18n): warn on missing Chinese, document English source text i18n:check lists entries without Traditional Chinese and passes; it still fails on placeholder mismatches and an out-of-date zh-TW.ts. i18n:strict (nightly on main) fails on missing Chinese. AGENTS.md, CONTRIBUTING.md and docs/design-system.md say English is the source text. schema/design.schema.json descriptions are now English (191 lines). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: follow the English source text Assertions on message ids and raw example data use the English ids; assertions on zh-TW output stay and go through t(). The text dump used for the acceptance check runs only with DUMP_OUT set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(i18n): allow English-only contributions with empty Chinese An example added in English only leaves its zh-TW entry empty, so zh-TW shows English. Only require Chinese when the catalog has it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the zh-complete check in nightly.yml PR #2 added nightly.yml with its own schedule and concurrency group, so ci.yml goes back to main's version and the nightly translation check becomes a job next to the slow tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(i18n): show a user's own text with @@ as written t() stripped the @@context from any text, so a scenario named "壓測@@臺北" showed as "壓測" in both languages. Only a catalog id has a context mark now; other text stays as written. Catalog lookups use Object.hasOwn. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(oncall): check the English title for words that name the fault The check only looked at the Traditional Chinese title, so an English title with no translation ("Database failure") passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(i18n): clearer English for three labels "Replica reads" -> "Read replica count", "Replica applies write cost" -> "Replica write-apply cost", "Not enough result" -> "Insufficient data" (with the three import messages that start with it). The Traditional Chinese moves with each id and does not change. Regenerated schema/ and the AI guide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(app): the scenario menu finds a preset by its old Chinese name Built-in scenario names are English now. An old share link or a saved version still has the Traditional Chinese name, so the menu did not mark the current preset. presetOf() (src/app/presetName.ts) also matches the Chinese name in zh-TW.ts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: remove the one-time codemod and the text-dump test The English source text conversion is done and checked; copies stay with the validation data outside the repo. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The full unit tests and the e2e tests move from the developer's machine to GitHub Actions.
scripts/accuracy.ts --check,scripts/ai-guide.ts --check, or 8 of the 10 files ofnpm run test:slow.Install Chromiumstep (apt, exit code 100). None failed in a test.What changes
Only
.github/workflows/. No source, test or config file changes.ci.yml,checkjob (blocks a pull request)node scripts/accuracy.ts --check(docs/accuracy.mdis what the script generates).node scripts/ai-guide.ts --check.ci.yml,e2ejob (blocks a pull request)mcr.microsoft.com/playwright:v1.61.0-noble. The image has Chromium and its system libraries, so the apt step is gone.@playwright/testinpackage-lock.jsonis not the same version as the container. Change both together.npx playwright test --retries=2 --trace=on-first-retry --reporter=list,github. A test that passes only on a retry is reported as flaky, with a trace.playwright.config.tsis unchanged, so a local run behaves as before.nightly.yml(new; does not block a pull request)Run workflow).format:tests/format.test.tstiming:tests/timescale.test.ts,tests/import/limit.test.tsexamples:tests/learn.test.ts,tests/constraints.test.ts,tests/components.test.ts,tests/oncall.test.ts,tests/queueing.test.tsnpx vitest run --retry=1. A failed scheduled run sends GitHub's default mail.How it was checked
v1.61.0-nobleexists in the registry.--retries,--trace,--reporter(playwright) and--retry(vitest) flags exist in the installed versions.actionlint(not installed). The workflows have not run on GitHub yet; this pull request is their first run.What to look at in the first runs
e2e: the job starts in the container and finishes in about the same time as before (6.5 to 8.5 minutes).nightlydoes not run on a pull request. After the merge, start it once by hand and note the time of each group.tests/timescale.test.ts(15 s), the 120 s wall-clock limit intests/import/limit.test.ts, the 30 s per-example timeout intests/format.test.ts. If one fails, raise that limit in its own commit with the measured time.🤖 Generated with Claude Code
https://claude.ai/code/session_015uxaDTazJv7ozbPd3u5Cub