Skip to content

testing.md

Latest commit

 

History

History
101 lines (82 loc) · 4.41 KB

testing.md

File metadata and controls

101 lines (82 loc) · 4.41 KB
Raw

Testing

Layer Tool Where Runs in CI
Backend unit JUnit 5 backend/src/test/.../*Test.java yes
Backend integration Spring Boot test, MockMvc, Testcontainers (PostgreSQL 17) backend/src/test/.../*IT.java yes
Judge against real Piston JUnit, tag piston PistonIT no (opt-in)
Frontend unit and component Vitest, Testing Library, jsdom frontend/tests/unit, frontend/tests/components yes
End-to-end, mocked API Playwright, axe-core frontend/tests/e2e/smoke.spec.ts yes
End-to-end, full stack Playwright, axe-core frontend/tests/e2e/stack.spec.ts yes (e2e job)
Load k6 performance/k6 no (manual, performance.md)
Smoke after deploy sh + curl scripts/smoke.sh yes (e2e job)

Backend

cd backend
./mvnw verify            # Spotless check, unit (Surefire), integration (Failsafe), JaCoCo report
./mvnw test              # unit tests only, no Docker needed

Integration tests need Docker for Testcontainers. Each test class shares one PostgreSQL container, and tables are truncated between tests. The judge is replaced by FakeCodeRunner, which picks the verdict from a keyword in the source (sum, zero, echo, ce, tle, crash, down). The clock is MutableClock, so contest windows and token expiry are tested without sleeping. Judge workers are off (app.judge.workers=0); tests call judgeAll() to process the queue synchronously.

What the integration tests cover:

  • AuthFlowIT: registration, login, logout, CSRF, verification and reset mails (captured by RecordingMailService), token expiry, rate limits, session revocation, cookie flags.
  • AuthorizationIT: admin API, unpublished contests, hidden problems before the start, submission privacy, profile privacy, comment permissions.
  • JudgingIT: queue, every verdict, hidden test data, idempotent submits, retries and INTERNAL_ERROR, stale-row recovery, rejudge, run endpoint.
  • StandingsIT: ICPC scoring and penalties, shared ranks, paging, registration rules, the contest list filters.

Against a real Piston (all four languages, compile errors, time and memory limits):

docker compose up -d piston && docker compose run --rm piston-install
# Piston has no host port in compose; publish one for the test, e.g. with a
# compose.override.yml containing: services: { piston: { ports: ["127.0.0.1:2000:2000"] } }
PISTON_URL=http://localhost:2000 ./mvnw verify \
  -Dit.groups=piston -Dit.excludedGroups=none -Dit.test=PistonIT

Coverage report: backend/target/site/jacoco/index.html.

Frontend

cd frontend
npm run lint && npm run typecheck
npm test                 # Vitest
npm run test:cov         # with coverage (v8)
npx playwright install chromium   # once; or E2E_CHANNEL=chrome to use installed Chrome
npm run test:e2e         # mocked-API end-to-end tests against the Vite dev server

Full-stack end-to-end tests run when E2E_BASE_URL points at a running stack with demo data; Playwright then starts no dev server:

docker compose up -d && docker compose run --rm piston-install
docker compose --profile seed run --rm seed
cd frontend && E2E_BASE_URL=http://localhost:3200 npm run test:e2e

They log in as a demo user, submit a solution that is judged by the real Piston, check the verdict and the standings, run axe on each page, and check that public pages do not scroll sideways at 390 px. Each run logs in once; the login limit is 10 per minute per IP, so back-to-back runs together with other logins from the same machine can hit 429.

Results

Recorded on 2026-11-10 on the development machine (Windows 11, Docker Desktop, 12 CPUs):

Suite Result
Backend unit 19 passed
Backend integration 35 passed
PistonIT (real Piston) 6 passed
Backend coverage (JaCoCo, unit + integration) 74.0 % lines (1348/1822), 51.7 % branches (233/451)
Frontend Vitest 119 passed in 14 files
Frontend coverage (v8) 41.9 % statements, 84.8 % branches, 60.2 % functions
Playwright, mocked API 2 passed (full-stack specs skipped)
Playwright, full stack (E2E_BASE_URL) 5 passed, including axe checks and a 390 px layout check

Frontend statement coverage is low because pages are mostly covered by the end-to-end tests, which Vitest coverage does not count.

The Playwright runs above used the installed Google Chrome (E2E_CHANNEL=chrome); CI uses Playwright's own Chromium.