restful-booker-api-testing

Production-grade API test automation framework for the Restful Booker Platform — a Spring Boot microservices Bed & Breakfast booking system. Built to demonstrate a modular, maintainable, and CI-integrated approach to testing a multi-service API, including cross-service contract validation.

Status: feature-complete. All eleven milestones (M0–M10) are closed; the project board records how it was built. Ongoing work is upkeep — Dependabot updates and the nightly monitoring jobs.

What it demonstrates

A single framework covering a microservices API from several angles, each earning its place:

• Layered client — typed HttpClient over Axios with a fluent request builder, a redacting exchange logger (the file log carries no bodies or headers; the opt-in console log redacts credentials), bounded retry on transient failures, and a readiness gate for the cold-starting shared demo.
• Nine test layers — unit, smoke, schema/contract, negative, data-driven, property-based (fast-check), security (OWASP-oriented), consumer contracts (Pact, verified against running providers), and performance (k6 with enforced budgets).
• Two targets, one suite — the hosted platform and a dockerized RBP that run different versions of the same API; every difference is declared in one profile rather than papered over, and a nightly job watches the two for drift.
• Defects as executable records — twelve platform bugs, each a written report paired with a guardsDefect test that stays green while the bug exists and fails the moment it is fixed or the request stops completing.
• CI that gates and CI that monitors — pull requests are gated on static checks, unit, Pact and the full live suite with coverage thresholds; four staggered nightly jobs (live, drift, ZAP, k6) report against the moving target and publish artifacts.

The design decisions, and several corrected mistakes, are written up in docs/.

System Under Test

The Restful Booker Platform is composed of independent services, each owning a slice of the domain:

Service	Port	Responsibility
auth	3004	Issue, validate, destroy tokens
room	3001	Manage bookable rooms
booking	3000	Manage bookings and availability
message	3006	Guest contact messages
branding	3002	Site identity / branding
report	3005	Collate rooms and bookings

Each service exposes Swagger UI and an /actuator/health endpoint. Mutating operations are protected by a token issued by the auth service (default credentials: admin / password).

Tech Stack

Concern	Choice
Language	TypeScript (strict)
HTTP client	Axios (wrapped in a typed HttpClient)
Test runner	Vitest
Schema & contract	Zod → JSON Schema
Consumer contracts	Pact (pact-js) + dockerized Pact Broker
Property-based testing	fast-check
Performance / load	k6 (TypeScript, @types/k6)
Test data	@faker-js/faker
Deterministic target	Dockerized RBP via docker-compose
Reporting	Allure + JUnit, published to GitHub Pages
CI/CD	GitHub Actions

Architecture

A layered design keeps tests declarative and decoupled from transport and service topology:

flowchart TB
  suites["9 test layers · Vitest<br/>unit · smoke · contract · pact · negative<br/>data-driven · property · security · perf"]

  subgraph client["Layered client"]
    direction LR
    builder["Request builder"] --> http["HttpClient · Axios"] --> resilience["Bounded retry ·<br/>redacting logger ·<br/>readiness gate"]
  end

  svc["Service layer<br/>Auth · Room · Booking · Message · Branding · Report"]
  zod["Zod schemas →<br/>JSON Schema contracts"]

  suites --> client --> svc
  zod -. validate responses .-> svc
  svc --> live[("live target<br/>hosted platform")]
  svc --> local[("local target<br/>Docker RBP 2.2")]

Loading

The layout on disk:

src/
  config/     Typed, validated per-service configuration
  client/     HttpClient (Axios), request builder, token auth, error model
  models/     Domain types (Room, Booking, Message, Branding, Report, AuthToken)
  schemas/    Zod schemas and generated JSON Schema contracts
  services/   AuthService, RoomService, BookingService, MessageService,
              BrandingService, ReportService
  factories/  faker-based builders and fast-check arbitraries
  support/    session and provisioning helpers for suites
tests/
  unit/       Hermetic framework tests (no network)
  smoke/      Behavioural happy paths per service
  contract/   Schema, drift & cross-service consistency
  pact/       Consumer-driven contracts (hermetic, no platform needed)
  negative/   Auth, authorization, boundary, malformed input
  data-driven/ JSON-dataset driven room & booking matrices
  property/   fast-check property-based suites
  security/   OWASP-oriented authz, token & injection checks
  data/       External test-case datasets
perf/         k6 smoke-load harness (TypeScript), thresholds, nightly CI
docs/         Architecture, test strategy, bug reports

Coverage

• Auth — login, validate, logout; negative and authorization paths
• Room — full CRUD
• Booking — create, get/list (by room), availability summary, update, delete
• Message — contact, list, read, delete, unread count
• Branding / Report — read/update branding; report consistency vs room + booking
• Health — /actuator/health gate across all services
• Contract — every response validated against its schema; cross-service consistency
• Consumer contracts — Pact contracts for auth, room and booking, verified against the running providers
• Negative & data-driven — authorization, boundary, malformed, table-driven, and property-based (incl. double-booking)
• Security — BFLA/IDOR authorization matrix, token tampering, header hygiene, secret non-leakage (OWASP API-oriented)
• Performance — k6 smoke load on the booking flow with enforced p95-latency and error-rate budgets (perf/)

Getting Started

nvm use
npm install
cp .env.example .env    # configure per-service URLs, credentials, TEST_MODE
npm test

Scripts

Running tests

Script	Purpose
npm test	Everything — unit, pact and every live suite
npm run test:unit	Hermetic framework tests, no network
npm run test:live	All six live suites against the platform
npm run test:local	The same suites against the Docker stack, sequentially
npm run test:smoke	Behavioural happy paths per service
npm run test:contract	Schema, drift and cross-service checks
npm run test:negative	Credentials, authorization matrix, boundary, malformed
npm run test:data-driven	Room and booking matrices from external JSON datasets
npm run test:property	fast-check properties, including double-booking
npm run test:security	OWASP-oriented authorization, token and injection checks
npm run test:watch	Watch mode
npm run coverage	Everything, with thresholds enforced
npm run coverage:local	The same against the Docker stack

Consumer contracts (details)

Script	Purpose
npm run test:pact	Generate the pacts — hermetic, nothing needs to be running
npm run pact:broker:up	Start the ephemeral broker and its database
npm run pact:publish	Publish the pacts, versioned by git sha and branch
npm run pact:verify	Replay every pact against the running providers
npm run pact:can-i-deploy	Refuse to proceed unless every result is present and green
npm run pact:broker:down	Stop the broker and drop its volume

The dockerized platform

Script	Purpose
npm run docker:up	Start the six RBP services
npm run docker:down	Stop them and drop volumes
npm run docker:ps	Show container state
npm run docker:logs	Follow the container logs

Performance (details)

Script	Purpose
npm run perf:smoke	k6 smoke load against the local stack, thresholds enforced
npm run perf:smoke:live	The same against live — read-only; never for the write flow
npm run perf:typecheck	Type-check the k6 scripts against @types/k6

Reporting and diagnostics

Script	Purpose
npm run test:report	Full suite with Allure results
npm run allure:generate	Render the Allure HTML report
npm run allure:open	Open it locally
npm run diagnose:exchanges	Summarise an HTTP_LOG_FILE exchange log — see #67
npm run schema:export	Render the Zod schemas to JSON Schema under schemas/

Quality gates

Script	Purpose
npm run typecheck	TypeScript, strict
npm run lint	ESLint
npm run lint:fix	ESLint with autofix
npm run format	Prettier, write
npm run format:check	Prettier, check only — CI uses this

Contracts

Zod schemas in src/schemas/ are the single source of truth for every service response. npm run schema:export renders them to language-agnostic JSON Schema files under schemas/. The @contract suite validates live responses against these schemas, detects drift (unexpected fields, malformed dates) via strict parsing, and asserts cross-service consistency (a booking surfaces in the room report).

On top of that, Pact contracts state what this framework's client layer requires of auth, room and booking, and CI replays them against the running services before a merge. Schemas describe a response that arrived; a pact describes a request that must keep working. See docs/contract-testing.md.

Resilience

The platform is a shared public demo that cold-starts, so the framework treats transient failure as an expected condition rather than a test result:

• Readiness gate — globalSetup polls /actuator/health across all six services until they are UP or READINESS_TIMEOUT_MS elapses, so a cold start no longer reds an otherwise healthy pipeline.
• Retry with backoff — idempotent requests retry on 408/425/429/502/503/504 and transport failures, with exponential backoff and jitter. POST never retries, and 500 is treated as a real defect.
• Observable — every exchange log entry carries its attempt, so retried calls stay visible in the report.
• Opt-out — negative suites use createServicesWithoutRetry() so a failure assertion always observes the first response.

Variable	Default	Purpose
RETRY_MAX_ATTEMPTS	3	Attempts per idempotent request
RETRY_BASE_DELAY_MS	300	First backoff delay
RETRY_MAX_DELAY_MS	3000	Backoff ceiling
READINESS_TIMEOUT_MS	90000	Total wait for the platform to boot
READINESS_INTERVAL_MS	3000	Interval between health polls

Set RETRY_MAX_ATTEMPTS=1 and READINESS_TIMEOUT_MS=0 to reproduce the pre-retry, fail-fast behaviour.

Test Targets

The suite runs against two targets that implement different versions of the same API:

Target	Command	What it is
live	npm run test:live	The hosted platform at automationintesting.online
local	npm run test:local	RBP 2.2 images via docker compose, offline and disposable

cp .env.local.example .env.local
npm run docker:up      # start the six services
npm run test:local     # run every live suite against the container stack
npm run docker:down    # stop and remove volumes

The local target runs nightly in CI, never on pull requests: 20 of the 140 tests skip against it, so a green run there is not evidence about the deployed platform. Its job is to catch the two targets drifting further apart — see test-strategy.md.

The target is selected by ENV_FILE; each env file sets TEST_MODE, which drives the expectation profile.

They are not the same API

The hosted platform runs code that is not in the open-source project — verified against the 2.2 images, the latest images and upstream trunk source, all three of which agree with each other and disagree with live. So the suite does not pretend one set of expectations fits both:

expect(response.status).toBe(expectedStatus('auth.rejected'))   // 401 live, 403 local
itWhenSupported('auth.tokenInBody')('returns the token in the body', …)

src/profiles/target-profile.ts holds every difference in one place. Full inventory, including why the defect guards are live-only, in docs/target-differences.md.

Writing a Test

Tests never touch HTTP. A service method returns a typed ApiResponse<T>, and a non-2xx is a value to assert rather than an exception to catch:

const { room } = createServices()

const response = await room.create(roomPayload(), token)

expect(response.status).toBe(expectedStatus('resource.created'))

Services are thin and declarative — the fluent builder carries the token, so a negative test is just the same call without one:

export class RoomService {
  async create(
    payload: RoomPayload,
    token?: string,
  ): Promise<ApiResponse<SuccessResponse | ErrorsResponse>> {
    return this.client.request(RequestBuilder.post('').withBody(payload).withToken(token).build())
  }
}

withToken(undefined) omits the header entirely, which is why room.create(payload) reads as "create without authenticating" instead of needing a separate code path.

A cross-service test

The most valuable assertions span services — a booking created through one service must surface in another's report:

it('a booking created via BookingService is reflected in the room report', async () => {
  const response = await report.getByRoom(testRoom.roomid, token)
  const validated = assertValid(reportSchema, response.data)

  expect(validated.report).toContainEqual({
    start: testBooking.bookingdates.checkin,
    end: testBooking.bookingdates.checkout,
    title: 'Unavailable',
  })
})

assertValid() parses the response through its Zod schema before the assertion runs, so a shape change fails as a contract violation with a precise path — not as a confusing undefined three lines later.

Guarding a known defect

A platform bug is encoded as the behaviour that should happen. The suite stays green while the defect exists and turns red the moment it is fixed:

guardsDefect('BUG-002', 'returns 404 for a deleted room', async () => {
  const created = await createRoom(roomPayload())
  await room.delete(created.roomid, token)
  createdRoomIds.forget(created.roomid)

  const response = await room.getById(created.roomid)

  expect(response.status).toBe(404)
})

guardsDefect classifies the outcome rather than inverting it: a failed assertion means the defect still reproduces, a clean pass means it is fixed (and names the report to close), and a timeout or any other error fails the test. Its predecessor, it.fails, accepted any failure at all — so a request that never completed looked exactly like a defect still present. That cost three false greens before it was replaced (why).

Each guard is paired with a written report in docs/bug-reports/ — twelve reports, twelve guards, and a unit test asserting both directions of that parity.

Defects Found

Twelve confirmed platform defects, each with reproduction steps, evidence and a guarding test:

ID	Defect	Severity
BUG-001	Auth token remains valid after logout	Major
BUG-002	Fetching a deleted room returns 500 instead of 404	Minor
BUG-003	Booking update errors leak internal implementation details	Major
BUG-004	Message inbox is readable without authentication	Major
BUG-005	A booking can be created for a non-existent room	Major
BUG-006	Branding cannot be round-tripped	Minor
BUG-007	An invalid token returns 500 instead of 401	Major
BUG-008	Booking summary accepts any non-empty token	Major
BUG-009	Report stalls ~31 s before rejecting an invalid token	Major
BUG-010	Responses leak infrastructure details in headers	Minor
BUG-011	API responses carry no standard security headers	Minor
BUG-012	Oversized string input returns 500 instead of 400	Minor

Reporting

• JUnit XML is produced by the CI pipeline for every run.
• Allure results are generated with npm run test:report; npm run allure:generate renders the HTML report (npm run allure:open to view it locally).
• The Report workflow runs the full suite, builds the Allure report and publishes it to GitHub Pages on every main build and on a nightly schedule (03:00 UTC), so the live report always reflects the latest run against the platform.

Documentation

• Architecture — layering, request flow, run lifecycle, retries
• Test Strategy — scope, risk prioritisation, suite taxonomy, CI strategy
• Target Differences — live vs local, and why they are not the same API
• Contract Testing — Pact consumer contracts, provider verification, and who the consumer is
• Security Scan — OWASP ZAP baseline in CI, and how it complements the security suite
• Performance — k6 smoke-load harness, budgets, and why load is its own layer
• Bug Reports — twelve defects with evidence and guarding tests

Contributing

See CONTRIBUTING.md for the branching model, commit conventions, and workflow, and CODE_OF_CONDUCT.md for community expectations.

License

MIT © Andrii Kohut