Skip to content

Repository files navigation

Expense Tracker

QA automation portfolio project (Project B). A Python-native expense tracker (FastAPI, Jinja2, PostgreSQL) built as the target application for a layered test automation suite: Selenium for the UI, Schemathesis for the API contract, and Locust for performance. It is one of two portfolio projects built with different tooling than its counterpart (Node/React, Playwright, Pact) to demonstrate range across both major test automation stacks.

Stack

  • Backend: FastAPI
  • Database: PostgreSQL, accessed via SQLAlchemy
  • Frontend: FastAPI with Jinja2 server rendered templates
  • UI testing: Selenium (Python), Page Object Model, explicit waits
  • API testing: pytest with Schemathesis (property based testing against the generated OpenAPI schema)
  • Performance testing: Locust
  • CI/CD: GitHub Actions, three parallel jobs (API schema, UI, performance smoke)
  • Containerization: Docker and docker-compose (app and Postgres)

Design decisions

This section documents the non-obvious choices made in this project and the reasoning behind them.

src layout (src/app/) instead of a flat layout. A flat layout lets import app silently resolve to an uninstalled local directory. With a src layout, the package is only importable once it is actually installed (pip install -e .), so tests run against the real installed package rather than whatever happens to be in the working directory. This is recommended by the Python Packaging Authority and is cheap to set up correctly, even though it matters more for library packaging than for a plain web application.

pyproject.toml (PEP 621) with hatchling instead of requirements.txt. Hatchling was chosen over Poetry because it needs almost no configuration for a single src layout package and does not introduce a separate lockfile or tooling layer that would be hard to justify for a project of this size.

psycopg2-binary instead of psycopg3. psycopg2 remains the more widely used and better documented synchronous driver for SQLAlchemy with Postgres, and was chosen for that reason over the newer psycopg3.

No Alembic yet. The schema is created with Base.metadata.create_all() on startup instead of migrations. This is a scoping decision for a two day project rather than an oversight. Alembic would be the natural next step for a production grade version.

amount is Numeric(10, 2), not Float. Floats use binary floating point and accumulate rounding error on money (for example, 0.1 + 0.2 != 0.3). Numeric/Decimal performs exact base 10 arithmetic, which is the standard choice for currency values.

category is a fixed Python Enum, not free text. Invalid categories are rejected at the API boundary (FastAPI returns 422 automatically) instead of letting arbitrary strings accumulate in the database. The trade-off is that adding a category requires a code change rather than just new data, which is acceptable since the category set is small and stable.

Jinja2 pages call crud.py directly instead of making an internal HTTP call to the JSON API. Both run in the same process. The JSON API and the HTML pages are two presentation layers over the same business logic, so there is no reason to round trip through HTTP to talk to itself.

Page routes use include_in_schema=False. Without this they would be pulled into the OpenAPI spec and Schemathesis would fuzz HTML form endpoints as if they were part of the JSON contract. Page routes are covered by the Selenium suite instead; Schemathesis is scoped to the JSON API only.

The create form uses Post/Redirect/Get (a 303 redirect on success). This prevents a page refresh after submission from resubmitting the same expense.

Bugs found while testing

tests/api/test_schema.py runs Schemathesis's property based fuzzing against the application's own generated OpenAPI schema, in process via schemathesis.openapi.from_asgi (no separate server process is needed). This surfaced real issues within a handful of runs, which is the main value of schema based testing over hand written example tests: it exercises inputs that would not otherwise be covered. Each finding was investigated individually rather than simply changed until the fuzzer stopped complaining.

Finding Verdict Resolution
OPTIONS /api/expenses returns a 405 whose Allow header omits GET Framework behavior. Starlette computes the Allow header per matched route rather than per path when multiple methods share a path. Excluded the allow_header_conformance check, documented inline.
Undeclared query parameters (for example ?x=1) are accepted instead of rejected Accepted design trade-off. This is FastAPI's default permissive, forward compatible behavior; strict rejection would trade away tolerance for unrecognized but harmless parameters. Excluded the negative_data_rejection check, documented inline.
amount's decimal_places=2 constraint has no OpenAPI equivalent, so a value with three or more decimal places that Schemathesis considered schema valid was rejected by the API Schema and implementation mismatch. Added multipleOf: 0.01 to the schema and tightened the lower bound from gt=0 to ge=0.01, the actual business minimum, removing a floating point edge case near zero.
amount had no upper bound. A large value passed validation but crashed at the database layer (NumericValueOutOfRange) with an unhandled 500. Real bug. Added le=99999999.99, matching the Numeric(10, 2) column exactly.
Pydantic's JSON schema for Decimal is anyOf[number, string]. Only the number branch carried the min/max, so an oversized value sent as a string still validated as valid data. Schema generation quirk in how Pydantic represents Decimal. Overrode json_schema_extra to pin the schema to a single number representation, and added a JSON mode field_serializer so the response now emits a number as well (Pydantic serializes Decimal to a string by default, which would have contradicted the corrected schema).
skip had no upper bound. A value beyond Postgres's bigint range crashed the OFFSET clause with an unhandled 500. Real bug. Added le=1_000_000.
A malformed, non-JSON request body returns Starlette's own 400, which FastAPI does not document automatically (it only adds 422 for validation failures on an already parsed body). Documentation gap. Declared 400 explicitly via the route's responses=.
A NUL byte (\u0000) in description passed length and type validation but crashed the Postgres driver (ValueError: string literal cannot contain NUL). Real bug. Replaced with a pattern constraint (^[^\x00]*$) instead of a bespoke validator, so the exclusion is enforced and documented in the schema. Schemathesis then treats the rejection as expected rather than as a contract violation.

Three genuine crash bugs were found and fixed before any Selenium or Locust test was written: two missing upper bounds that let bad input reach the database, and a NUL byte input that crashed the driver. Two further schema accuracy issues were corrected, and two checks were deliberately excluded with the reasoning documented above rather than silently disabled.

UI test suite (Selenium)

tests/ui/ drives the Jinja2 pages through a real browser, which is the layer Schemathesis does not touch (page routes use include_in_schema=False, see above). The suite is structured as a Page Object Model: tests/ui/pages/base_page.py holds the driver, a shared explicit wait, and navigation through the nav bar common to every page; expense_list_page.py and expense_form_page.py each model one page's locators and actions.

Notable implementation decisions:

  • Selenium Manager instead of webdriver-manager. Selenium 4.6 and later auto-resolves the matching chromedriver on its own, which makes a separate driver management dependency unnecessary.
  • Test data is seeded by writing to the database directly (via the application's own crud.py and SessionLocal), the same approach tests/api/conftest.py already uses, rather than seeding through the JSON API over HTTP. This is faster and avoids giving the UI suite a hard dependency on the API layer being available, at the cost of not being a fully black box suite.
  • The suite runs against a real uvicorn process, unlike Schemathesis, which talks to the ASGI application in process. A browser cannot drive an application that is not actually listening on a port. Locally this is docker compose up; in CI, uvicorn is started as a background step (see CI below).
  • The amount=0 validation test submits the form by calling the <form> element's native submit() method through JavaScript instead of clicking the Save button. Per the HTML specification, calling submit() directly skips the browser's own constraint validation (min="0.01" on the input), unlike a real click. This is intentional: any non-browser client, such as curl or a hand crafted request, skips the same client side checks, so this is how the suite proves that the ge=0.01 minimum is enforced by the server rather than relying on the page's HTML attributes.
  • Date fields are set by assigning element.value directly and dispatching input and change events, rather than using send_keys. A native <input type="date"> expects locale formatted keystrokes (for example MM/DD/YYYY on en-US Chrome), which is a well known source of Selenium flakiness across environments.

Load test (Locust)

tests/perf/locustfile.py load tests the two busiest JSON endpoints, GET /api/expenses (list, paginated) and POST /api/expenses (create), weighted 3:1 read to write to approximate real usage. This is a smoke test rather than a capacity benchmark: its purpose is to reveal anything worth knowing under light concurrent load, not to establish a maximum sustainable throughput.

Run against the containerized application:

locust -f tests/perf/locustfile.py --headless -u 20 -r 5 -t 30s --host http://localhost:8000

Result at 20 concurrent users over 30 seconds: 956 requests, 0 failures.

Endpoint Median 95th percentile Max
GET /api/expenses (list) 11 ms 22 ms 114 ms
POST /api/expenses (create) 59 ms 68 ms 166 ms

Finding: creating an expense is consistently about five times slower than listing them, even though both operate at single row scale against a table of only a few hundred rows. The likely cause is in crud.py: create_expense calls db.add(), then db.commit(), then db.refresh(). The commit is a real disk synced write (fsync), and refresh() issues a second round trip SELECT afterward to reload the server generated id and created_at, which ExpenseRead returns. This means every create is a write plus a read, while list is a single read. This is not a bug; the endpoint still responds well under 100 ms at this scale, which is comfortably fine for the traffic this application would ever see. It is the expected shape of a write versus read latency difference, documented here rather than left unexplained.

Running locally

docker compose up --build

Then check http://localhost:8000/health. It should return {"status": "ok"} once the application has connected to Postgres.

CI

.github/workflows/ci.yml runs three jobs in parallel on every push and pull request to master:

  • api-schema: runs the Schemathesis contract suite against a Postgres service container.
  • ui-tests: starts the application as a background uvicorn process against the same Postgres service, waits for /health to respond, then runs the Selenium suite against it.
  • perf-smoke: uses the same setup as ui-tests to run a short (15 second) headless Locust smoke run.

ui-tests and perf-smoke each repeat the Postgres service block and the "start application, wait for health" steps rather than factoring them into a composite action. Extracting a small number of duplicated lines across two jobs was not judged worth the added indirection for a workflow this size.

Status

Both days of the original plan are complete.

Day 1: Expense model with POST and GET /api/expenses (paginated), a Schemathesis contract suite that is stable across repeated runs (see findings above), a Jinja2 frontend (list and add-expense form, Post/Redirect/Get on submit) verified manually in the browser including the server-side validation error path, and CI running the schema suite on every push.

Day 2: a Selenium UI suite covering the expense list (empty state, populated state, date descending ordering) and the add-expense form (the happy path through to the redirect, and the server-side validation path via a client-side validation bypass), five tests in total. A Locust smoke test run against the containerized application produced zero failures at 20 concurrent users, with the create versus list latency difference documented above. CI runs all three jobs in parallel on every push and pull request, confirmed passing on GitHub Actions.

About

API contract, UI & performance test suite for a FastAPI/PostgreSQL app — Schemathesis, Selenium, Locust

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages