Skip to content

opendatadurban/ocpo-api

Repository files navigation

OCPO FastAPI Template

This is as a realtime quey API for any entity that want's to get CSD data. Built on FASTAPI, a lightweight Python API framework.

Technology Stack

  • ⚑ FastAPI
    • 🧰 SQLAlchemy V2 for the Python SQL database interactions (ORM).
    • πŸ” Pydantic, used by FastAPI, for the data validation and settings management.
    • πŸ’Ύ PostgreSQL as the SQL database.
    • πŸ” JWT Authentication - JSON Web Token based authentication
  • πŸ‹ Docker Compose for optional containerized development and builds (the app can also be run directly on the host; see below)
  • βœ… Tests with Pytest.
  • 🏭 CI (continuous integration) and CD (continuous deployment) based on GitHub Actions.

✨ Features

  • User authentication with JWT tokens
  • Role-based access control (superuser and regular users)
  • Full CRUD operations for users and items
  • Secure password hashing
  • Email uniqueness validation
  • Self-service user management
  • Administrative user management
  • Item ownership and permissions
  • Async database operations
  • Input validation and error handling

Configure

You can then update configs in the .env files to customize your configurations.

Before deploying it, make sure you change at least the values for:

  • POSTGRES_PASSWORD

You can (and should) pass these as environment variables from secrets

Running without Docker

You can run the API and Alembic on your machine with a PostgreSQL instance you provide (local install, managed service, or Postgres only in Docker).

Prerequisites

  • Python 3.12
  • PostgreSQL (16 or newer is a good match for the Docker image in compose.yaml)

Python environment

From the repository root:

python -m venv .venv

Activate the virtual environment:

  • Linux or macOS: source .venv/bin/activate
  • Windows (PowerShell): .venv\Scripts\Activate.ps1

Install dependencies:

python -m pip install --upgrade pip
python -m pip install -r requirements.txt
python -m pip install -r requirements-dev.txt

The second requirements file is optional for running the API alone; include it for tests, linting, and other dev tooling.

Database and .env

Create an empty database (and role, if needed) that matches the values you will put in .env. Copy .env.example to .env at the repository root and set at least POSTGRES_*, SECRET_KEY, FIRST_SUPERUSER, and FIRST_SUPERUSER_PASSWORD as described in Configure.

For PostgreSQL on the same machine using the default port, use POSTGRES_SERVER=localhost and POSTGRES_PORT=5432.

Settings load the root .env when commands are run with a working directory of src (see app/core/config.py).

Migrations and API server

Alembic and Uvicorn expect the working directory to be src so the app package and alembic.ini resolve correctly (this mirrors the layout under /usr/src in the container).

cd src
alembic upgrade head
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

Then open the interactive docs at http://localhost:8000/docs (ReDoc: http://localhost:8000/redoc).

To install and configure ocpo-api as a Windows service using NSSM, see docs/nssm-windows-service.md.

Hybrid: Postgres in Docker, API on the host

If you want Docker only for the database, start the dev database service and leave the API on the host:

docker compose --profile dev up db-dev -d

With the default port mapping in compose.yaml, Postgres is available on the host at localhost:8081. Point .env at it with POSTGRES_SERVER=localhost, POSTGRES_PORT=8081, and the same POSTGRES_USER, POSTGRES_PASSWORD, and POSTGRES_DB values your Compose stack uses. Then run the Python steps above from src.

Tests without Docker

With requirements-dev.txt installed and PostgreSQL reachable using the connection details in .env, from src run:

cd src
python -m pytest

The test suite configures the engine from the same settings as the app and, for the session-scoped engine fixture, drops and recreates tables in that database. Use a dedicated database (for example a separate POSTGRES_DB value in .env while testing) or keep using ./scripts/test-local.sh, which runs tests against an isolated Compose profile.

Windows service (NSSM)

To run the API as a Windows service with NSSM (including use of the repository run_api.bat wrapper for PYTHONPATH and the virtualenv), see docs/nssm-windows-service.md.

πŸ‹ Running with Docker

If requirements change build a new base image

docker buildx build -f Dockerfile.base -t ocl_base_image:latest .

Start the stack with Docker Compose:

./scripts/dev-local.sh

To seed data into the db. (Make sure to change the SEED_FILE variable in your .env)

# Option 1 (CLI): copy the XLSX into the running container, then run the seeder
docker cp "Incident_DATA.xlsx" ocl-api-oag-api-dev-1:/usr/src/Incident_DATA.xlsx
docker exec -it ocl-api-oag-api-dev-1 python -m app.seed_incident_db --file "/usr/src/Incident_DATA.xlsx"

Alternatively (recommended for non-technical users), upload the monthly XLSX and seed via the API:

  • Swagger: open /docs on the same base URL you use for the API (for example http://localhost:8000/docs without Docker, or the host port from compose.yaml with Docker) β†’ POST /api/v1/incidents/seed
  • curl (replace the base URL and port to match your setup):
curl -X POST "http://localhost:8000/api/v1/incidents/seed" \
  -H "Authorization: Bearer <access_token>" \
  -F "file=@Incident_DATA.xlsx"

This will:

  • Build the API container
  • Start PostgreSQL
  • Set up the database
  • Start the FastAPI server

πŸ§ͺ Testing

Running Tests

Execute the test suite:

./scripts/test-local.sh

This will:

  • Set up a test database
  • Run the pytest suite

Test Coverage

The project includes:

  • Unit tests for CRUD operations
  • Integration tests for API endpoints
  • Fixture-based test setup
  • Transaction rollback for test isolation
  • Async test support

πŸ“š API Documentation

Once running, access the interactive API documentation at:

  • Without Docker (Uvicorn on port 8000 as above): Swagger UI at http://localhost:8000/docs, ReDoc at http://localhost:8000/redoc
  • With Docker Compose: use the host port mapped to the API container in compose.yaml (for example 8085 β†’ http://localhost:8085/docs)

πŸ“ Project Structure

.
β”œβ”€β”€ Dockerfile           # Main Dockerfile for the application
β”œβ”€β”€ Dockerfile.base      # Base Dockerfile for common dependencies
β”œβ”€β”€ README.md           # Project documentation
β”œβ”€β”€ compose.yaml        # Docker Compose configuration
β”œβ”€β”€ development.md      # Development guidelines and setup
β”œβ”€β”€ requirements-dev.txt # Development dependencies
β”œβ”€β”€ requirements.txt    # Production dependencies
β”œβ”€β”€ scripts/           # Utility scripts
β”‚   β”œβ”€β”€ create_alembic_revision.sh  # Database migration script
β”‚   β”œβ”€β”€ dev-local.sh               # Local development startup
β”‚   └── test-local.sh             # Test execution script
└── src/               # Source code root
    β”œβ”€β”€ alembic.ini    # Alembic configuration
    └── app/           # Main application package
        β”œβ”€β”€ alembic/   # Database migrations
        β”‚   β”œβ”€β”€ README
        β”‚   β”œβ”€β”€ env.py
        β”‚   β”œβ”€β”€ script.py.mako
        β”‚   └── versions/
        β”œβ”€β”€ api/       # API layer
        β”‚   β”œβ”€β”€ deps.py           # FastAPI dependencies
        β”‚   β”œβ”€β”€ main.py          # API configuration
        β”‚   └── routes/         # API endpoints
        β”‚       β”œβ”€β”€ items.py    # Item endpoints
        β”‚       β”œβ”€β”€ login.py    # Authentication endpoints
        β”‚       β”œβ”€β”€ users.py    # User endpoints
        β”‚       └── utils.py    # Route utilities
        β”œβ”€β”€ core/      # Core functionality
        β”‚   β”œβ”€β”€ config.py      # Application configuration
        β”‚   β”œβ”€β”€ db.py         # Database connection
        β”‚   └── security.py   # Security utilities
        β”œβ”€β”€ crud/      # Database operations
        β”‚   β”œβ”€β”€ base.py      # Base CRUD operations
        β”‚   β”œβ”€β”€ items.py     # Item CRUD operations
        β”‚   └── users.py     # User CRUD operations
        β”œβ”€β”€ main.py    # Application entry point
        β”œβ”€β”€ models/    # Data models
        β”‚   β”œβ”€β”€ api/         # API models (Pydantic)
        β”‚   β”‚   β”œβ”€β”€ generic.py
        β”‚   β”‚   β”œβ”€β”€ items.py
        β”‚   β”‚   └── users.py
        β”‚   └── db/          # Database models (SQLAlchemy)
        β”‚       β”œβ”€β”€ base.py
        β”‚       β”œβ”€β”€ items.py
        β”‚       └── users.py
        β”œβ”€β”€ pytest.ini  # Pytest configuration
        β”œβ”€β”€ tests/      # Test suite
        β”‚   β”œβ”€β”€ api/          # API tests
        β”‚   β”‚   └── routes/
        β”‚   β”‚       β”œβ”€β”€ test_items.py
        β”‚   β”‚       β”œβ”€β”€ test_login.py
        β”‚   β”‚       └── test_users.py
        β”‚   β”œβ”€β”€ conftest.py   # Test fixtures
        β”‚   β”œβ”€β”€ crud/         # CRUD tests
        β”‚   β”‚   β”œβ”€β”€ test_item.py
        β”‚   β”‚   └── test_user.py
        β”‚   └── utils/        # Test utilities
        β”‚       β”œβ”€β”€ item.py
        β”‚       β”œβ”€β”€ user.py
        β”‚       └── utils.py
        └── utils.py    # General utilities

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages