Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
214 lines (185 loc) · 13 KB

ARCHITECTURE.md

File metadata and controls

214 lines (185 loc) · 13 KB
Raw

Architecture

How this Rust rebuild is put together, and why. It realizes a structure-agnostic design: the wire contracts, data provenance, algorithms, and invariants are fixed; the stack here is one valid way to honor them.

Shape

The backend is a Rust coordinator in front of two external engines. Routing and geocoding are mature, memory-heavy JVM engines (OpenTripPlanner, Komoot Photon) — we orchestrate them, we don't reimplement them. Everything else (tile serving, styles, overlays, offline, live-trains, health, and the data pipeline) is Rust.

                 external proxy (TLS, domain, prod CORS, rate-limit)   ── out of scope (P3)
                                     │
        ┌────────────────────────────┼─────────────────────────────┐
        │                 internal container network                │
        │                                                           │
        │   ┌─────────────┐   reverse-proxy    ┌──────────────┐     │
client ─┼──▶│ iter-gateway │──────────────────▶│ OTP  :8080   │     │  routing
        │   │   (edge/BFF) │──────────────────▶│ Photon :2322 │     │  geocoding
        │   │    :8090     │                    └──────────────┘     │
        │   │  serves directly:                                      │
        │   │   tiles · styles · glyphs · sprite · overlays          │
        │   │   offline · live-trains · health · manifest            │
        │   └─────────────┘                                          │
        │          ▲ reads                                           │
        │   ┌──────┴───────────────┐         ┌──────────────┐        │
        │   │  read-only artifacts │◀────────│ iter-pipeline│ (build)│
        │   │  (graph, index,      │  writes │ iter-worker  │ (jobs) │
        │   │   tiles, overlays)   │         └──────────────┘        │
        │   └──────────────────────┘                                 │
        └───────────────────────────────────────────────────────────┘

Components

iter-gateway (edge / BFF) — stateless, scale-wide

The single ingress for everything the Rust side owns, plus a reverse proxy to the engines. It is stateless across requests (no per-client state), so replicas scale horizontally with no coordination. Surfaces:

  • Tiles — GET /tiles/*.pmtiles via HTTP byte-range (gzip off; the archive is internally compressed and must be clustered).
  • Styles / glyphs / sprite — the four MapLibre styles with per-request __BASE_URL__ substitution; glyph stacks fall back to NotoSans-Regular (never 404).
  • Overlays — GET /overlays/{metro-stations,transit-lines}.geojson, fail-soft (missing file → client draws nothing, no error).
  • Reliability — GET /reliability/{route}/{direction}/{stop} serves the worker-written Tier-2 archive (p50/p85/p90 delay, on-time rate, sample count per tod-bucket/day-type), fail-soft (absent/missing/corrupt store → empty cells, never an error) and path-safe; the read side lives in iter-core so the gateway never depends on the worker (ADR 0024).
  • Offline — GET /offline/{extract,bundle}; range-reads the clustered PMTiles (via the pinned go-pmtiles CLI) and zips a bundle. Abuse guards (6 deg² area cap, z14 clamp, 3 concurrent) are the only protection on this public, auth-less surface.
  • Live-trains — GET /trenitalia/*; a normalized, TTL-cached, single-flighted proxy over ViaggiaTreno.
  • Health — client-facing freshness health.json, the GET /manifest per-artifact freshness document, plus the orchestration probes (/livez, /readyz).
  • Routing / geocoding — reverse-proxied to OTP / Photon; the BFF already hosts place enrichment/correlation (below). Soft itinerary re-ranking now starts here: the opt-in POST /otp/gtfs/v1?rerank=reliability path buffers the plan and stably reorders its itineraries by a reliability score, default-off and fail-soft so the passthrough never regresses (ADR 0026).
  • Places — GET /places/enrich fuses open sources (Wikipedia summary, Wikidata, Wikimedia Commons) into the normalized Place DTO with per-field provenance; GET /places/image proxies a Commons image through the BFF; GET /places/related correlates the places sharing a searched civico via an in-memory address-bucket index (ADRs 0011, 0012).

iter-pipeline (build tier) — one-shot / scheduled

The idempotent data-prep orchestrator: fetch → clip → build → render → import → write-health, every step skipped when its output exists and forceable individually (FORCE_<step> / SKIP_<step>). It coordinates external tools (osmium, planetiler, the OTP graph build, the Photon import) — which live inside the build image, never on the host — alongside Rust-native steps (glyph fetch, style render, build-config generation, overlay generation, health write).

Implemented steps: OSM (fetch the regional PBF) → CLIP (osmium carves the routing extent) → GTFS (fetch the region's feeds) → BUILD_CONFIG (pin OTP's inputs with stable feedIds) → GRAPH (OTP --build --save) → OVERLAY (transit overlays from the clip) → STYLES (render the four MapLibre styles) → CIVICI (Overture house numbers via DuckDB) → PHOTON (geocoding index import) → PLACES (addressed POIs for the correlation index) → TILES (planetiler render) → HEALTH. OTP's inputs and its graph.obj share /data/graph (loaded read-only by OTP, ADR 0009); the Photon index lives at /data/photon and is served read-write (embedded OpenSearch; ADR 0010), with civici baked in as low-importance house docs so location bias picks the right number. OVERLAY generates the transit overlays from the OSM clip in pure Rust — transit-lines (way-union line geometry) and metro-stations (concave-hull concourses smoothed into organic footprints via Chaikin corner-cutting + Visvalingam-Whyatt simplification, then dissolved with their overlapping platform strips into one footprint via geo::unary_union — ADR 0031 — plus per-direction platform strips + exits), no shapely (ADR 0014). STYLES renders the four whitelisted MapLibre styles (Standard / Transit × light / dark) into output/styles/, each wired to the region's tile source, the glyph endpoint, the sprite (Standard only), and the region's overlay sources, all via the literal __BASE_URL__ token the gateway rewrites per request — deterministic and byte-stable (ADR 0025). Routing/geocoding/overlay steps no-op for a region lacking that config.

iter-worker (background tier)

Long-running scheduled jobs: FL NeTEx→GTFS (fl-gtfs, on startup + every 24 h) — a streaming quick-xml parser converts the official Lazio NeTEx into a routable GTFS for the OTP graph build (ADR 0016) — and GTFS-RT ingestion (rt-reliability, every 30 s) — polls ATAC's trip-updates feed, decodes it (a vendored prost GTFS-RT subset, no protoc), and derives validated stop-delay events on the stable (route, direction, stop, date) key (ADR 0015). Those events feed the persistent reliability rollup tier (reliability-rollup, ADR 0022): derived events are teed into a hot Tier-0 partition, folded into bounded warm Tier-1 aggregates (mergeable moments + a fixed-bin delay histogram), then merged into a tiny permanent Tier-2 keyed on (route, direction, stop, tod-bucket, day-type) — the worker's first persistent state, holding vehicle/road aggregates only (never a user/device/ session key). The pure rollup core and the path-safe Tier-2 reader live in iter-core, so the gateway serves this archive (GET /reliability/*, ADR 0024) without depending on the worker. Jobs are a name/interval/run abstraction — a job failure is logged and the schedule continues (unlike a pipeline step, which aborts), so a transient upstream blip never takes the worker down; it scales independently of the request path.

Shared crates

  • iter-core — config helpers, the {error:{code,message,details?}} envelope, operator-local tracing, SIGINT/SIGTERM graceful shutdown, the liveness/readiness model.
  • iter-contracts — the wire DTOs (camelCase field names the client greps for): geo::BBox, health documents, live-trains board/station, offline manifest + caps + error codes.
  • iter-region — the region model: profile schema + the root→leaf resolver (see Region model).

(Tile range-extract is done by the pinned go-pmtiles CLI rather than a Rust PMTiles reader; see ADR 0007.)

Region model

Region is a first-class, region-generic abstraction — not hardcoded constants (ADR 0008). A region is a node in a tree of declarative profiles (regions/<path>/region.toml); a deployment targets a node (ITER_REGION=italy/lazio/rome) and iter-region resolves the chain root→leaf into one effective config: the decoupled extents, geocoding, live-trains provider, feeds, and overlays. Data is placed by service area, not operator — the all-Italy basemap + geocoding + ViaggiaTreno boards at the italy root, COTRAL/COTRAL-FERRO/FL at lazio, ATAC + overlays at rome. The pipeline, gateway, and worker consume the resolved config and stay region-generic, so adding a region (Milan, Paris, all of Europe) is config + data, no recompile.

Scaling model

The design follows the build/serve asymmetry and scaling asymmetry of a data-heavy backend:

  • Build ≠ serve. Data-prep is a multi-GB, CPU-saturating, minutes-long peak that exits; serving is light and runs forever. They never run together, so builds live in iter-pipeline (a one-shot/cron tier) and ship read-only artifacts to the serving tier.
  • Stateless wide, stateful narrow. The edge holds no dataset → scale it wide for throughput. OTP/Photon load their whole dataset into each instance → keep them narrow (a replica is for HA, not throughput, because it duplicates the dataset).
  • Artifacts are externalized and regenerable. Nothing the backend stores is irreplaceable (P7); "delete and rebuild" is always valid. On one host the artifact tree is a shared mount; on a fleet it is baked into versioned images.

This is why the same code runs as a single compose stack (docker/compose.yaml) and scales to Kubernetes replicas + a worker tier: statelessness + externalized state + graceful drain + readiness gating are designed in, not bolted on.

A deliberate evolution. The proof-of-concept's design fixed "single host, no Kubernetes" (its P2). This rebuild keeps single-host "clone + up" as the default and makes the code K8s-ready (stateless replicas, worker tier, blue/green-friendly artifact delivery). The advanced orchestration manifests themselves live in a separate iter-maps/deploy repo, so single-host stays the default and orchestration is opt-in.

Configuration

Entirely environment-driven (.env for "clone + up"); no host-side state. The active region (ITER_REGION) selects a node in the region tree, whose resolved profile supplies the decoupled extents (basemap vs routing vs overlay), feeds, geocoding, and overlays — so the map can go nationwide while routing stays scoped. The pipeline's per-step FORCE_*/SKIP_* knobs make refresh granular, and env knobs (e.g. PMTILES_BOUNDS) can override a profile value for a small host.

Invariants honored

  • No commercial keys — every tile/geocode/route comes from open, self-hostable data and runtimes.
  • No proxy/TLS/host-ports in the deployment — an external proxy owns those; the stack exposes no host ports in production (a dev override publishes them).
  • Zero-touch idempotent setup — clone + up, every step skip-if-present.
  • No persistent user state — the server is stateless; personalization arrives as request params. (A future opt-in, end-to-end-encrypted sync blob store is the one scoped exception — it holds only opaque ciphertext.)
  • Host-agnostic artifacts — styles/bundles carry the literal __BASE_URL__ placeholder, rewritten per-request online and to file:// offline.

Roadmap

The external-engine integration (OTP graph build, Photon import, planetiler render, overlay geometry, FL NeTEx→GTFS) and the planned capabilities (personalized planning, place discovery, traffic, crowd telemetry, reliability archive, Italy/Europe scaling) are tracked in roadmap/, each linked to its design source.