Parsnip Backend API

Parsnip is a Flask + PostgreSQL backend for protocol parser configuration data, with:

• a full REST API (blueprints under api/blueprints/)
• canonical Parsnip IL (.pil) export and import in a single call
• atomic tree-upsert endpoints for switches and object-field advanced children
• Scalar API documentation as the full docs surface
• a full pytest suite, including IL parity + preflight tests

This repo is self-contained — ./tooling/parsnip.sh handles local dev, docs, tests, and production entry points without needing anything outside this directory. It also slots in as a submodule of the parsnip superproject when you want to run it alongside the frontend and compiler.

Resource scoping note:

• Enums, bitfields, objects, and switches use scope_id.
• If a request omits scope_id, the API resolves a default general scope automatically.

One Command Entry Point

./tooling/parsnip.sh

This opens a simplified interactive menu with four choices:

• Dev Backend
• Production Backend
• Demo
• Exit

Common Commands

./tooling/parsnip.sh backend-dev
./tooling/parsnip.sh backend-prod
./tooling/parsnip.sh demo

Blueprints

Registered in api/api.py:

• parsers — parsers.py — parsers, parser configuration, /parsers/<id>/generate_il, /parsers/import, /parsers/compile
• objects — objects.py — objects, object fields, inputs, until/conditional children, /object_fields/<id>/upsert-advanced
• switches — switches.py — switches, options, actions, /switches/<id>/upsert-tree
• enums — enums.py — enums + enum values
• bitfields — bitfields.py — bitfields + bitfield fields
• dependencies — dependencies.py — parser/switch/object dependencies
• configuration_associations — configuration_associations.py — configurations, scopes, port tuples, user types, copyrights

IL Export / Import

• GET /parsers/<id>/generate_il — exports the parser as a .pil zip (per-scope config.json / enums.json / bitfields.json / objects.json / switches.json). Runs il_preflight.run_preflight first and returns 422 with structured diagnostics if any invariant fails (missing references, unresolved entry point, sub-byte standalone enums, etc.).
• POST /parsers/import — materializes a .pil bundle into a new parser tree in one transaction. multipart/form-data with file; optional on_conflict=error|rename|replace and parser_name overrides. Runs preflight against the freshly-built graph and rolls back on any violation. Implementation in api/il_import.py.

Compile

• POST /parsers/compile — proxies an uploaded .pil (multipart/form-data with file) to the parsnip compiler microservice. The service runs the compiler and then spicyz to produce a Zeek-loadable .hlto, and returns the artifact tree (base64 zip) plus the captured toolchain logs. No DB writes; the upload is ephemeral. Override the microservice URL with PARSNIP_COMPILER_SERVICE_URL (default http://parsnip-compiler-service:5001). Failures surface as HTTP 422 with the log payload; an unreachable service returns 502.

Atomic Upsert Endpoints

Tree-shaped edits ship as one transaction to avoid frontend-choreographed partial-failure windows:

• POST /switches/<id>/upsert-tree — replace switch name, options, per-option actions + inputs, default action, enum-derived + additional dependencies.
• POST /object_fields/<id>/upsert-advanced — replace an object field's inputs, conditional field + tuples, and until conditional.

Scalar Docs (Primary Docs System)

Start the dev workbench (recommended):

./tooling/parsnip.sh backend-dev

This command:

• starts Docker services (api + db) with docs-safe CORS
• resets the local Docker database volume so the docs workbench always uses the clean current schema
• waits for API health
• installs/updates Python dependencies in .venv
• seeds docs example data for valid Try-It defaults
• regenerates docs/openapi.json
• serves Scalar docs locally
• shuts down everything cleanly on Ctrl+C

This is the prefilled testing mode. It seeds valid example records so Scalar Try it requests already have working scope_id, parser, and association defaults.

Then open:

• http://127.0.0.1:8000/ (full Scalar docs experience)

Regenerate the spec manually:

source .venv/bin/activate
PYTHONPATH=api .venv/bin/python tooling/generate_openapi.py

Build static docs snapshot:

./tooling/build-scalar-docs.sh

Output is written to site/.

Validate all Scalar prefilled operations against a live API:

./tooling/run-scalar-prefill-validation.sh

Audit OpenAPI contract completeness against the Flask route map:

source .venv/bin/activate
PYTHONPATH=api .venv/bin/python tooling/audit_openapi_consistency.py

Migrations

Alembic migrations live under api/migrations/versions/. The Docker entrypoint runs alembic upgrade head on container start.

Add a migration:

source .venv/bin/activate
cd api && alembic revision -m "short description"

Keep revision IDs ≤ 32 characters — the alembic_version column caps there.

Testing

./tooling/run-tests.sh

Test tree under test/: parsers/ (endpoints, generate_il, preflight), roundtrip/ (per-entity CRUD), validation/, schema/, scopes/, conditionals/, availability/, impacted/, enums/, enumvalues/, objects/, bitfields/, configuration_associations/.

Production Notes

• backend-prod uses docker-compose.yml + docker-compose.prod.yml.
• Production override removes API source bind mounts and enables restart policies.
• backend-prod does not seed Scalar example data or run the local Scalar server.
• Set environment values explicitly for production deployments (DB_*, CORS_ORIGINS, and related secrets/config).

Repo Requirements

• Docker + Docker Compose (docker compose or docker-compose)
• Python 3.9+