Shoppipeline — ML jobs + Supabase

This repo implements the late-delivery pipeline from docs/PIPELINE_PLAN.md: ETL → train → batch inference writing to order_predictions, automated in GitHub Actions. The Next.js app lives under apps/web/ (see docs/WEB_APP_PLAN.md) — set Vercel Root Directory to apps/web when your teammate adds it.

What you need

• Python 3.11+ locally (match CI).
• Supabase Postgres with migrations applied from supabase/migrations/.
• GitHub secret DATABASE_URL on the repo: Supabase Settings → Database → URI (use session mode or pooler per Supabase docs; append ?sslmode=require if needed).

Database

# If using Supabase CLI linked to the project:
supabase db push
# Optional dev seed:
# psql "$DATABASE_URL" -f supabase/seed.sql

Shared contract with the web app: tables customers, orders, order_items, products, order_predictions; priority-queue SQL is in docs/WEB_APP_PLAN.md.

Local pipeline

cp .env.example .env   # add DATABASE_URL
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
export ARTIFACTS_DIR="$(pwd)/artifacts"
mkdir -p artifacts
export DATABASE_URL="postgresql://..."
python jobs/etl_build_warehouse.py
python jobs/train_model.py
python jobs/run_inference.py

GitHub Actions

Workflow: .github/workflows/pipeline.yml.

Trigger	Behavior
Daily cron (06:00 UTC)	full: ETL → train → inference; uploads artifact late-delivery-model.
workflow_dispatch mode: full	Same as above.
workflow_dispatch mode: inference-only	Downloads the latest successful late-delivery-model artifact from this workflow, then runs inference only.

First-time setup: run full once (Actions tab or push) so a model artifact exists before inference-only (what the Vercel “Run Scoring” button uses).

Web app integration (for your teammate)

• Dispatch body: POST GitHub API with "inputs": { "mode": "inference-only" } — must match the workflow input name mode and values full | inference-only.
• Secrets: DATABASE_URL in GitHub for the workflow; Vercel env vars per docs/WEB_APP_PLAN.md.
• Flow: app inserts/updates operational rows → workflow scores unfulfilled orders → order_predictions upserted → priority queue reads join order_predictions.

Part 2 (assignment)

Notebook outline: notebooks/part2_is_fraud_crispdm.ipynb (is_fraud — separate from late-delivery features).