Skip to content

t10 docker + demo: containerised recommend app, compose one-command demo — completes M1 - #5

Merged
vlobus merged 3 commits into
mainfrom
t10-docker
Aug 17, 2026
Merged

vlobus merged 3 commits into
mainfrom
t10-docker

Conversation

@vlobus

@vlobus vlobus commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Final M1 task. docker compose up on a clean machine → migrations apply → /health/ready 200 → a seeded anomaly flows to Telegram end-to-end.

what

  • packages/recommend/Dockerfile — python:3.14-slim + the uv binary; two-phase uv sync (deps layer cached on the lockfile, then source + workspace install), so a code change doesn't re-download torch. WORKDIR is the recommend package so alembic resolves; ENTRYPOINT applies migrations then execs the CMD.
  • docker-entrypoint.shalembic upgrade head then exec "$@" (migrations are idempotent, safe every boot).
  • docker-compose.yml — the existing db plus a new app service: builds the image, waits for db health, reaches Postgres by service name, binds 0.0.0.0, runs in demo mode, persists FinBERT weights in a volume, and health-checks /health/ready with a 240s start period for the first model download. (Also fixed the db volume mount to the pg18+ layout, /var/lib/postgresql.)
  • composition.py — extracted the composition root out of main.py into build_components(settings), shared by serve and demo. select_bar_source/select_news_source branch on demo_mode: offline seed bars + null news vs live Massive/Finnhub. Everything else (FinBERT, Anthropic, Telegram, scheduler) is identical to production.
  • demo.py — one-shot backfill → explain → notify, prints each step. docker compose run --rm app python -m hodlin_recommend.demo delivers the anomaly in one command, no waiting for ticks.
  • connectors/offline.pyNullNewsSource (the anomaly self-cites, so zero news still explains).
  • connectors/seed_bars.py — the seed source now returns its full committed series regardless of the requested window: it's a fixed June-2024 fixture standing in for "recent history", and the scheduled backfill asks for the last N days, which those dates never fall inside. Symbol/interval still filter; the live providers still honour the window.
  • config.py — non-secret runtime toggles with defaults (host, port, demo_mode); secrets stay default-less (D17). main.py now binds settings.host/port.
  • README — a real "run it" (compose up + the one-shot demo) and "develop" section.

demo mode

Prices come from the committed seed CSV and news is skipped, so only Anthropic + Telegram need real credentials — the Finnhub/Massive placeholders stay as-is. cp .env.example .env, fill two creds, docker compose up.

verified locally (real Docker)

Image builds; docker compose up applies migrations (→ 0001); /health/ready returns 200; the scheduled backfill ran (50 seed bars, the BTC-USD down anomaly detected, notify ticking); the demo command runs and — on a deliberately fake Anthropic key — the explain step catches the 401 as LLMUnavailable and degrades to an error outcome with no crash and no key leaked (D26). With real creds, explain + notify complete the delivery.

tests

  • tests/test_composition.py — source selection per demo_mode + the non-secret defaults; hermetic (_env_file=None), no live API.
  • Updated the seed-source test to the fixed-series semantics.
  • subagent code review: clean, no must-fixes (two optional polish notes applied — demo-mode marker, pg-volume reset note).
  • full gate green: ruff, mypy strict, import-linter, pytest (117 passed).

M1 acceptance

With this merged, T1–T10 are all green; the import-linter boundary holds; the recommend domain holds no keys/authority; uv.lock is committed; CI passes.

second review round (fc29565)

A pre-merge review pass over the full branch found no must-fix but eight real should-fixes, all now applied:

  • Image 5.53 GB → 1.22 GB. PyPI's torch pulls ~2.9 GB of nvidia-* wheels + triton under sys_platform == 'linux', i.e. on every Docker build, for CPU-only FinBERT inference. Routed torch to the pytorch-cpu index via [[tool.uv.index]] + [tool.uv.sources] (explicit = true, so nothing else resolves there). 37 CUDA packages left the lock; .venv 4.8 GB → 1.1 GB. Re-resolution moved torch 2.12.1 → 2.13.0 — the opt-in real-FinBERT test passes on it.
  • Non-root container. uid 10001, with /models created and chowned in the image so a fresh weights volume inherits that ownership and HF can populate it unprivileged. Verified in the running container: uid=10001(hodlin), /models writable.
  • Pinned the uv binary to 0.9.9 (the version that wrote uv.lock); :latest under uv sync --frozen is exactly the non-reproducibility the lockfile exists to prevent.
  • The demo ordering was wrong. The README presented the one-shot as a follow-up to docker compose up — but by then the scheduler has already claimed and notified the anomaly, so the reviewer's next command prints Nothing delivered (reproduced). It's now the primary path, with up as the "run it as a real service" alternative and the notify-once semantics stated. Also dropped compose's false "within a minute or two" (EXPLAIN_EVERY_S = 300, no immediate first fire) and added restart: unless-stopped.
  • Two test bugs. The client fixture did async with ...: return c, handing out an already-closed client (passed only because no test made a request) → yield + an is_closed assertion. And _env_file=None blocks dotenv but not os.environ, so HOST=0.0.0.0 DEMO_MODE=true — what this branch's own compose exports — failed the defaults test; now delenv'd.
  • .dockerignore cache patterns are context-root-anchored, so nested packages/*/.ruff_cache were shipping → **/ anchored.
  • Doc drift from the extraction: config.py/scheduler.py/serving/app.py still named main.py as the composition root, and backfill.py's docstring claimed the demo passes a fixed end that the seed source ignores.

Re-verified on the rebuilt image (throwaway Postgres, dummy credentials): migrations apply, /health/ready 200, demo-first on a fresh DB gives 50 bars + 1 anomaly, explain degrades to an error outcome on a bad key with no crash and no credential in ingest_runs.detail (0 rows matched a scan for the fake values), image carries no caches or .env. Gate green (117 passed) and CI green.

Deliberately deferred as optional: eager FinBERT construction in the demo path (demo mode never calls it), unused Components fields, AsyncExitStack assembly ordering, and the unclosed AsyncAnthropic transport.

vlobus added 3 commits July 22, 2026 19:58
… service w/ migrations entrypoint + demo mode, composition root extracted for main/demo, seed source as fixed-series fixture, README run-me
…ontainer, pinned uv, honest demo ordering

- torch from the pytorch-cpu index: pypi's linux resolution dragged 2.9gb of
  nvidia/cuda wheels + triton into a cpu-only image for no benefit
- drop root: uid 10001, /models chowned so a fresh weights volume inherits it
- pin the uv binary to 0.9.9 (the version that wrote uv.lock) - :latest under
  --frozen is the non-reproducibility the lockfile exists to prevent
- readme/compose: the one-shot demo is the alternative to `up`, not the sequel;
  running it after `up` printed "nothing delivered" because the scheduler had
  already claimed the anomaly. also drop the false "within a minute or two"
  (explain ticks every 5 min) and add restart: unless-stopped
- test_composition: the client fixture returned from inside `async with`, so
  tests got a closed client; delenv HOST/PORT/DEMO_MODE so a sourced app env
  can't fail the defaults assertion
- **/ anchor the cache patterns in .dockerignore (nested ones were shipping)
- composition root is composition.py now, not main.py; backfill's docstring
  claimed the demo passes a fixed `end`, which the seed source ignores
@vlobus
vlobus merged commit 7deac05 into main Aug 17, 2026
1 check passed
@vlobus
vlobus deleted the t10-docker branch August 20, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant