diff --git a/README.md b/README.md index a9132ad..684b885 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,20 @@ AI-enhanced dividend investing. Track your portfolio, set target prices on your radar, plan purchases, and get intelligent insights — all in one place. -Built on Elixir + Phoenix LiveView. Consolidates the previous multi-stack ecosystem (Rails dividend-portfolio, Phoenix pulse, Go logo-service, NestJS trends) into a single monolith. +Built on Elixir + Phoenix LiveView. Consolidates a previous multi-stack ecosystem (Rails dividend-portfolio, Phoenix pulse, Go logo-service, NestJS trends) into a single monolith. + +## Features + +- **Portfolio** — holdings with live prices and multi-currency valuation (yield-on-cost + current yield). +- **Radar** — a watchlist with per-stock buy/sell target prices. +- **Buy plan** — ranks holdings and radar stocks by a composite interest score, with tunable per-factor weights and one-click checkout into holdings. +- **Dividends** — a cash ledger with IBKR / MyInvestor statement import, per-currency charts (12-month + full-history), a payment calendar, and upcoming ex-dividend tracking. +- **Path to Freedom** — a FIRE projection (three capital pools, inflation, adaptive drawdown) with live sensitivity levers. +- **Community** — opt-in public portfolio (`/p/:slug`) and radar (`/r/:slug`) pages with share images, plus a `/community` dashboard and trending. +- **AI** — portfolio/radar insights and per-stock summaries (Gemini, provider-agnostic), with a per-user daily quota. +- **Telegram** — an account-linked bot (queries + a daily digest of ex-divs, dividends, and target hits). +- **Seven languages** — en, es, ca, pt, de, fr, it. +- **Demo mode** — "Try a sample portfolio" seeds a real, ephemeral account so every feature is visible without signing up. ## Quickstart @@ -27,23 +40,26 @@ docker compose up -d # 3. Install dependencies and set up the database mix setup -# 4. (Optional, for login to work) Set Google OAuth credentials +# 4. (Optional) Configure integrations cp .env.example .env -# fill in GOOGLE_OAUTH_CLIENT_ID and GOOGLE_OAUTH_CLIENT_SECRET, then export them +# - GOOGLE_OAUTH_CLIENT_ID / _SECRET → enable Google sign-in +# - GEMINI_API_KEY → enable AI insights/summaries +# (all optional in dev; magic-link emails land in the local mailbox) # 5. Start the server mix phx.server ``` -Open . +Open . Magic-link sign-in works out of the box in dev — the email shows up at . ## Project structure ``` lib/ quantic/ # contexts (bounded contexts a la DDD/hex) - accounts/ # users, sessions, OAuth - ... # portfolio, radar, market_data, community, events, logos, ai + accounts/ # users, sessions, OAuth, magic links + ... # portfolio, radar, market_data, community, + ... # events, logos, ai, content, demo quantic_web/ # web layer controllers/ # JSON / OAuth callbacks live/ # LiveViews (the main UI) diff --git a/docs/architecture.md b/docs/architecture.md index a28d450..3f7eae9 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,6 +1,6 @@ # Architecture -> **Status**: in active construction. The pre-consolidation ecosystem (Rails `dividend-portfolio`, Phoenix `pulse`, Go `logo-service`, NestJS `trends`) is being folded into this single Phoenix monolith one bounded context at a time. See [`docs/migration/plan.md`](migration/plan.md) for the strategy and [`docs/migration/status.md`](migration/status.md) for current progress. +> **Status**: feature-complete and ready to cut over. The pre-consolidation ecosystem (Rails `dividend-portfolio`, Phoenix `pulse`, Go `logo-service`, NestJS `trends`) has been folded into this single Phoenix monolith — every bounded context is ported and live on beta. The only remaining work is the production cutover (an ops window, not code): see the [cutover runbook](deploy.md#production-cutover-step-9). Migration history lives in [`docs/migration/plan.md`](migration/plan.md) (strategy) and [`docs/migration/status.md`](migration/status.md) (per-slice log). ## High-level shape @@ -9,18 +9,24 @@ Quantic is a single Phoenix 1.8 application organized by **bounded contexts** in ``` lib/ quantic/ # core: contexts + their domain/use cases - accounts/ # users (+ preferred_currency), sessions, OAuth, magic links (live) - portfolio/ # holdings + valuation + dividends ledger (live); yields, buy plans, projections (planned) - radar/ # watchlist + target prices (live); alerts (planned) - market_data/ # quotes (live + batched + cache + Oban refresh) + FX (live + 2-tier cache: ETS + Postgres) (live) - community/ # public surface (← pulse): /p/:slug, /r/:slug, /community + visit analytics (live) - events/ # time-series ingest + read API (← trends) (live) - logos/ # logo cache + serving + acquisition pipeline (live); LLM fallback (live) - ai/ # provider-agnostic AI (insights, summaries) + Telegram bot + daily digest (live) - content/ # founder social-post drafts (← legacy content_drafts): topic select → build → generate (live) + accounts/ # users (+ preferred_currency, admin, sharing), sessions, OAuth, magic links + portfolio/ # holdings + valuation + yields + dividends ledger/importers/charts + buy plan + freedom projection + radar/ # watchlist + target prices (target-hit alerts ride the Telegram digest) + market_data/ # quotes (batched + cache + Oban refresh) + FX (2-tier cache: ETS + Postgres) + stock enrichment + community/ # public surface (← pulse): /p/:slug, /r/:slug, /community + visit analytics + events/ # time-series ingest + read API (← trends) + logos/ # logo cache + serving + acquisition pipeline (GitHub → Wikidata → LLM) + ai/ # provider-agnostic AI (insights, summaries) + per-user quota + Telegram bot + daily digest + content/ # founder social-post drafts (← legacy content_drafts): topic select → build → generate + demo/ # ephemeral seeded demo users + nightly sweep quantic_web/ # web layer: LiveViews, controllers, layouts ``` +Everything above is **live on beta**; nothing is stubbed. The web layer +covers the full UI (Phoenix 1.8 LiveView): home (anon + dashboard), +portfolio, radar, buy plan, dividends, path-to-freedom, community, +settings, login, FAQ, and the admin dashboard + content-drafts surface. + Each context owns its Ecto schemas, use cases, and external adapters. **Cross-context calls go through the context module's public API** — nothing outside `Quantic.Portfolio` may reach into `Quantic.Portfolio.Holdings.Repo`. Inter-context async coordination uses `Phoenix.PubSub`. ## Stack @@ -30,7 +36,7 @@ Each context owns its Ecto schemas, use cases, and external adapters. **Cross-co | Language / runtime | Elixir 1.20, Erlang/OTP 27 | | Web framework | Phoenix 1.8 + LiveView 1.1 | | Persistence | PostgreSQL 17 + Ecto 3.13 | -| Background jobs | Oban 2.23 (Postgres-backed; `oban_jobs` schema v14). First consumer: the periodic quote refresher | +| Background jobs | Oban 2.23 (Postgres-backed; `oban_jobs` schema v14). Consumers: quote refresher, logo acquisition, stock enrichment, Telegram daily digest, demo sweep | | Real-time | `Phoenix.PubSub` + LiveView (server-pushed) | | Auth | OAuth-only (Google for v1) via Ueberauth + session tokens | | Asset pipeline | esbuild + Tailwind v4 + DaisyUI | @@ -42,12 +48,12 @@ Every third-party vendor sits behind an Elixir **behaviour** (port). Concrete ad | Category | Behaviour | Today | |---|---|---| -| AI providers | `Quantic.AI.Provider` | Gemini, Anthropic, OpenAI (planned) | -| Stock quotes / fundamentals | `Quantic.MarketData.Provider` (`get_quote/1`, `get_quotes/1`) | Yahoo Finance (via `yahoo_finance_ex` hex pkg); AlphaVantage (planned) | -| FX rates | `Quantic.MarketData.Provider` (`get_fx_rate/2`) | Yahoo Finance (via `yahoo_finance_ex`); AlphaVantage (planned) | -| Logo resolvers | `Quantic.Logos.Resolver` | GitHub ticker-icons + Wikidata (live); LLM fallback (with Step 7) | +| AI providers | `Quantic.AI.Provider` | Gemini (live); Anthropic / OpenAI drop in via two callbacks | +| Stock quotes / fundamentals | `Quantic.MarketData.Provider` (`get_quote/1`, `get_quotes/1`) | Yahoo Finance (via `yahoo_finance_ex` hex pkg, live); AlphaVantage (live fallback — flip `:provider`) | +| FX rates | `Quantic.MarketData.Provider` (`get_fx_rate/2`) | Yahoo Finance (via `yahoo_finance_ex`, live); AlphaVantage (live fallback) | +| Logo resolvers | `Quantic.Logos.Resolver` | GitHub ticker-icons → Wikidata → LLM (all live) | | Email | Swoosh adapter | Resend in prod, Swoosh local adapter in dev | -| Telegram | `Quantic.AI.Telegram` (wraps ExGram) | Telegram Bot API (planned) | +| Telegram | `Quantic.AI.Telegram` (wraps ExGram) | Telegram Bot API (live, ships dormant — webhook registered at cutover) | AI tool specs stay provider-neutral. Provider-specific concepts never leak into context APIs or domain models. @@ -101,7 +107,49 @@ For each holding the value is `shares × quote.price × fx(quote.currency → pr ## Logos -`Quantic.Logos` (← logo-service, Go → Elixir) serves ticker logos from a two-part cache: metadata rows in Postgres (provenance, per-size flags, status) and PNG bytes on disk in the legacy `{SYMBOL}/{size}.png` layout (five square sizes, 16–256px). `GET /logos/:symbol?size=` serves cache-only with day-long cache headers; a miss for a known stock enqueues the Oban `AcquireWorker` (unique per symbol), which runs the resolver chain — GitHub ticker-icons → Wikidata (P154 → Commons), LLM fallback arriving with Step 7 — then validates (magic bytes; SVG/HTML rejected), resizes via `image`/vix (precompiled libvips), and stores. Chain exhaustion negative-caches as `not_found`. The `<.stock_logo>` component renders logos everywhere with an initials fallback; the 404 that shows the fallback is the same signal that triggers acquisition. Cutover: rsync the legacy image volume + `mix quantic.import_logo_service`. +`Quantic.Logos` (← logo-service, Go → Elixir) serves ticker logos from a two-part cache: metadata rows in Postgres (provenance, per-size flags, status) and PNG bytes on disk in the legacy `{SYMBOL}/{size}.png` layout (five square sizes, 16–256px). `GET /logos/:symbol?size=` serves cache-only with day-long cache headers; a miss for a known stock enqueues the Oban `AcquireWorker` (unique per symbol), which runs the resolver chain — GitHub ticker-icons → Wikidata (P154 → Commons) → LLM (Gemini, grounded) — then validates (magic bytes; SVG/HTML rejected), resizes via `image`/vix (precompiled libvips), and stores. Chain exhaustion negative-caches as `not_found`. The `<.stock_logo>` component renders logos everywhere with an initials fallback; the 404 that shows the fallback is the same signal that triggers acquisition. Cutover: rsync the legacy image volume + `mix quantic.import_logo_service`. + +## Portfolio: dividends, buy plan, freedom + +Beyond holdings + valuation, `Quantic.Portfolio` owns the dividend-investing core: + +- **Dividends ledger** (`Portfolio.Dividend`, ← legacy 1:1) — gross/withholding/net per payment, provenance `manual | ibkr | myinvestor`, a partial dedup unique index for broker imports. All rows are user-deletable (re-import restores broker rows via the natural-key upsert). +- **Broker importers** (`Portfolio.DividendImport.{Ibkr,Myinvestor}` + dispatcher) — byte-level format sniffing (no extension trust; MyInvestor's `.xls` is HTML), ISIN → symbol → provider → unmatched resolution with a manual-match UI, non-destructive preview → apply. The detected broker is stored as `source` (never defaulted — unlike legacy). +- **Charts** (`dividend_chart_data/2`) — per-currency monthly buckets, 12-month forward projections from the inferred payment schedule, rendered server-side (bars for the year view, an axised line chart for full history; no JS charting lib). +- **Buy plan** — ranks holdings ∪ radar by a composite interest score (quality rating, % below target, underweight, averaging-down, 52-week position, missing-months, underowned-sector), with per-user importance weights; checkout merges picks into holdings at weighted-average cost. +- **Path to Freedom** (`Portfolio.FreedomProjection`, ← legacy `motivation.ts`) — three capital pools, accumulation + adaptive-SWR distribution, inflation throughout; the `/freedom` page is a verdict-led narrative with sensitivity levers that re-run the projection. + +Stock **enrichment** (sector/industry + inferred dividend payment schedule) runs as a one-shot Oban job (`MarketData.EnrichStockWorker`) when an un-enriched stock is first upserted, so adding a holding never blocks on the extra provider round-trips. + +## AI + +`Quantic.AI` is a provider-agnostic seam over LLM calls. The `Quantic.AI.Provider` behaviour has just two generic callbacks — `chat/2` (with an internally-walked tool loop) and `generate_json/3` — so adding a provider is two functions, not a per-feature re-port. The Gemini adapter (Req, `Req.Test`-stubbed) is the live implementation. + +Every scoped call (`chat_as/3`, `generate_json_as/4`) enforces a **per-user daily quota** (`ai_requests` table, 3/day UTC, config-overridable) recorded *before* the call (a failed LLM call still costs money), with one ordering invariant pinned by test: the **cache probe runs before the quota check**, so a rate-limited user still sees cached results. **Admins bypass the quota** (legacy `AiRateLimiter` parity). System-initiated calls (`generate_json/3`, logo LLM resolver, content drafts) bypass the quota entirely. + +Features: portfolio/radar **insights** and per-stock **summaries** (`AI.Insights`, 6h input-digest cache), the **Telegram bot's brain**, and **content drafts** (below). All need `GEMINI_API_KEY` to light up; without it each AI feature degrades independently. + +## Telegram + +`Quantic.Telegram` ports legacy's bot (`Handler`/`Nlu`/`Tools`/`Client`) + linking lifecycle (`user_telegram_links`, one-shot expiring codes → `/start `). Seven user-scoped tools close over the linked user's data (the privacy property the system prompt promises). A **daily digest** (Oban cron, 06:30 UTC) sends upcoming ex-divs, dividends received, and throttled radar target-hit alerts, in the user's profile locale. + +**Ships dormant**: the monolith never registers the webhook on boot (doing so on the shared prod bot token would detach legacy's webhook), and the notifier is config-gated off. The one explicit cutover step — `Telegram.Client.register_webhook/1` + `TELEGRAM_NOTIFICATIONS=true` — attaches the bot to the monolith. Same bot token everywhere; verified by simulated-update tests since beta can't exercise it live. + +## Content drafts (founder tool) + +`Quantic.Content` (← legacy `ContentDrafts`) generates social posts for the brand's X/LinkedIn accounts, admin-only at `/admin/content-drafts`. Pipeline: `TopicSelector` (weighted auto-pick across stock-of-the-day / dividend-calendar / community-pulse with per-category repeat windows, plus one-off feature announcements) → `TopicDataBuilder` (assembles inputs from the app's own contexts) → `ContentGenerator` (`AI.generate_json/3` with brand voice rules, char-limit truncation, and a privacy guard that rejects any draft leaking an email / slug / profile URL). + +## Admin & founder tooling + +`users.admin` (boolean, never cast from user input — set by the legacy import or `mix quantic.set_admin EMAIL [--revoke]`) gates a `live_session :admin` via `UserAuth.on_mount(:ensure_admin)`; non-admins are bounced to `/` with no hint the area exists. `Quantic.Admin.dashboard_stats/0` aggregates across every context (users, catalog, portfolios, dividends, community, activity trend, AI usage, Telegram) with demo users excluded. `/admin` also offers the users table with delete and a "refresh all stocks" action. + +## Demo mode + +`Quantic.Demo` — "Try a sample portfolio" (`POST /demo`) creates a **real ephemeral user** seeded with curated multi-currency data and signs the visitor in, so every page renders through the real templates (no separate demo UI to maintain). Writable per-visitor, isolated, excluded from community aggregates and sharing, and swept after 24h by a nightly Oban job. + +## Internationalization + +Seven locales (en, es, ca, pt, de, fr, it) via Gettext. `QuanticWeb.Locale` resolves the language in one place — session ← header switch → signed-in user's preference → en — as both a plug and a LiveView `on_mount` hook. Adding a locale is one `.po` tree + two lines in `Locale`; untranslated strings fall back to English. Translations are community-improvable (plain `.po` files); the README documents the contribution path. ## Authentication (current) @@ -120,7 +168,7 @@ Sessions are persisted as opaque random tokens in the `users_tokens` table, inde - `QuanticWeb.UserAuth` provides the `fetch_current_scope_for_user` plug, `log_in_user/3`, `log_out_user/1`, and the `:mount_current_scope` / `:ensure_authenticated` LiveView `on_mount` hooks. - `QuanticWeb.AuthController` handles `/auth/google` + callback (Ueberauth), `/auth/magic_link/:token` (magic-link consume), and `DELETE /auth/log_out`. - `QuanticWeb.LoginLive` is the `/login` page (email form → "check your inbox" state). -- `QuanticWeb.SettingsLive` is the `/settings` page (authenticated). Backed by `Accounts.update_user_preferences/2`, which uses `User.preferences_changeset/2` — scoped strictly to user-editable fields (currently `preferred_currency`) so identity columns are unreachable from the settings form path. +- `QuanticWeb.SettingsLive` is the `/settings` page (authenticated): preferred currency + display language, community sharing (public link + per-surface toggles), and the Telegram connect/notifications card (shown only when the bot is configured). Each card has its own narrowly-scoped changeset (`User.preferences_changeset/2`, `sharing_changeset/2`) that casts only its user-editable fields, so identity columns are unreachable from any settings form path. **External credentials & email infra**: @@ -132,7 +180,11 @@ See `.env.example` for the full set. ## Deployment -Single Phoenix release built via `mix release`, packaged as a Docker image, deployed via Kamal v2 (the existing ecosystem deployment tool). One Postgres accessory holds all bounded contexts' data. +Single Phoenix release built via `mix release`, packaged as a Docker image (Debian both stages — vix's precompiled libvips is glibc-only; arm64 target for the ARM VPS), deployed via **Kamal v2** (pinned to 2.7.0 ecosystem-wide). One Postgres accessory (localhost-only) holds all bounded contexts' data; logo PNGs live on a mounted volume in the legacy `{SYMBOL}/{size}.png` layout. Migrations run automatically on boot (`bin/server` → `Release.migrate`), and releases have no Mix — one-off tasks (legacy imports, `set_admin`) run via `bin/quantic eval`. + +**Beta** (`beta.quantic.es`) runs the monolith on the shared ecosystem VPS alongside the legacy services; **merging to `main` deploys to beta** after CI passes (`.github/workflows/deploy.yml`, native ARM runner). Full runbook + the standing safety rules (never reboot the shared kamal-proxy, never touch legacy volumes) in [`docs/deploy.md`](deploy.md). + +**Cutover** to `quantic.es` is a **kamal-proxy host re-route, not a DNS change** — legacy and the monolith share one VPS/IP and the proxy routes by `Host` header, so the flip is atomic at the proxy and instantly reversible. See the [cutover runbook](deploy.md#production-cutover-step-9). ## What lives outside this repo