Operate useSend as the multi-brand email platform

[vladbisceanu]

Current state

The production service is live at send.growthpath.systems from this public fork's growthpath-mail branch. Amazon SES production access is approved, the bisceanu.com sending identity and callback path are working, and the first product integration is prepared but must be reconciled with current bisceanu.com main before cutover.

Before real subscriber traffic, finish the native Railway health gate and a verified Postgres recovery path. Reconcile this fork with upstream v1.9.7 separately so the local BullMQ connection fix is not carried twice. Then deploy and prove the bisceanu integration before migrating the remaining products and retiring Loops.

Spec

Outcome. Self-hosted useSend on Railway is the shared email platform for bisceanu.com, growthpath.systems, heralded.ai, and Bookling marketing. One internal team owns the brands, separated by sending domain and contact book. It supports transactional sends and campaigns; product workflows remain in each product backend.

Phase 1. Operate useSend with Postgres and Redis at send.growthpath.systems, using Amazon SES in eu-west-1. Harden production, migrate the bisceanu contact form completely, and prove transactional delivery, campaign unsubscribe behavior, bounce suppression, and idempotent retries.

Phase 2. Migrate each remaining product through an independently closable issue in its repository. Product email that has a deliberate separate abstraction remains outside this program.

Done when. Every in-scope Loops send and list runs on useSend, deliverability is verified per brand from received mail, recoverability and health gates are proven, and the obsolete Loops accounts are closed.

Appetite. One focused slice for production hardening, one for the bisceanu cutover, then roughly two to four days across the remaining product repositories.

Design

• Ownership: this fork owns the application and the complete service lifecycle: upstream reconciliation, pull requests, deployments, Railway, SES/SNS, sending-domain DNS, Postgres, Redis, backups, health, and this program.
• Consumer boundary: each product repository owns its useSend integration, templates, runtime variables, product deployment, workflow scheduling, and acceptance proof.
• Setup boundary: setup-works owns Builder OS, fleet setup, and shared credential plumbing only.
• Access model: one internal team contains first-party sending domains and contact books. External customers do not receive dashboard accounts; customer-facing tenancy is a separate decision.
• Hosting: Railway provides an isolated service lifecycle. send.growthpath.systems is the dashboard; mail.<sending-domain> remains available for SES custom MAIL FROM.
• Delivery gate: every PR to growthpath-mail runs one stable GitHub Actions gate covering type-checking, tests, and the production Docker build. Protect the branch with that exact Actions context, required PRs, admin enforcement, no approval count, and no force-push or deletion bypass. Railway deploys only the protected post-merge push.
• Readiness: startup fails closed on migration errors. Railway runs the migration command before deploy and waits for /api/health; that route uses bounded Postgres and Redis probes and reports the Railway commit SHA so provider metadata and live response identify the same artifact.
• Migration safety: review every Prisma migration for compatibility with the still-running deployment. Use expand/contract changes when both versions can overlap; a backup is recovery evidence, not permission for a destructive migration.
• Recovery: Railway PITR is the primary Postgres control. Wait for the initial base backup and healthy WAL coverage, then restore a named timestamp into a private sibling service. Attach no app or worker, validate only redacted schema and aggregate evidence, and delete the temporary service after proof.
• Security and licence: the public AGPL fork remains the complete corresponding source. Provider callbacks stay authenticated, external requests bounded, and suppression and unsubscribe behavior preserved.
• Data safety: pin application versions, review Prisma migrations, and verify a fresh recoverable Postgres backup before any production upgrade because startup applies migrations.
• Idempotency: Redis protects short retry windows; product backends retain durable uniqueness for sends that must never repeat.

[vladbisceanu]

This issue is the canonical public epic because GitHub does not permit transferring the prior issue from a private repository into this public fork. The existing four product/setup issues were reparented here without copying private comments. Service governance, production hardening, and upstream reconciliation are now native sub-issues #4, #6, and #7. The superseded private epic is closed.

[vladbisceanu]

Adversarial design critique for production hardening:

1. docker/start.sh does not fail fast. A failed Prisma migration can fall through to a running server, while the current health route always returns 200. Make startup fail closed and make readiness depend on bounded Postgres and Redis probes.
2. Startup migrations mutate the shared database while the old deployment still serves traffic. Treat migration compatibility as an explicit expand/contract gate and use Railway's pre-deploy command so migration failure stops the deployment before startup.
3. Recovery must be proven before the first hardening merge because every growthpath-mail merge deploys and startup applies migrations.
4. The required workflow currently tests source behavior but does not lint or build the production Docker path. Add those to the stable required job.
5. Bootstrap the check on a real PR, read back its exact GitHub Actions context, then protect growthpath-mail; target that base explicitly and avoid path filters that leave required checks pending.
6. Use a solo-safe protection policy: PRs required, zero impossible self-approval requirements, required Actions check, admin enforcement, no force pushes, and no deletion. Railway only consumes post-merge pushes and needs no bypass.
7. Prefer PITR because it restores into a sibling service without touching production. Wait for initial base-backup coverage, name the restore time, validate the fork, and record retention and metered cost.
8. The restored database contains credentials, addresses, message bodies, and webhook data. Keep it private, attach no app or worker, emit only redacted aggregate proof, and delete the temporary service after validation.
9. Bind live proof to one Railway deployment ID and commit SHA. Add the runtime commit SHA to the health response or verify it directly from the active deployment metadata.

These findings change the hardening design but not the overall architecture. I am folding them into the epic once, then proceeding without another critique loop.

[vladbisceanu]

Updated the epic design after the one adversarial pass. The changes make startup and readiness fail closed, define the exact solo-safe PR protection, require production-artifact coverage, bind health to the deployed SHA, and make PITR restore validation private and non-disruptive.

[vladbisceanu]

Disposition update: the repository-wide lint gate is not part of #6 because both current lint entry points fail on baseline debt. Issue #9 owns restoring it. The required production gate will use the checks that are green and meaningful now: type-checking, unit/tRPC/API/integration tests, and the production Docker build.

[vladbisceanu]

Production-service hardening and the v1.9.7 fork reconciliation are complete.

• Make the production fork the native service owner #4 / PR chore: make the fork the service owner #8: fork-native service ownership merged and deployed
• Harden GrowthPath Mail production before the first cutover #6 / PR feat: gate production on readiness #10: required always-running gate, protected growthpath-mail, bounded Postgres/Redis readiness, fail-fast migrations, Railway healthcheck, PITR, and an actual private-sibling recovery drill completed
• Reconcile the production fork with upstream useSend v1.9.7 #7 / PR chore: reconcile upstream v1.9.7 #11: main advanced to v1.9.7, the equivalent BullMQ patch reconciled without a duplicate tree delta, a locked pre-release backup created, and exact-SHA production deployment proven
• current live SHA: d1e6e5ad1d7550a11e768562174d66cd443f999d
• current health: HTTP 200 with that same SHA

The service foundation is ready for the consumer cutover slices described by this epic. #9 is the separately tracked pre-existing lint debt and does not weaken the required production gate.

[vladbisceanu]

The bisceanu.com consumer cutover is complete in vladbisceanu/bisceanu-com#67.

PR usesend#68 merged at exact SHA f22ef5f99b68c74d61d1b45878fa01160ffaaef9. Post-merge CI and both production deployment proofs passed. Live acceptance covered the contact-form messages, pending-to-confirmed double opt-in, a one-recipient campaign with confirmed SES delivery, browser unsubscribe, and no silent resubscribe. Vlad also ran an independent successful production submission.

The five retired Loops Production variables were removed, the same SHA was redeployed, and the post-cleanup form check passed. Vercel Production now retains only the three useSend/contact values required by the current application.