chore: establish production deployment and recovery operations

backend/Dockerfile

@@ -17,6 +17,18 @@ COPY backend ./backend
 
 RUN npm run build --workspace=@wc-telegram/backend
 
+FROM dependencies AS migration
+
+COPY backend ./backend
+COPY scripts/ops/run-migrations.sh /usr/local/bin/wctm-run-migrations
+
+RUN chmod 0555 /usr/local/bin/wctm-run-migrations
+
+WORKDIR /app
+USER node
+
+ENTRYPOINT ["/usr/local/bin/wctm-run-migrations"]
+
 FROM node:24.20.0-alpine3.24@sha256:e67514e5d0f6c46656005e1b693b2ec9d52e80b641307de684d4a015ba7a4eaf AS production
 
 ARG ALPINE_OPENSSL_VERSION=3.5.8-r0

docker-compose.yml

@@ -83,6 +83,24 @@ services:
     networks:
       - wc-telegram-net
 
+  migrate:
+    profiles: ['operations']
+    build:
+      context: .
+      dockerfile: backend/Dockerfile
+      target: migration
+    init: true
+    read_only: true
+    tmpfs:
+      - /tmp:size=16m,mode=1777
+    security_opt:
+      - no-new-privileges:true
+    depends_on:
+      postgres:
+        condition: service_healthy
+    networks:
+      - wc-telegram-net
+
   telegram-bot:
     build:
       context: .

docs/DECISIONS.md

@@ -1173,4 +1173,92 @@ the staged D-030 architecture or start P7.2+.
 
 ---
 
-Next decision number: D-031.
+## D-031
+
+Date
+
+2026-09-16
+
+Decision
+
+P7.2 separates production schema migration from normal application runtime.
+The backend continues to authenticate only as restricted `wctm_runtime` and
+never runs a migration during startup. An independently invokable, non-root,
+read-only Compose operations job contains the exact checked-out Prisma CLI,
+schema, and migrations and accepts a distinct existing owner/migration
+credential only for that invocation through a protected external file. Its
+entrypoint rejects `wctm_runtime`, Prisma Migrate deploy remains authoritative,
+and the ephemeral container is removed after success or failure.
+
+The supported deployment sequence is revision-locked and fail-fast: clean and
+synchronized repository, production configuration audit, verified P7.3 backup,
+image build, explicit migration before application cutover, backend/bot
+recreation, health/readiness, restricted-role verification, and bounded
+functional smoke. PostgreSQL and Redis are reconciled separately onto the
+reviewed immutable references only after backup and exact named-volume
+verification; their volumes are preserved and never deleted. Schema migrations
+are not assumed reversible, and database recovery delegates to P7.3.
+
+Reason
+
+Keeping migration authority out of long-lived runtime preserves P7.1 least
+privilege while providing one deterministic, auditable production path without
+a new deployment platform or automatic schema mutation.
+
+Status
+
+Accepted for repository implementation. Production acceptance remains pending
+A-owned live validation.
+
+---
+
+## D-032
+
+Date
+
+2026-09-16
+
+Decision
+
+P7.3 uses timestamped PostgreSQL custom-format backups with restrictive local
+permissions, atomic finalization, dump-list readability validation, SHA-256,
+and non-secret metadata. Retention is explicit, count-based, defaults to 14
+valid daily sets, never removes the newest valid set, and targets only the exact
+WCTM backup naming pattern. A provider-neutral executable hook supplies
+off-host transfer. The repository rclone implementation requires the remote
+dump content to match the locally generated SHA-256: it uses a compatible
+native remote SHA-256 when available, otherwise streams the remote dump through
+rclone and hashes it locally without replacing the local backup. Remote size
+remains a secondary check for all three transferred artifacts. Copy, metadata,
+size, content-integrity, or integrity-verification failures exit non-zero.
+
+Daily scheduling uses repository-controlled systemd service/timer templates
+whose external environment file contains paths and an optional remote name but
+no committed credential. Restore validation always targets a uniquely generated
+PostgreSQL 16.15 container and volume with no network or published port; it has
+no production URL/overwrite option, verifies checksum and dump readability,
+restores migration/schema/data state, performs configurable critical row-count
+checks, and removes only its own isolated resources.
+
+Database recovery also requires the matching repository revision and valid APP
+encryption key. A previous APP key is relevant only to a backup taken during a
+controlled historical rotation window. JWT, callback, internal service, bot,
+and database credentials restore their own trust boundaries and do not decrypt
+stored ciphertext. Initial targets are a 24-hour RPO and eight-hour RTO, not
+contractual guarantees or an SLA.
+
+Reason
+
+The P7.1 one-off restore proof did not provide repeatable backup, retention,
+off-host, scheduling, guarded restore, or disaster-recovery operations. This
+bounded repository mechanism supplies those controls without a backup catalog,
+new application table, public endpoint, provider lock-in, or multi-region HA.
+
+Status
+
+Accepted for repository implementation. Production scheduling, destination,
+restore exercise, and milestone acceptance remain pending A-owned validation.
+
+---
+
+Next decision number: D-033.

docs/HANDOFF.md

@@ -2,7 +2,7 @@
 
 **Generated:** 2026-07-19
 
-**Updated:** 2026-09-06
+**Updated:** 2026-09-16
 
 **Reason:** Maintaining implementation continuity
 
@@ -43,7 +43,7 @@ n8n is **NOT** part of the production architecture (D-008, prototype only).
 
 ---
 
-## 3. Architectural Decisions (D-001–D-030)
+## 3. Architectural Decisions (D-001–D-032)
 
 | ID    | Decision                                                                                                                                                                                                                   | Status   |
 | ----- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
@@ -77,8 +77,10 @@ n8n is **NOT** part of the production architecture (D-008, prototype only).
 | D-028 | One backend-authoritative Tenant ACTIVE/SUSPENDED lifecycle with derived expiry, operator-only mutation, explicit product gates, continued projections, terminal notification suppression, and no commercial billing scope | Accepted |
 | D-029 | Phase 7 production readiness precedes deferred Phase 6; P7.1 establishes the bounded repository security baseline while host changes, credential rotation, deployment, and P7.2+ remain separately controlled              | Accepted |
 | D-030 | P7.1 uses a temporary decrypt-only previous key, current-key-only writes, and a trusted-shell row-transactional rotation command for the exact five encrypted database fields                                              | Accepted |
+| D-031 | P7.2 separates restricted backend runtime from an ephemeral explicit Prisma migration identity and defines revision-locked fail-fast deployment plus volume-preserving immutable data-service reconciliation               | Accepted |
+| D-032 | P7.3 establishes atomic verified PostgreSQL backups, explicit retention, verified provider-neutral off-host copy, daily systemd scheduling, isolated guarded restore, DR dependencies, and target-only RPO/RTO             | Accepted |
 
-Next decision number: **D-031**, if a future task produces a genuine
+Next decision number: **D-033**, if a future task produces a genuine
 architectural or product decision.
 
 ---
@@ -1179,9 +1181,62 @@ Dependency and automated evidence:
   missing or excessive privileges across all 20 application tables, while the
   existing privileged owner remains available for Prisma migration execution.
 
-This closure branch changes documentation only. It performs no VPS, production,
-secret, database, deployment, schema, migration, application, product, P7.2+,
-or Phase 6 action.
+#### P7.2 — Production Migration & Deployment Path (repository complete; awaiting validation)
+
+- Implementation commit:
+  `91e16fd5933ffa4e6386d649f21f9e2592833ed5 chore(ops): establish production
+migration and deployment path`.
+- D-031 keeps `wctm_runtime` as the backend-only restricted identity and moves
+  Prisma tooling into an explicit ephemeral Compose operations target. The
+  wrapper reads one protected external migration URL, rejects `wctm_runtime`,
+  runs exact checked-out migrations, removes the container, and never exposes
+  that credential to normal backend runtime.
+- `deploy-production.sh` is revision-locked and fail-fast across clean/synced
+  Git, Compose, config audit, fresh verified P7.3 backup, image build, explicit
+  migration before cutover, backend/bot recreation, health/readiness, and
+  restricted-role verification. Backend startup remains `node dist/main.js`
+  with no automatic schema mutation or `db push`.
+- `reconcile-data-images.sh` requires the verified backup plus explicit current
+  PostgreSQL/Redis volume names, preserves those volumes, recreates only those
+  services on the P7.1 immutable references, verifies health, migration/key
+  continuity and exact image configuration, and leaves apps stopped for cutover.
+- Failure guidance distinguishes safe application-image restoration from
+  irreversible/forward-only schema effects and hands database recovery to P7.3.
+  Repository implementation is ready for B review; production acceptance is
+  pending A's combined runbook.
+
+#### P7.3 — Backup, Restore & Disaster Recovery (repository complete; awaiting validation)
+
+- Implementation commit:
+  `ddde3c705b249a29a131a3bfc96763f17b6e9cbc chore(ops): add production backup
+and recovery path`.
+- D-032 adds atomic mode-0600 custom-format dumps with UTC unique names,
+  `pg_restore --list`, SHA-256, and non-secret revision/migration/size/version
+  metadata. Explicit retention defaults to 14 valid sets, always protects the
+  newest, and ignores unrelated/incomplete/corrupt files.
+- A provider-neutral executable hook is supported; the supplied rclone hook
+  copies dump/checksum/metadata, verifies their remote sizes, and requires the
+  remote dump to match the local SHA-256 through a compatible native remote
+  hash or a locally hashed rclone stream. Failure is non-zero.
+  Daily Ubuntu systemd templates load paths/remote name from an external file,
+  expose failures through service status/journal, and contain no credential.
+- Restore accepts one explicit local backup only and creates a generated
+  PostgreSQL 16.15 container/volume with `--network none`. It verifies checksum,
+  dump readability, exact repository migration count, public schema and chosen
+  critical table rows, and removes only its own isolated resources.
+- DR covers application/container loss, failed deployment, database corruption,
+  and new-host recovery. Stored ciphertext requires the matching APP key; other
+  service/signing/database credentials do not decrypt it. Initial targets are a
+  24-hour RPO and eight-hour RTO, not SLA guarantees.
+- The executable combined A session is
+  `docs/operations/P7_2_P7_3_RUNBOOK.md`: preflight; fresh verified/off-host
+  backup; migration identity proof; immutable PostgreSQL/Redis reconciliation;
+  app deployment; bounded smoke; post-deployment backup; isolated restore; and
+  final permission/migration/backup/secret/health scans.
+
+This repository cycle performs no VPS, production, secret, live database,
+deployment, schema, migration, or product mutation. P7.2 and P7.3 are not
+production-complete. P7.4+, Phase 6, and product work remain unstarted.
 
 ## 5. Current Repository Structure
 
@@ -1190,25 +1245,26 @@ NestJS API, `telegram-bot/` for the grammY process, and `wp-content/plugins/` fo
 the lightweight connector. The larger `apps/`, `packages/`, and
 `infrastructure/` layout remains a planned target rather than current structure.
 
-Current branch: `docs/p7.1-production-security-closure`.
+Current branch: `chore/p7.2-p7.3-production-operations`.
 
 ---
 
 ## 6. Current Blockers
 
 No open Phase-5 or P7.1 blocker remains. Public launch remains gated by
-unstarted P7.2–P7.8. Historical provenance remains unresolved and assigned to
-P7.8. Generalized backup/restore/DR remains P7.3, monitoring P7.4, DATE-001
+P7.2/P7.3 production acceptance and unstarted P7.4–P7.8. Historical provenance
+remains unresolved and assigned to P7.8. Monitoring remains P7.4, DATE-001
 P7.5, network/runtime resilience P7.6, and AuditLog structural immutability
 P7.7.
 
 ---
 
 ## 7. Current Task
 
-P7.1 — Production Security Baseline is complete. P7.2 — Production Migration &
-Deployment Path is next but unstarted pending separate approval. Do not start
-P7.2+, Phase 6, or product expansion from this docs-only closure.
+P7.1 is complete. P7.2 and P7.3 repository implementation is complete and ready
+for B review; both await A-owned live validation and neither is production-
+complete. P7.4 is next only after both acceptances and remains unstarted. Do not
+start P7.4+, Phase 6, or product expansion.
 
 ### Last completed product milestone: M22
 

docs/MASTER-ROADMAP.md

@@ -294,17 +294,23 @@ commercial/product expansion.
 Milestones, in order:
 
 - P7.1 — Production Security Baseline — ✅ Complete
-- P7.2 — Production Migration & Deployment Path — ⬜ Next / unstarted
-- P7.3 — Backup, Restore & Disaster Recovery
+- P7.2 — Production Migration & Deployment Path — 🟡 Repository implemented;
+  awaiting B review and A-owned production validation
+- P7.3 — Backup, Restore & Disaster Recovery — 🟡 Repository implemented;
+  awaiting B review and A-owned production validation
 - P7.4 — Monitoring & Alerting
 - P7.5 — Data & Time Correctness
 - P7.6 — Network & Runtime Reliability
 - P7.7 — Audit & Operational Integrity
 - P7.8 — Final Launch Readiness Gate
 
-P7.1 is complete following accepted production validation. P7.2 is the next
-allowed Phase 7 milestone and remains unstarted. P7.3–P7.8 and Phase 6 remain
-unstarted; Phase 7 is not complete.
+P7.1 is complete following accepted production validation. P7.2 and P7.3 now
+have repository-side implementation for explicit least-privilege migration,
+deterministic deployment, immutable database/cache image reconciliation,
+verified backup/off-host retention/scheduling, isolated restore, and disaster
+recovery. Both remain open pending B review and A-owned live validation; neither
+is production-complete. P7.4 is next only after both are accepted. P7.4–P7.8
+and Phase 6 remain unstarted; Phase 7 is not complete.
 
 ---
 

docs/PHASE_BREAKDOWN.md

@@ -758,7 +758,40 @@ Node 24.20.0 now backs both application images; Node, PostgreSQL 16, and Redis 7
 use exact patch/distro tags plus immutable digests. The initial overprivileged
 production identity was replaced for backend runtime by the validated
 restricted role. Historical release-provenance findings remain without history
-rewrite and are still assigned to P7.8. P7.2 is next but unstarted; P7.3 through
+rewrite and are still assigned to P7.8.
+
+### P7.2 — Production Migration & Deployment Path 🟡 Repository Complete / Awaiting Validation
+
+- D-031 establishes an ephemeral, non-root Compose migration job containing the
+  exact checked-out Prisma migration state. A distinct protected migration URL
+  is supplied only for the operation; `wctm_runtime` is rejected and remains
+  the backend-only restricted identity with no migration-table or DDL access.
+- Revision-locked deployment fails before cutover on preflight, config audit,
+  backup, build, or migration failure. Backend startup performs no migration or
+  `db push`; health/readiness and restricted-role verification follow cutover.
+- Guarded PostgreSQL/Redis reconciliation requires a verified backup and exact
+  named-volume confirmation, preserves volumes, recreates only those services
+  on the reviewed immutable references, and verifies health/data continuity.
+- Repository implementation is ready for B review. Production validation and
+  A acceptance remain required; P7.2 is not production-complete.
+
+### P7.3 — Backup, Restore & Disaster Recovery 🟡 Repository Complete / Awaiting Validation
+
+- D-032 establishes atomic PostgreSQL custom-format backups, SHA-256 and
+  readability verification, non-secret metadata, explicit 14-valid-set default
+  retention, verified provider-neutral off-host transfer, and daily systemd
+  templates with no committed credentials.
+- Guarded restore always targets an isolated PostgreSQL 16.15 container/volume
+  with no network exposure, verifies repository migration state, schema and
+  configurable critical row counts, and cannot target production by default.
+- The DR runbook covers application/container, deployment, database, and total
+  VPS loss; distinguishes APP-key decryption dependency from other service
+  credentials; and sets target-only 24-hour RPO/eight-hour RTO objectives.
+- Repository implementation is ready for B review. A must choose the off-host
+  destination, install/enable scheduling, and execute the combined live backup,
+  deployment and isolated-restore runbook. P7.3 is not production-complete.
+
+P7.4 remains next only after P7.2/P7.3 production acceptance. P7.4 through
 P7.8 are unstarted. Phase 7 is not complete.
 
 ### Approved Phase 7 order