Skip to content

feat(portal): delete former clients' records after three years - #616

Merged
devantler merged 6 commits into
mainfrom
claude/portal-retention
Nov 15, 2026
Merged

devantler merged 6 commits into
mainfrom
claude/portal-retention

Conversation

@devantler

@devantler devantler commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

What

The privacy notice (#614) keeps a former client's portal records for 3 years after their last portal membership ended. That is the general Danish limitation period (forældelsesloven §3), and the maintainer confirmed it. This PR adds the deletion that keeps that promise. It is the second of the two gaps that blocked switching the portal on.

  • identity.PurgeExpiredClients. A client is due when:

    • all their memberships were client memberships, so operator workspaces never are;
    • none is still active, and no invitation for them is pending;
    • no order still bills: monthly billing has ended or been withdrawn, and every card subscription has a confirmed end in the past, so the purge never strands a subscription the portal could no longer stop;
    • their last membership change lies more than identity.RetentionYears (3) years back. A member's move to another client is recorded under the new client, so the event that follows each of the client's events counts too. The period is one constant, so another period is a one-line change.

    Each due client goes in one transaction. It first locks invitations and memberships (SHARE ROW EXCLUSIVE, invitations first, the order redemption writes them) and checks the client again. An invitation redeemed during the purge therefore either shows in the recheck or waits and finds no invitation. Then it deletes, in dependency order:

    • billing: orders, charges, card checkouts, ends, reversals, withdrawals, completions and settlements;
    • email: the outbox and deliveries;
    • work approvals, legacy invoice and credit records, proposals, visibility, reviews and acceptances;
    • requests, contacts, Numina contact links, invitations and memberships;
    • the whole history of each of the client's memberships, one chain of events linked by previous_incarnation across renames and moves, whose last change is also past the period. A chain stays while it names another client that still has records (so that client can still be found), or while a remaining work approval cites it.
  • --retention-purge --migration-database-file <file>, or --migration-database-projection <file> for a Kubernetes Secret projection as feat(portal): let a Kubernetes Job run the schema migration #618 added for the migration. An exclusive one-shot mode of the executable, with exactly one connection input. It runs as the migration role and first checks that the role owns the complete schema. The web runtime still has no DELETE on this history, and every other option is refused. Like --migrate, it discards ambient PG* settings, has bounded deadlines, and prints only how many former clients lost records, or a generic failure.

  • Docs. server/README.md has a new Retention purge job section. MIGRATIONS.md records this purge as the one pruning of membership history. server/AGENTS.md notes the job.

Invoices and other accounting records live in Numina and Stripe for 5 years, and backups age out after 30 days.

Rollout

The daily CronJob is #620, in the package beside the migration Job (#619). It runs this command through --migration-database-projection and must merge after this PR.

Codex review

Each finding has a test that fails without its fix. The three findings at 43ceefd are fixed in 82f17c5:

  • P1, recheck under lock. TestRetentionRechecksUnderLock holds an uncommitted membership insert, sees the purge wait on the lock, commits, and the purge deletes nothing. A pending invitation keeps a former client too.
  • P2, moves to another client. TestRetentionCountsMoveToAnotherClient: a client whose member moved away today is kept; three years after the move it goes, and the moved member's membership and history stay.
  • P2, history cited elsewhere. TestRetentionKeepsHistoryOtherApprovalsCite: a former operator who approved another client's work becomes a former client. The purge succeeds and keeps the cited history and the approval.

The two findings at 82f17c5 are fixed in 71c241e:

  • P2, evidence another client needs. TestRetentionKeepsEvidenceOtherClientsNeed: a member moves from A to B and the membership is deleted. Both clients are purged in one run and the shared history goes; before, purging A removed the history B needed to be found.
  • P2, renamed identities. TestRetentionKeepsHistoryARenamedIdentityContinues: the history a renamed membership continues from stays, so the self-referencing foreign key holds.

The finding at 71c241e is fixed in b0decb9:

  • P2, rename and return. The purge now deletes whole event chains rather than identities, so mutually dependent identities need no special case. TestRetentionDeletesARenamedMembershipsWholeHistory renames a member away and back, and the chain goes as a whole.

The two findings at f7fecc0 are fixed in 440ba9e:

  • P1, live subscriptions. TestRetentionWaitsForLiveSubscriptions keeps a former client whose card subscription has no confirmed end and purges it once the end has passed.
  • P2, the purge restarting the clock. A chain's age ignores the deletion of an already revoked membership. TestRetentionDeletionDoesNotRestartTheClock purges two clients that share a moved member in one run, and the shared history goes.

Tests run locally (PostgreSQL 17, Go 1.27.4)

  • TestRetentionPurgesFormerClientsAfterThreeYears seeds every portal relation (the fixture-reset seed, now a shared helper) and checks:
    • a client who left today, or less than three years ago, is kept;
    • once three years have passed, every record of the former client goes, across 28 tables;
    • a current client, the operator workspace and Stripe event IDs stay;
    • a second run finds nothing.
  • TestActualExecutableRetentionPurge runs the real binary:
    • a schema not yet migrated fails;
    • a former client is purged and a current one kept;
    • extra or repeated flags, both connection inputs, --migrate, --runtime-role and a relative file are refused.
  • TestActualMigrationAcceptsKubeletProjection also runs the purge through a kubelet-shaped Secret projection, which the strict file option refuses.
  • The full Go suite passes: all four cohorts, contactintegration and contacttrialintegration.
  • golangci-lint, vet and gofmt are clean, and so are spelling and jscpd.

Part of #21

🤖 Generated with Claude Code

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review for Retention purge: which clients are due (all client memberships, none active, last change older than 3 years), the dependency-ordered per-client deletion, the membership history of identities with no membership left, and the exclusive --retention-purge command running as the migration role

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T23:25:36.971272Z 440ba9e Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 43ceefd569

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread server/internal/identity/retention.go Resolved
Comment thread server/internal/identity/retention.go Resolved
Comment thread server/internal/identity/retention.go Resolved
Base automatically changed from claude/portal-email to main November 15, 2026 21:59
@devantler
devantler force-pushed the claude/portal-retention branch from 43ceefd to 82f17c5 Compare November 15, 2026 22:10
devantler added a commit that referenced this pull request Nov 15, 2026
Codex review on #616:
- Lock invitations and memberships, then recheck the client inside
  the purge transaction, so an invitation redeemed after the candidate
  query keeps the client. A pending invitation also keeps it.
- Date a client's last membership change by the event that follows
  each of its events, so a member's move to another client counts.
- Delete an identity's membership history only when its last change
  is also past the period, it holds no other client's membership, and
  no remaining work approval cites it.
- Report how many clients lost records.

The CronJob ships in the package beside the portal migration Job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review for Re-review of the retention purge after the three findings at 43ceefd: per-client recheck under a SHARE ROW EXCLUSIVE lock on invitations and memberships, successor events dating a client's last change, and history deletion only for identities whose whole history is past the period, hold no other membership and are cited by no remaining work approval.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 82f17c51a6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread server/internal/identity/retention.go Resolved
Comment thread server/internal/identity/retention.go Resolved
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

✅MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Max errors Warnings Elapsed time
✅ COPYPASTE jscpd yes no no 1.94s
✅ CSS stylelint 1 0 0 0 2.5s
✅ GO golangci-lint yes yes no no 183.6s
✅ GO revive 183 0 0 1.84s
✅ HTML djlint 5 0 0 2.41s
✅ HTML htmlhint 5 0 0 0.32s
✅ JSON jsonlint 2 0 0 0.27s
✅ JSON prettier 2 0 0 0 0.32s
✅ JSON v8r 2 0 0 1.66s
✅ MARKDOWN markdownlint 17 0 0 0 2.97s
✅ MARKDOWN markdown-table-formatter 17 0 0 0 0.24s
✅ REPOSITORY betterleaks yes no no 1.31s
✅ REPOSITORY checkov yes no no 26.02s
✅ REPOSITORY git_diff yes no no 0.02s
✅ REPOSITORY grype yes no no 255.5s
✅ REPOSITORY osv-scanner yes no no 220.85s
✅ REPOSITORY secretlint yes no no 1.79s
✅ REPOSITORY syft yes no no 4.02s
✅ REPOSITORY trivy yes no no 15.26s
✅ REPOSITORY trivy-sbom yes no no 0.48s
✅ REPOSITORY trufflehog yes no no 5.88s
✅ SPELL cspell 210 0 0 8.5s
✅ SPELL lychee 24 0 0 194.65s

See detailed reports in MegaLinter artifacts

Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining FLAVOR_SUGGESTIONS: false)

  • Documentation: Custom Flavors
  • Command: npx mega-linter-runner@10.2.0 --custom-flavor-setup --custom-flavor-linters COPYPASTE_JSCPD,CSS_STYLELINT,GO_GOLANGCI_LINT,GO_REVIVE,HTML_DJLINT,HTML_HTMLHINT,JSON_JSONLINT,JSON_V8R,JSON_PRETTIER,MARKDOWN_MARKDOWNLINT,MARKDOWN_MARKDOWN_TABLE_FORMATTER,REPOSITORY_CHECKOV,REPOSITORY_GIT_DIFF,REPOSITORY_BETTERLEAKS,REPOSITORY_GRYPE,REPOSITORY_OSV_SCANNER,REPOSITORY_SECRETLINT,REPOSITORY_SYFT,REPOSITORY_TRIVY,REPOSITORY_TRIVY_SBOM,REPOSITORY_TRUFFLEHOG,SPELL_CSPELL,SPELL_LYCHEE

MegaLinter is provided by OX Security
Show us your support by starring ⭐ the repository

@github-code-quality

github-code-quality Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Code coverage overview

Languages: Go

Go / code-coverage/go

main claude/portal-retent... 440ba9e +/-
— 78% —

Line coverage data for the main branch is not yet available.

View coverage of the 10 most covered files.
File main claude/portal-retent... 440ba9e +/-
internal/identity/browser.go — 90% —
internal/webapp/app.go — 89% —
internal/identity/contact.go — 89% —
internal/custom...ity/material.go — 85% —
internal/identity/request.go — 82% —
internal/identi...ork_approval.go — 79% —
internal/identi...l_acceptance.go — 77% —
internal/identity/card.go — 77% —
cmd/business-site/main.go — 73% —
internal/identity/billing.go — 69% —

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.


Updated November 15, 2026 23:28 UTC

devantler added a commit that referenced this pull request Nov 15, 2026
Codex review on #616 at 82f17c5:
- Keep an identity's history while any other client its events name
  still has records, so purging one former client never removes the
  evidence that makes another due. That also covers a membership the
  identity still holds elsewhere.
- Keep it while another identity's history continues from it (a
  renamed membership), so the self-referencing foreign key holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review for Re-review after the two findings at 82f17c5: identity history now stays while another client its events name still has records, or while another identity's history continues from it via previous_incarnation.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 71c241ee4c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread server/internal/identity/retention.go Resolved
devantler and others added 5 commits November 15, 2026 23:53
The privacy notice (#614) keeps a former client's portal records for 3
years after their last portal membership ended, the general Danish
limitation period, and the maintainer confirmed that period.

- identity.PurgeExpiredClients deletes, one client per transaction, the
  records of every client whose memberships were all client memberships,
  none still active, last changed more than RetentionYears (3) ago:
  billing, email, work approvals, proposals, requests, contacts,
  invitations and memberships, and the membership history of identities
  with no membership left.
- The executable's exclusive --retention-purge --migration-database-file
  mode runs it as the migration role, which owns the complete schema;
  the web runtime still has no DELETE on this history. It prints only
  how many former clients it purged. The platform runs it daily beside
  the migration job.

Invoices and accounting stay 5 years in Numina and Stripe.

Part of #21

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex review on #616:
- Lock invitations and memberships, then recheck the client inside
  the purge transaction, so an invitation redeemed after the candidate
  query keeps the client. A pending invitation also keeps it.
- Date a client's last membership change by the event that follows
  each of its events, so a member's move to another client counts.
- Delete an identity's membership history only when its last change
  is also past the period, it holds no other client's membership, and
  no remaining work approval cites it.
- Report how many clients lost records.

The CronJob ships in the package beside the portal migration Job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex review on #616 at 82f17c5:
- Keep an identity's history while any other client its events name
  still has records, so purging one former client never removes the
  evidence that makes another due. That also covers a membership the
  identity still holds elsewhere.
- Keep it while another identity's history continues from it (a
  renamed membership), so the self-referencing foreign key holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex review on #616 at 71c241e: an identity renamed away and back made
two identities depend on each other, so neither was ever deleted.

A membership's history is one chain of events linked by
previous_incarnation, across renames and moves. The purge now decides
per chain and deletes whole chains, so no remaining event loses its
predecessor and renames need no special case. A chain still stays while
another client it names has records, or a remaining work approval
cites it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion

After #618, the migration Job reads its connection through
--migration-database-projection. The retention purge takes the same
option, through the shared openMigrationPool, so its CronJob can mount
the same Secret. Exactly one connection input is admitted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devantler
devantler force-pushed the claude/portal-retention branch from 71c241e to f7fecc0 Compare November 15, 2026 22:58
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review for Re-review after rebasing onto #618: membership history is now deleted per event chain (recursive CTE over previous_incarnation), and --retention-purge also takes --migration-database-projection through openMigrationPool.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7fecc0d2c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread server/internal/identity/retention.go Resolved
Comment thread server/internal/identity/retention.go Resolved
…etions out of the clock

Codex review on #616 at f7fecc0:
- A former client whose card subscription has no confirmed end, or
  whose monthly billing has not ended or been withdrawn, is not due:
  purging it would leave a subscription the portal could no longer stop.
- Deleting a membership that was already revoked records an event
  today. A chain's age no longer counts such deletions, so a chain that
  had to wait for another client's purge is not kept three more years.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review for Re-review after the two findings at f7fecc0: a client with billing that has not ended (or a card subscription without a confirmed past end) is not due, and a chain's age ignores deletion events of already revoked memberships.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 440ba9ef84

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@devantler
devantler marked this pull request as ready for review November 15, 2026 23:38
@devantler
devantler requested a review from a team as a code owner November 15, 2026 23:38
@devantler
devantler merged commit b210677 into main Nov 15, 2026
40 of 53 checks passed
@devantler
devantler deleted the claude/portal-retention branch November 15, 2026 23:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant