Skip to content

retention: opt-in pruning for inactive_nodes, node_changes, ping_triggers and soft-deleted observers #329

Closed

Description

@dborup

Some tables are never pruned today, so an operator cannot honour a fixed retention period for node and observer data:

Table Today
inactive_nodes Nodes move here after nodeDays; there is no DELETE.
node_changes Permanent history of name/role/position changes.
ping_triggers Holds sender names. Not covered by the transmission prune batches (maintenance.go); the only DELETE is a hash-migrate dedupe.
observers Rows past observerDays are only soft-deleted.

Proposal

  • Add opt-in retention knobs in retention, for example inactiveNodeDays, nodeChangeDays, pingTriggerDays and observerPurgeDays. When unset, behaviour stays as today (upstream-compatible).
  • Run the deletes in the ingestor's existing daily prune and at startup, in batches inside WriterTx, like pruneBatches.
  • ping_triggers should follow the transmission prune (delete triggers whose transmission is gone, or apply its own age).
  • Log the counts, the same way the existing [prune] lines do.

Acceptance

  • For each knob, a test that is red on master and green after: old rows are removed, recent rows are kept, and an unset knob changes nothing.
  • No regression in the Ping Scores history: entries already marked data_pruned must still render.
  • Every write stays in cmd/ingestor.
  • Document the knobs in config.example.json.

Activity

  1. dborup commented on Oct 7, 2026

    @dborup
    OwnerAuthor

    Implementation note: #34 is a port of upstream#1886 (opt-in observerPurgeDays hard-delete). Use it as the basis for the observers part, rebased onto current master. Close #34 when this lands.

  2. dborup commented on Oct 7, 2026

    @dborup
    OwnerAuthor

    Scope change (operator decision 2026-10-07): ping_triggers is excluded from this issue. It must keep its rows indefinitely, because Ping Scores all-time records join it with the history sidecar (see #349 / #241). Only inactive_nodes, node_changes and soft-deleted observers get opt-in retention here.

  3. added a commit that references this issue on Oct 7, 2026
  4. dborup commented on Oct 7, 2026

    @dborup
    OwnerAuthor

    Fixed by #350 (merged as 987c220d3ffbb6c89cf02f325a22ad628ffa9b14).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions