Problem
retention.packetDays prunes all transmissions older than N days. Channel messages (GRP_TXT, payload_type = 5) are transmissions too, so a channel's history disappears after N days.
The ingestor runs this prune at every startup ([prune] startup pruned N transmissions older than N days) and then once a day. To users it therefore looks as if a restart deletes channel messages. In fact a restart only removes, all at once, everything that has aged past packetDays since the last prune.
Deployments that keep a short packetDays to bound database size, for example 14 days, lose chat history that users expect to keep. Channel messages are a small fraction of all traffic, so keeping them longer is cheap.
Proposal
Add retention.channelDays (ingestor):
- When it is set and greater than
packetDays, the regular packet prune skips channel messages (payload_type = 5) younger than channelDays. A separate prune removes channel messages older than channelDays.
- When it is
0 or unset, behaviour is unchanged: channel messages follow packetDays.
- Observations of a kept channel message are kept with it, so the region filter and observer counts stay consistent.
- Both prunes log separately, for example
[prune] startup pruned N channel messages older than M days.
Acceptance
Problem
retention.packetDaysprunes all transmissions older than N days. Channel messages (GRP_TXT,payload_type = 5) are transmissions too, so a channel's history disappears after N days.The ingestor runs this prune at every startup (
[prune] startup pruned N transmissions older than N days) and then once a day. To users it therefore looks as if a restart deletes channel messages. In fact a restart only removes, all at once, everything that has aged pastpacketDayssince the last prune.Deployments that keep a short
packetDaysto bound database size, for example 14 days, lose chat history that users expect to keep. Channel messages are a small fraction of all traffic, so keeping them longer is cheap.Proposal
Add
retention.channelDays(ingestor):packetDays, the regular packet prune skips channel messages (payload_type = 5) younger thanchannelDays. A separate prune removes channel messages older thanchannelDays.0or unset, behaviour is unchanged: channel messages followpacketDays.[prune] startup pruned N channel messages older than M days.Acceptance
packetDays = 14andchannelDays = 90, a 30-day-old channel message and its observations survive the startup prune. A 30-day-old advert or other packet is pruned. A 100-day-old channel message is pruned.channelDaysunset or0, behaviour is identical to today.GET /api/channelsand/api/channels/{hash}/messages. These read SQL with full history, so no server change should be needed. Verify it.EXPLAIN QUERY PLAN. Measure it on a realistic fixture.cmd/ingestor. No newmap[string]interface{}.config.example.jsondescribe the option. Note that the server's in-memory packet store window (packetStore.retentionHours) is independent of it.