Skip to content

fix(chatwoot-adapter): bound seen-marker growth with a 3-day TTL prune (#620)#32

Merged
rmyndharis merged 3 commits into
mainfrom
feat/seen-marker-ttl-prune
Jul 3, 2026
Merged

fix(chatwoot-adapter): bound seen-marker growth with a 3-day TTL prune (#620)#32
rmyndharis merged 3 commits into
mainfrom
feat/seen-marker-ttl-prune

Conversation

@rmyndharis

Copy link
Copy Markdown
Owner

Summary

The Chatwoot adapter records one ctx.storage marker per relayed message, used to de-duplicate WhatsApp re-deliveries and to suppress the operator's own echoed sends. These markers were written once and never removed, so on a long-running instance the plugin's storage grew without bound, and the inbound-retry drain's periodic directory scan (storage.list, a full readdirSync plus a per-file key decode) grew steadily more expensive as the marker count climbed.

This change bounds the seen: keyspace with a time-based retention window.

Closes #620.

Changes

  • Timestamped markers. markSeen now stores { t: <ms> } instead of a bare 1; hasSeen is unchanged (it still checks presence only). This gives every marker an age.
  • pruneSeen(now, ttl). A new MappingStore primitive that streams the seen: keys and deletes any older than the retention window. A pre-existing marker (stored as a bare 1, with no timestamp) is adopted — re-stamped with the current time — rather than deleted, so the migration can never re-post a message as a duplicate; those adopted markers then age out one window later.
  • Hourly prune. Wired onto the existing 30-second inbound-retry timer (no second timer), guarded by its own in-flight flag and kept independent of the retry drain so a prune never blocks a retry. It runs regardless of Chatwoot config validity, since it neither reads config nor relays.
  • Retention window SEEN_TTL_MS is 3 days and the prune cadence is hourly — hardcoded constants, matching the existing retry-timer tuning.

Impact

  • Fixes unbounded plugin-storage growth and keeps the retry drain's directory scan cheap on long-running instances.
  • No configuration or operator action is required; existing markers migrate automatically on the first prune.
  • The 3-day window comfortably outlasts any realistic WhatsApp re-delivery or own-send echo, so normal live de-duplication is unaffected.
  • Adapter-only: no core, permission, or minOpenWAVersion change. Version 0.5.1 → 0.5.2 (patch).

Testing

  • New mapping-store unit tests cover pruneSeen: TTL expiry, keeping recent markers, legacy-marker adoption (stamps rather than deletes), and touching only seen:-prefixed keys.
  • Full plugin test suite: 272/272 passing. tsc --noEmit clean, catalog in sync, and chatwoot-adapter packages cleanly.

@rmyndharis rmyndharis merged commit 652e246 into main Jul 3, 2026
1 check passed
@rmyndharis rmyndharis deleted the feat/seen-marker-ttl-prune branch July 3, 2026 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant