Skip to content

filter: advert window, path block, dry-run, sender/text rules, message age limit - #11

Merged
Elektr0Vodka merged 6 commits into
dmc-devfrom
enhancement/dmc-dev-filtering
Sep 26, 2026
Merged

Elektr0Vodka merged 6 commits into
dmc-devfrom
enhancement/dmc-dev-filtering

Conversation

@Elektr0Vodka

Copy link
Copy Markdown

Packet-filter enhancements for the repeater, partly ported from the jhuebert repeater-filter fork and rewritten in our filter grammar, plus the message age limit requested in #8.

New commands

  • filter dryrun on|off: count every drop but keep forwarding, to size a setting before enforcing it. The status line ends in (dry-run) after the bracket, so existing Toolbox parsers keep working.
  • filter advert <0-720h>|clear: forward each node's advert at most once per window (per origin, 256-entry cache).
  • filter path add|remove|list <2-8 hex>: drop flood packets whose path contains a blocked prefix (up to 8).
  • filter sender add <name> [secs] [prob] / filter text add <pattern> [secs] [prob]: block, throttle or partially drop group texts by sender name or text (up to 8 each).
  • filter watch add|remove|list <#name>: up to 4 # channels the rules may read besides Public.
  • filter age <minutes>|off (closes [Feature request] Repeater Filter on timestamp of incomming message #8): drop group texts on Public and watched channels whose sender timestamp is older than the limit (1-10080 min, default off).
    • Inactive while the repeater clock is not set (before 2026-01-01; an RTC-less repeater boots at May 2024), and filter age then says so.
    • Direct messages and other channels are encrypted with keys the repeater does not have, so their age cannot be read.
  • filter stats advert|path|air|sender|text|age: new stats topics; air is the estimated airtime the drops saved.

Fix

The malformed scan's ±1 week timestamp check now gets the same clock guard. Before, a repeater with an unset clock dropped every current Public message as time malformed once filter malformed on was set.

Compatibility

New settings are appended to FilterPrefs; each field defaults when an older /filter_prefs does not cover it. Help strings are guarded against the 160-byte reply buffer at compile time.

Testing

  • Native: 179/179 (new test_filter_policy, test_filter_rules, extended test_filterstats, MessageAge tests).
  • Heltec_v3_repeater and RAK_4631_repeater build.
  • Not tested on a device.

Before merging

  • prebuilts/ holds ~3.7 MB of test binaries (Heltec v3, RAK 4631) committed for testers. Decide whether they should land on dmc-dev or be dropped first.
  • Toolbox (Dutch-Meshcore-Toolbox) has no UI for the new commands yet.

Observer counterpart: enhancement/dmc-observer-dev-filtering → dmc-observer-dev.

🤖 Generated with Claude Code

…irtime

Ports the four ideas from the jhuebert/MeshCore repeater-filter fork that
fit our per-type filter without adopting its rule engine or grammar:

- `filter advert <hours>`: each origin's flood advert is forwarded at most
  once per window (0-720 h, 256-entry ring keyed on 4 pubkey bytes), ahead
  of the per-type advert limiter so repeats never eat the legit budget.
- `filter path add|remove|list <hex>`: drop flood packets whose path holds
  a repeater ID starting with one of up to 8 prefixes (2-8 hex digits),
  compared aligned to the packet's hash size.
- `filter dryrun on|off`: every drop is counted but still forwarded; the
  status line gains a trailing `(dry-run)` marker after the bracket.
- `filter stats air`: estimated time-on-air the drops saved, via the
  radio's own estimate, billed only on drops.

New stats topics `advert`, `path`, `air`; existing replies are unchanged
except the condensed `filter help` line. New prefs are appended to
/filter_prefs and defaulted per field when an older file is shorter.

Header-only AdvertLimiter/PathBlock plus the FilterStats additions are
covered by native tests (test_filter_policy, test_filterstats).
Adapts the remaining jhuebert-fork ideas into our grammar without its rule
engine or regex:

- `filter sender add <name> [secs] [prob]` / remove / list: exact name or
  `Prefix*`; secs=0 blocks, otherwise one match per secs passes and the
  excess is dropped; prob 1-100 is the share of matches the rule decides.
- `filter text add <pattern> [secs] [prob]`: substring, or `^prefix`.
- `filter watch add|remove|list <#name>`: up to 4 channels whose key is
  derived from the name and decrypted for the rules; Public is always read.
- `filter stats sender|text`: drops per rule plus throttle passes.

Rules are evaluated top to bottom (senders, then texts); a failed roll or a
within-budget throttle pass steps aside so a later rule can still decide.
Only plain group texts (`Sender: text`) match. The one decrypt per packet is
shared with the malformed scan and only happens when a rule exists.

Header-only SenderRules.h (parse, match, evaluate, arg parsing) and the
FilterStats formatter are covered by test_filter_rules and test_filterstats.
Prefs append 8+8 rules and 4 watch channels behind the earlier fields with
the same per-field load defaults.
…341f1)

Heltec V3 (ESP32, app + merged) and RAK4631 (nRF52, uf2 + zip) repeater
bins built locally as v1.17.1-dev so testers can flash without a release.
See prebuilts/README.md.
`filter age <minutes>|off` drops group texts whose sender timestamp is older
than the limit (1-10080 min, default off), on every channel the repeater can
read: Public plus the watch list. Direct messages and other channels are
encrypted with keys the repeater does not hold, so their age cannot be read.
Drops are counted under `filter stats age`; the setting is appended to
FilterPrefs and defaults to off when an older /filter_prefs is loaded.

The check trusts the repeater clock, so it is inactive while that clock is
unset (before 2026-01-01; an RTC-less repeater boots at 15 May 2024) and
`filter age` then says so. The malformed scan's +-1 week window gets the
same guard: before, a repeater with an unset clock dropped every current
Public message as "time" malformed once the scan was on.

Timestamp logic lives in header-only MessageAge.h with native tests.
The cert bundle, cacert.pem, generated WebConfigHtml.h and a .pyc are
generated by observer builds and were left over in the worktree.
@Elektr0Vodka
Elektr0Vodka merged commit 34d1c16 into dmc-dev Sep 26, 2026
1 check passed
Comment thread prebuilts/README.md

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Message Age not in this build right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants