Repository navigation
feat: foreign advert detection — alert when out-of-region nodes appear in local mesh #730
Copy link
Copy link
Open
Labels
area: analyticsAnalytics pageAnalytics pagearea: packetsPackets browser pagePackets browser pageenhancementNew feature or requestNew feature or requestpriority: minorNice to fix — polish and consistencyNice to fix — polish and consistency
Description
Problem
When two regional MeshCore meshes get bridged (intentionally or accidentally), observers in one region start seeing adverts from nodes with GPS coordinates in a completely different region. Today, the ingestor silently drops these out-of-region adverts via NodePassesGeoFilter() (main.go L311). This means:
- No visibility — operators don't know bridging is happening
- Data loss — foreign adverts are discarded, not logged
- No forensics — can't determine when bridging started, which nodes are entry points, or whether it's intentional
Operators need to know when this is happening and which nodes are the bridge entry points.
Current Behavior
internal/geofilter/providesPassesFilter()— polygon + buffer zone check- Ingestor calls
NodePassesGeoFilter()on every ADVERT with GPS - If the node is outside the configured region → silent
return, no log, no counter, no record - Nodes without GPS (
0,0) pass through (correct — can't classify them)
Proposed Feature
M1: Detect and log foreign adverts
Scope: Ingestor + API. No frontend changes.
At ingest time, when an ADVERT has GPS and fails the geofilter:
- Log it:
log.Printf("MQTT [%s] foreign advert: node=%s lat=%.4f lon=%.4f", tag, pubkey, lat, lon) - Increment a per-node counter in a new
foreign_advertstable:CREATE TABLE foreign_adverts ( pubkey TEXT NOT NULL, name TEXT, lat REAL, lon REAL, observer_id TEXT NOT NULL, first_seen DATETIME DEFAULT CURRENT_TIMESTAMP, last_seen DATETIME DEFAULT CURRENT_TIMESTAMP, count INTEGER DEFAULT 1, UNIQUE(pubkey, observer_id) );
- Do NOT drop the advert — insert it into the normal pipeline too, but flag it. This preserves data for forensics. (The current silent drop is the wrong behavior for an analytics tool.)
- New API endpoint:
GET /api/analytics/foreign-adverts— returns list of foreign nodes with counts, locations, observer IDs, first/last seen - Query params:
?hours=24(default 24h window),?observer=<id>(filter by observer)
M2: Dashboard card + map overlay
Scope: Frontend analytics page + map.
Analytics page — new "Foreign Adverts" card:
- Count of foreign adverts in last 24h
- Table: node name, pubkey (truncated), GPS, observer(s) that heard it, count, first/last seen
- Timeline sparkline showing when foreign adverts started appearing (detect bridging onset)
- Link to map view filtered to foreign nodes
Map overlay:
- Foreign nodes shown with distinctive marker (red circle with warning icon or pulsing red dot)
- Different from normal out-of-region markers — these are nodes that are actively being heard by local observers
- Tooltip: "Foreign node — heard by [observer] at [time], GPS: [lat, lon]"
M3: Smart alerting + bridge detection
Scope: Analytics engine + optional webhook/notification.
Baseline tracking:
- Track normal out-of-region noise (GPS jitter on border nodes near the polygon edge)
- Configurable buffer zone already exists (
BufferKmin geofilter config) — nodes within buffer are NOT foreign
Alert logic:
- Threshold: X foreign adverts from Y unique nodes in Z hours (all configurable)
- Distinguish between:
- Single node roaming: one node briefly appears outside region → probably moved, low priority
- Systematic bridge: multiple foreign nodes from a consistent distant region → active bridge, high priority
- Bridge entry point detection: cross-reference foreign node pubkeys against path bytes in relayed packets. If a foreign node appears in paths of packets relayed through local mesh → active bridge relay, not just leaked adverts
Notification:
- Optional webhook URL in config for bridge detection alerts
- Include: foreign node count, estimated bridge region (cluster foreign GPS coords), affected observers, onset time
Design Decisions to Resolve
| Decision | Options | Recommendation |
|---|---|---|
| What counts as "foreign"? | GPS outside geofilter polygon (+ buffer) | Use existing PassesFilter() — it already handles polygon + buffer zone |
| Nodes with no GPS? | Can't classify | Skip — only flag nodes with HasLocation && !(lat==0 && lon==0) |
| Nodes near boundary? | GPS jitter causes false positives | BufferKm already handles this; M3 adds noise baseline on top |
| Drop or flag foreign adverts? | Currently drops silently | Flag, don't drop — an analyzer should never discard data silently. Store with a foreign flag |
| Where to store? | In-memory vs SQLite | SQLite (foreign_adverts table) — needs to survive restarts, support time-range queries |
| Performance impact? | Extra geofilter check already happens | Negligible — PassesFilter() is O(polygon edges), typically <20 vertices. The change is: instead of return, do a cheap INSERT OR UPDATE |
Protocol Notes
Per MeshCore firmware behavior:
from_node/ originator pubkey is only available on ADVERT packets — this feature only applies to adverts, which is correct- Adverts include GPS when the node has a fix (
HasLocationflag in decoded payload) - Repeaters flood adverts every 12-24h; companions advertise on app open — foreign advert frequency indicates bridge persistence
- Path bytes in forwarded adverts show the relay chain —
path[last]is closest to the observer, which helps identify the local-side bridge node
Risk Considerations
- Fat tail scenario: A misconfigured bridge floods thousands of foreign adverts per hour. M1's
foreign_advertstable must have reasonable eviction (e.g., keep last 7 days, configurable). Without eviction, SQLite bloats silently. - Silent failure today: The current silent
returnis itself a silent risk — bridges happen with zero indication. This feature converts an invisible risk into an observable one. - False positive management: Border nodes with GPS jitter will trigger if buffer zone is too small. M3's baseline tracking prevents alert fatigue.
- Backward compatibility: M1 changes ingestor behavior (no longer drops foreign adverts). Deployments that rely on the drop behavior need a config flag:
"foreign_adverts": { "mode": "flag" | "drop" }, defaulting to"flag".
Milestones
- M1 — Low effort, high value. Can ship independently.
- M2 — Medium effort, depends on M1 API.
- M3 — Higher effort, depends on M1 data accumulation for baseline.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area: analyticsAnalytics pageAnalytics pagearea: packetsPackets browser pagePackets browser pageenhancementNew feature or requestNew feature or requestpriority: minorNice to fix — polish and consistencyNice to fix — polish and consistency
Activity
kizniche commentedon Apr 14, 2026
Excellent! Dropping of data is why I haven't used the geofilter feature, because I value what can be learned about outside meshes. If the geofilter simply identified the main region of the mesh, but still allowed additional analysis of outer regions, this would be ideal.
kizniche commentedon Apr 18, 2026
Do you think creating a new foreign_adverts table is better than maybe adding a new column to the existing table, to identify entries as foreign? If I were to change my region, this would require potentially moving entries from one table to another, rather than simply changing the bool value of a column for relevant entries.
This brings up another consideration: the ability to change a region and have a way to reanalyze the coordinates and properly reclassify data.
feat: add optional MQTT region field (#788) (#1012)
fix(#730): server-side ensure for nodes.foreign_advert column
feat(#730): foreign-advert detection — flag instead of silent drop (#…
merge: resolve upstream/master conflicts for PR Kpa-clawbot#736
Kpa-clawbot commentedon Jun 17, 2026
Severity: P2 — silent data drop in ingest pipeline.
Claim: confirmed.
cmd/ingestor/main.go:311callsNodePassesGeoFilter()which silentlyreturns on out-of-region adverts.Code today:
internal/geofilter/geofilter.go—PassesFilter()polygon+buffer check;cmd/ingestor/main.go:311— silent drop.Findings:
Fix path: M1 only as first PR: add
foreign_advertstable + flag-don't-drop + GET endpoint. M-L effort (schema migration + ingestor change + tests).Dup check: none.
Triage verdict: needs-operator-input (confirm "flag don't drop" reverses #1?? historical decision; M1-only scope).
fix: add GeoFilterAppliesToNodeList escape hatch for the geoFilter op…
fix: flip geo_filter node-list default polarity — safe-by-default for…
feat: Domestic/Foreign filter on the Nodes page
feat: add All/Domestic/Foreign origin toggle to New Nodes tool
feat: optional user accounts (part A: foundation) (#2129)
efiten commentedon Oct 8, 2026
Status update. M1 (detect and keep foreign adverts) was already in through
foreignAdverts.mode: "flag" stores them withnodes.foreign_advert, "drop" keeps the old behaviour.New since #2140: an admin can turn on a mail for every new foreign node (once per node). That is a first piece of M3's alerting.
Still open: M2 (dashboard card and map overlay) and the bridge detection from M3. Leaving this issue open for those.