Skip to content

feat(channels): show the sender's path hash size on each message - #2089

Open
sylr wants to merge 2 commits into
Kpa-clawbot:masterfrom
sylr:feat/channel-msg-hash-size
Open

sylr wants to merge 2 commits into
Kpa-clawbot:masterfrom
sylr:feat/channel-msg-hash-size

Conversation

@sylr

@sylr sylr commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

What

Each message in the Channels view now shows the path hash size its sender used. It sits right before the region scope chip:

17/05/2026 18:01 · 6 hops · 2-bytes · #belgium · View packet →
17/05/2026 18:03 · 6 hops · 1-byte · View packet →

The label is 1-byte, 2-bytes or 3-bytes, with a tooltip saying "Path hash size the sender used". It's left out when the packet doesn't encode a size.

Where the size comes from

The size is read from the path byte in raw_hex (bits 7-6, plus 1), following the firmware:

  • Mesh::sendFlood sets setPathHashSizeAndCount(size, 0) before the first hop. So a flood packet carries the size even at 0 hops, including a message heard directly from its sender.
  • Mesh::sendZeroHop sets path_len = 0 on a direct route. That's the zero-hop marker, not a 1-byte size, so no size is shown.
  • TRACE path bytes are SNR readings (PathBytesAreHops), so no size is shown.
  • Size bits 0b11 are reserved (the firmware rejects sizes above 3), so no size is shown.

Repeaters don't change the hash size, so it's the same for every observation of a message.

Changes

  • internal/packetpath: a new HashSize(rawHex) int that returns 0 when the size is unknown. It decodes only the header and path bytes.
  • cmd/server: /api/channels/{hash}/messages returns hash_size from both sources, the in-memory store (store.go) and the SQLite fallback (db.go). The SQLite query selects substr(t.raw_hex, 1, 12) instead of the whole packet.
  • public/app.js: a new pathHashSize(rawHex) that returns null when unknown. It sits next to getPathLenOffset and follows the same rule as the Go helper.
  • public/channels.js:

The WebSocket broadcast payload is unchanged.

Performance

  • REST: one call to HashSize per message built. It's O(1), reads at most 6 bytes, and runs only for the page of messages returned, not for every packet. The SQLite path reads 12 characters of raw_hex per row instead of none.
  • WebSocket: one O(1) pathHashSize call per GRP_TXT message in the selected channel. There's no extra fetch, and the broadcast carries no new field.
  • The render adds one span per message line and no new DOM passes.

Tests

  • internal/packetpath/path_test.go TestHashSize: flood / transport flood / direct at 0 and more hops, zero-hop, reserved bits, TRACE, malformed input.
  • tests/unit/test-frontend-helpers.js: the same case table for pathHashSize, plus null and undefined, so the two helpers must agree.
  • cmd/server/channel_message_hash_size_test.go: both the store and the SQLite path return the expected hash_size, including 0 for a zero-hop packet.
  • tests/unit/test-issue-1851-channel-message-scope.js: the REST, WebSocket and in-browser-decrypted paths render the exact label before the scope chip, render nothing when unknown, and re-decrypt a cache saved before this change. I checked that these fail with the channels.js change reverted.
  • Two existing channels test setups (test-frontend-helpers.js, test-channel-live-decrypt-userprefix.js) now provide pathHashSize.

Local runs:

  • go test ./... in cmd/server and internal/packetpath: pass.
  • go vet: clean.
  • All 183 files in test-all.sh pass except test-issue-1956-release-routing.js, which fails locally because my sandbox blocks mktemp. It doesn't touch this code.

Browser check

I ran the server against a migrated copy of test-fixtures/e2e-fixture.db, with one message given a #belgium scope so the ordering shows. In #/channels/%23test, 34 messages show 1-byte and one shows 2-bytes · #belgium, and the console has no errors. The fixture has no 3-byte traffic; that case is covered by the unit tests.

Customizer

No new configurable values.

Each channel message now shows the path hash size its sender used
("1-byte", "2-bytes", "3-bytes"), right before the region scope chip.

The size comes from the path byte of raw_hex. Mesh::sendFlood sets it
before the first hop, so a flood packet carries it even at 0 hops; a
direct packet with path byte 0x00 is sendZeroHop's marker and carries
none, and TRACE path bytes are SNR readings. packetpath.HashSize (Go)
and pathHashSize (app.js) implement that rule and share one test table.

/api/channels/{hash}/messages returns hash_size from both the in-memory
store and the SQLite fallback. Live WebSocket messages and channels
decrypted in the browser compute it from raw_hex, which both already
carry, so the broadcast payload is unchanged.

Constraint: Hash size is set by the originator and repeaters keep it, so it is per message, not per observation
Rejected: Add hash_size to the WS broadcast maps | every packet broadcast would grow for a value the client can read from raw_hex
Rejected: Show the size whenever path byte bits 7-6 are 00 | a direct zero-hop packet would claim 1-byte
Directive: packetpath.HashSize and pathHashSize must stay in agreement; their tests use the same cases
Confidence: high
Scope-risk: narrow
Not-tested: Real 3-byte traffic in the browser (the fixture has only 1- and 2-byte messages; 3-byte is covered by unit tests)
channels.js calls pathHashSize() from app.js; like every other app.js
global it has to be listed in .eslintrc.json, or the frontend lint step
fails with no-undef.

Confidence: high
Scope-risk: narrow
@dborup

dborup commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Some field data in support of the "repeaters don't change the hash size" assumption, in case it's useful for review.

On a downstream fork we built a variant that derives the width from the relayed hops in every stored observation's path_json (instead of the path byte of one frame) and flags a message as mixed if different observations disagree. It runs on our staging instance, https://stg.meshview.dk/#/channels, with real traffic from several observers.

Why we went that way:

  • transmissions.raw_hex keeps only the first frame ingested for a content hash. We didn't want the label to depend on which observer happened to be first, in case two frames with the same content hash could carry different path bytes (for example a resend on another route).
  • We wanted the label to describe what was actually seen on the air (hop tokens of a given width in relayed paths) rather than assert the sender's configuration, so it errs on the side of showing nothing: direct/zero-hop copies and malformed paths give no badge.
  • It reuses the observation scan the channel view already does, so it needed no schema change or extra query.

What we found on real traffic:

Channel Messages 1-byte 2-byte 3-byte No relayed hops Mixed widths
#wardriving (latest 500) 500 1 232 244 23 0

No message showed different widths across its observations, so the case we guarded against didn't occur, which matches the firmware behaviour described here. And the conservative choice has a cost this PR doesn't have: the 23 zero-hop floods got no width from us, while the path byte would still carry the sender's size.

@efiten

efiten commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator

Reviewed against upstream/master at 9eb30988. I checked the central assumption against the firmware clone rather than taking it on trust, and it holds, more strongly than the PR claims.

The code never infers the width from hop-token widths in path_json. It reads bits 7-6 of the path byte, which the originator writes and nothing downstream rewrites:

  • src/Packet.h:79-83 — hash_size = (path_len >> 6) + 1, count in the low 6 bits.
  • src/Mesh.cpp:403-410 — a relaying repeater appends its hash then calls setPathHashCount(n+1), which is path_len &= ~63; path_len |= n. Bits 7-6 are untouched. So the preservation is verified, not assumed.
  • src/Mesh.cpp:332-341 — removeSelfFromPath uses the same setter on direct-route forwarding.
  • src/Mesh.cpp:644-649 — sendFlood validates 1..3 then calls setPathHashSizeAndCount(path_hash_size, 0) before any hop. This is the headline claim and it is correct.
  • src/Packet.cpp:13-17 — isValidPathLen rejects hash_size == 4 and readFrom rejects the whole frame, so the reserved value never reaches the air.

For this surface specifically the value is even more direct than the PR says: a channel message is built by createGroupDatagram(PAYLOAD_TYPE_GRP_TXT, ...) and sent through sendFloodScoped to sendFlood(pkt, delay, _prefs.path_hash_mode + 1) (companion_radio/MyMesh.cpp:489-520), so it is literally the sender's own configured path_hash_mode. The tooltip is accurate.

I also went looking for a counter-example to "a repeater cannot change it" and there is none. The only mechanism that rewrites the size bits is setPathHashSizeAndCount, and its only callers originate a new packet. The sendFloodReply(..., packet->getPathHashSize()) calls in simple_repeater/MyMesh.cpp:606-754 mirror the requester's size onto a new reply, not onto a relay of this packet.

On @dborup's point: it is correct at the schema level and sharper than stated. cmd/ingestor/decoder.go:1070 ComputeContentHash deliberately skips the path bytes, so one transmissions row aggregates frames from many observers and raw_hex is whichever arrived first. The label therefore comes from one arbitrary observer's frame. It is benign only because of the preservation invariant above, and it matches the "first observation wins" semantic db.go already uses for hops, snr and observer in the same function. His 500-message sample finding 0 mixed is consistent with that, and his own table shows this approach recovers the 23 zero-hop floods his conservative variant drops.

Absent or ambiguous evidence shows nothing rather than a confident wrong number, and the tests cover short hex, transport-truncated hex, invalid hex, empty, null, undefined, the reserved bits, TRACE and direct zero-hop. The value is computed once per message at build time, not per render, and substr(t.raw_hex, 1, 12) moves fewer bytes than selecting the column.

One thing should be fixed before this merges.

The same packet reports two different sizes on two pages, one click apart

public/packets.js:3367, unchanged by this PR:

const hashSize = (isNaN(rawPathByte) || (rawPathByte & 0x3F) === 0) ? null : ((rawPathByte >> 6) + 1);

That suppresses the size whenever the hop count is 0, for every route type including FLOOD. Per Mesh.cpp:649 that is wrong, and your new helper gets it right by only suppressing pathByte === 0 on DIRECT and TRANSPORT_DIRECT.

Trigger: any 0-hop flood channel message, one an observer heard directly from its sender.

  • Channels page: 2-bytes
  • View packet → in the same meta line (channels.js:2305): the Hash Size row is absent (packets.js:3521)
  • Hex breakdown: packets.js:3749 labels that byte hash_count=0 (direct advert) and :3780 suppresses Advertised Hash Size

@dborup's sample puts this at 23/500, about 5%, and it is precisely the case this PR advertises as its advantage.

The fix is to have packets.js:3367 call the new pathHashSize(pkt.raw_hex) instead of open-coding the rule, which also resolves the next point.

The rule now has a sixth and seventh copy

public/app.js:22 (new) and internal/packetpath/path.go:63 (new) join public/app.js:79 (57 lines below the new helper, same file), public/packets.js:3367, :3746, :3753, public/hop-filter.js:89, internal/packetpath/path.go:40, cmd/ingestor/decoder.go:241 and :249, cmd/server/decoder.go:230, and cmd/decrypt/main.go:268.

AGENTS.md names this exact failure as its cautionary tale, and the point above is that bug factory producing its next bug. I am not asking for a refactor of all eleven sites. I am asking that the new helper become the single frontend source and that packets.js:3367 call it. Worth knowing: cmd/server/decoder.go already ships path.hashSize in decoded_json for every packet, and channels.js has the parsed dj in hand at :585, so a value was available without a new parser at all.

One measured divergence to resolve

Two Go helpers in the same binary disagree on one input. I probed raw 1640DEADBEEF (DIRECT, size bits 0b01, hash_count 0): packetpath.HashSize returns 2, cmd/server/decoder.go:655 path.hashSize returns 0. This PR keys on pathByte == 0; decoder.go keys on pathByte & 0x3F == 0. Neither test table has the case.

sendZeroHop always writes exactly 0x00, but sendDirect preserves whatever path_len byte the stored path carried and Packet::copyPath returns it unchanged, so a size-tagged empty path can survive. I could not confirm this shape occurs in real traffic. Which rule is intended, and can the case go into both tables?

For floods at 0 hops the two agree (1580 gives 3/3, 1500 gives 1/1), so my first suspicion of a broader divergence was wrong.

Minor, no action needed

  • hash_size as a JSON key already means something else: cmd/server/store.go:8543-8545 ships hash_size / hash_size_inconsistent / hash_sizes_seen on nodes, the size a node is observed to use with an explicit inconsistency flag. Same key, two semantics across two API domains. Both new helpers are named pathHashSize/HashSize, so path_hash_size as the wire key would have avoided the collision. That existing machinery is also CoreScope's own evidence that per-node inconsistency is real over time, which makes the per-message value more precise than the node aggregate, not less.
  • ch-msg-hash-size has no CSS rule anywhere in public/. The label renders as bare text between · separators, which matches your example, so it may be deliberate; then the class is dead markup.
  • Only the isV3 query branch is tested. The non-v3 obsSQL at db.go:2237 also gained the substr column. Column positions are symmetric so the risk is low, and setupTestDBv2 exists and would cover it cheaply.
  • channels.js:2289 guards with if (msg.hash_size) before Number(...), so a string "0" would render 0-bytes. Not reachable today; const hs = Number(msg.hash_size) || 0 is the order AGENTS.md's cast-at-the-boundary rule asks for.
  • The JS helper uses === 9 for TRACE and routeType === 2 || routeType === 3 where the Go side uses named constants, and app.js:5-7 already has ROUTE_TYPES/PAYLOAD_TYPES.
  • The commit message says the two implementations must stay in agreement and share one case table, but the table is duplicated across internal/packetpath/path_test.go and tests/unit/test-frontend-helpers.js rather than read from one fixture. Go gates on PathBytesAreHops(payloadType); JS hardcodes === 9. They agree today, and nothing fails if PathBytesAreHops ever changes.

I ran internal/packetpath tests, TestDBGetChannelMessagesCarriesHashSize, TestStoreGetChannelMessagesCarriesHashSize, test-frontend-helpers.js (743 passed) and test-issue-1851-channel-message-scope.js (11 passed). All green, gofmt -l clean. I did not run the full server or ingestor suites and did no browser validation of my own.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants