channels: hide revoked shared channels from the channel list (historical messages keep them visible), and decide case handling of proposal names

[dborup]

Relates to #99, #196, #232

Problem

On production (v0.2.1, with channelProposals.autoApprove on) an administrator revoked a shared channel, but it stayed in the channel list:

Time (UTC)	Event
09:38:39	#HelloWorld suggested and auto-approved
09:41:31	#helloworld suggested and auto-approved
09:41:57	one message on #helloworld received and decrypted
09:44:19 / 09:44:43	both revoked (removed from channel keys)

After the revoke, #helloworld is gone from approvedChannels and is no longer decrypted. It still shows in GET /api/channels (messageCount: 1) and in the channel list, because the already-decoded message is stored. That matches the documented contract ("Historical messages already decoded and stored are NOT deleted or hidden"). For an administrator who revokes a channel to get rid of it, though, the result looks like the revoke failed. The channel only disappears once the message ages out of packet retention.

A second issue is visible in the same timeline: #HelloWorld and #helloworld became two separate proposals and two approvals.

Proposal

1. Hide revoked channels from the channel list. A channel whose proposal is revoked, and that is not decrypted through the built-in or config list, should be left out of GET /api/channels. The server does not delete anything (it stays read-only); it only filters. Optionally offer a toggle or an admin view to still see them.
   • Decide the behaviour for a channel that is revoked and later re-approved: it is shown again, with its history.
   • Make sure a channel that the ingestor decrypts through hashChannels, channelKeys or the rainbow table is never hidden by a revoked proposal for the same name.
2. Case in proposal names. Check in the MeshCore firmware or companion app source how hashtag channel secrets are derived from the name, and whether the name is case-sensitive.
   • If the key derivation is case-insensitive (or the apps normalise case): normalise proposal names, so that #HelloWorld and #helloworld are one proposal.
   • If it is case-sensitive: keep them separate, but show the administrator that a near-duplicate exists. Do not guess: cite the source.

Acceptance

• After a revoke, the channel disappears from GET /api/channels and the channel list within one refresh, even though historical messages exist. A test covers this.
• Re-approval shows it again. A built-in or config-decrypted channel with the same name is never hidden.
• Case handling of proposal names is decided from the firmware or app source and pinned by a test.
• The API docs (openapi.go, docs/api-spec.md) describe the new behaviour. cmd/server stays read-only.

[dborup]

Plan — CS-MacBook #251

1. Filtering revoked channels (server stays read-only)

• GET /api/channels is built in handleChannels (cmd/server/routes.go). The list comes from db.GetChannels (60 s cache, grouped by transmissions.channel_hash) or store.GetChannels. In both, hash == name == "#helloworld" (the name of the key that decrypted the message), so a proposal name matches a list entry exactly.
• Source of revoked names: the channel_proposals table the server already reads. approvedChannels() already keeps a 10 s snapshot of the approved names, and noteDecision already drops it early when a request-status read sees an approval or revocation. I add one more read-only query (status = 'revoked', capped at MaxListed) to that same refresh and store the result as an immutable hidden set in the same snapshot. No per-request DB scan; with no revoked rows the filter is a no-op.
• The hidden set is computed once per refresh: hidden = revoked − approved − builtin.
  • builtin is the file the ingestor already publishes (builtin-channels.json: built-in keys, rainbow table, hashChannels, channelKeys). Those names are decrypted whether or not a proposal is approved, so they are never hidden.
  • Fail-open: if that file is missing or unreadable, the hidden set is empty. We only hide a channel when we positively know it is not config-decrypted.
• Per request: O(channels) map lookups on a copy (the cached slice is never mutated). Filtering is applied after the cache, so no cache invalidation is needed.
• Re-approval: the ingestor resurrects the same row (UNIQUE name) to pending, then approved. As soon as the status is no longer revoked, the name leaves the hidden set and the channel is listed again with its history. Known edge: it reappears at the pending step, not only at approval (a resurrected row cannot be told apart from a new pending one). I will document it.
• Retention: revoked rows are pruned after RetentionDays; a hidden channel with surviving messages then reappears. Documented as a limit, not changed (it would need an ingestor schema change).
• GET /api/channels/{hash}/messages is unchanged: history stays readable by direct request. Nothing is deleted.

2. Case handling of proposal names (from source, not guessed)

• Firmware docs/companion_protocol.md:439-444: the hashtag key is "the first 16 bytes of sha256("#test")" of the name, with no case folding. The firmware's own hashtag-derived key (src/helpers/TransportKeyStore.cpp:44-47) hashes the exact bytes with strlen.
• meshcore-open (companion app) lib/models/channel.dart:78-85, derivePskFromHashtag: sha256(utf8('#name'))[:16], no lowercasing; the caller (lib/screens/channels_screen.dart:1229-1252) only trims and adds #. (The community variant lowercases, but it is an HMAC scheme, not a public hashtag channel.)
• meshcore-cli help text (meshcore_cli.py:4794) says it does not lowercase, and warns that the Android app and Ripple do. So key derivation is case-sensitive on the air; some clients lowercase before deriving.
• Decision: #HelloWorld and #helloworld derive different keys, so they are genuinely different channels. Merging them would derive the wrong key for one of them. Keep separate proposals (NormalizeName already says so). New: the admin list marks a proposal whose case-folded name equals another proposal's or a built-in name (nearDuplicateOf), and the admin dialog shows it. A test pins both: no normalization, and the warning.

3. Frontend

• The channel list is rebuilt from the server answer; mergeClientChannelState (fix(channels): keep client-only state (PSK selection, My Channels, unread) across a channel-list refresh #153) only enriches rows the fresh list contains and never resurrects missing ones, so a revoked channel disappears at the next loadChannels() (client cache 15 s). A user's own PSK row for the same name stays (their key). A test pins "revoked row not carried over from the previous list".
• Gap: after an admin revoke the list is not refreshed (only approval triggers onApproved). I add the same refresh on revoke so "within one refresh" holds for the revoking tab.
• An open conversation on a channel that becomes revoked is closed by the existing reconcileSelectionAfterChannelRefresh() ("Select a channel", URL reset to #/channels). I keep that and document it.
• No hardcoded colours, no per-item API calls; the near-duplicate note is built with textContent.

4. Tests and mutants

• Go (red first): revoked name hidden (DB path and in-memory path); re-approval shows it; builtin name never hidden (and fail-open when the builtin file is missing); snapshot cached (no per-request query); includeEncrypted unaffected; cached slice not mutated; ListRevokedNames; near-duplicate marking; case pinned (NormalizeName keeps case, #HelloWorld/#helloworld stay two proposals).
• Node: revoked row not resurrected by the client merge; revoke triggers a list refresh; near-duplicate rendering.
• Mutants (at least four): filter removed; builtin channel hidden; re-approval ignored (approved names still hidden); case handling swapped (names folded).
• Also: go vet, gofmt -l, sh test-all.sh, test-channel-proposals-e2e.js, channel E2Es against a local server with e2e-fixture.db, scripts/check-xss-sinks.sh --diff, fork guards unchanged (9 and 1).

[dborup]

Fixed by #257 (merged as c6b356de).