Feature proposal: user-suggested shared hashtag channels with admin approval and revoke

[dborup]

Summary

On a downstream fork we added a way for visitors to suggest a public hashtag channel (e.g. #mycity) on the Channels page. An administrator reviews the suggestions and can approve, reject, or later revoke a channel. Approved channels are then decrypted by the ingestor and listed for everyone, even before they have traffic.

It has run on our staging instance for a few days: https://stg.meshview.dk/#/channels (the suggestion form is in the add-channel dialog; the admin review view needs the instance's API key).

Before porting anything, we'd like to know whether you want this upstream, and if so how you'd like it split. It is a sizeable change (about 6.7k lines incl. tests, in our fork), so we don't want to open an unsolicited PR.

What it does

Visitors

• Suggest a hashtag channel name in the add-channel dialog. The name is validated client- and server-side; the visitor sees the request status (queued → pending/approved/rejected).
• Approved channels appear in the channel list for everyone (GET /api/channels gains approvedChannels).

Admins (existing apiKey, X-API-Key header)

• Review view at #/channels?view=proposals: approve, reject, and revoke an approved channel (with a confirmation dialog).
• Revoke stops future decryption only; messages already decoded stay as they are.

Design

• Server stays read-only (the mode=ro invariant). It only writes small command files to a bounded, atomic file queue next to the database. No new write path on the server's DB handle; a static test fails if write SQL appears in cmd/server.
• Ingestor does all writes. It applies each command in one transaction, writes a result file, and updates its live key layer in the same step. It re-validates everything and is the authority, so races between the server's quick checks and the ingestor are safe.
• New table channel_proposals, created by the ingestor's schema step. It is deliberately not required at server startup, so a new server can run against an older database during a rolling upgrade (a missing table is treated as empty).
• Keys: hashtag keys are derived from the name as today (sha256("#name")[:16]). Built-in and configured channels always win over approved suggestions; the ingestor publishes their names, so the admin UI can mark a suggestion as "already decrypted" and revoking it doesn't stop that decryption.
• State machine:

From	Action	To
pending	approve	approved (key added)
pending	reject	rejected
approved	revoke	revoked (key removed unless also configured)
revoked	suggest again	pending (never auto-approved)
rejected	suggest again	stays rejected until retention removes the row

• Name rules (shared by server, ingestor and frontend): at most 31 UTF-8 bytes including # (firmware limit), case preserved, no control/bidi characters, no line/paragraph separators, no invisible format characters (Unicode Cf) except ZWJ, so emoji sequences still work.

Limits and config

All off by default; enabling requires a strong apiKey:

"channelProposals": {
  "enabled": false,
  "maxPending": 100,
  "maxApproved": 128,
  "maxQueuedRequests": 256,
  "retentionDays": 30,
  "submissionsPerHour": 20
}

The submission rate limit is global, not per client: per-IP limits are unreliable behind reverse proxies. Rejected and revoked rows are pruned after retentionDays; approved rows are kept.

Trust and privacy

• Approving a hashtag channel makes its messages readable by everyone on that instance. Hashtag keys are public by construction, but an operator should decide deliberately which ones are shown; that is why nothing is auto-approved.
• Suggestions carry no identity; the only abuse control is the global rate limit plus admin review.

Code

Everything is in our fork, pinned to commit 5f493f1d so the links stay valid:

• Pull request: feat(channels): shared hashtag channel proposals, with admin revoke dborup/CoreScope#99 (description in Danish; code, comments and tests in English), plus the name-trim follow-up Fix channel-name trim parity between frontend and server dborup/CoreScope#112
• Shared module (name rules, config, file queue, read-only SQL helpers): https://github.com/dborup/CoreScope/tree/5f493f1de6bbf1eff4078790476c72667f608a74/internal/channelregistry
• Ingestor (applies commands, key layer, built-in list): https://github.com/dborup/CoreScope/blob/5f493f1de6bbf1eff4078790476c72667f608a74/cmd/ingestor/channel_proposals.go, https://github.com/dborup/CoreScope/blob/5f493f1de6bbf1eff4078790476c72667f608a74/cmd/ingestor/hot_reload.go
• Schema (table, and the migration that adds revoked to an older CHECK): https://github.com/dborup/CoreScope/blob/5f493f1de6bbf1eff4078790476c72667f608a74/internal/dbschema/dbschema.go
• Server (endpoints, read-only): https://github.com/dborup/CoreScope/blob/5f493f1de6bbf1eff4078790476c72667f608a74/cmd/server/channel_proposals.go
• Frontend: https://github.com/dborup/CoreScope/blob/5f493f1de6bbf1eff4078790476c72667f608a74/public/channel-proposals.js
• Docs: https://github.com/dborup/CoreScope/blob/5f493f1de6bbf1eff4078790476c72667f608a74/docs/user-guide/channels.md, https://github.com/dborup/CoreScope/blob/5f493f1de6bbf1eff4078790476c72667f608a74/docs/api-spec.md

How we would split a port

1. Name validation, config and queue module + schema (no user-visible change).
2. Ingestor: apply commands, key layer, built-in list.
3. Server: endpoints, OpenAPI/API docs, approvedChannels.
4. Frontend: suggestion form, admin review view, tests (unit, E2E).

Questions

• Is this something you'd want in CoreScope?
• Any constraints on the file-queue approach, or would you prefer a different server → ingestor channel?
• Would you rather keep "shared channels" config-only (operator edits hashChannels) and skip the suggestion flow?