Repository navigation
feat(channels): shared hashtag channel proposals, with admin revoke - #99
Merged
Merged
Conversation
Anyone can suggest a public hashtag channel on the Channels page (a new
section in the Add Channel dialog); the administrator approves or rejects
suggestions at #/channels?view=proposals with the existing apiKey.
Approved channels are decrypted by the ingestor and listed for everyone
via GET /api/channels approvedChannels, even before they carry traffic.
Architecture:
- internal/channelregistry: shared name validation (firmware limit of 31
UTF-8 bytes incl. '#', case preserved, no control/bidi characters), the
config/limits, a bounded atomic file queue between the read-only server
and the ingestor, and read-only SQL helpers that treat a missing table
as empty (rolling upgrade).
- Server (read-only): /api/channel-proposals{,/config,/requests/{id}} and
/api/admin/channel-proposals{,/{id}/approve,/{id}/reject}; submissions
open only with channelProposals.enabled and a strong apiKey; global
hourly rate limit and queue bound. approvedChannels comes from a short
cache and never touches the cached channel list; the includeEncrypted
append no longer writes into that cached slice's spare capacity.
- Ingestor: channel_proposals table (dbschema.Apply, BINARY unique name),
idempotent command replay after a crash, limits, retention, and an
approved-channel layer in hotKeys that SIGHUP cannot drop (configured
keys keep priority).
- Frontend: channel-proposals.js (suggest form, admin dialog with keyboard,
Escape and focus handling, admin key in memory only, single bounded
poller stopped on navigation) and O(n+m) merge of approved channels,
shown under Network with a Shared label and no local remove control.
Tests: registry, ingestor and server Go tests (limits, case semantics,
replay, retention, restart, SIGHUP, missing table, cache immutability),
frontend unit tests on the production file, and a Playwright E2E that runs
the real ingestor and server with independent browser sessions. CI runs
all of them; Dockerfile copies the new module. Docs, OpenAPI and
config.example.json updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a revoke action so an administrator can undo a previous approval of a publicly suggested hashtag channel, on top of the existing submit/approve/reject flow (internal/channelregistry, cmd/ingestor, cmd/server, public/channel-proposals.js). State machine: pending --approve--> approved --revoke--> revoked; pending --reject--> rejected; revoked --resuggest same name--> pending (never auto-approved; the row's original id is reused, so the resuggestion is idempotent under command replay the same way plain submit already is). Approving, rejecting or revoking a row outside its required source status is a conflict; repeating the exact decision already stored is idempotent. - internal/channelregistry: StatusRevoked/OpRevoke/RequestRevoked; the queue accepts revoke commands the same way as approve/reject. - internal/dbschema: the channel_proposals CHECK constraint accepts 'revoked'. This table has never shipped to any real database (part of the same unreleased commit that introduced it), so the CREATE TABLE literal is changed in place — no migration path was needed. - cmd/ingestor: hotKeys.RemoveApproved drops a name from the approved layer only, never touching base, so a manually configured key for the same channel keeps decrypting it. reviewChannelProposal now takes an explicit source status (approve/reject: pending; revoke: approved) so approving a revoked row is correctly a conflict, not a silent re-approval. submitChannelProposal resurrects a revoked row to pending on resuggestion instead of leaving it revoked or auto-approving it. PruneChannelProposals prunes revoked rows on the same retention rule as rejected ones (reviewed_at is reused as the revoke timestamp). - cmd/server: a dedicated revoke handler performs a synchronous precondition check (only an approved proposal may be revoked) before enqueueing anything — a deliberate exception to this package's usual "let the ingestor validate asynchronously" pattern, since the caller wants a side-effect-free 409 for the common case. A real TOCTOU window remains between that check and the ingestor's own re-validation, which is the true source of truth. GET /api/channels' approvedChannels cache is invalidated on an observed revoke the same way it already is on an observed approval. - public/channel-proposals.js: a Revoked filter tab; an approved row gets a Remove action guarded by a confirmation dialog reusing the admin dialog's own overlay, focus trap and single keydown listener (no second, independently-stacked Escape/Tab handler). Historical messages already decoded and stored while a channel was approved are never deleted or hidden by a revoke — only future decryption stops. Documented in docs/api-spec.md and docs/user-guide/channels.md alongside the retention behavior. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A code-review pass over the revoke feature (previous commit) found a leftover "#TBD" placeholder in the comment explaining why the status CHECK constraint was widened in place. No behavior change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…nel-proposals Confirmed master SHA f8ee2be, CI green (Go Build & Test, Playwright E2E, Docker image) before merging. One conflict, in internal/dbschema/dbschema.go's Apply(): both branches inserted a new migration call at the same point (this branch's ensureChannelProposalsTable, master's ensureTransmissionsRouteMaskColumn + ensureRouteMaskChangesTable). Resolved by keeping both, in sequence; neither depends on the other. channel_proposals remains outside AssertReady/assertReady in the merged version, so a new server still starts against a database the ingestor has not migrated yet. Verified after resolving: cmd/server, cmd/ingestor, internal/dbschema and internal/channelregistry build and their full test suites (incl. -race, and the read-only-invariant and OpenAPI-completeness tests) pass against the merged tree. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…-channel-proposals Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nvisible names Fixes from the independent review of the shared channel proposals. 1. XSS: the packet detail inserted decoded.channel (and channel_idx, path_len, SNR) into innerHTML unescaped; a publicly suggested name such as "#<img src=x onerror=...>" ran on open. The message block is now buildDetailMessageHtml with every meta part escaped. Also escaped: data-supplied channelHashHex in the list preview, detail and byte breakdown, the analytics channel table hash, the known-channels catalogue region, and the (currently dead) global search channel link. 2. dbschema: a channel_proposals table created by 4868622 has a status CHECK without 'revoked', which CREATE TABLE IF NOT EXISTS never fixes. Apply now rebuilds it in one transaction, keeping every row and the index, and logs it; a second run is a no-op. 3. Channel names reject invisible formatting characters (Unicode Cf, except ZWJ; variation selectors are not Cf) and U+2028/U+2029, in channelregistry (server and ingestor revalidation) and the frontend. This is CoreScope's own presentation rule; the firmware only limits the length. 4. Built-in names: the ingestor publishes the hashtag names of its config-derived key layer (built-in, channel-rainbow.json, hashChannels, channelKeys) to builtin-channels.json in the queue directory at startup and after each reload. The server marks them builtIn in the admin list and request status; the UI labels them, warns the submitter and says in the Remove confirmation that decryption continues. Stale hot_reload comment ("only ever grows") corrected. 5. A rejected name stays blocked until retention; documented and tested. 6. channels.js CSS.escape's the hash in the time-ticker selectors; the approved-channel cache is only invalidated by a result that completed after the snapshot was read (repeated anonymous polls no longer flush it); revoke conflict reads "suggestion is not approved (it is X)"; submit refuses trailing data and unknown fields; the queue fsyncs its directory after rename; a static test fails on new write SQL in cmd/server; api-spec/OpenAPI list 'revoked' and the 409. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Review feedback addressed (commit
Mutationer: 18/18 røde. E2E: test-channel-proposals-e2e 21/21; test-e2e-playwright 132/135 (3 skip, 0 fejl) mod master 131/135 (4 skip, 0 fejl). |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The preflight XSS gate flagged dlg.innerHTML = confirmDialogHtml(id, name, builtIn). The name was escaped with esc(), but the gate only accepts an escape on the sink line itself. Instead of opting out, the confirmation is now built with createElement, textContent and setAttribute, so the channel name never reaches an HTML string. Structure, classes, ids, roles, aria attributes, focus, Escape and the Tab trap are unchanged. The string-based confirmDialogHtml test is replaced by a DOM test that opens the dialog with "#\"><img src=x onerror=__x=1>" and checks the name is text in the title and a plain attribute value on the confirm button, plus a source check that openConfirm uses no HTML-string sink. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
CI gate fix (commit
|
dborup
marked this pull request as ready for review
September 26, 2026 13:03
This was referenced Sep 26, 2026
dborup
added a commit
that referenced
this pull request
Sep 26, 2026
This was referenced Sep 29, 2026
efiten
added a commit
to Kpa-clawbot/CoreScope
that referenced
this pull request
Oct 8, 2026
…2139) Part D of #2128: logged-in users propose a hashtag channel, an admin approves, rejects or later revokes it, and the ingestor decrypts approved channels without a restart. It is the first consumer of a generic proposal table, and answers the request in #2092. Stacked on #2138 (part C). Review the commits after that branch; GitHub shows C's commits here until it is merged. ## The situation - An instance decrypts only the hashtag channels listed in `hashChannels`. Users who want a city or club channel shown have to ask the operator, who edits `config.json` and restarts the ingestor. - #2092 asked for a suggest-and-approve flow; a downstream fork (dborup#99) runs one with anonymous suggestions and a file queue. ## What this PR adds **Name rules (`internal/channel`).** `ValidateHashtagName`, mirrored in the browser: - at most 31 UTF-8 bytes including `#` (firmware `ChannelDetails.name[32]`), case preserved, `Public` refused in any case; - control, bidi, separator and format characters refused except ZWJ, plus invisible fillers (`Other_Default_Ignorable_Code_Point`, U+2800, non-ASCII spaces). Go and JS give the same verdict for every code point. **Store.** Schema v4, one generic `proposals` table (kind, subject, status pending/approved/rejected/revoked, proposer, reviewer, note). One row per (kind, subject); limits are checked in the same transaction as the change. **Server** (only with `userManagement.channelProposals.enabled`) - `POST /api/proposals`, `GET /api/account/proposals`, `GET /api/admin/proposals`, `POST /api/admin/proposals/{id}/approve|reject|revoke`. - 400 for an invalid name. 409 for a duplicate, a rejected name, or a name already in `hashChannels`/`channelKeys` ("this channel is already decrypted on this instance"). 429 over a limit (`maxPending` 100, `maxApproved` 128, `perUserPerDay` 5). - `GET /api/channels` gains `approvedChannels`, so approved channels are listed before they have traffic. With the feature off the response is byte-identical. **Ingestor.** Opens `users.db` with `mode=ro` every 60 seconds, re-validates the names, and swaps its key map through an `atomic.Pointer`. Configured channels always win, and a failed read keeps the last good set. The ingestor never writes `users.db`; AGENTS.md now says so. **Frontend.** "Propose for everyone" in the add-channel dialog, an "approved" marker in the channel list, "My proposals" on the account page, and an admin "Proposals" tab with a warning on approve. The packet detail pane now escapes the channel name: until now only the operator chose those names. Spec: [`docs/specs/2026-10-07-channel-proposals-design.md`](https://github.com/efiten/CoreScope/blob/feat/channel-proposals/docs/specs/2026-10-07-channel-proposals-design.md). ## Performance Every key is tried on each GRP_TXT that no key opens (`decodeGrpTxt`). Benchmark of that worst case: | Keys | Time per packet | |---|---| | 320 configured | 204 to 216 µs | | 320 configured + 128 approved | 272 to 320 µs | That is about +33%, linear in the key count and capped by `maxApproved`. With the feature off the decoder gets the identical map and no ticker runs. ## Verification - `internal/channel`, `internal/users` (with `-race`), `cmd/server` and `cmd/ingestor` pass locally, 53 new Go tests. The ingestor's `TestWriteStatsAtomic_SymlinkAtDestIsReplaced` needs symlink rights on Windows and fails on `master` too. - `sh test-all.sh` exits 0; the XSS gate in diff mode passes. - User-management E2E: 21 of 21 steps locally (propose, approve, the channel appears, revoke removes it). - On our staging and production instance since 7 October 2026. ## Not in this PR - Private (PSK) channels: their keys stay in the browser (#725). - Anonymous proposals. - Names that need ZWNJ (some Persian and Urdu spellings) and subdivision flags are refused, because ZWNJ and tag characters are format characters. Allowing them later only widens the rule, so no approved name gets stranded. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Formål
Anyone can suggest a public hashtag channel on the Channels page; an
administrator (the existing
apiKey) reviews suggestions at#/channels?view=proposals— approve, reject, and now also revoke achannel that was already approved.
Produktbeslutninger
(
GET /api/channels→approvedChannels), selv før de har trafik.per-klient grænse ville være upålidelig bag en reverse proxy.
udvidelse — admin kan altså altid fortryde en godkendelse.
gemte beskeder slettes eller skjules ikke — de forbliver synlige på
Channels-siden i deres eksisterende form, uændret.
Arkitektur
internal/channelregistry: delt navnevalidering (firmware-grænsen på 31UTF-8 bytes inkl.
#, case bevares, ingen kontrol-/bidi-tegn, linjeskiftU+2028/U+2029 eller usynlige formateringstegn), config/limits,
en bounded atomic fil-kø mellem den read-only server og ingestoren, og
read-only SQL-hjælpere der behandler en manglende tabel som tom (rolling
upgrade).
siden af databasen. Ingen ny skrive-sti på
*DB— håndhævet afcmd/server/readonly_invariant_test.go.skriver et resultat, og opdaterer
hotKeys(den levendedekrypteringsnøgle-lag) i samme operation.
channel_proposalsoprettes afinternal/dbschema.Apply(kuningestoren), og er bevidst uden for
AssertReady— en ny serverstarter altså stadig fint mod en database uden tabellen (rolling upgrade).
Tilstandstabel
hotKeys.AddApproved(nøgle tilføjes)hotKeys.RemoveApproved(nøgle fjernes, medmindre samme kanal også er konfigureret manuelt — den vinder stadig)retentionDays, standard 30 dage efter afvisningen)Fjernede rækker slettes ikke ved selve fjernelsen (status ændres blot,
reviewed_atbevares som audit-tidsstempel), men følger sammeretention-regel som afviste rækker. Godkendte rækker beskæres aldrig.
Endpoints og auth
GET /api/channel-proposals/config,POST /api/channel-proposals,GET /api/channel-proposals/requests/{requestId}— åbne (kræver kun atchannelProposals.enableder sat og en stærkapiKeyer konfigureret).GET /api/admin/channel-proposals[?status=],POST /api/admin/channel-proposals/{id}/approve|reject|revoke— kræverX-API-Key.revokeer den eneste beslutning med et synkront forhåndstjek: kun enaktuelt godkendt kanal kan fjernes — er den ikke det, svares
409med detsamme, og intet skrives til kø-mappen. Der er stadig et reelt kapløb mellem
dette tjek og ingestorens faktiske anvendelse; ingestoren genvalidere
status fra bunden og er den egentlige autoritet.
Rettelser efter det uafhængige review (BLOCK)
decoded.channelblev indsatuescaped i
innerHTML(public/packets.js, meta-linjen underbeskeden). Beskedblokken er nu den rene funktion
buildDetailMessageHtml,hvor hver del (kanal,
channel_idx, hops, SNR) escapes medescapeHtml.Gennemgået og rettet i samme omgang:
packets.js: data-leveretchannelHashHexi listevisningens preview(GRP_TXT og GRP_DATA), i detaljens GRP_TXT-blok og i byte-oversigten.
channels.js: tidsopdateringensquerySelectorbruger nuCSS.escape(et navn med"kastede før en SyntaxError hvert sekundog stoppede resten af rækkerne); katalogets
regionfraknownChannelsUrlescapes nu.analytics.js: kanal-tabellensc.hashidata-valueog celle.app.js: den (i dag døde) kanal-gren i global søgning.channels.js,channel-proposals.js,live.js,map.js/route-view,channel-qr.js, farvevælgeren.'revoked'. Commit 4868622 blev pushet medCHECK(status IN ('pending','approved','rejected')), ogCREATE TABLE IF NOT EXISTSretter ikke en eksisterende tabel.internal/dbschema.Applylæser nu tabellens DDL i
sqlite_master; mangler'revoked', genopbyggestabellen i én transaktion (ny tabel, kopiér rækker, drop, rename, genskab
indeks) og logger antallet af bevarede rækker. En anden kørsel er no-op.
(U+200D) — bl.a. ZWSP, BOM, soft hyphen og tag-tegnene
U+E0000–U+E007F — samt U+2028/U+2029 (Zl/Zp, men linjeskift). Variation
selectors er ikke Cf og forbliver tilladt, så emoji med ZWJ/VS16 virker.
Samme regel i
channelregistry(server og ingestorens genvalidering) ogi frontendens validering. Det er CoreScopes egen præsentationsregel;
firmwaren begrænser kun længden. Konsekvens: emoji-flag med tag-sekvens
(fx subdivision-flag) afvises.
channel-rainbow.jsonmed ca. 320 navne som#test,#chat,#general, samthashChannels/channelKeys) vinder over godkendteforslag. Ingestoren skriver nu disse hashtag-navne til
builtin-channels.jsoni kø-mappen ved opstart og efter hver SIGHUP;serveren markerer dem
builtIn: truei admin-listen og irequest-status. Admin-UI'et viser "Built in: already decrypted",
bekræftelsen ved Remove siger at dekrypteringen fortsætter, og den der
foreslår får en advarsel. Den forældede kommentar i
cmd/ingestor/hot_reload.go("only ever grows") er rettet.standard); dokumenteret og testet.
CSS.escapeichannels.js(se 1); approved-cachen tømmeskun af et resultat, der er afsluttet efter snapshot'et blev læst (gentagne
anonyme polls tømmer den ikke længere); revoke-konflikten siger nu
"suggestion is not approved (it is pending)"; submit afviser
trailing data og ukendte felter; kø-mappen fsync'es efter rename; en
statisk test fejler ved ny skrivende SQL i
cmd/server; api-spec ogOpenAPI nævner
revoked,builtInog409.CI / Docker
.github/workflows/deploy.yml: nye testtrin forinternal/channelregistryog
internal/dbschema,test-channel-proposals.js,test-packet-detail-channel-xss.jsogtest-channels-name-escaping.jstilføjet til unit-blokken, bygger en ny
corescope-ingestor-binær til E2E, og én nyE2E-linje for
test-channel-proposals-e2e.js.Dockerfile:COPY internal/channelregistry/ ...tilføjet til bådeserver- og ingestor-build-stages, samme mønster som eksisterende interne
pakker.
additive CI-/build-trin.
Tests
-race): fuld dækning af tilstandstabellenovenfor, idempotens, crash-replay, SIGHUP, genstart, manglende tabel,
genforslag-efter-fjernelse, retention. Read-only-invariant og
OpenAPI-completeness kørt eksplicit.
test-channel-proposals.js(26 tests),test-packet-detail-channel-xss.js(8) ogtest-channels-name-escaping.js(4) — inkl. XSS-escape afkanalnavnet og fokus/Escape/Tab-trap for bekræftelsesdialogen ved fjernelse.
test-channel-proposals-e2e.js(21 tests) — hele flowet: foreslå→ godkend → en anden session ser den delt → admin fjerner (med
bekræftelsesdialog) → sessionen ser den ikke længere → genstart af
ingestor+server genopliver den ikke → genforslag lander som pending.
Desktop lys/mørk + mobil 390×844 valideret, ingen horisontal overflow.
RemoveApproved-kald, SIGHUP der nulstiller den godkendte nøgle-lag,approvedChannelsder ikke filtrerer fjernede kanaler, ogauto-godkendelse ved genforslag.
Kendte begrænsninger
builtInkræver en ingestor fra denne PR; en ældre ingestor skriver ikkebuiltin-channels.json, og så markeres intet. Filen skrives kun vedopstart og SIGHUP — slettes den manuelt, kommer den igen ved næste
genstart/reload.
cmd/serverhar fra før denne PR skrivende SQL mod den delte database ihash_migrate.googroutes.go(samtping_score_history.gomod sin egendatabase og
backup.gosVACUUM INTO). Den nye statiske test tilladerpræcis disse og fejler på alt nyt; selve stederne er ikke ændret her.
Frontendens Cf-tjek bruger
\p{Cf}; en browser uden Unicode propertyescapes springer tjekket over, og serveren afviser så navnet.
Den globale rate limit for forslag kan lukke for nye forslag i op til 1
time, hvis grænsen nås.
En klient kan i op til ~15 sekunder blive ved med at vise en kanal som
delt/godkendt efter admin har fjernet den (cache-TTL), før næste
hentning/genindlæsning viser den korrekte tilstand.
Grænseværdier (
maxPending,maxApproved,retentionDays,submissionsPerHour) er kun iconfig.jsonfor nu — de bør eksponeres icustomizeren i en senere milestone (jf. AGENTS.md regel 8).
Rækkefølge
Cache-rettelsen i #98 ligger også (uændret) inde i denne branchs historik,
fordi den oprindeligt blev fundet og rettet her, før den blev udskilt til sin
egen branch. #98 bør merges først — når den merges til master, falder
den sammen med den identiske rettelse her ved næste merge fra master.
Staging
Denne PR tilføjer en ny tabel (
channel_proposals) viainternal/dbschema.Apply. Et staging-deploy af denne branch bør derforhave en frisk DB-snapshot klar til rollback, i tilfælde af at
migrationen eller den nye kø-mekanisme opfører sig uventet mod rigtig data.
🤖 Generated with Claude Code