Skip to content

feat: propose and approve hashtag channels (user management part D) - #2139

Merged
efiten merged 41 commits into
Kpa-clawbot:masterfrom
efiten:feat/channel-proposals
Oct 8, 2026
Merged

efiten merged 41 commits into
Kpa-clawbot:masterfrom
efiten:feat/channel-proposals

Conversation

@efiten

@efiten efiten commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Part D of #2128: logged-in users propose a hashtag channel, an admin approves, rejects or later revokes it, and the ingestor decrypts approved channels without a restart. It is the first consumer of a generic proposal table, and answers the request in #2092.

Stacked on #2138 (part C). Review the commits after that branch; GitHub shows C's commits here until it is merged.

The situation

What this PR adds

Name rules (internal/channel). ValidateHashtagName, mirrored in the browser:

  • at most 31 UTF-8 bytes including # (firmware ChannelDetails.name[32]), case preserved, Public refused in any case;
  • control, bidi, separator and format characters refused except ZWJ, plus invisible fillers (Other_Default_Ignorable_Code_Point, U+2800, non-ASCII spaces).

Go and JS give the same verdict for every code point.

Store. Schema v4, one generic proposals table (kind, subject, status pending/approved/rejected/revoked, proposer, reviewer, note). One row per (kind, subject); limits are checked in the same transaction as the change.

Server (only with userManagement.channelProposals.enabled)

  • POST /api/proposals, GET /api/account/proposals, GET /api/admin/proposals, POST /api/admin/proposals/{id}/approve|reject|revoke.
  • 400 for an invalid name. 409 for a duplicate, a rejected name, or a name already in hashChannels/channelKeys ("this channel is already decrypted on this instance"). 429 over a limit (maxPending 100, maxApproved 128, perUserPerDay 5).
  • GET /api/channels gains approvedChannels, so approved channels are listed before they have traffic. With the feature off the response is byte-identical.

Ingestor. Opens users.db with mode=ro every 60 seconds, re-validates the names, and swaps its key map through an atomic.Pointer. Configured channels always win, and a failed read keeps the last good set. The ingestor never writes users.db; AGENTS.md now says so.

Frontend. "Propose for everyone" in the add-channel dialog, an "approved" marker in the channel list, "My proposals" on the account page, and an admin "Proposals" tab with a warning on approve. The packet detail pane now escapes the channel name: until now only the operator chose those names.

Spec: docs/specs/2026-10-07-channel-proposals-design.md.

Performance

Every key is tried on each GRP_TXT that no key opens (decodeGrpTxt). Benchmark of that worst case:

Keys Time per packet
320 configured 204 to 216 µs
320 configured + 128 approved 272 to 320 µs

That is about +33%, linear in the key count and capped by maxApproved. With the feature off the decoder gets the identical map and no ticker runs.

Verification

  • internal/channel, internal/users (with -race), cmd/server and cmd/ingestor pass locally, 53 new Go tests. The ingestor's TestWriteStatsAtomic_SymlinkAtDestIsReplaced needs symlink rights on Windows and fails on master too.
  • sh test-all.sh exits 0; the XSS gate in diff mode passes.
  • User-management E2E: 21 of 21 steps locally (propose, approve, the channel appears, revoke removes it).
  • On our staging and production instance since 7 October 2026.

Not in this PR

efiten and others added 30 commits October 7, 2026 12:00
… schema v3

AuditList filters by action (exact or a ".*" group), user as actor or
target, and an inclusive period, newest first, keyset-paginated on id
(default 100, at most 500). PruneAudit deletes rows of the listed actions
older than an age and keeps every other action. UsersByID resolves the
users of a page in one query. Schema v3 adds an index on audit_log(at).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Account counts by status, active admins, stuck pending accounts (older
than 24 hours), bouncing addresses, registrations over 7 and 30 days and
per UTC day for 30 days (from user.register audit rows), active users,
logins and failed logins in 24 hours, mail of the last 7 days by latest
delivery event, and accounts with 5 or more failed logins in 24 hours.
Thresholds are constants for now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
A successful login writes user.login; a refused login for an existing
account writes user.login.failed with reason wrong_password, pending or
disabled. An unknown address writes nothing and no IP is stored. The
audit write follows the decided answer and a failure is only logged, so
every failure answer stays identical. The hourly janitor deletes login
rows older than 90 days and keeps all other actions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Admin-only and registered only with user management on. Filters by
action or a ".*" group, user as actor or target, and an RFC 3339
period; keyset-paginated with next/before, default 100, capped at 500.
Actor and target are resolved in one query, a removed account becomes
{id, deleted: true}. Invalid parameters answer 400.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Admin-only and registered only with user management on. Serves the
users.Stats figures as a typed struct; newPerDay always holds 30 days
and guessing is always an array.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
The login audit INSERT ran before the buffered answer reached the client,
so a refusal for an existing account took one extra SQLite commit (up to
the 5 s busy timeout on a locked users.db) that an unknown address never
paid: a timing signal for account existence. user.login and
user.login.failed are now written by auditAsync after the answer is
decided, best-effort, tracked by a WaitGroup that closeUserManagement
drains after stopping the janitor. Response bytes, session creation,
rate limiting and the dummy hash are unchanged. Spec amendment 7 now
says so, including that a row can be lost if the process stops right
after a login.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Filters (action or group, period, numeric user id) live in the hash and
are validated on read; filter changes use replaceState. Rows are rendered
newest first, Load more appends the next keyset page and hides at the
end. Account names link to the Users tab. Not reachable yet: the admin
shell mounts it in a later commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
load() clears auditMsg first. A filter change drops the paging cursor,
hides Load more and empties the table before the new request, so a click
cannot mix pages and a failed request leaves no rows of the old filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Needs attention (stuck activations, bouncing mail, password guessing,
MQTT sources down for 10 minutes or never heard), the Users card from
/api/admin/stats with a 30-day registrations chart, and the System card
composed from /api/health, /api/healthz, /api/mqtt/status and
/api/observers. A failing source shows Could not load in its own block;
healthz 503 while warming up is shown as warming up. Refreshes every
60 s while the browser tab is visible; the timer skips /api/healthz,
which is read on open and on Refresh only. Not reachable yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Page admin is now a tab shell at #/admin?tab=overview|users|audit; the
user table becomes the Users tab and gains a Bouncing mail only filter.
Old #/admin/users links are rewritten to the Users tab with
replaceState, filters and detail id kept. The account menu entry and
the account page button read Admin. Non-admins see Admins only; with
the feature off the page stays Not found. The shell unmounts the
active tab on every init and on destroy, and an init whose auth check
resolves after the page was left mounts nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
An admin sees the user figures and a five-attempt failed-login burst
under Needs attention, follows it to the audit tab and finds the row.
The old #/admin/users link lands on the Users tab with its filter. Axe
runs on all three admin tabs. Menu assertions now look for Admin.

The axe steps park the mouse at 0,0 first: a hovered row tints its
cells to the row-hover colour, where the active status text is 4.48:1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Green status words in the user and admin tables used --status-green-text
(green-700 #15803d), which is 4.48:1 on the row-hover tint #eef2ff. A
scoped token --status-green-tint-text (light green-800 #166534, dark
keeps #22c55e in both dark blocks) now colours them: measured 6.38:1 on
#eef2ff, 6.54:1 on --surface-0 in light, 5.75:1 on #2d2d50 in dark.
The global --status-green-text is unchanged.

The token is mapped in the customizer THEME_CSS_MAP. The contrast unit
test gains the hovered-row pairs for green, yellow and red, and the E2E
mouse-park comment now says parking only avoids hover noise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
The audit tab accepted /^\d+$/ for the user id, so user=0 in the hash
reached the server and came back as a 400 message instead of being
ignored like other invalid URL filters. Both the hash reader and the
user id input now accept /^[1-9]\d*$/: user=0 and leading zeros such
as 007 are ignored in the hash and rejected in the input (the previous
value is restored, no reload).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
The 250 ms debounced search read #umQ without a guard, so typing and
then switching to another admin tab within 250 ms threw a TypeError
when the timer fired. The callback now carries a mount token: it
returns when the tab was unmounted or remounted since, or when #umQ is
gone. Two unit tests cover type then unmount (no throw, no request, hash
untouched) and a stale timer from a previous mount.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Every Refresh or Retry click started a full refresh including
/api/healthz, which walks the packet store under its read lock, so
repeated clicks queued repeated walks. Open, Refresh and Retry now go
through one guard: while a full refresh is in flight further clicks are
ignored and the Refresh button is disabled until it settles. The
60-second timer is unchanged (it never reads /api/healthz).

Unit test: clicks during the mount refresh start nothing, the button is
disabled then re-enabled, and two rapid clicks make one /api/healthz
request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
The amendment said neither the audit write nor a locked users.db
changes response timing. That holds for the login response itself, but
users.db uses one connection (internal/users/store.go SetMaxOpenConns(1)),
so the background INSERT can delay the next users.db request by one
commit. The text now says so, and that the login rate limits (10 per
15 minutes per address, and per IP when IPs can be told apart) keep it
from being a useful signal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
The Users tab applied bouncing=1 in the browser to the list the server
returns, which is capped at 1000 rows, so on a large instance it could
show fewer users than the overview's bouncing count. users.ListFilter
gains Bouncing (email_bouncing != 0, combined with the other filters),
GET /api/admin/users accepts bouncing=1 (any other value is 400, like
the status and role filters), and the Users tab sends it instead of
filtering client-side. OpenAPI and docs/api-spec.md describe the
parameter.

Tests: internal/users List with Bouncing alone and with Role,
cmd/server bouncing=1 rows and 400 for 0/yes/true, UI request carries
bouncing=1 and the browser no longer drops rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
ValidateHashtagName: 31-byte firmware limit, no invisible or control
characters except ZWJ, case preserved, Public refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
One generic proposals table, kind hashtag_channel. Propose, Decide,
ListProposals, ProposalsByUser, ApprovedSubjects, PruneProposals; limits
are checked inside the state-change transaction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
…oposals

POST /api/proposals, GET /api/account/proposals, GET /api/admin/proposals
and approve/reject/revoke, audited as proposal.*; the janitor prunes
decisions after 90 days. Off by default: no routes registered.

A name config.json already decrypts (hashChannels, channelKeys) is
refused with 409; the server reads channelKeys for its names only and
drops the key values. Every decision reloads the approved-name snapshot
from users.db under a mutex.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
…g load

LoadConfig falls back to pure defaults on any parse error, so the
names-only channelKeys type made a string or array value start the
server without apiKey or userManagement. It now logs and keeps no names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
approvedChannels from an in-memory snapshot refreshed on approve and
revoke, capped at maxApproved like the ingestor; client config gains
userManagement.channelProposals. Absent when the feature is off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Reads users.db read-only once a minute, re-validates every name, merges
below the configured keys and swaps the key map atomically; a failed read
keeps the last good set. Off by default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
window.CSProposals mirrors the hashtag name rules, posts proposals,
merges approved channels into the channel list, and renders the account
page's My proposals. Inert unless the server advertises the feature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
The Add Channel dialog gains Propose for everyone; approved hashtag
channels are listed with an approved marker, also without traffic.
Channel names are now escaped in row data attributes and in the
timestamp timer's selectors, since approved names may contain quotes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Approve, reject or revoke proposed hashtag channels with an optional
note; approving asks first because the channel becomes readable for every
visitor. Shown only with channel proposals on; deep link
#/admin?tab=proposals&status=. The audit filter gains proposal.*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
…rows

A decision in flight ignores a second click on the same row and disables
its buttons until it settles. A refused decision (for example another admin
decided first) reloads the list and keeps the error message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
efiten and others added 10 commits October 7, 2026 17:24
A user proposes #e2e-test, the admin approves it after the warning, the
channel appears with its marker, revoke removes it from approvedChannels.
The Propose button stays hidden while logged out, and the approved channel
is listed although it has no traffic. CI's user-management server runs with
channelProposals on. Operator and user docs, and the spec's amendments from
the plan and the controller rulings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
Approved channel proposals make decoded.channel a name a logged-in user
chose, and the name rules allow "<" and ">". The message meta line of the
packet detail pane interpolated it as HTML. Escape the line at the sink and
pin it with a unit test that drives the real renderDetail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
…g names

The name rules let invisible characters that are not Cf through: U+3164,
U+115F, U+FFA0, U+034F and the rest of Other_Default_Ignorable_Code_Point,
U+2800 BRAILLE PATTERN BLANK, and interior space separators such as NBSP
and U+3000. "#mesh" plus U+3164 then passed for a configured "#mesh".

ValidateHashtagName now refuses those with the existing invisible-character
message. ZWJ and variation selectors stay allowed, so "#❤️" and ZWJ
emoji sequences still pass. The browser mirror spells out Go's
Other_Default_Ignorable table, since JS has no property escape for it; a
scratch run over every code point gave identical verdicts (4029 refused on
both sides). Tightening now is free: the ingestor drops any stored name the
current rule refuses, so a later change would orphan approved channels.

internal/users/validate.go points at the stricter variant; the display-name
rules are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
…stor

Both startup lines printed the configured path. With a relative path and
different working directories, the server and the ingestor printed the
same string for two different files, which is the case the log line
exists to expose. Both now resolve the path with filepath.Abs and fall
back to the path as given. The log prefixes are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
At 375 px the Actions column (note input and buttons) sat past the edge
of the table and was reachable only by scrolling inside it. Mark the
Proposed by and Decision columns um-col-optional, the pattern the Users
table already uses, so they hide below 760 px and Actions fits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
…w screens

With Proposed by and Decision hidden, the note input (max-width 16em)
still held the Actions cell 43 px past the table edge at 375 px (table
scrollWidth 398 in a 355 px wrapper). Below the same 760 px breakpoint
the input now takes the cell width, so the table fits (scrollWidth 355)
and the buttons wrap under it. Measured in Chrome against a local server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
The spec amendments and the accounts guide now quote the 409 as the
server sends it ("this channel is already decrypted on this instance"),
say that both users.db log lines show the absolute path, and record the
name-rule tightening (Other_Default_Ignorable fillers, U+2800 and non-ASCII
space separators refused; ZWJ and variation selectors kept) as amendment 14.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FGYZDCxPrcM9FC6VHksY71
@efiten
efiten merged commit abbe684 into Kpa-clawbot:master Oct 8, 2026
12 checks passed
efiten added a commit to efiten/CoreScope that referenced this pull request Oct 8, 2026
@efiten
efiten deleted the feat/channel-proposals branch October 8, 2026 19:52
efiten added a commit that referenced this pull request Oct 8, 2026
…2140)

Part E of #2128: a logged-in user watches nodes and gets one mail when a
watched node goes offline, comes back, or reports a low battery. Admins
can add instance events: a new foreign node, and an observer going
offline or back. This covers the per-user mail part of #775.

Stacked on #2139 (part D, which builds on #2138). Review the commits
after that branch.

## The situation

A sysop learns that a repeater went silent, or that its battery is
running down, only by opening CoreScope. #775 asks for notifications.
Accounts (part A) now give a verified address to mail and a mailer with
delivery status.

## What this PR adds

**Events.** They use the instance thresholds, so a mail does not
disagree with the node page:
- `node.offline`: silent for the role's `healthThresholds` window. Last
heard comes from the packet store, and repeaters and rooms count relayed
traffic (#1598).
- `node.battery`: advert telemetry below `batteryThresholds.lowMv`,
recovered at `lowMv + 100`.
- Admins only: `foreign.new` (once per node) and `observer.offline`.

**Store.** Schema v5 with `notification_prefs`, `notification_watches`
and `notification_state`.

**Server** (only with `userManagement.notifications.enabled`)
- A notifier next to the janitor, every `intervalMinutes` (5). The
evaluator is a pure function of watches, prefs, stored states and node
snapshots.
- No burst after a restart or a new watch: a subject's first evaluation
stores its state without mailing. The loop waits for the startup load.
While ingest is stale (newest packet older than 30 minutes) the offline
checks pause, and after recovery they wait one silent window.
- One mail per user per check. Limits: 20 per user and 100 per instance
per rolling 24 hours, 50 watches per user. A change over a limit is
recorded and never mailed later.
- Every mail has a one-click unsubscribe (`List-Unsubscribe` and
`List-Unsubscribe-Post`). The GET only redirects to a confirm page, so
link scanners cannot unsubscribe anyone. Node names are cleaned of
control and bidi characters before they go into a mail.
- Routes: `GET`/`PUT /api/account/notifications`, `PUT`/`DELETE
/api/account/notifications/watches/{pubkey}`, `POST
/api/account/notifications/watch-my-nodes` (copies the synced "my nodes"
list), `GET`/`POST /api/notifications/unsubscribe`. `mailer.Message`
gains `Headers`.

**Frontend.** A "Notify me" toggle on the node pages, a Notifications
section on the account page (`#/account?section=notifications`), an
unsubscribe page, and the notification mail count on the admin overview.

Spec:
[`docs/specs/2026-10-07-node-notifications-design.md`](https://github.com/efiten/CoreScope/blob/feat/notifications/docs/specs/2026-10-07-node-notifications-design.md).

## Performance

Each check reads watches, prefs and states from `users.db` (one query
each), the watched nodes from the analyzer DB in chunks of 500, and
last-heard times from the packet store.

| Measurement | Result |
|---|---|
| One check on our staging instance, 130,000 packets in memory, 1
watcher | 6.9 ms, of which 0.93 ms holding the store read lock |
| Evaluator benchmark, 100 users with 50 watches each, 2,000 nodes |
1.75 ms |

No work is added to ingest, broadcast or any request path.

## Verification

- `internal/users` and `internal/mailer` with `-race`; the `cmd/server`
suite plus `-race` on the notifier tests; 74 new Go tests. The same
Windows-only failure as noted in #2138 applies.
- `sh test-all.sh` exits 0; the XSS gate in diff mode passes.
- User-management E2E: 26 of 26 steps locally. With notifications off
the new steps fail, so they do test the flag.
- On our staging and production instance since 7 October 2026.

## Not in this PR

- Other channels (Discord, Telegram, webhooks). Detection is separate
from delivery, so one can be added.
- The topology, RF and anomaly alerts of #775.
- If the whole server starts after a feed outage that already ended, the
grace window is unknown and a watcher can get one wrong offline mail.
The user guide says so.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant