Skip to content

Feat/message import tool - #26

Closed
svscr wants to merge 16 commits into
YSelim0:mainfrom
svscr:feat/message-import-tool
Closed

svscr wants to merge 16 commits into
YSelim0:mainfrom
svscr:feat/message-import-tool

Conversation

@svscr

@svscr svscr commented Sep 4, 2026

Copy link
Copy Markdown

No description provided.

svscr and others added 16 commits September 4, 2026 05:10
Processor now checks each normalized chat message's sender username
against an optional in-memory watchlist and sends an SMTP alert email
on a match, at most once per sender per cooldown window. Fully opt-in:
disabled unless WATCHED_SENDER_USERNAMES, SMTP_HOST, and NOTIFY_EMAIL_TO
are all set. Notification is fire-and-forget from the processor's hot
path so a slow/blocked mail server cannot delay JetStream ack.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MH1ojHmQkjd5x8UeZKbVN7
Add architecture/decisions/recent_changes entries for the new
opt-in processor notification feature.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MH1ojHmQkjd5x8UeZKbVN7
Replace the static WATCHED_SENDER_USERNAMES env var with an
admin-managed SQLite list: new watched_senders table, CRUD service,
and GET/POST/DELETE /admin/watched-senders routes (auto-covered by
the existing admin-read/admin-write rate-limit policies). The
processor now polls the list every WATCHLIST_REFRESH_INTERVAL_SECONDS
and pushes it into WatchlistService via SetUsernames, so adding or
removing a watched account takes effect without a restart. The
notification feature now activates on SMTP_HOST + NOTIFY_EMAIL_TO
alone.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MH1ojHmQkjd5x8UeZKbVN7
New /admin/notifications page and WatchedSenderAdmin component: add,
list, and remove watched Kick usernames against the new
/admin/watched-senders API. New "Notifications" nav item in the
admin sidebar.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MH1ojHmQkjd5x8UeZKbVN7
Update architecture/decisions/recent_changes for the move from a
static WATCHED_SENDER_USERNAMES env var to an admin-panel-managed
SQLite watchlist.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MH1ojHmQkjd5x8UeZKbVN7
feat(notify): watched-sender email notification with admin panel
Replace the process-wide NOTIFY_EMAIL_COOLDOWN_SECONDS env var with an
admin-managed SQLite setting: new notification_settings table (single
row, mirrors retention_settings), NotificationSettingsRepository, and
GET/PUT /admin/notification-settings routes (auto-covered by the
existing admin-read/admin-write rate-limit policies). Cooldown is
validated to 30-86400 seconds. WatchlistService gained SetCooldown,
and the processor now polls the setting every
WATCHLIST_REFRESH_INTERVAL_SECONDS via a new refreshCooldownForever
goroutine, so a cooldown change takes effect without a restart. The
env var now only seeds the row's first-run value.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MH1ojHmQkjd5x8UeZKbVN7
New "Bekleme s眉resi (cooldown)" card on /admin/notifications, above
the watched-user list: shows the current cooldown and lets an admin
save a new value in minutes (1-1440) against GET/PUT
/admin/notification-settings via new getNotificationSettings /
updateNotificationSettings API functions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MH1ojHmQkjd5x8UeZKbVN7
Update architecture/decisions/recent_changes for the move from a
process-wide NOTIFY_EMAIL_COOLDOWN_SECONDS env var to an
admin-panel-managed SQLite cooldown setting.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MH1ojHmQkjd5x8UeZKbVN7
feat(notify): manage email cooldown from the admin panel
Backfills chat_messages in ClickHouse from a JSON message export
(items/count/max_rows/truncated shape). Append-only: existing rows are
matched by kick_message_id and never modified; a real write requires
-dry-run=false plus an exact confirm phrase. -limit caps how many rows
are processed and -verify-csv cross-checks kick_message_id sets against
a CSV export as an informational sanity check only.

Row id reuses the live listener's deterministic fnv64a(kick_message_id)
hash so backfilled rows match what live ingestion would have produced.
Adds a tools-profile compose service (mirrors migrate-go) and an
operations runbook covering backup-first, dry-run, and real-run usage.
Extracts export parsing, field mapping, validation, dedup, and the
append-only insert out of cmd/importmessages into
internal/usecase/messageimport so the CLI and the upcoming admin panel
endpoints share one implementation and report identical dry-run counts.

Preview never writes; Confirm requires the exact "IMPORT MESSAGES"
phrase and inserts only rows whose kick_message_id is absent. A test
pins the deterministic row id against a real exported id so it cannot
drift from the live listener's hash.
POST /admin/data-management/import/preview and /import/confirm accept a
multipart JSON export upload (file, optional limit, confirmation_text)
and follow the same preview-then-exact-confirmation-text contract as
data cleanup.

The upload is bounded because the API parses it in memory on a small
container: MESSAGE_IMPORT_MAX_UPLOAD_BYTES (16MB, enforced with
MaxBytesReader) and MESSAGE_IMPORT_MAX_ROWS (5000). Confirm re-analyzes
the upload so the existence check runs immediately before the insert.
import/confirm gets cleanup/confirm's tight rate limit; import/preview
gets 10/min burst 3 since each preview parses an upload.
/admin/data gains a "Mesaj 陌莽e Aktarma" panel: pick a JSON export, set
an optional limit, run a dry-run, review the four counts (eklenecek,
zaten mevcut, dosya i莽i tekrar, hatal谋), then type the exact
confirmation phrase to import. The confirm form is hidden entirely when
the preview reports nothing to insert.

api-client now passes a FormData body through untouched instead of
JSON-stringifying it, so multipart uploads work through the shared
client.
Updates the message import runbook to cover both entry points and their
different size limits, records the design decisions behind the shared
usecase and the bounded upload, and adds the two new routes to the API
contract and architecture route lists.
# Conflicts:
#	docs/context/decisions.md
#	docs/context/recent_changes.md
@svscr svscr closed this Sep 4, 2026
@svscr
svscr deleted the feat/message-import-tool branch September 4, 2026 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant