Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
22f7b54
fix(live-alerts): isolate a rejected Twitch login or Kick slug from t…
Ferdinand99 Nov 15, 2026
43c3cda
fix(tickets): bound the member lookups one DM can cause
Ferdinand99 Nov 15, 2026
7c6e243
fix(events): give each server a fair share of the event image cache
Ferdinand99 Nov 15, 2026
f3e8864
feat(backup): warn about a privileged database role and confirm befor…
Ferdinand99 Nov 15, 2026
91ae820
test(tickets): cover the ticket access guard in the v2 tickets router
Ferdinand99 Nov 16, 2026
f5e4916
fix(live-alerts): decide item fault or request fault from the poll's …
Ferdinand99 Nov 16, 2026
e4e2776
fix(tickets): route a DM by open tickets, never by a cut-short lookup
Ferdinand99 Nov 16, 2026
76c2a7a
fix(events): evict along the limit that was exceeded
Ferdinand99 Nov 16, 2026
d3f5f22
fix(backup): enforce the restore confirmation on the server, check su…
Ferdinand99 Nov 16, 2026
a7d037d
test(tickets): make the ported access guard tests sharper
Ferdinand99 Nov 16, 2026
ad4059e
docs(postgres): the server asks for the acknowledgement of an uploade…
Ferdinand99 Nov 16, 2026
6b649e8
fix(live-alerts): poll every batch first, search only the failed ones…
Ferdinand99 Nov 16, 2026
8f0c0f5
fix(tickets): cap the open ticket checks and make the lookup budget f…
Ferdinand99 Nov 16, 2026
c7a0bcb
fix(backup): link the docs on the default branch, test the Postgres r…
Ferdinand99 Nov 16, 2026
dcfc449
fix(live-alerts): search failed batches depth first so every poll mak…
Ferdinand99 Nov 16, 2026
f4f634a
fix(tickets): do not open a new ticket elsewhere on the strength of a…
Ferdinand99 Nov 16, 2026
27877b4
fix(live-alerts): batch servers that save rejected values apart, ask …
Ferdinand99 Nov 16, 2026
12571cc
fix(tickets): offer the remembered server in the picker and let the p…
Ferdinand99 Nov 16, 2026
47263a1
test(backups): pin db.roleRisks in the health report for app clients
Ferdinand99 Nov 16, 2026
df245c1
Merge remote-tracking branch 'origin/v5.3' into HEAD
Ferdinand99 Nov 16, 2026
c8c86e0
fix(health): show the database role warning first in its section and …
Ferdinand99 Nov 16, 2026
2a90488
fix(tickets): accept a DM menu pick only from the DM's own platform
Ferdinand99 Nov 16, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs/mobile-app.md
Original file line number Diff line number Diff line change
Expand Up @@ -617,6 +617,9 @@ its account is an operator.
is stored as a snapshot only.
The import is sent as raw `application/octet-stream` (a JSON body is cut at
256 kB) of up to 128 MB, read from the file as it goes.
A client detects that the server asks for the restore confirmation (`confirm` and
`acknowledgeSql`) by `db.roleRisks` being an array in the health report from
`GET /api/v2/health` (5.3 and later); an older server does not send it.
- **Bot Personalizer**: the bot's name and pictures, its presence, and the
announcement everyone sees.
- **Stats cache**: the game-stats lookups every server shares (read only).
Expand All @@ -638,7 +641,7 @@ its account is an operator.
| `POST /api/v2/health/dev-log-error-test` | Operators only | Logs a real error to test the pipeline; `{ ok }` |
| `POST /api/v2/health/backups` | Operators only | Takes a backup; `{ name }` |
| `GET /api/v2/health/backups/:name` | Operators only | The backup file, as a download |
| `POST /api/v2/health/backups/:name/delete`, `/restore` | Operators only | `{ ok }` (a restore then restarts the bot) |
| `POST /api/v2/health/backups/:name/delete`, `/restore` | Operators only | `{ ok }` (a restore then restarts the bot). `/restore` takes an optional `{ confirm: <name> }`, which must match when sent. An uploaded (`sylo-imported-*`) dump on Postgres needs `{ confirm: <name>, acknowledgeSql: true }` (its SQL runs with the rights of the database role), else 400 |
| `POST /api/v2/health/backups/import` | Operators only | A raw body (`application/octet-stream`, up to 128 MB); `{ name }` |
| `GET /api/v2/personalizer`, `/personalizer/presence` | Operators only | The bot; the presence and its choices |
| `POST /api/v2/personalizer/identity`, `/presence`, `/announcement`, `/announcement/clear` | Operators only | Bot-wide settings |
Expand Down
13 changes: 13 additions & 0 deletions docs/modules/tickets.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,18 @@ server. Staff reply as **Staff** from the ticket page; the member sees Sylo's DM
**Close ticket** sends a final reply (or the default closing notice if the box is
empty) and closes it. The member can open a new one by DMing again.

To find the servers a member shares with the bot, Sylo first looks at the servers where the member
has an open ticket (from its own database, so this survives a restart): a follow-up message always
goes to the open ticket. Those servers are checked with a few requests per message (5, and 10 a
minute per member); a member who left such a server is remembered as gone for ten minutes. Then it
uses its member cache: a server whose full member list is cached (Server Members intent on)
answers without any request to Discord. For the other servers it asks Discord for the one member:
at most 8 servers per message, 16 requests a minute per member and 120 a minute for the whole bot
(the last 30 are kept for members who have used little), and it remembers "not a member" for three
minutes (a join clears it at once where Sylo sees the join). A server that refuses the lookup is
skipped for ten minutes. Without the Server Members intent a member is still found this way. If a
message could not check every server yet, Sylo never opens a new ticket from it: it asks the member
to send the message again a few seconds later, which continues with the servers not checked yet.

`/forget` removes the member's ticket history: every message in their tickets,
staff replies included. Replies already delivered by DM are not clawed back.
54 changes: 53 additions & 1 deletion docs/postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ be a real bottleneck. If that's not you, skip this page —
- [Should you use this?](#should-you-use-this)
- [Enabling Postgres for a new install](#enabling-postgres-for-a-new-install)
- [Moving an existing SQLite install over](#moving-an-existing-sqlite-install-over)
- [Least-privilege database role](#least-privilege-database-role)
- [Backups and restore on Postgres](#backups-and-restore-on-postgres)
- [Rolling back](#rolling-back)
- [Troubleshooting](#troubleshooting)
Expand All @@ -35,7 +36,8 @@ Set `DATABASE_URL` before the first start:
DATABASE_URL=postgres://user:pass@host:5432/dbname
```

That's the whole setup. On first connect, Sylo creates every table it needs
Use a database role with few rights for this, not a superuser: see
[Least-privilege database role](#least-privilege-database-role). That's the whole setup. On first connect, Sylo creates every table it needs
automatically — no separate schema step, no migration to run by hand. Leave
`DATABASE_URL` unset (the default) and nothing changes: Sylo uses
`DATABASE_PATH` exactly as it always has.
Expand Down Expand Up @@ -121,6 +123,52 @@ giveaway's entries) stay intact — confirmed by inserting a new row through
Sylo's own code right after a migration and checking it didn't collide with
a copied one.

## Least-privilege database role

The role in `DATABASE_URL` should own the Sylo database and have no other rights. The reason is
backup import. A Postgres dump is a file of SQL, and **restoring one (including a file an operator
uploaded on the Health page) runs that SQL with the rights of the role Sylo connects as**:
`pg_restore` sends every statement in the file to the database as that role. Inside Sylo's own
database that adds nothing an operator cannot already do, because a restore replaces the whole
database. With a superuser role, or a role that is a member of `pg_execute_server_program`,
`pg_read_server_files` or `pg_write_server_files`, the SQL can also reach the database server's
own system (`COPY ... PROGRAM`, server files). Sylo cannot change the role from code, so set it up
once when you create the database. At start-up Sylo reads the role and logs one warning if it has
`SUPERUSER`, `CREATEROLE`, `CREATEDB` or one of those three memberships; the Health page shows the
same note.

Create the database and the role as an administrator of the Postgres server (with the official
`postgres` image that is its `POSTGRES_USER`, which is a superuser: use it for this setup only,
never in `DATABASE_URL`):

```sql
CREATE ROLE sylo LOGIN PASSWORD 'choose-a-long-random-password'
NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS;
CREATE DATABASE sylo OWNER sylo;
REVOKE ALL ON DATABASE sylo FROM PUBLIC;
-- then, connected to the new database (\c sylo):
ALTER SCHEMA public OWNER TO sylo;
```

Then point Sylo at it: `DATABASE_URL=postgres://sylo:choose-a-long-random-password@host:5432/sylo`.
Sylo creates its tables itself on first start, so the role owns them, and `pg_restore --clean`
can drop and recreate them without extra rights. To check an existing setup, connect with the same
credentials Sylo uses and run:

```sql
SELECT rolsuper, rolcreaterole, rolcreatedb FROM pg_roles WHERE rolname = current_user;
SELECT pg_has_role(current_user, 'pg_execute_server_program', 'MEMBER') AS exec_program,
pg_has_role(current_user, 'pg_read_server_files', 'MEMBER') AS read_files,
pg_has_role(current_user, 'pg_write_server_files', 'MEMBER') AS write_files;
```

Every value should be `f`. Restores are never blocked because of the role. Before it restores an
uploaded dump the Health page says that its SQL runs with the role's rights, and the server does
not restore one unless the request names the snapshot (`confirm`) and sets `acknowledgeSql: true`
(the dashboard sends both after its warning; other clients of the API must do the same). Even with a least-privilege role, a restored dump can still do anything that role may do
inside the Sylo database (create functions and triggers, change or delete any row), so restore only
files you made or trust.

## Backups and restore on Postgres

The Health page's backup/restore works the same way on both drivers — same
Expand All @@ -139,6 +187,10 @@ restores onto the driver it came from — trying to restore a `.dump` while
running on SQLite (or vice versa) fails with a clear error instead of doing
something destructive.

A restore runs the SQL in the dump with the rights of the database role Sylo connects as, so
use the [least-privilege role](#least-privilege-database-role) above and restore only dumps you
made or trust.

`pg_dump`/`pg_restore` are bundled in the Docker image, pinned to the newest
supported Postgres major version — that covers dumping from and restoring to
any older server version too (a newer client is safe against an older
Expand Down
2 changes: 1 addition & 1 deletion docs/self-hosting.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ At least one platform must be configured.
| `GAMETOOLS_API_BASE` | `https://api.gametools.network` | Stats API base URL |
| `STATS_CACHE_TTL_MINUTES` | `5` | How long stats lookups are cached |
| `DATABASE_PATH` | `./data/sylo.db` | SQLite file path |
| `DATABASE_URL` | — | Optional: a `postgres://` URL to use Postgres instead of SQLite. Hosted-scale deployments only — see [docs/postgres.md](postgres.md). Unset (default) means nothing about the setup below changes. |
| `DATABASE_URL` | — | Optional: a `postgres://` URL to use Postgres instead of SQLite. Hosted-scale deployments only — see [docs/postgres.md](postgres.md), and use a [role with few rights](postgres.md#least-privilege-database-role), not a superuser. Unset (default) means nothing about the setup below changes. |
| `BACKUP_INTERVAL_HOURS` | `24` | Scheduled DB snapshot interval; `0` disables it (pre-migration + manual still run) |
| `BACKUP_RETENTION` | `14` | How many DB snapshots to keep in `<data>/backups` |
| `BACKUP_DIR` | `<db dir>/backups` | Where DB snapshots are written |
Expand Down
72 changes: 51 additions & 21 deletions src/bot/events/dmTickets.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,10 @@
// a DM questionnaire (bot/lib/dmQuestionnaire.js), which takes their DMs first. When the
// user shares several ticket-enabled servers and has no open ticket, the bot
// asks which server with a select menu.
import { guildKey, guildByKey, platformOfClient } from '../../platform/platformOf.js';
import { guildKey, guildByKey, platformOfClient, platformOfGuild } from '../../platform/platformOf.js';
import { config } from '../../config.js';
import { Events, StringSelectMenuBuilder, ActionRowBuilder } from 'discord.js';
import { ticketGuildsForUser, ingestUserDM } from '../../modules/tickets.js';
import { getOpenTicket } from '../../db/tickets.js';
import { lookupTicketGuilds, canOpenTicketIn, forgetNotMember, ingestUserDM } from '../../modules/tickets.js';
import { registerComponent } from '../lib/components.js';
import { handleQuestionnaireMessage, looksLikeQuestionnairePrompt } from '../lib/dmQuestionnaire.js';
import { lateAnswerWindowMs } from '../../modules/applications.js';
Expand Down Expand Up @@ -70,23 +69,34 @@ async function handleDM(message) {
if (!payload.content && payload.attachments.length === 0) return;

// Only the servers on the platform this DM arrived on: a user id means nothing on the other one.
const guilds = await ticketGuildsForUser(message.author, message.client);
if (guilds.length === 0) {
const { guilds, openGuilds, openComplete, openRemembered, rememberedGuilds, incomplete } =
await lookupTicketGuilds(message.author, message.client);

// The open tickets come from the database, so they are certain. A follow-up goes to the one
// server where the member has an open ticket, however much of the rest could be checked.
if (openComplete && openGuilds.length === 1) {
await ingestUserDM(openGuilds[0], message.author, payload);
await message.react('✅').catch(() => {});
return;
}
// Some servers could not be checked yet (the lookup is capped per message). Never open a new
// ticket from that: the right server may be one of those. Ask for the message again; the
// retry continues with the servers not checked yet.
if (incomplete && openGuilds.length === 0) {
await message
.reply("I'm not set up to take messages for any server you're in right now.")
.reply('I could not check all servers yet. Please send your message again in a few seconds.')
.catch(() => {});
return;
}

// If there's exactly one open ticket already, keep the conversation there.
const openFlags = await Promise.all(guilds.map((g) => getOpenTicket(guildKey(g), message.author.id)));
const openGuilds = guilds.filter((g, i) => openFlags[i]);
if (openGuilds.length === 1) {
await ingestUserDM(openGuilds[0], message.author, payload);
await message.react('✅').catch(() => {});
if (guilds.length === 0) {
await message
.reply("I'm not set up to take messages for any server you're in right now.")
.catch(() => {});
return;
}
if (openGuilds.length === 0 && guilds.length === 1) {
// Not when an open ticket server was skipped from memory ("left"): the member may have come back
// to it, and a new ticket in the one other server could be the wrong one. They pick instead.
if (openGuilds.length === 0 && guilds.length === 1 && !openRemembered) {
await ingestUserDM(guilds[0], message.author, payload);
await message.react('✅').catch(() => {});
return;
Expand All @@ -98,7 +108,8 @@ async function handleDM(message) {
.setCustomId(SELECT_ID)
.setPlaceholder('Which server is this about?')
.addOptions(
(openGuilds.length ? openGuilds : guilds)
// A server skipped from memory ("left") is offered too: the member may have come back to it.
[...new Set([...(openGuilds.length ? openGuilds : guilds), ...rememberedGuilds])]
.slice(0, 25)
.map((g) => ({ label: g.name.slice(0, 100), value: guildKey(g) }))
);
Expand All @@ -112,13 +123,28 @@ async function handleDM(message) {

async function handleSelect(interaction) {
const picked = guildByKey(interaction.values[0]);
// Still one the member may open a ticket in (a member there, tickets on): the menu could be old.
const allowed = picked
? await ticketGuildsForUser(interaction.user, interaction.client ?? picked.client)
: [];
const guild = allowed.find((g) => guildKey(g) === guildKey(picked));
const stashed = pending.get(interaction.user.id);
if (!stashed || Date.now() - stashed.at > PENDING_TTL) {
pending.delete(interaction.user.id);
await interaction
.update({ content: 'That request expired — send your message again.', components: [] })
.catch(() => {});
return;
}
// Only a server on the platform this menu was shown on: a user id means nothing on the other one (the menu
// only offers those, so this guards against a value that was not from it). Then still one the member may
// open a ticket in (a member there, tickets on): the menu could be old.
const samePlatform = picked && platformOfGuild(picked) === platformOfClient(interaction.client);
const guild = samePlatform && (await canOpenTicketIn(interaction.user, picked)) ? picked : null;
if (!guild) {
// The message stays stashed and the menu stays up, so the member can pick again.
await interaction
.reply({ content: 'I could not confirm that server right now. Pick it again in a moment.' })
.catch(() => {});
return;
}
const payload = takeStash(interaction.user.id);
if (!guild || !payload) {
if (!payload) {
await interaction
.update({ content: 'That request expired — send your message again.', components: [] })
.catch(() => {});
Expand All @@ -137,6 +163,10 @@ registerComponent('tickets', SELECT_ID, handleSelect);

/** @param {import('discord.js').Client} client */
export function register(client, { isCommand = () => false } = {}) {
// A person who joins a server is no longer "not a member" there (Discord, with the intent on).
client.on(Events.GuildMemberAdd, (member) => {
if (member?.guild) forgetNotMember(member.id, guildKey(member.guild));
});
client.on(Events.MessageCreate, async (message) => {
try {
if (message.guild || message.author?.bot) return;
Expand Down
29 changes: 29 additions & 0 deletions src/db/backup.js
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,35 @@ export async function inspectDbFile(filePath) {
return { ok: false, error: 'not a recognised Sylo backup file (SQLite snapshot or Postgres dump)' };
}

/** Was this snapshot uploaded (Health page import) rather than made by Sylo? */
export const isImportedBackup = (name) => /^sylo-imported-/.test(String(name));

/**
* What a restore request must say before it is carried out, or null when it said enough. A
* `confirm` that is sent has to be the snapshot's name. An uploaded dump on Postgres is restored by
* running the SQL in it with the rights of the database role, so that one needs `confirm` and
* `acknowledgeSql: true`. Other snapshots need no body (the app sends none).
* @param {string} name
* @param {{ confirm?: unknown, acknowledgeSql?: unknown } | undefined} body
* @param {{ postgres: boolean }} o
* @returns {string | null}
*/
export function restoreConfirmationError(name, body, { postgres }) {
const { confirm, acknowledgeSql } = body ?? {};
if (confirm !== undefined && confirm !== name) {
return 'confirm must be the name of the snapshot to restore';
}
if (postgres && isImportedBackup(name)) {
if (confirm !== name) {
return 'Restoring an uploaded dump needs { "confirm": "<snapshot name>", "acknowledgeSql": true }';
}
if (acknowledgeSql !== true) {
return 'An uploaded Postgres dump runs its SQL with the rights of the database role: send "acknowledgeSql": true to restore it';
}
}
return null;
}

/**
* Validate an uploaded database and, if good, store it as a normal snapshot the
* operator can then restore. Returns `{ ok, name?, error? }`.
Expand Down
Loading
Loading