diff --git a/README.md b/README.md index b85a430f..fefe7843 100644 --- a/README.md +++ b/README.md @@ -89,14 +89,14 @@ tier. iOS app with push notifications for new tickets.
-All 43 modules +All 44 modules | Area | Modules | |---|---| | **Moderation & safety** | [Moderation](docs/modules/moderation.md) (case log, warnings, auto-actions) · [Auto-moderation](docs/modules/automod.md) (can sync to Discord's native AutoMod) · [Server logging](docs/modules/logging.md) · [Verification](docs/modules/verification.md) (button or Turnstile captcha) · [Ban appeals](docs/modules/appeals.md) · [Tickets / modmail](docs/modules/tickets.md) · [Applications](docs/modules/applications.md) (forms for staff to accept or deny) · [Channel cleanup](docs/modules/channel-cleanup.md) · Honeypot (beta) | | **Welcome & roles** | [Welcome & leave](docs/modules/welcome.md) · [Welcome channel](docs/modules/welcome-channel.md) · [Introductions](docs/modules/introductions.md) (membership cards) · [Reaction roles & autoroles](docs/modules/roles.md) (emoji, buttons or dropdown) · [Temporary roles](docs/modules/temp-roles.md) (beta) · [Birthdays](docs/modules/birthdays.md) | | **Engagement** | [Leveling](docs/modules/leveling.md) (image rank cards, public leaderboard) · [Economy](docs/modules/economy.md) (currency and shop, beta) · [Invite tracker](docs/modules/invite-tracker.md) · [Starboard](docs/modules/starboard.md) · [Quote board](docs/modules/quote-board.md) (beta, optional public page) · [Polls](docs/modules/polls.md) · [Giveaways](docs/modules/giveaways.md) · [Events](docs/modules/events.md) (beta) · [Suggestions](docs/modules/suggestions.md) (beta) · [Counting](docs/modules/counting.md) · [Game stats](docs/modules/game-stats.md) (Battlefield, OSRS, RS3) | -| **Automation** | [Custom commands](docs/modules/custom-commands.md) · [Autoresponder](docs/modules/autoresponder.md) · [Auto-react](docs/modules/auto-react.md) · [Auto threads](docs/modules/auto-threads.md) (beta, Discord only) · [Reminders](docs/modules/reminders.md) · [Sticky messages](docs/modules/sticky.md) · [AFK](docs/modules/afk.md) · [Temporary voice channels](docs/modules/temp-voice.md) | +| **Automation** | [Custom commands](docs/modules/custom-commands.md) · [Autoresponder](docs/modules/autoresponder.md) · [Auto-react](docs/modules/auto-react.md) · [Auto threads](docs/modules/auto-threads.md) (beta, Discord only) · [Bridge](docs/modules/bridge.md) (Discord ↔ Fluxer, beta) · [Reminders](docs/modules/reminders.md) · [Sticky messages](docs/modules/sticky.md) · [AFK](docs/modules/afk.md) · [Temporary voice channels](docs/modules/temp-voice.md) | | **Alerts** | [Twitch](docs/modules/twitch-alerts.md) · [YouTube](docs/modules/youtube-alerts.md) · [Kick](docs/modules/kick-alerts.md) · [RSS / Atom](docs/modules/rss.md) (also Reddit, Mastodon, Bluesky) · [GitHub](docs/modules/github.md) · [Free games](docs/modules/free-games.md) · [App Store Connect](docs/modules/apple-connect.md) (instance operator only) | | **Insights** | [Server statistics](docs/modules/server-stats.md) (live counter channels) · [Server insights](docs/modules/insights.md) (activity charts, aggregate counts only) | @@ -366,6 +366,7 @@ are in [CONTRIBUTING.md](CONTRIBUTING.md). | [docs/modules/](docs/modules/README.md) | One page per module: settings, commands, permissions, intents | | [docs/self-hosting.md](docs/self-hosting.md) | Install, environment variables, reverse proxy, Docker, Unraid, backups, upgrades, troubleshooting | | [docs/fluxer.md](docs/fluxer.md) | Running on Fluxer, alone or next to Discord | +| [docs/mirror.md](docs/mirror.md) | Mirroring a Discord server to a Fluxer server and keeping it in step | | [docs/mobile-app.md](docs/mobile-app.md) | The iOS app: what an instance needs, sign-in, endpoints, notifications | | [docs/dashboard-v2.md](docs/dashboard-v2.md) | The React dashboard (beta) | | [docs/postgres.md](docs/postgres.md) | Optional Postgres backend and moving an SQLite install over | diff --git a/docs/dashboard-v2.md b/docs/dashboard-v2.md index 3c55bf9f..d8ac55ed 100644 --- a/docs/dashboard-v2.md +++ b/docs/dashboard-v2.md @@ -26,7 +26,7 @@ note. ## What the dashboard shows -- **Sidebar shows only what's turned on.** Instead of listing all 43 modules +- **Sidebar shows only what's turned on.** Instead of listing all 44 modules with an on/off dot, the sidebar lists just the modules enabled for whichever server you're looking at, grouped the same way the Dashboard/Overview page groups them. Turn a module on from Overview and it appears in the sidebar @@ -36,6 +36,9 @@ note. - **A server selection follows you** onto bot-wide pages (Bot Personalizer, Health) that aren't scoped to any one server, instead of clearing when you navigate there. +- **Mirror to Fluxer.** A page in the Server group of the sidebar that copies a Discord + server to a Fluxer server and keeps it in step: ask, agree on both servers, preview, + copy, then pause or stop. See [Mirror to Fluxer](mirror.md). - **A setup wizard for new servers.** A server with fewer than three modules set up gets a "Set up Sylo" card on its Overview; the wizard can also be run again any time from **Settings → Server data → Setup wizard**. Sylo looks at diff --git a/docs/fluxer.md b/docs/fluxer.md index b9625071..06cdae38 100644 --- a/docs/fluxer.md +++ b/docs/fluxer.md @@ -238,6 +238,7 @@ does not register anything with Fluxer for these. | `/intro create` and the **Create my intro** button | **Introductions** uses `!intro create`, then asks the questions in DM, as on Discord. Sylo answers "Check your DMs" in the channel and removes that message after about 15 seconds. The **Create my intro** button of the prompt message, and **Post** and **Start over** under the preview in DM, are shown as reactions. | | `/apply start` and the **Applications** panel | **Applications** uses `!apply start
` (and `!apply withdraw`), then asks the questions in DM, as on Discord. Choice and role questions are numbered options (at most 10), answered with the numbers. The panel message, the choice between DM and web, **Submit** and **Start over** under the preview, and **Accept** and **Deny** on each application are shown as reactions. Staff give a reason with `!application accept ` or `!application deny `. A staff thread is not opened. | | `/balance`, `/daily`, `/pay` and the other **Economy** commands | **Economy** uses chat commands with the same names (`!balance`, `!daily`, `!pay @member 100`, `!eco add @member 500 reason:"Prize"`). The leaderboard is `!economy-leaderboard`; its pages turn with the ⬅️ and ➡️ reactions. The shop is `!shop` (a numbered list that pages the same way), `!buy `, `!inventory` and `!collection` (collectibles, with their pictures as embeds; the pages turn with the same reactions; with collections private it is a private reply, so it comes to your DMs); a giveaway entry fee works with the Enter reaction. A robbery result is posted in the channel as on Discord, and the victim gets the same DM. | +| A Discord channel's conversation in a Fluxer channel | The **Bridge** module shares one channel with a Discord channel through webhooks, with a one-time code both servers agree on. It is set up in the dashboard; no chat command is needed. A Discord poll shows on Fluxer as text. See [Bridge](modules/bridge.md). | | Message menu **Save quote** | Not available. The **Quote board** module saves a quote with `!quote save `, or `!quote save` as a reply to the message. | | Threads and auto-threads | Not available yet. The **Auto threads** module does nothing on Fluxer. Lock and lockdown skip the thread permissions. | | Moving members between voice | Not usable from a bot. **Temporary voice channels** are still created, but the member gets a DM with a link to join, and a channel nobody joins within two minutes is removed. | @@ -256,6 +257,9 @@ Two things to know about modules that reach outside Fluxer: - Some feed hosts rate limit shared server addresses. Reddit in particular answers with `HTTP 429` now and then. That shows up in the error log, not as a bot fault. +- A Discord server can be mirrored to a Fluxer server (channels, roles, emoji, stickers and Sylo's settings, + kept in step afterwards): see [Mirror to Fluxer](mirror.md). + --- ## Server ids diff --git a/docs/mirror.md b/docs/mirror.md new file mode 100644 index 00000000..7387ebe9 --- /dev/null +++ b/docs/mirror.md @@ -0,0 +1,200 @@ +# Mirror to Fluxer + +A server that lives on Discord can get a copy on Fluxer. Sylo reads the Discord server and builds the same +categories, channels, roles, emoji and stickers on a Fluxer server you made, then keeps the copy in step when you +change the Discord server. The copy goes one way: Discord is the source, Fluxer is the copy. + +Like every Sylo feature it is free. It needs a Sylo that runs on both platforms (a Discord login and a Fluxer +login, see [Fluxer](fluxer.md)), and Sylo has to be a member of both servers. Find it in the dashboard under +**Mirror to Fluxer**, in the Server group of the sidebar. The page also works on the Fluxer server, where it is +used to agree to a mirror and to pause or stop it. + +## Only the server owner + +**Only the owner of the Discord server and the owner of the Fluxer server can set up and run a mirror.** An +Administrator, someone with Edit access to the dashboard and a bot master cannot ask for a mirror, agree to one, +change what is mirrored, start the copy, pause, resume, stop or remove it. Everyone with dashboard access (View or +more) can look at the page, the preview and the report. + +The reason is that a mirror copies the whole server, hidden channels and all, to another place, and keeps that place +acting on it. The people who agreed are recorded by their account on each platform, and Sylo checks before every +piece of work that each server still has the owner who agreed. If a server changes hands, the mirror pauses until +the new owner has confirmed it on that server's page. + +## What you need + +1. A Fluxer server for the copy. Create an empty one: Sylo never deletes anything it did not make itself, so an + empty server gives the cleanest result. +2. Sylo in that server, with a role that has the **Administrator** permission, **at the very top of the role list**. + Sylo can only order the roles below its own. Fluxer ranks roles by position and then by age, and every new role (a + bot's too) starts at position 1, so a position number says nothing: what counts is the rank. On a fresh server where + Sylo has just been invited, its role is already the top one and nothing needs doing. If another role ranks above or + level with Sylo's, the preview refuses to start and names those roles: the Fluxer server's owner opens Server + Settings, then Roles, and drags Sylo's role to the very top (Sylo cannot move its own role), and you look again. + (Without this, a mirrored role with Manage Roles could end up above the mirrored Administrator role.) +3. To be the owner of the Discord server, and the owner of the Fluxer server to agree (or to own both). + +## The steps + +1. **Ask.** The Discord server's owner opens the Mirror page and enters the id of the Fluxer server. +2. **Agree.** Nothing is created until the owner of the Fluxer server has agreed: + - If you own both servers with your own account (you signed in with Discord and also connected Fluxer under + Account), both sides agree at once. + - Otherwise Sylo shows a one-time code, once. The Fluxer server's owner opens the dashboard for that server, + goes to **Mirror to Fluxer** and enters the code. The code works for 15 minutes, once, only for that Fluxer + server, and five wrong tries cancel the request. +3. **Preview.** The dry run reads both servers and lists what the copy would do: what it creates, what it + changes, what it skips, and everything that cannot be carried over. It creates nothing. You choose what is + copied: the name, icon and banner, roles, channels, emoji, stickers, Sylo's own settings, whether Sylo keeps the + copy in step afterwards, and whether deletions are mirrored (off by default, see below). +4. **Copy.** After you confirm, Sylo works through the list at a pace Fluxer accepts. The page shows the + progress. You can pause the copy, resume it and stop it from either server's page. +5. **Report.** Every step and what became of it, the items that failed, and the list of what could not be + carried over. + +## Linking the channels through the bridge + +The last step of a copy links every copied text and announcement channel through the [Bridge](modules/bridge.md), +so a message written on one side shows up on the other. Voice channels, categories and forums are not linked. + +- **Both owners already agreed.** The mirror is the consent: no link code is used. Each link records the two + people who agreed to the mirror (shown as "Name (server mirror)" on the Bridge page and in both audit logs), and + both channels get the usual notice, which says the link was made by the server mirror. +- **The Bridge module is switched on in both servers** if it was off, but only the first time the mirror links + channels (it has to be on for a link to carry anything). Both audit logs say so. If a server's owner switches it + off afterwards, that stays off: the mirror reports the pairs it could not link ("the Bridge module is switched + off") and does not switch it on again, and the links carry nothing. +- **Every rule of the bridge holds.** An age-restricted channel is linked only to an age-restricted channel; a + private channel only to a private one (a channel everyone can read is never linked to one that only some can); + each owner must be able to see, read and send in the channel, and Sylo needs Manage Webhooks, View Channel and + Send Messages in both; bots are off; nobody is pinged across. The privacy rule is checked again whenever a + paused link resumes and on every message: a channel that turns private later stops the link ("no longer open + to the same people"). A pair that fails a rule is skipped, the reason + is in the report ("Linking channels through the bridge" and the notes), and the rest are linked. +- **Safe to run again.** "Copy again" links only the pairs that have no link yet. A channel that an owner + unlinked on the Bridge page stays unlinked, and a pair the owners bridged by hand is never taken over or + changed (it is reported as skipped). A server can have at most 50 bridged channels this way (10 when + linking by hand), and a copy links 25 channels at a time. A channel made on Discord later (continuous mirroring) + is linked too. +- **The links follow the mirror.** While the mirror is paused (by an owner, by Sylo after a failure, or because a + server changed owner) or stopped, its links are paused, with no notice in each channel, and they run again when + the mirror does. If a server changed owner, its link stays paused until the new owner confirms it on the Bridge + page. **Removing** a mirror pauses its links and leaves them, paused by the server that removed it: only that + server's owner can resume them on the Bridge page, and the other server's owner can unlink them. While the + mirror is not running, none of its links can be resumed on the Bridge page. + +## What is copied + +| From Discord | To Fluxer | +| --- | --- | +| Server name, icon, banner | The same (each can be switched off). A missing icon or banner is left as it is on Fluxer. | +| Roles | Name, colour, hoist, mentionable, order and permissions. Managed roles (a bot's, an integration's) are not copied. | +| @everyone | Its permissions are set on the Fluxer @everyone role, without powerful permissions (Administrator, Manage Roles and the like are never copied onto @everyone). The role itself is never created or deleted. | +| Categories and channels | Text, voice, announcement, forum and media channels: name, topic, order, category, slowmode, age restriction (a server marked age-restricted gives every copied text, announcement, forum and media channel the flag; voice channels and categories carry no flag), bitrate, user limit, and the permission overwrites. A stage channel becomes a voice channel. Threads are not copied. | +| Emoji and stickers | Up to 500 of each, within 512 KB per emoji and per sticker. Lottie stickers are not copied. | +| Sylo's own settings | The settings and on/off state of each module (not the operator's own modules, and not the bridge), with channels and roles pointing at the new ones. See the checks below. A module that already has settings on the Fluxer server is left alone. | + +Not copied: members (they join Fluxer themselves), the roles people hold, message history, threads, permission +overwrites for one specific member (accounts are not linked between the platforms), and anything Fluxer has no +counterpart for. + +### Never more open than on Discord + +A channel that @everyone cannot see on Discord is hidden from @everyone on Fluxer by an overwrite of its own, +whatever else is copied: with roles switched off, or if the @everyone step fails, a private channel is still private +(Fluxer's @everyone can see channels by default). A voice channel that @everyone cannot join also denies Connect. + +### Sylo's settings get the dashboard's checks + +The copied settings pass the same rules as saving them in the dashboard. A role with powerful permissions is not +handed out by the copied settings (autoroles, reaction roles, level rewards, the verified role, the birthday role, +command actions): that part is left out and the report says so. A giveaway that pings @here or @everyone is copied +with the ping off. The economy module comes over switched off (its settings are copied, so you can look at the income and the shop +first). Other ids that mean nothing on Fluxer (the ids of people, of messages, of things that were not +copied) are cleared, and the report counts them. Set them again on the Fluxer server if you need them. + +### Permissions, bit by bit + +Fluxer's permissions are laid out like Discord's, so most map one to one, including the thread permissions (Manage +Threads, Create Public and Private Threads, Send Messages in Threads). Discord permissions that Fluxer does not +have are dropped, and the preview and the report name them wherever they were dropped: View Server Insights, +Use Application Commands, Request to Speak, Manage Events, Use Activities, View Creator Analytics, Use +Soundboard, Create Events, Use External Sounds, Send Voice Messages, Set Voice Channel Status, Create Polls and Use +External Apps. Two Fluxer permissions have no Discord counterpart (Update RTC Region and View Channel Members): the +mirror never touches them, so a role or channel that already holds them on Fluxer keeps them. + +A role with Administrator on Discord gets Administrator on Fluxer (except on @everyone); the report points it out. + +## Keeping the copy in step + +With **Keep the Fluxer server in step afterwards** on, a change on Discord is queued and applied on Fluxer: +a channel or role created, renamed, moved or deleted, a permission overwrite changed, an emoji or sticker added, +renamed or removed, and the server's name, icon or banner. Many changes to one item in a short time become one +update, and a drag that moves many channels becomes one reorder. + +- The copy goes **one way**. Sylo reads Discord and writes Fluxer; it does not read Fluxer back. A change you make + on Fluxer to a mirrored channel or role is overwritten the next time that item changes on Discord. Changing a + mirrored category also changes the channels on Fluxer that are synced with it, including ones you put there + yourself. +- **Deleting on Discord does not delete on Fluxer by default.** The Fluxer copy is locked instead: a channel is hidden + from @everyone and made read-only (its messages are kept: deleting a Fluxer channel deletes its messages), a + role loses its permissions, and it is listed on the Mirror page as "Removed on Discord" for the Fluxer owner to + look at and delete. Only with **Delete on Fluxer what is deleted on Discord** switched on is the copy really + deleted (with its messages). The option is off by default and carries a warning. The Discord owner can only *ask* for + it: it stays off until the owner of the Fluxer server allows it on their Mirror page, because deleting a Fluxer channel + destroys messages that exist only there. The Fluxer owner can switch it off again, and so can the Discord owner. +- **A burst of deletions pauses the mirror.** More than five deletions on Discord within ten minutes (a nuke, a + mistaken clean-up) pause the mirror, with the reason in Health on both servers. The deletions wait in the queue and + are applied only when the owner confirms in the dashboard, in a dialog that lists what will be locked or deleted + (with real deletion on, only the Fluxer owner can confirm). The same confirmation is asked when a paused mirror is + resumed with more than five deletions waiting. +- Sylo **never deletes something on Fluxer that it did not create.** A Fluxer channel that was there before the + mirror stays, even with the same name as a copied one. Only what Sylo made (and what Sylo's id map shows as made + by Sylo) is ever locked or deleted, and only when its Discord original is deleted. +- Channels and roles that were already on the Fluxer server are never adopted, with one exception: a job that was + running when Sylo stopped, or whose answer was lost in a timeout, looks for what it may have made just before + (matching, and made since the job started) and adopts it, as something Sylo may not delete. +- Settings of Sylo's modules are copied once with the copy and when you copy again; they are not followed. + +## Safe by design + +- **Only the owners**, who agreed on each side and are checked again before every piece of work (see above). Either + owner can pause or stop the mirror at any time; a mirror one owner paused can be resumed only by that owner. + Stopping removes the agreement; the copy that exists stays. +- **Per server.** Each mirror has its own queue, pace and failure count: one mirror going wrong never slows another. + A few mirrors run at a time so that mirror work cannot starve the bridge and the other modules. +- **Paced.** Sylo spaces its calls to Fluxer, waits as long as Fluxer asks when it says to slow down, and backs + off with a growing delay when Fluxer cannot be reached. +- **Resumable, and no second copy.** The work is a queue in the database. After a restart (or a crash) it goes on where + it stopped. Only a real "not found" from Fluxer means a copy is gone: a timeout, a rate limit or a server error + never makes a second one, and a role created but not finished is tracked and updated, not made again. +- **A failing mirror pauses itself** after a few failures in a row, with the reason, and shows up in + [Health](dashboard-v2.md). Fix the cause (for example give Sylo's role the missing permission on Fluxer), then + press **Resume**. A single item Fluxer refuses (a name it does not accept) is reported and the copy goes on. +- **The role order is exact, and powerful permissions come last.** Sylo sends a distinct position to every mirrored + role, in Discord's order, all under its own role. A new role is first made without its powerful permissions + (Administrator, Manage Roles and the like), because a new role starts at the bottom where a member with Manage Roles + could hand it out; only after the order is set does Sylo give it its permissions. If a role ranks above Sylo's (the + order cannot be set) the mirror pauses with the roles named, and the new role stays without its permissions until the + Fluxer owner has fixed it and the mirror is resumed. +- **Limits.** Sylo plans with Fluxer's defaults (500 emoji and 500 stickers per server, 512 KB per emoji and per + sticker). If Fluxer refuses more because of its own limit, Sylo stops that kind and lists the rest in the report. + +## Who can do what + +Looking at the page, the preview and the report needs View access. Everything that changes anything needs the +server's owner (see the top), on the server the action is made on. Every action looks only at the mirror the server +is part of, so a code or an id of another server reaches nothing. + +## For the app + +A Sylo that has the mirror lists `mirror` in the `features` of `GET /api/mobile/info`. See the +[mobile app API](mobile-app.md) for the routes. + +## Self-hosting + +Nothing needs setting up besides both platforms' logins. The tables are created by the usual migration +(SQLite) or on start (Postgres); they hold the pair, an id map between the two servers and the queue, and are +removed with a server's data when Sylo leaves a server or the data is deleted. With sign-in switched off (a +private instance with no logins) there are no accounts to check, so the owner rule is not enforced there, as +Edit access is not. diff --git a/docs/mobile-app.md b/docs/mobile-app.md index d03cd0c3..6090a6f6 100644 --- a/docs/mobile-app.md +++ b/docs/mobile-app.md @@ -317,7 +317,7 @@ in there: | `login` | Whether sign-in is available (false in open mode) | | `loginProviders` | The configured logins, for example `["discord", "fluxer"]`; empty when `login` is false | | `platforms` | The platforms the bot runs on, for example `["discord"]` | -| `features` | What this version's app API supports, so the app can hide what an older Sylo lacks. Today: `app-login`, `live-access`, `changelog`, `collectibles`, `welcome-trigger`, `member-search`, `access-levels`, `health`, `app-connect`, `setup-wizard`, `config-history`, `inbox-search`, `config-import`, `duplicate`, `message-templates` and `message-move`, plus `push` and `push-pigeon` when this instance can send notifications. See the list below | +| `features` | What this version's app API supports, so the app can hide what an older Sylo lacks. Today: `app-login`, `live-access`, `changelog`, `collectibles`, `welcome-trigger`, `member-search`, `access-levels`, `health`, `app-connect`, `setup-wizard`, `config-history`, `inbox-search`, `config-import`, `duplicate`, `message-templates`, `message-move`, `bridge` and `mirror`, plus `push` and `push-pigeon` when this instance can send notifications. See the list below | The feature strings: @@ -338,6 +338,8 @@ The feature strings: - `duplicate`: the `duplicate` routes for reminders, embed messages, reaction-role sets, custom commands and shop items. - `message-templates`: the built-in message templates and the raw message JSON import and export. - `message-move`: when the channel of a posted **embed message**, the **Welcome channel** message or a **role panel** changes and is published, the server posts a fresh message in the new channel, stores it and then deletes the old one, so the app must not unpublish first. Other bot posts (verification, honeypot traps, applications and so on) are not covered. A publish that loses a race with another publish of the same embed message answers `409` with the current record. +- `bridge`: the [Bridge module](modules/bridge.md): `GET /api/v2/guilds/:id/modules/bridge/config` (the links, waiting requests and channels of the server; View level, with `owner` saying whether the caller may change anything) and, for the server owner only (Edit access is not enough: a `403` otherwise), `POST` to `…/modules/bridge/request` (a code for another server), `…/modules/bridge/accept` (use a code), `…/modules/bridge/requests/:id/cancel`, and `…/modules/bridge/links/:id/options`, `…/pause`, `…/resume` and `…/unlink`. Every answer carries the fresh `overview`. +- `mirror`: the [Mirror to Fluxer](mirror.md) routes under `/api/v2/guilds/:id/mirror`: `GET` the state (the pair, its progress and the Fluxer servers the person manages), `GET …/preview` (the dry run) and `GET …/report`, and `POST …/request`, `…/accept`, `…/options`, `…/deletes` (the Fluxer server's owner allows real deletion that the Discord owner asked for), `…/start`, `…/pause`, `…/resume`, `…/stop` and `…/remove`. Reading needs View; changing anything needs Edit and, on top of that, the owner of the server the call is made on (an Administrator is not enough): any other person gets `403` with `code: "owner-only"`. The state answer carries `isOwner`, so the app can disable the controls. - `app-connect`: [adding the other login](#adding-the-other-login). Only where sign-in is on and both Discord and Fluxer logins are set up. - `push`, `push-pigeon`: this instance can send notifications to signed-in apps. diff --git a/docs/modules/README.md b/docs/modules/README.md index 5fc9c316..d501b7f3 100644 --- a/docs/modules/README.md +++ b/docs/modules/README.md @@ -43,6 +43,7 @@ platform gaps (native AutoMod rules, threads, moving members between voice). | [Quote board](quote-board.md) | Save memorable messages (message menu or `/quote save`), show them with `/quote`, optional public page (beta) | off | — | | [Introductions](introductions.md) | Members answer a few questions in DM and Sylo posts a membership card in the introductions channel (beta) | off | Server Members | | [Applications](applications.md) | Forms members fill in, in DM or on the web, for staff to accept or deny, with requirements, roles and messages (beta) | off | Server Members | +| [Bridge](bridge.md) | Share one conversation between a Discord channel and a Fluxer channel: a one-time code both servers agree on, webhooks, replies, files, edits and deletes, no pings across (beta) | off | Message Content | | [Economy](economy.md) | A currency members earn by chatting and in voice, keep in cash or a bank, pay each other and compete for; optional robbing; staff tools (beta) | off | Message Content | | [AFK](afk.md) | Members mark themselves away; Sylo replies to mentions | off | — | | [Temporary voice channels](temp-voice.md) | Join-to-create voice hubs controlled with `/voice-*` | off | — | diff --git a/docs/modules/bridge.md b/docs/modules/bridge.md new file mode 100644 index 00000000..f49d3e65 --- /dev/null +++ b/docs/modules/bridge.md @@ -0,0 +1,124 @@ +# Bridge (beta) + +Share one conversation between a channel on **Discord** and a channel on **Fluxer**. Members on both +sides write as they always do; Sylo shows each message on the other side with the author's name and +picture, and carries replies, files, edits and deletes along. + +This is Sylo's own implementation. A bridge needs a Sylo that is connected to both platforms (the hosted +Sylo is) and that is a member of both servers. + +**Dashboard:** `/v2/guilds//m/bridge` - the linked channels, a form to ask for a link, a form to use a +code, what this server shares, and pause, resume and unlink. + +## Who can use it: the server owner + +**Only the owner of a server can use the bridge** (the Discord server owner, the Fluxer server owner), on +both sides. Administrators, people with **Edit** access to the dashboard and the bot-master roles cannot ask +for a link, use a code, change what a server shares, or pause, resume or unlink a bridge. People with **View** +access see the status of the links on the page, read-only. Ownership is checked again on every action, from +the owner the platform reports for the server. + +Sylo records who owns each server when the link is made. **If the owner of either server changes, the link +pauses** (on both sides, with a notice and an audit entry) **until the new owner confirms it**: the new +owner opens the Bridge page and presses **Resume**. If both servers changed owner, both owners have to confirm. +A code that was made by a previous owner is refused. Messages check this as they come, and a check every five +minutes covers a quiet link. + +## Linking: both servers agree + +A link is never made from one side alone. + +1. In the first server, the owner opens the Bridge page, picks a channel and presses + **Get a code**. The code (`XXXX-XXXX-XXXX`) works once and for **15 minutes**. Sylo stores only a hash of + it, so it is shown this one time. +2. They give the code to the owner of the other server. That owner opens the Bridge + page there, pastes the code, picks the channel that should join it and presses **Link the channels**. +3. Sylo makes a webhook in each channel and posts a notice in both: "Bridge on. Messages in this channel are + shared with #chat in Fluxer Home on Fluxer ...". + +A link joins a Discord channel to a Fluxer channel, never two on the same platform. A channel is in one link +at a time, and a server has at most 10 links and 5 waiting requests. The Bridge module must be on in both +servers. Wrong codes are rate limited, and a few wrong ones in a row lock the attempts for a while. + +**Either side can unlink at any time** (the owner). Both webhooks are deleted, the message map goes, and +both channels are told. If Sylo is removed from a server, the server on the other end loses its webhook and +is told too. + +**Links made by the server mirror.** A [server mirror](../mirror.md) links its copied text channels through the +bridge without codes, because both owners already agreed to the mirror. Those links show "Name (server mirror)" +as who made them, follow the mirror (paused with it, running again with it; the Bridge page cannot resume one on +its own while the mirror is paused) and follow every rule on this page. A server can have 50 of them (10 when you +link by hand). Unlink works for them as for any other link, and an unlinked channel is not linked again. If the +mirror is removed, its links stay, paused by the server that removed it: only that server's owner can resume +them, and either owner can unlink them. The mirror switches this module on only the first time it links, and a +server that switches it off keeps it off. + +**Pause** stops the bridge from either side. Only the server that paused it can resume it. A link that Sylo +stopped itself (see below) can be resumed by either server once the cause is fixed. + +## What crosses + +| | | +|---|---| +| Messages | Posted by a webhook in the other channel under the author's display name and picture, with the platform in the name ("Alice [Discord]"). | +| Replies | A reply starts with a short quote of the message it answers, with a link to it on the other side when Sylo knows the copy. | +| Files | Re-uploaded up to **8 MB** each and 8 MB per message (at most 10 files). Larger files, files that cannot be fetched, and every file when a server turns uploads off, cross as a link. | +| Emoji | Unicode emoji as they are. A custom emoji crosses as the custom emoji **with the same name** in the other server, otherwise as `:name:`. | +| Stickers | As a picture link. | +| Embeds | Rich embeds (from bots) with their text made harmless. A link preview is left out: the other platform makes its own. | +| Polls | As text: the question and the answers. Fluxer has no polls and a vote cannot count on both sides, so the vote stays where the poll is. | +| Edits and deletes | Mirrored through a message map that keeps a message for **30 days** (the daily retention job removes older rows). An edit or delete of an older message is not followed. Deleting a copy does not delete the original. | +| Joins | Optional, per server: a line "👋 Zed joined Discord Home" on the other side. Needs the Server Members intent on Discord. | + +## Safety rules + +- **No ping crosses.** Every copy is sent with allowed mentions set to none, and the text itself is changed: + `@everyone` and `@here` get an invisible character that breaks them, and users, roles and channels become + plain names ("@Bobby", "@moderators", "#rules"). +- **No loops or echoes.** A message posted by any Sylo bridge webhook, or by Sylo itself, is never relayed. + A channel is in only one link, so a copy cannot find its way back. +- **Webhook messages never cross**, whatever the options say: they are what bridges post (this one and any other + bridge on the same channels), and relaying them would make two bridges feed each other forever. Bot accounts + whose name already carries a platform tag are skipped too. +- **Bots are off by default.** Each server decides for itself what it sends: **bot messages** (bot accounts), **who + joins** and **copy files** are options of that server's side only, and the other server cannot change them. + System messages (joins, pins, boosts) are never relayed. +- **Age restriction.** An age-restricted channel can only be linked to an age-restricted channel. On Fluxer a + channel counts as age-restricted when its own setting, its category's or the server's says so (what Sylo cannot + read counts as restricted); a Discord server marked age-restricted counts for all its channels. It is checked + when linking, when resuming, and on **every message**: if one of the channels changes, the link stops. +- **Visible notice.** Both channels get a notice while a bridge is on, and when it is paused, stopped or + ended. +- **Rate limits.** Each webhook has its own queue: sends are spaced out, a rate limit from the platform is + waited out, a short outage is retried with growing pauses, and a message that waited more than five + minutes is dropped. A queue holds at most 100 messages; a flooded channel drops the newest and counts it. + One server's queue never slows another's. +- **A failing link stops.** No permission, a webhook Sylo cannot make again, a deleted channel, or five + failed deliveries in a row stop the link. Sylo says so in both channels, the **Health** page shows it as a + problem with the reason, and it is not retried until someone presses **Resume**. +- **Webhooks are Sylo's.** Sylo makes and manages the webhook of each channel and makes a new one if it is + deleted. Its token is stored sealed (AES-256-GCM, keyed from the session secret) and never sent to the + browser. If the session secret changes, the tokens cannot be read and Sylo makes new webhooks (also when it + only wanted to edit a copy). A plain-text token from a development database is sealed the first time its link + is read; accepting plain text at all is removed in a later version. +- **You can only bridge what you can see.** The person who asks for a link, and the one who uses a code, must + be able to view, read and send in their channel themselves (an owner normally can). +- **Deletes always follow.** A copy is removed even if the link is paused or stopped, when the delete comes + before the copy was posted, after a webhook was replaced (Sylo then deletes it as the bot, which needs Manage + Messages), and for a purge of many messages. After an unlink the message map is gone, so copies already + posted stay and a later delete of the original no longer reaches them. +- **File downloads share one budget** (4 files held at a time and 32 MB for the whole Sylo): a file is held from + its download until its copy is sent, and a file that does not fit crosses as a link. +- **A delete that hits a temporary error is tried again** (with growing pauses), and the message map entry stays + until the copy is really gone. + +## Needs + +- **Manage Webhooks**, View Channel and Send Messages in each bridged channel (the Health page checks them). +- The **Message Content** intent (`INTENT_MESSAGE_CONTENT`, on by default). +- Being the owner of the server that makes the request and of the server that uses the code. + +## API + +The Sylo app can read and change a bridge through `/api/v2` (`bridge` in the app's feature list). A GET at +View level, everything else for the server owner only (Edit access is not enough). See [mobile-app.md](../mobile-app.md). diff --git a/src/bot/events/mirror.js b/src/bot/events/mirror.js new file mode 100644 index 00000000..cb4224d1 --- /dev/null +++ b/src/bot/events/mirror.js @@ -0,0 +1,74 @@ +// Feeds the server mirror (src/mirror/): every change to a Discord server's channels, roles, emoji, stickers +// and its name or icon is passed to the engine, which queues work only when that server is the source of a +// running mirror. Also runs the engine's timer and, once at start, puts work that a restart interrupted +// back in the queue. Nothing here writes to Discord: the mirror only reads it. +import { Events } from 'discord.js'; +import { guildKeyOf } from '../../platform/platformOf.js'; +import { log } from '../../lib/log.js'; +import { every } from '../../modules/schedule.js'; +import { mirror } from '../../mirror/index.js'; + +/** How often the engine looks for pairs with work to do. */ +const TICK_MS = 3000; + +/** @param {import('discord.js').Client} client */ +export function register(client) { + const note = (entity, change) => { + const key = guildKeyOf(entity); + if (!key) return; + mirror + .noteSourceChange(key, change) + .catch((err) => log.error('mirror', `could not queue a ${change.kind} change:`, err)); + }; + const positionChanged = (a, b) => a.rawPosition !== b.rawPosition || a.parentId !== b.parentId; + + client.on(Events.ChannelCreate, (ch) => + note(ch, { kind: 'channel', op: 'sync', id: ch.id, reorder: 'channels' }) + ); + client.on(Events.ChannelUpdate, (oldCh, newCh) => + note(newCh, { + kind: 'channel', + op: 'sync', + id: newCh.id, + ...(positionChanged(oldCh, newCh) ? { reorder: 'channels' } : {}), + }) + ); + client.on(Events.ChannelDelete, (ch) => note(ch, { kind: 'channel', op: 'delete', id: ch.id })); + + client.on(Events.GuildRoleCreate, (role) => + note(role, { kind: 'role', op: 'sync', id: role.id, reorder: 'roles' }) + ); + client.on(Events.GuildRoleUpdate, (oldRole, newRole) => + note(newRole, { + kind: 'role', + op: 'sync', + id: newRole.id, + ...(oldRole.rawPosition !== newRole.rawPosition ? { reorder: 'roles' } : {}), + }) + ); + client.on(Events.GuildRoleDelete, (role) => note(role, { kind: 'role', op: 'delete', id: role.id })); + + client.on(Events.GuildEmojiCreate, (e) => note(e, { kind: 'emoji', op: 'sync', id: e.id })); + client.on(Events.GuildEmojiUpdate, (_old, e) => note(e, { kind: 'emoji', op: 'sync', id: e.id })); + client.on(Events.GuildEmojiDelete, (e) => note(e, { kind: 'emoji', op: 'delete', id: e.id })); + + client.on(Events.GuildStickerCreate, (s) => note(s, { kind: 'sticker', op: 'sync', id: s.id })); + client.on(Events.GuildStickerUpdate, (_old, s) => note(s, { kind: 'sticker', op: 'sync', id: s.id })); + client.on(Events.GuildStickerDelete, (s) => note(s, { kind: 'sticker', op: 'delete', id: s.id })); + + client.on(Events.GuildUpdate, (oldGuild, newGuild) => { + if ( + oldGuild.name === newGuild.name && + oldGuild.icon === newGuild.icon && + oldGuild.banner === newGuild.banner + ) + return; + note(newGuild, { kind: 'guild', op: 'sync', id: 'identity' }); + }); + + // Work that was running when the process last stopped waits in the queue again. + client.once(Events.ClientReady, () => { + mirror.resumeAfterRestart().catch((err) => log.error('mirror', 'could not resume the queue:', err)); + }); + every('mirror', TICK_MS, () => mirror.tick()); +} diff --git a/src/db/bridge.js b/src/db/bridge.js new file mode 100644 index 00000000..5a820d01 --- /dev/null +++ b/src/db/bridge.js @@ -0,0 +1,524 @@ +// Bridge storage: the links between a Discord channel and a Fluxer channel, the one-time codes that +// start a link, and the map from a message to its copy on the other side. +// +// A link has two sides. Side "a" is the server that asked for the link (it made the code), side "b" +// is the server that accepted it with the code. Each side has its own channel, its own options (what +// leaves THAT server) and its own webhook, so each server controls what it sends and neither side can +// change what the other shares. +// +// bridge_links guild_a/channel_a, guild_b/channel_b server keys (a Discord id, or "fluxer:") +// status active | paused (a server paused it) | failed (stopped by Sylo) +// paused_side 'a' or 'b' when status is paused: only that server may resume it +// options_x JSON text, see normaliseOptions in modules/lib/bridgeText.js +// hook_x_id / hook_x_token the webhook Sylo made in that channel. The token is a +// secret: it is stored sealed (AES-256-GCM, lib/seal.js) and never leaves +// this table (the dashboard only gets "has a webhook"). A token that cannot +// be opened reads as empty, and Sylo makes a new webhook. +// requested_by_name / accepted_by_name display names for the audit trail, not user ids +// bridge_codes a pending request: only the SHA-256 of the code is stored, it expires, and it is +// used up by the first server that accepts it +// bridge_messages a_message_id <-> b_message_id for edits, deletes and replies; swept after 30 days +// +// The server keys are in `guild_a` / `guild_b`, not `guild_id`, so purgeBridgeGuild() below deletes by +// either side (the guild-leave purge in purge.js calls it). bridge_codes has a plain guild_id and is in +// GUILD_TABLES. +import { createHash, randomBytes } from 'node:crypto'; +import { prepare, registerPostgresBootstrap } from './driver.js'; +import { config } from '../config.js'; +import { isSealed, open, seal } from '../lib/seal.js'; + +registerPostgresBootstrap(` + CREATE TABLE IF NOT EXISTS bridge_links ( + id SERIAL PRIMARY KEY, + guild_a TEXT NOT NULL, + channel_a TEXT NOT NULL, + guild_b TEXT NOT NULL, + channel_b TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'active', + paused_side TEXT NOT NULL DEFAULT '', + fail_reason TEXT NOT NULL DEFAULT '', + failure_count INTEGER NOT NULL DEFAULT 0, + options_a TEXT NOT NULL DEFAULT '{}', + options_b TEXT NOT NULL DEFAULT '{}', + hook_a_id TEXT NOT NULL DEFAULT '', + hook_a_token TEXT NOT NULL DEFAULT '', + hook_b_id TEXT NOT NULL DEFAULT '', + hook_b_token TEXT NOT NULL DEFAULT '', + requested_by_name TEXT NOT NULL DEFAULT '', + accepted_by_name TEXT NOT NULL DEFAULT '', + owner_a TEXT NOT NULL DEFAULT '', + owner_b TEXT NOT NULL DEFAULT '', + created_at BIGINT NOT NULL, + updated_at BIGINT NOT NULL + ); + CREATE UNIQUE INDEX IF NOT EXISTS idx_bridge_links_a ON bridge_links (guild_a, channel_a); + CREATE UNIQUE INDEX IF NOT EXISTS idx_bridge_links_b ON bridge_links (guild_b, channel_b); + CREATE INDEX IF NOT EXISTS idx_bridge_links_guild_b ON bridge_links (guild_b); + CREATE TABLE IF NOT EXISTS bridge_codes ( + id SERIAL PRIMARY KEY, + guild_id TEXT NOT NULL, + channel_id TEXT NOT NULL, + code_hash TEXT NOT NULL, + options TEXT NOT NULL DEFAULT '{}', + requested_by_name TEXT NOT NULL DEFAULT '', + guild_owner TEXT NOT NULL DEFAULT '', + expires_at BIGINT NOT NULL, + created_at BIGINT NOT NULL + ); + CREATE UNIQUE INDEX IF NOT EXISTS idx_bridge_codes_hash ON bridge_codes (code_hash); + CREATE INDEX IF NOT EXISTS idx_bridge_codes_guild ON bridge_codes (guild_id); + CREATE TABLE IF NOT EXISTS bridge_messages ( + link_id INTEGER NOT NULL, + origin TEXT NOT NULL, + a_message_id TEXT NOT NULL, + b_message_id TEXT NOT NULL, + copy_hook_id TEXT NOT NULL DEFAULT '', + content_hash TEXT NOT NULL DEFAULT '', + extra TEXT NOT NULL DEFAULT '{}', + created_at BIGINT NOT NULL, + PRIMARY KEY (link_id, a_message_id) + ); + CREATE UNIQUE INDEX IF NOT EXISTS idx_bridge_messages_b ON bridge_messages (link_id, b_message_id); + CREATE INDEX IF NOT EXISTS idx_bridge_messages_created ON bridge_messages (created_at); +`); + +/** How long a message map row is kept: the window in which an edit, a delete or a reply still follows. */ +export const MESSAGE_MAP_TTL_MS = 30 * 86_400_000; + +const s = { + insertLink: prepare( + `INSERT INTO bridge_links + (guild_a, channel_a, guild_b, channel_b, status, options_a, options_b, hook_a_id, hook_a_token, + hook_b_id, hook_b_token, requested_by_name, accepted_by_name, owner_a, owner_b, created_at, updated_at) + VALUES (@guildA, @channelA, @guildB, @channelB, 'active', @optionsA, @optionsB, @hookAId, @hookAToken, + @hookBId, @hookBToken, @requestedBy, @acceptedBy, @ownerA, @ownerB, @now, @now)`, + { returningId: true } + ), + link: prepare('SELECT * FROM bridge_links WHERE id = ?'), + linksOfGuild: prepare('SELECT * FROM bridge_links WHERE guild_a = ? OR guild_b = ? ORDER BY id'), + linkByChannelA: prepare('SELECT * FROM bridge_links WHERE guild_a = ? AND channel_a = ?'), + linkByChannelB: prepare('SELECT * FROM bridge_links WHERE guild_b = ? AND channel_b = ?'), + linkByHook: prepare( + "SELECT * FROM bridge_links WHERE (hook_a_id = @id AND hook_a_id <> '') OR (hook_b_id = @id AND hook_b_id <> '')" + ), + countLinks: prepare('SELECT COUNT(*) AS n FROM bridge_links WHERE guild_a = ? OR guild_b = ?'), + setStatus: prepare( + `UPDATE bridge_links SET status = @status, paused_side = @pausedSide, fail_reason = @reason, + failure_count = @failures, updated_at = @now WHERE id = @id` + ), + setOptionsA: prepare('UPDATE bridge_links SET options_a = @options, updated_at = @now WHERE id = @id'), + setOptionsB: prepare('UPDATE bridge_links SET options_b = @options, updated_at = @now WHERE id = @id'), + setOwnerA: prepare('UPDATE bridge_links SET owner_a = @owner, updated_at = @now WHERE id = @id'), + setOwnerB: prepare('UPDATE bridge_links SET owner_b = @owner, updated_at = @now WHERE id = @id'), + resealA: prepare('UPDATE bridge_links SET hook_a_token = @token WHERE id = @id AND hook_a_token = @plain'), + resealB: prepare('UPDATE bridge_links SET hook_b_token = @token WHERE id = @id AND hook_b_token = @plain'), + setHookA: prepare( + 'UPDATE bridge_links SET hook_a_id = @id2, hook_a_token = @token, updated_at = @now WHERE id = @id' + ), + setHookB: prepare( + 'UPDATE bridge_links SET hook_b_id = @id2, hook_b_token = @token, updated_at = @now WHERE id = @id' + ), + delLink: prepare('DELETE FROM bridge_links WHERE id = ?'), + delLinkMessages: prepare('DELETE FROM bridge_messages WHERE link_id = ?'), + activeLinks: prepare("SELECT * FROM bridge_links WHERE status = 'active'"), + linkIdsOfGuild: prepare('SELECT id FROM bridge_links WHERE guild_a = ? OR guild_b = ?'), + delLinksOfGuild: prepare('DELETE FROM bridge_links WHERE guild_a = ? OR guild_b = ?'), + + insertCode: prepare( + `INSERT INTO bridge_codes (guild_id, channel_id, code_hash, options, requested_by_name, guild_owner, expires_at, created_at) + VALUES (@guildId, @channelId, @hash, @options, @requestedBy, @guildOwner, @expiresAt, @now)`, + { returningId: true } + ), + codeByHash: prepare('SELECT * FROM bridge_codes WHERE code_hash = ? AND expires_at > ?'), + codesOfGuild: prepare( + 'SELECT id, guild_id, channel_id, requested_by_name, expires_at, created_at FROM bridge_codes WHERE guild_id = ? AND expires_at > ? ORDER BY id' + ), + countCodes: prepare('SELECT COUNT(*) AS n FROM bridge_codes WHERE guild_id = ? AND expires_at > ?'), + codeChannel: prepare( + 'SELECT id FROM bridge_codes WHERE guild_id = ? AND channel_id = ? AND expires_at > ?' + ), + delCode: prepare('DELETE FROM bridge_codes WHERE id = ?'), + delCodeOfGuild: prepare('DELETE FROM bridge_codes WHERE id = ? AND guild_id = ?'), + delExpiredCodes: prepare('DELETE FROM bridge_codes WHERE expires_at <= ?'), + + insertMap: prepare( + `INSERT INTO bridge_messages (link_id, origin, a_message_id, b_message_id, copy_hook_id, content_hash, extra, created_at) + VALUES (@linkId, @origin, @aId, @bId, @hookId, @hash, @extra, @now) + ON CONFLICT (link_id, a_message_id) DO UPDATE SET + origin = excluded.origin, b_message_id = excluded.b_message_id, + copy_hook_id = excluded.copy_hook_id, content_hash = excluded.content_hash, + extra = excluded.extra` + ), + mapByA: prepare('SELECT * FROM bridge_messages WHERE link_id = ? AND a_message_id = ?'), + mapByB: prepare('SELECT * FROM bridge_messages WHERE link_id = ? AND b_message_id = ?'), + setHash: prepare( + 'UPDATE bridge_messages SET content_hash = ? WHERE link_id = ? AND a_message_id = ? AND b_message_id = ?' + ), + delMapByA: prepare('DELETE FROM bridge_messages WHERE link_id = ? AND a_message_id = ?'), + delMapByB: prepare('DELETE FROM bridge_messages WHERE link_id = ? AND b_message_id = ?'), + sweepMap: prepare('DELETE FROM bridge_messages WHERE created_at < ?'), + countMap: prepare('SELECT COUNT(*) AS n FROM bridge_messages WHERE link_id = ?'), +}; + +// --- links ----------------------------------------------------------------------------------------- + +// A webhook token is a bearer credential for the channel, so it is sealed in the table. A plain value (no +// v1. prefix) is sealed in place the first time its row is read (readLink below), and until then read as it +// is. No release ever wrote plain tokens, so this only covers development and test databases. +// TODO(5.4): drop the plain branch of openToken and the resealing, so a plain value can never persist. +const TOKEN_PURPOSE = 'bridge-webhook'; +const sealToken = (token) => (token ? seal(config.sessionSecret, TOKEN_PURPOSE, token) : ''); +const openToken = (stored) => { + if (!stored) return ''; + return isSealed(stored) ? (open(config.sessionSecret, TOKEN_PURPOSE, stored) ?? '') : stored; +}; + +const parseJson = (text) => { + try { + const value = JSON.parse(text); + return value && typeof value === 'object' && !Array.isArray(value) ? value : {}; + } catch { + return {}; + } +}; + +/** A stored row as the code uses it (numbers as numbers, options parsed). */ +export function linkFrom(row) { + if (!row) return null; + return { + id: Number(row.id), + guildA: row.guild_a, + channelA: row.channel_a, + guildB: row.guild_b, + channelB: row.channel_b, + status: row.status, + pausedSide: row.paused_side || '', + failReason: row.fail_reason || '', + failureCount: Number(row.failure_count) || 0, + optionsA: parseJson(row.options_a), + optionsB: parseJson(row.options_b), + hookAId: row.hook_a_id || '', + hookAToken: openToken(row.hook_a_token), + hookBId: row.hook_b_id || '', + hookBToken: openToken(row.hook_b_token), + requestedBy: row.requested_by_name || '', + acceptedBy: row.accepted_by_name || '', + // The platform user id of each server's owner when the link was made (or last confirmed). The bridge is + // for the server owners only, so a change of owner pauses the link until the new owner confirms. + ownerA: row.owner_a || '', + ownerB: row.owner_b || '', + createdAt: Number(row.created_at), + updatedAt: Number(row.updated_at), + }; +} + +/** A link from its row, sealing a plain webhook token in place on the way (see openToken). */ +async function readLink(row) { + if (!row) return null; + for (const side of ['a', 'b']) { + const stored = row[`hook_${side}_token`]; + if (stored && !isSealed(stored)) { + await (side === 'a' ? s.resealA : s.resealB).run({ + id: row.id, + token: sealToken(stored), + plain: stored, + }); + } + } + return linkFrom(row); +} + +/** + * Create a link. Returns its id, or null when either channel is already in a link (the unique indexes + * decide, so two requests that race cannot both win). + * @param {{ guildA: string, channelA: string, guildB: string, channelB: string, optionsA?: object, + * optionsB?: object, hookA?: { id: string, token: string }, hookB?: { id: string, token: string }, + * requestedBy?: string, acceptedBy?: string, ownerA?: string, ownerB?: string }} link + */ +export async function createLink(link) { + try { + const res = await s.insertLink.run({ + guildA: link.guildA, + channelA: link.channelA, + guildB: link.guildB, + channelB: link.channelB, + optionsA: JSON.stringify(link.optionsA ?? {}), + optionsB: JSON.stringify(link.optionsB ?? {}), + hookAId: link.hookA?.id ?? '', + hookAToken: sealToken(link.hookA?.token), + hookBId: link.hookB?.id ?? '', + hookBToken: sealToken(link.hookB?.token), + requestedBy: String(link.requestedBy ?? '').slice(0, 100), + acceptedBy: String(link.acceptedBy ?? '').slice(0, 100), + ownerA: String(link.ownerA ?? ''), + ownerB: String(link.ownerB ?? ''), + now: Date.now(), + }); + return Number(res.lastInsertRowid); + } catch (err) { + if (/unique|duplicate/i.test(String(err?.message ?? err))) return null; + throw err; + } +} + +export const getLink = async (id) => readLink(await s.link.get(Number(id))); +export const linksOfGuild = async (key) => + Promise.all((await s.linksOfGuild.all(key, key)).map((row) => readLink(row))); +/** Every running link (the owner sweep). */ +export const activeLinks = async () => Promise.all((await s.activeLinks.all()).map((row) => readLink(row))); +export const linkCount = async (key) => Number((await s.countLinks.get(key, key)).n); + +/** + * The link a channel is in and which side the channel is, or null. + * @returns {Promise<{ link: ReturnType, side: 'a' | 'b' } | null>} + */ +export async function linkForChannel(guildKey, channelId) { + const a = await s.linkByChannelA.get(guildKey, channelId); + if (a) return { link: await readLink(a), side: 'a' }; + const b = await s.linkByChannelB.get(guildKey, channelId); + if (b) return { link: await readLink(b), side: 'b' }; + return null; +} + +/** The link that owns a webhook id (one of Sylo's bridge webhooks), or null. */ +export async function linkForHook(hookId) { + if (!hookId) return null; + return readLink(await s.linkByHook.get({ id: String(hookId) })); +} + +export const STATUSES = Object.freeze(['active', 'paused', 'failed']); + +/** Change the state of a link. `pausedSide` only means something while paused. */ +export async function setLinkStatus(id, { status, pausedSide = '', reason = '', failures = 0 }) { + await s.setStatus.run({ + id: Number(id), + status, + pausedSide: status === 'paused' ? pausedSide : '', + reason: String(reason).slice(0, 300), + failures, + now: Date.now(), + }); +} + +export const setLinkOptions = (id, side, options) => + (side === 'a' ? s.setOptionsA : s.setOptionsB).run({ + id: Number(id), + options: JSON.stringify(options ?? {}), + now: Date.now(), + }); + +/** Record who owns a server now (the new owner confirmed the bridge). */ +export const setLinkOwner = (id, side, owner) => + (side === 'a' ? s.setOwnerA : s.setOwnerB).run({ + id: Number(id), + owner: String(owner ?? ''), + now: Date.now(), + }); + +/** Store the webhook of one side (after Sylo made a new one). */ +export const setLinkHook = (id, side, hook) => + (side === 'a' ? s.setHookA : s.setHookB).run({ + id: Number(id), + id2: hook?.id ?? '', + token: sealToken(hook?.token), + now: Date.now(), + }); + +/** Delete a link and every message-map row of it. */ +export async function deleteLink(id) { + await s.delLinkMessages.run(Number(id)); + return (await s.delLink.run(Number(id))).changes; +} + +/** + * Delete every link a server is in, with the message maps of them. For the guild-leave purge and + * /forget-style erasure. Returns the removed links so the caller can tell the other side. + */ +export async function purgeBridgeGuild(key) { + const ids = (await s.linkIdsOfGuild.all(key, key)).map((r) => Number(r.id)); + for (const id of ids) await s.delLinkMessages.run(id); + await s.delLinksOfGuild.run(key, key); + return ids.length; +} + +// --- codes ----------------------------------------------------------------------------------------- + +/** No 0/O/1/I/L: a code read out loud or copied by hand is not mistaken. */ +const CODE_ALPHABET = 'ABCDEFGHJKMNPQRSTUVWXYZ23456789'; +export const CODE_LENGTH = 12; +export const CODE_TTL_MS = 15 * 60_000; + +/** A new random code, shown as XXXX-XXXX-XXXX (12 characters of a 31-letter alphabet: about 59 bits). */ +export function newCode() { + const bytes = randomBytes(CODE_LENGTH); + let out = ''; + for (let i = 0; i < CODE_LENGTH; i += 1) { + out += CODE_ALPHABET[bytes[i] % CODE_ALPHABET.length]; + if (i % 4 === 3 && i < CODE_LENGTH - 1) out += '-'; + } + return out; +} + +/** The text of a code as typed: upper-case, without dashes or spaces. */ +export function normaliseCode(text) { + return String(text ?? '') + .toUpperCase() + .replace(/[\s-]/g, ''); +} + +// Only the hash is stored; the code itself is shown once, when it is made. +const hashOf = (text) => + createHash('sha256') + .update(`sylo-bridge-code:${normaliseCode(text)}`) + .digest('hex'); + +const CODE_SHAPE = new RegExp(`^[${CODE_ALPHABET}]{${CODE_LENGTH}}$`); + +/** Is this text the right shape for a code (checked before any database lookup)? */ +export const looksLikeCode = (text) => CODE_SHAPE.test(normaliseCode(text)); + +/** + * Store a new request and return the code (shown once: only its hash is kept). + * @returns {Promise<{ id: number, code: string, expiresAt: number }>} + */ +export async function createCode({ + guildId, + channelId, + options = {}, + requestedBy = '', + guildOwner = '', + ttlMs = CODE_TTL_MS, +}) { + const now = Date.now(); + const code = newCode(); + const res = await s.insertCode.run({ + guildId, + channelId, + hash: hashOf(code), + options: JSON.stringify(options), + requestedBy: String(requestedBy).slice(0, 100), + guildOwner: String(guildOwner ?? ''), + expiresAt: now + ttlMs, + now, + }); + return { id: Number(res.lastInsertRowid), code, expiresAt: now + ttlMs }; +} + +/** + * Use a code up: returns the request it belonged to, or null when it is unknown, expired or already + * used. The DELETE decides who gets it, so two servers accepting at once cannot both succeed. + */ +export async function claimCode(text, now = Date.now()) { + if (!looksLikeCode(text)) return null; + const row = await s.codeByHash.get(hashOf(text), now); + if (!row) return null; + if ((await s.delCode.run(row.id)).changes !== 1) return null; + return { + guildId: row.guild_id, + channelId: row.channel_id, + options: parseJson(row.options), + requestedBy: row.requested_by_name, + guildOwner: row.guild_owner || '', + expiresAt: Number(row.expires_at), + }; +} + +export async function pendingCodes(guildId, now = Date.now()) { + return (await s.codesOfGuild.all(guildId, now)).map((r) => ({ + id: Number(r.id), + channelId: r.channel_id, + requestedBy: r.requested_by_name, + expiresAt: Number(r.expires_at), + createdAt: Number(r.created_at), + })); +} + +export const pendingCodeCount = async (guildId, now = Date.now()) => + Number((await s.countCodes.get(guildId, now)).n); + +/** Is a request for this channel already waiting? */ +export const channelHasPendingCode = async (guildId, channelId, now = Date.now()) => + Boolean(await s.codeChannel.get(guildId, channelId, now)); + +/** Cancel one of a server's own requests. @returns {Promise} */ +export async function cancelCode(guildId, id) { + return (await s.delCodeOfGuild.run(Number(id), guildId)).changes === 1; +} + +/** Drop requests past their time (the daily retention sweep). */ +export const sweepCodes = async (now = Date.now()) => (await s.delExpiredCodes.run(now)).changes; + +// --- the message map ------------------------------------------------------------------------------- + +/** + * Remember that a message on one side has a copy on the other. + * @param {number} linkId + * @param {'a' | 'b'} origin the side the ORIGINAL message was on + * @param {{ sourceId: string, copyId: string, hookId?: string, hash?: string }} ids + */ +export async function saveMapping(linkId, origin, { sourceId, copyId, hookId = '', hash = '', extra = {} }) { + await s.insertMap.run({ + linkId: Number(linkId), + origin, + aId: origin === 'a' ? sourceId : copyId, + bId: origin === 'a' ? copyId : sourceId, + hookId, + hash, + extra: JSON.stringify(extra ?? {}).slice(0, 4000), + now: Date.now(), + }); +} + +const mapFrom = (row) => + row && { + linkId: Number(row.link_id), + origin: row.origin, + aMessageId: row.a_message_id, + bMessageId: row.b_message_id, + copyHookId: row.copy_hook_id || '', + contentHash: row.content_hash || '', + extra: parseJson(row.extra), + createdAt: Number(row.created_at), + }; + +/** + * The mapping a message on `side` takes part in, whether it is the original or the copy, with the id + * of its counterpart on the other side. Null when it is not mapped (older than the map, or never relayed). + * @returns {Promise<(NonNullable> & { counterpartId: string, isOriginal: boolean }) | null>} + */ +export async function mappingOf(linkId, side, messageId) { + const row = mapFrom(await (side === 'a' ? s.mapByA : s.mapByB).get(Number(linkId), String(messageId))); + if (!row) return null; + return { + ...row, + counterpartId: side === 'a' ? row.bMessageId : row.aMessageId, + isOriginal: row.origin === side, + }; +} + +export const setMappingHash = (linkId, aId, bId, hash) => s.setHash.run(hash, Number(linkId), aId, bId); + +/** Read a request by its code without using it up (the accepting server's checks run before the claim). */ +export async function peekCode(text, now = Date.now()) { + if (!looksLikeCode(text)) return null; + const row = await s.codeByHash.get(hashOf(text), now); + if (!row) return null; + return { + guildId: row.guild_id, + channelId: row.channel_id, + options: parseJson(row.options), + requestedBy: row.requested_by_name, + guildOwner: row.guild_owner || '', + expiresAt: Number(row.expires_at), + }; +} + +/** Forget a message's mapping (its original was deleted). */ +export const deleteMapping = (linkId, side, messageId) => + (side === 'a' ? s.delMapByA : s.delMapByB).run(Number(linkId), String(messageId)); + +export const mappingCount = async (linkId) => Number((await s.countMap.get(Number(linkId))).n); + +/** Delete map rows older than the retention window (the daily sweep in retention.js). */ +export const sweepMappings = async (now = Date.now()) => + (await s.sweepMap.run(now - MESSAGE_MAP_TTL_MS)).changes; diff --git a/src/db/index.js b/src/db/index.js index 5991616a..4310a0b9 100644 --- a/src/db/index.js +++ b/src/db/index.js @@ -1594,6 +1594,157 @@ export const MIGRATIONS = [ if (!has.has('decided_by_user_id')) database.exec('ALTER TABLE appeals ADD COLUMN decided_by_user_id TEXT;'); }, + + // Versions 81 and 82 are taken by other 5.3 changes (the look of the public member pages, and the + // tickets index). Their branches add their own steps here; until they are merged these two stay empty, + // so the Bridge below keeps the version it was given (83) and a database never skips a step. + () => {}, + () => {}, + + // Bridge (src/db/bridge.js, which bootstraps the same tables on Postgres): the links between a Discord + // channel and a Fluxer channel, the one-time codes that start one (only the hash is kept) and the map + // from a message to its copy on the other side (swept after 30 days by the retention job). + (database) => { + database.exec(` + CREATE TABLE bridge_links ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + guild_a TEXT NOT NULL, + channel_a TEXT NOT NULL, + guild_b TEXT NOT NULL, + channel_b TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'active', + paused_side TEXT NOT NULL DEFAULT '', + fail_reason TEXT NOT NULL DEFAULT '', + failure_count INTEGER NOT NULL DEFAULT 0, + options_a TEXT NOT NULL DEFAULT '{}', + options_b TEXT NOT NULL DEFAULT '{}', + hook_a_id TEXT NOT NULL DEFAULT '', + hook_a_token TEXT NOT NULL DEFAULT '', + hook_b_id TEXT NOT NULL DEFAULT '', + hook_b_token TEXT NOT NULL DEFAULT '', + requested_by_name TEXT NOT NULL DEFAULT '', + accepted_by_name TEXT NOT NULL DEFAULT '', + owner_a TEXT NOT NULL DEFAULT '', + owner_b TEXT NOT NULL DEFAULT '', + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL + ); + CREATE UNIQUE INDEX idx_bridge_links_a ON bridge_links (guild_a, channel_a); + CREATE UNIQUE INDEX idx_bridge_links_b ON bridge_links (guild_b, channel_b); + CREATE INDEX idx_bridge_links_guild_b ON bridge_links (guild_b); + CREATE TABLE bridge_codes ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + guild_id TEXT NOT NULL, + channel_id TEXT NOT NULL, + code_hash TEXT NOT NULL, + options TEXT NOT NULL DEFAULT '{}', + requested_by_name TEXT NOT NULL DEFAULT '', + guild_owner TEXT NOT NULL DEFAULT '', + expires_at INTEGER NOT NULL, + created_at INTEGER NOT NULL + ); + CREATE UNIQUE INDEX idx_bridge_codes_hash ON bridge_codes (code_hash); + CREATE INDEX idx_bridge_codes_guild ON bridge_codes (guild_id); + CREATE TABLE bridge_messages ( + link_id INTEGER NOT NULL, + origin TEXT NOT NULL, + a_message_id TEXT NOT NULL, + b_message_id TEXT NOT NULL, + copy_hook_id TEXT NOT NULL DEFAULT '', + content_hash TEXT NOT NULL DEFAULT '', + extra TEXT NOT NULL DEFAULT '{}', + created_at INTEGER NOT NULL, + PRIMARY KEY (link_id, a_message_id) + ); + CREATE UNIQUE INDEX idx_bridge_messages_b ON bridge_messages (link_id, b_message_id); + CREATE INDEX idx_bridge_messages_created ON bridge_messages (created_at); + `); + }, + + // Migration 84 of the 5.3 line (it follows the bridge's 83; the entry is the next one in this list whichever + // migrations sit before it): the server mirror (src/db/mirror.js, which bootstraps + // the same tables on Postgres). Pairs of a Discord and a Fluxer server, the id map between their channels, + // roles, emoji and stickers, and the queue of work that makes a copy resumable. + (database) => { + database.exec(` + CREATE TABLE IF NOT EXISTS mirror_pairs ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + source_guild TEXT NOT NULL, + target_guild TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'pending', + resume_status TEXT NOT NULL DEFAULT '', + options TEXT NOT NULL DEFAULT '{}', + code_hash TEXT NOT NULL DEFAULT '', + code_expires_at INTEGER NOT NULL DEFAULT 0, + code_attempts INTEGER NOT NULL DEFAULT 0, + requested_by TEXT NOT NULL DEFAULT '', + source_consent_by TEXT NOT NULL DEFAULT '', + source_consent_at INTEGER NOT NULL DEFAULT 0, + target_consent_by TEXT NOT NULL DEFAULT '', + target_consent_at INTEGER NOT NULL DEFAULT 0, + source_consent_user TEXT NOT NULL DEFAULT '', + target_consent_user TEXT NOT NULL DEFAULT '', + pause_kind TEXT NOT NULL DEFAULT '', + paused_by TEXT NOT NULL DEFAULT '', + copy_run INTEGER NOT NULL DEFAULT 0, + copied_at INTEGER NOT NULL DEFAULT 0, + failures INTEGER NOT NULL DEFAULT 0, + last_error TEXT NOT NULL DEFAULT '', + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL + ); + CREATE UNIQUE INDEX IF NOT EXISTS idx_mirror_pairs_source ON mirror_pairs (source_guild); + CREATE UNIQUE INDEX IF NOT EXISTS idx_mirror_pairs_target ON mirror_pairs (target_guild); + + CREATE TABLE IF NOT EXISTS mirror_ids ( + pair_id INTEGER NOT NULL, + kind TEXT NOT NULL, + source_id TEXT NOT NULL, + target_id TEXT NOT NULL, + owned INTEGER NOT NULL DEFAULT 1, + spec_hash TEXT NOT NULL DEFAULT '', + label TEXT NOT NULL DEFAULT '', + removed_at INTEGER NOT NULL DEFAULT 0, + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL, + PRIMARY KEY (pair_id, kind, source_id) + ); + CREATE INDEX IF NOT EXISTS idx_mirror_ids_target ON mirror_ids (pair_id, kind, target_id); + + CREATE TABLE IF NOT EXISTS mirror_jobs ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + pair_id INTEGER NOT NULL, + run INTEGER NOT NULL DEFAULT 0, + phase TEXT NOT NULL, + op TEXT NOT NULL, + kind TEXT NOT NULL, + source_id TEXT NOT NULL DEFAULT '', + step_key TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'pending', + attempts INTEGER NOT NULL DEFAULT 0, + next_at INTEGER NOT NULL DEFAULT 0, + recovering INTEGER NOT NULL DEFAULT 0, + first_claim_at INTEGER NOT NULL DEFAULT 0, + payload TEXT NOT NULL DEFAULT '{}', + detail TEXT NOT NULL DEFAULT '', + notes TEXT NOT NULL DEFAULT '[]', + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL + ); + CREATE INDEX IF NOT EXISTS idx_mirror_jobs_pair ON mirror_jobs (pair_id, status, id); + CREATE UNIQUE INDEX IF NOT EXISTS idx_mirror_jobs_open ON mirror_jobs (pair_id, step_key) WHERE status = 'pending'; + + CREATE TABLE IF NOT EXISTS mirror_links ( + pair_id INTEGER NOT NULL, + source_channel TEXT NOT NULL, + target_channel TEXT NOT NULL, + link_id INTEGER NOT NULL, + created_at INTEGER NOT NULL, + PRIMARY KEY (pair_id, source_channel) + ); + CREATE UNIQUE INDEX IF NOT EXISTS idx_mirror_links_link ON mirror_links (link_id); + `); + }, ]; /** Highest schema version this build knows how to run. */ diff --git a/src/db/mirror.js b/src/db/mirror.js new file mode 100644 index 00000000..1a4d17ac --- /dev/null +++ b/src/db/mirror.js @@ -0,0 +1,612 @@ +// The server mirror's data (src/mirror/): which Discord server is copied to which Fluxer server, the map +// between their channels, roles, emoji and stickers, and the queue of work that keeps them in step. +// +// mirror_pairs one row per mirror: the Discord server (source_guild, a bare id) and the Fluxer server +// (target_guild, "fluxer:"), the state (status), who agreed on each side, the hashed +// one-time code that asks the other side, the copy options and the last error. +// mirror_ids the id map of a pair: for each source id the id Sylo made (or adopted) on the target. +// `owned` is 1 for what Sylo created, and only owned items are ever deleted on the target. +// `spec_hash` is a fingerprint of what was last sent, so an unchanged item is not sent again. +// mirror_links the bridge links the mirror made for the channel pairs of a pair (see src/mirror/bridgeLink.js): +// which Discord channel and Fluxer channel, and the id of the bridge link. A row that stays after +// its link is gone means somebody unlinked that channel on purpose, so the mirror does not link +// it again. +// mirror_jobs the queue: one row per piece of work (copy one channel, update one role, delete one +// emoji ...). The queue is what makes a copy resumable: after a restart the unfinished rows +// are still there. Event rows coalesce: one open row per item. +// +// None of these tables has a guild_id column: a pair belongs to two servers, and purgeMirrorOf() (called +// from purge.js) removes every pair a server is part of, with its map and queue. +import { prepare, registerPostgresBootstrap } from './driver.js'; + +registerPostgresBootstrap(` + CREATE TABLE IF NOT EXISTS mirror_pairs ( + id SERIAL PRIMARY KEY, + source_guild TEXT NOT NULL, + target_guild TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'pending', + resume_status TEXT NOT NULL DEFAULT '', + options TEXT NOT NULL DEFAULT '{}', + code_hash TEXT NOT NULL DEFAULT '', + code_expires_at BIGINT NOT NULL DEFAULT 0, + code_attempts INTEGER NOT NULL DEFAULT 0, + requested_by TEXT NOT NULL DEFAULT '', + source_consent_by TEXT NOT NULL DEFAULT '', + source_consent_at BIGINT NOT NULL DEFAULT 0, + target_consent_by TEXT NOT NULL DEFAULT '', + target_consent_at BIGINT NOT NULL DEFAULT 0, + source_consent_user TEXT NOT NULL DEFAULT '', + target_consent_user TEXT NOT NULL DEFAULT '', + pause_kind TEXT NOT NULL DEFAULT '', + paused_by TEXT NOT NULL DEFAULT '', + copy_run INTEGER NOT NULL DEFAULT 0, + copied_at BIGINT NOT NULL DEFAULT 0, + failures INTEGER NOT NULL DEFAULT 0, + last_error TEXT NOT NULL DEFAULT '', + created_at BIGINT NOT NULL, + updated_at BIGINT NOT NULL + ); + CREATE UNIQUE INDEX IF NOT EXISTS idx_mirror_pairs_source ON mirror_pairs (source_guild); + CREATE UNIQUE INDEX IF NOT EXISTS idx_mirror_pairs_target ON mirror_pairs (target_guild); + + CREATE TABLE IF NOT EXISTS mirror_ids ( + pair_id INTEGER NOT NULL, + kind TEXT NOT NULL, + source_id TEXT NOT NULL, + target_id TEXT NOT NULL, + owned INTEGER NOT NULL DEFAULT 1, + spec_hash TEXT NOT NULL DEFAULT '', + label TEXT NOT NULL DEFAULT '', + removed_at BIGINT NOT NULL DEFAULT 0, + created_at BIGINT NOT NULL, + updated_at BIGINT NOT NULL, + PRIMARY KEY (pair_id, kind, source_id) + ); + CREATE INDEX IF NOT EXISTS idx_mirror_ids_target ON mirror_ids (pair_id, kind, target_id); + + CREATE TABLE IF NOT EXISTS mirror_jobs ( + id SERIAL PRIMARY KEY, + pair_id INTEGER NOT NULL, + run INTEGER NOT NULL DEFAULT 0, + phase TEXT NOT NULL, + op TEXT NOT NULL, + kind TEXT NOT NULL, + source_id TEXT NOT NULL DEFAULT '', + step_key TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'pending', + attempts INTEGER NOT NULL DEFAULT 0, + next_at BIGINT NOT NULL DEFAULT 0, + recovering INTEGER NOT NULL DEFAULT 0, + first_claim_at BIGINT NOT NULL DEFAULT 0, + payload TEXT NOT NULL DEFAULT '{}', + detail TEXT NOT NULL DEFAULT '', + notes TEXT NOT NULL DEFAULT '[]', + created_at BIGINT NOT NULL, + updated_at BIGINT NOT NULL + ); + CREATE INDEX IF NOT EXISTS idx_mirror_jobs_pair ON mirror_jobs (pair_id, status, id); + CREATE UNIQUE INDEX IF NOT EXISTS idx_mirror_jobs_open ON mirror_jobs (pair_id, step_key) WHERE status = 'pending'; + + CREATE TABLE IF NOT EXISTS mirror_links ( + pair_id INTEGER NOT NULL, + source_channel TEXT NOT NULL, + target_channel TEXT NOT NULL, + link_id INTEGER NOT NULL, + created_at BIGINT NOT NULL, + PRIMARY KEY (pair_id, source_channel) + ); + CREATE UNIQUE INDEX IF NOT EXISTS idx_mirror_links_link ON mirror_links (link_id); +`); + +/** Pair states. `pending`: waiting for both servers to agree. `ready`: agreed, not copied yet. */ +export const PAIR_STATUS = Object.freeze({ + PENDING: 'pending', + READY: 'ready', + COPYING: 'copying', + LIVE: 'live', + DONE: 'done', + PAUSED: 'paused', + STOPPED: 'stopped', +}); + +/** The states in which a mirror runs, and so do the bridge links it made. */ +export const PAIR_RUNNING = Object.freeze([PAIR_STATUS.COPYING, PAIR_STATUS.LIVE, PAIR_STATUS.DONE]); + +const num = (v) => Number(v ?? 0); + +const parseJson = (text, fallback) => { + try { + const v = JSON.parse(text); + return v && typeof v === 'object' ? v : fallback; + } catch { + return fallback; + } +}; + +const toPair = (row) => + row + ? { + id: num(row.id), + sourceGuild: row.source_guild, + targetGuild: row.target_guild, + status: row.status, + resumeStatus: row.resume_status, + options: parseJson(row.options, {}), + codeHash: row.code_hash, + codeExpiresAt: num(row.code_expires_at), + codeAttempts: num(row.code_attempts), + requestedBy: row.requested_by, + sourceConsentBy: row.source_consent_by, + sourceConsentAt: num(row.source_consent_at), + targetConsentBy: row.target_consent_by, + targetConsentAt: num(row.target_consent_at), + sourceConsentUser: row.source_consent_user, + targetConsentUser: row.target_consent_user, + pauseKind: row.pause_kind, + pausedBy: row.paused_by, + copyRun: num(row.copy_run), + copiedAt: num(row.copied_at), + failures: num(row.failures), + lastError: row.last_error, + createdAt: num(row.created_at), + updatedAt: num(row.updated_at), + } + : null; + +const toJob = (row) => + row + ? { + id: num(row.id), + pairId: num(row.pair_id), + run: num(row.run), + phase: row.phase, + op: row.op, + kind: row.kind, + sourceId: row.source_id, + stepKey: row.step_key, + status: row.status, + attempts: num(row.attempts), + nextAt: num(row.next_at), + recovering: num(row.recovering) === 1, + firstClaimAt: num(row.first_claim_at), + payload: parseJson(row.payload, {}), + detail: row.detail, + notes: Array.isArray(parseJson(row.notes, [])) ? parseJson(row.notes, []) : [], + createdAt: num(row.created_at), + updatedAt: num(row.updated_at), + } + : null; + +const s = { + insertPair: prepare( + `INSERT INTO mirror_pairs (source_guild, target_guild, status, options, code_hash, code_expires_at, + requested_by, source_consent_by, source_consent_at, target_consent_by, target_consent_at, + created_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + { returningId: true } + ), + pair: prepare('SELECT * FROM mirror_pairs WHERE id = ?'), + bySource: prepare('SELECT * FROM mirror_pairs WHERE source_guild = ?'), + byTarget: prepare('SELECT * FROM mirror_pairs WHERE target_guild = ?'), + ofGuild: prepare('SELECT * FROM mirror_pairs WHERE source_guild = ? OR target_guild = ? ORDER BY id'), + inStatus: prepare('SELECT * FROM mirror_pairs WHERE status = ? ORDER BY id'), + allPairs: prepare('SELECT * FROM mirror_pairs ORDER BY id'), + deletePair: prepare('DELETE FROM mirror_pairs WHERE id = ?'), + deleteIds: prepare('DELETE FROM mirror_ids WHERE pair_id = ?'), + deleteJobs: prepare('DELETE FROM mirror_jobs WHERE pair_id = ?'), + deleteLinks: prepare('DELETE FROM mirror_links WHERE pair_id = ?'), + linksOfPair: prepare('SELECT * FROM mirror_links WHERE pair_id = ? ORDER BY source_channel'), + linkRow: prepare('SELECT * FROM mirror_links WHERE pair_id = ? AND source_channel = ?'), + putLink: prepare( + `INSERT INTO mirror_links (pair_id, source_channel, target_channel, link_id, created_at) + VALUES (?, ?, ?, ?, ?) + ON CONFLICT (pair_id, source_channel) DO NOTHING` + ), + pairOfLink: prepare('SELECT pair_id FROM mirror_links WHERE link_id = ?'), + + getId: prepare('SELECT * FROM mirror_ids WHERE pair_id = ? AND kind = ? AND source_id = ?'), + getIdByTarget: prepare('SELECT * FROM mirror_ids WHERE pair_id = ? AND kind = ? AND target_id = ?'), + listIds: prepare('SELECT * FROM mirror_ids WHERE pair_id = ? AND kind = ? ORDER BY source_id'), + putId: prepare(` + INSERT INTO mirror_ids (pair_id, kind, source_id, target_id, owned, spec_hash, label, removed_at, created_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, 0, ?, ?) + ON CONFLICT (pair_id, kind, source_id) DO UPDATE SET + target_id = excluded.target_id, owned = excluded.owned, spec_hash = excluded.spec_hash, + label = excluded.label, removed_at = 0, updated_at = excluded.updated_at + `), + markRemoved: prepare( + 'UPDATE mirror_ids SET removed_at = ?, updated_at = ? WHERE pair_id = ? AND kind = ? AND source_id = ?' + ), + listRemoved: prepare( + 'SELECT * FROM mirror_ids WHERE pair_id = ? AND removed_at > 0 ORDER BY removed_at, source_id' + ), + countIds: prepare('SELECT COUNT(*) AS n FROM mirror_ids WHERE pair_id = ?'), + recentDeletes: prepare( + "SELECT COUNT(*) AS n FROM mirror_jobs WHERE pair_id = ? AND op = 'delete' AND created_at >= ?" + ), + pendingDeletes: prepare( + "SELECT * FROM mirror_jobs WHERE pair_id = ? AND op = 'delete' AND status = 'pending' ORDER BY id LIMIT ?" + ), + countPendingDeletes: prepare( + "SELECT COUNT(*) AS n FROM mirror_jobs WHERE pair_id = ? AND op = 'delete' AND status = 'pending'" + ), + deleteId: prepare('DELETE FROM mirror_ids WHERE pair_id = ? AND kind = ? AND source_id = ?'), + + insertJob: prepare( + `INSERT INTO mirror_jobs (pair_id, run, phase, op, kind, source_id, step_key, status, payload, next_at, + created_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, 'pending', ?, ?, ?, ?) + ON CONFLICT DO NOTHING` + ), + job: prepare('SELECT * FROM mirror_jobs WHERE id = ?'), + nextJob: prepare( + "SELECT * FROM mirror_jobs WHERE pair_id = ? AND status = 'pending' AND next_at <= ? ORDER BY id LIMIT 1" + ), + nextDue: prepare("SELECT MIN(next_at) AS at FROM mirror_jobs WHERE pair_id = ? AND status = 'pending'"), + claimJob: prepare( + `UPDATE mirror_jobs SET status = 'running', updated_at = ?, + first_claim_at = CASE WHEN first_claim_at = 0 THEN ? ELSE first_claim_at END + WHERE id = ? AND status = 'pending'` + ), + finishJob: prepare( + 'UPDATE mirror_jobs SET status = ?, detail = ?, notes = ?, recovering = 0, updated_at = ? WHERE id = ?' + ), + requeueJob: prepare( + "UPDATE mirror_jobs SET status = 'pending', attempts = ?, next_at = ?, detail = ?, recovering = ?, updated_at = ? WHERE id = ?" + ), + twin: prepare( + "SELECT id FROM mirror_jobs WHERE pair_id = ? AND step_key = ? AND status = 'pending' AND id <> ?" + ), + running: prepare("SELECT * FROM mirror_jobs WHERE status = 'running' ORDER BY id"), + skipOpenFor: prepare( + `UPDATE mirror_jobs SET status = 'skipped', detail = ?, updated_at = ? + WHERE pair_id = ? AND kind = ? AND source_id = ? AND op = ? AND status = 'pending'` + ), + dropOpen: prepare( + "UPDATE mirror_jobs SET status = 'skipped', detail = ?, updated_at = ? WHERE pair_id = ? AND status IN ('pending', 'running')" + ), + countByStatus: prepare( + 'SELECT status, COUNT(*) AS n FROM mirror_jobs WHERE pair_id = ? AND run = ? AND phase = ? GROUP BY status' + ), + countOpenSync: prepare( + "SELECT COUNT(*) AS n FROM mirror_jobs WHERE pair_id = ? AND phase = 'sync' AND status IN ('pending', 'running')" + ), + countOpen: prepare( + "SELECT COUNT(*) AS n FROM mirror_jobs WHERE pair_id = ? AND run = ? AND phase = 'copy' AND status IN ('pending', 'running')" + ), + jobsOfRun: prepare( + 'SELECT * FROM mirror_jobs WHERE pair_id = ? AND run = ? AND phase = ? ORDER BY id LIMIT ? OFFSET ?' + ), + failedJobs: prepare( + "SELECT * FROM mirror_jobs WHERE pair_id = ? AND status = 'failed' ORDER BY id DESC LIMIT ?" + ), + pruneJobs: prepare( + "DELETE FROM mirror_jobs WHERE status IN ('done', 'skipped') AND phase = 'sync' AND updated_at < ?" + ), + pruneOldRuns: prepare("DELETE FROM mirror_jobs WHERE pair_id = ? AND phase = 'copy' AND run < ?"), +}; + +// --- pairs ---------------------------------------------------------------------------------------- + +/** + * Create a pair. The caller has checked that neither server is in another pair. + * @param {{ sourceGuild: string, targetGuild: string, status?: string, options?: object, codeHash?: string, + * codeExpiresAt?: number, requestedBy?: string, sourceConsentBy?: string, targetConsentBy?: string, + * now?: number }} p + */ +export async function createPair(p) { + const now = p.now ?? Date.now(); + const res = await s.insertPair.run( + p.sourceGuild, + p.targetGuild, + p.status ?? PAIR_STATUS.PENDING, + JSON.stringify(p.options ?? {}), + p.codeHash ?? '', + p.codeExpiresAt ?? 0, + p.requestedBy ?? '', + p.sourceConsentBy ?? '', + p.sourceConsentBy ? now : 0, + p.targetConsentBy ?? '', + p.targetConsentBy ? now : 0, + now, + now + ); + return getPair(Number(res.lastInsertRowid)); +} + +export const getPair = async (id) => toPair(await s.pair.get(id)); +export const getPairBySource = async (key) => toPair(await s.bySource.get(key)); +export const getPairByTarget = async (key) => toPair(await s.byTarget.get(key)); +/** Every pair a server is part of, on either side. */ +export const pairsOfGuild = async (key) => (await s.ofGuild.all(key, key)).map(toPair); +export const listPairsIn = async (status) => (await s.inStatus.all(status)).map(toPair); +export const listAllPairs = async () => (await s.allPairs.all()).map(toPair); + +const COLUMNS = { + status: 'status', + resumeStatus: 'resume_status', + options: 'options', + codeHash: 'code_hash', + codeExpiresAt: 'code_expires_at', + codeAttempts: 'code_attempts', + requestedBy: 'requested_by', + sourceConsentBy: 'source_consent_by', + sourceConsentAt: 'source_consent_at', + targetConsentBy: 'target_consent_by', + targetConsentAt: 'target_consent_at', + sourceConsentUser: 'source_consent_user', + targetConsentUser: 'target_consent_user', + pauseKind: 'pause_kind', + pausedBy: 'paused_by', + copyRun: 'copy_run', + copiedAt: 'copied_at', + failures: 'failures', + lastError: 'last_error', +}; + +/** + * Change some fields of a pair. Only the listed fields exist; the SQL is built from fixed column names. + * @param {number} id + * @param {Partial>} patch + */ +export async function updatePair(id, patch, now = Date.now()) { + const sets = []; + const args = []; + for (const [key, column] of Object.entries(COLUMNS)) { + if (patch[key] === undefined) continue; + sets.push(`${column} = ?`); + args.push(key === 'options' ? JSON.stringify(patch[key]) : patch[key]); + } + sets.push('updated_at = ?'); + args.push(now, id); + await prepare(`UPDATE mirror_pairs SET ${sets.join(', ')} WHERE id = ?`).run(...args); + return getPair(id); +} + +/** Delete a pair with its id map and queue. */ +export async function deletePair(id) { + await s.deleteIds.run(id); + await s.deleteJobs.run(id); + await s.deleteLinks.run(id); + await s.deletePair.run(id); +} + +/** Remove every pair a server is part of (the server was purged, or Sylo left it). Returns the pairs removed. */ +export async function purgeMirrorOf(key) { + const pairs = await pairsOfGuild(key); + for (const p of pairs) await deletePair(p.id); + return pairs; +} + +// --- the bridge links of a pair --------------------------------------------------------------------- + +const toMirrorLink = (row) => + row + ? { + pairId: num(row.pair_id), + sourceChannel: row.source_channel, + targetChannel: row.target_channel, + linkId: num(row.link_id), + createdAt: num(row.created_at), + } + : null; + +/** The bridge links the mirror of a pair made (or tried to make and then lost: see the file header). */ +export const listMirrorLinks = async (pairId) => (await s.linksOfPair.all(pairId)).map(toMirrorLink); +export const getMirrorLink = async (pairId, sourceChannel) => + toMirrorLink(await s.linkRow.get(pairId, sourceChannel)); +/** Remember that a bridge link belongs to a pair's mirror. A channel that already has a row keeps it. */ +export const putMirrorLink = (pairId, sourceChannel, targetChannel, linkId, now = Date.now()) => + s.putLink.run(pairId, sourceChannel, targetChannel, Number(linkId), now); +/** The pair a bridge link was made for, or null (a link somebody made by hand). */ +export async function mirrorPairOfLink(linkId) { + const row = await s.pairOfLink.get(Number(linkId)); + return row ? num(row.pair_id) : null; +} + +// --- the id map ----------------------------------------------------------------------------------- + +const toMapping = (row) => + row + ? { + pairId: num(row.pair_id), + kind: row.kind, + sourceId: row.source_id, + targetId: row.target_id, + owned: num(row.owned) === 1, + specHash: row.spec_hash, + label: row.label, + removedAt: num(row.removed_at), + } + : null; + +export const getMapping = async (pairId, kind, sourceId) => + toMapping(await s.getId.get(pairId, kind, sourceId)); +export const getMappingByTarget = async (pairId, kind, targetId) => + toMapping(await s.getIdByTarget.get(pairId, kind, targetId)); +export const listMappings = async (pairId, kind) => (await s.listIds.all(pairId, kind)).map(toMapping); + +export async function putMapping({ + pairId, + kind, + sourceId, + targetId, + owned = true, + specHash = '', + label = '', + now, +}) { + const at = now ?? Date.now(); + await s.putId.run( + pairId, + kind, + sourceId, + targetId, + owned ? 1 : 0, + specHash, + String(label).slice(0, 100), + at, + at + ); +} + +/** Note that the Discord original is gone while the Fluxer copy stays (it was locked, or left alone). */ +export const markRemoved = (pairId, kind, sourceId, now = Date.now()) => + s.markRemoved.run(now, now, pairId, kind, sourceId); + +/** The copies whose Discord original was deleted, oldest first. */ +export async function listRemoved(pairId) { + return (await s.listRemoved.all(pairId)).map(toMapping); +} + +/** How many items a pair has in its id map (a stopped pair that made something is not thrown away). */ +export async function countMappings(pairId) { + return num((await s.countIds.get(pairId))?.n); +} + +/** How many delete jobs a pair queued since `since`: what the burst brake counts. */ +export async function countRecentDeletes(pairId, since) { + return num((await s.recentDeletes.get(pairId, since))?.n); +} + +export const deleteMapping = (pairId, kind, sourceId) => s.deleteId.run(pairId, kind, sourceId); + +// --- the queue ------------------------------------------------------------------------------------ + +/** + * Add a job unless an identical one is already waiting (same pair and step_key): a burst of events about + * one item is one job, because the job reads the item as it is when it runs. + * @returns {Promise} whether a job was added + */ +export async function enqueueJob({ + pairId, + run = 0, + phase, + op, + kind, + sourceId = '', + stepKey, + payload = {}, + now, +}) { + const at = now ?? Date.now(); + const res = await s.insertJob.run( + pairId, + run, + phase, + op, + kind, + sourceId, + stepKey, + JSON.stringify(payload), + at, + at, + at + ); + return res.changes > 0; +} + +export const getJob = async (id) => toJob(await s.job.get(id)); + +/** The oldest waiting job of a pair that is due. */ +export const nextDueJob = async (pairId, now = Date.now()) => toJob(await s.nextJob.get(pairId, now)); + +/** When the next waiting job of a pair is due, or null when none is waiting. */ +export async function nextDueAt(pairId) { + const row = await s.nextDue.get(pairId); + return row?.at == null ? null : num(row.at); +} + +/** Take a waiting job. False when someone else took it first. */ +export async function claimJob(id, now = Date.now()) { + return (await s.claimJob.run(now, now, id)).changes === 1; +} + +/** @param {'done'|'failed'|'skipped'} status */ +export const finishJob = (id, status, detail = '', notes = [], now = Date.now()) => + s.finishJob.run(status, String(detail).slice(0, 500), JSON.stringify(notes ?? []), now, id); + +/** + * Put a running job back in the queue. If a newer job for the same step is already waiting (a change came in + * while this one ran), that one will read the item as it is now, so this one is dropped instead: two waiting jobs + * for one step are not allowed. + */ +async function requeue(job, { attempts, nextAt, detail, recovering, now }) { + const twin = await s.twin.get(job.pairId, job.stepKey, job.id); + if (twin) { + await s.finishJob.run('skipped', 'a newer change replaced it', '[]', now, job.id); + return; + } + await s.requeueJob.run(attempts, nextAt, String(detail).slice(0, 500), recovering ? 1 : 0, now, job.id); +} + +/** + * Put a job back in the queue to run again at `nextAt`. `maybeDone`: the call may have gone through before it + * failed (a timeout), so the next run looks for what the last one made before it makes anything. + */ +export async function retryJob( + id, + attempts, + nextAt, + detail = '', + now = Date.now(), + { maybeDone = false } = {} +) { + const job = await getJob(id); + if (job) await requeue(job, { attempts, nextAt, detail, recovering: job.recovering || maybeDone, now }); +} + +/** After a restart: a job that was running when the process stopped waits again, marked as recovering. */ +export async function recoverRunningJobs(now = Date.now()) { + for (const row of await s.running.all()) { + const job = toJob(row); + await requeue(job, { + attempts: job.attempts, + nextAt: job.nextAt, + detail: job.detail, + recovering: true, + now, + }); + } +} + +/** Skip the waiting jobs of one item and operation (a delete replaces a pending sync). */ +export const skipWaitingJobs = (pairId, kind, sourceId, op, detail, now = Date.now()) => + s.skipOpenFor.run(detail, now, pairId, kind, sourceId, op); + +/** Stop all unfinished work of a pair. */ +export const dropOpenJobs = (pairId, detail, now = Date.now()) => s.dropOpen.run(detail, now, pairId); + +/** Counts of a pair's jobs by status for one copy run, e.g. { pending: 3, done: 40 }. */ +export async function jobCounts(pairId, run, phase = 'copy') { + const out = { pending: 0, running: 0, done: 0, failed: 0, skipped: 0 }; + for (const row of await s.countByStatus.all(pairId, run, phase)) out[row.status] = num(row.n); + return out; +} + +export const openSyncCount = async (pairId) => num((await s.countOpenSync.get(pairId))?.n); +export const openCopyCount = async (pairId, run) => num((await s.countOpen.get(pairId, run))?.n); + +/** The jobs of a copy run in order, a page at a time. */ +export async function listRunJobs(pairId, run, { phase = 'copy', limit = 1000, offset = 0 } = {}) { + return (await s.jobsOfRun.all(pairId, run, phase, limit, offset)).map(toJob); +} + +/** The latest failed jobs of a pair. */ +export async function listFailedJobs(pairId, limit = 50) { + return (await s.failedJobs.all(pairId, limit)).map(toJob); +} + +/** Housekeeping: finished event jobs older than `before`, and the jobs of copy runs before `keepRun`. */ +export async function pruneMirrorJobs({ before, pairId, keepRun }) { + if (before) await s.pruneJobs.run(before); + if (pairId && keepRun) await s.pruneOldRuns.run(pairId, keepRun); +} + +/** The deletes still waiting in a pair's queue: how many, and the first `limit` of them. */ +export async function listPendingDeletes(pairId, limit = 50) { + return { + count: num((await s.countPendingDeletes.get(pairId))?.n), + jobs: (await s.pendingDeletes.all(pairId, limit)).map(toJob), + }; +} diff --git a/src/db/purge.js b/src/db/purge.js index fcc06f41..efe51062 100644 --- a/src/db/purge.js +++ b/src/db/purge.js @@ -10,6 +10,7 @@ // deleted on the next attempt — self-healing, not a race. So both run as a // plain sequence of awaited statements instead of a transaction. import { prepare } from './driver.js'; +import { log } from '../lib/log.js'; import { clearCommandPrefixCache } from './guildSettings.js'; import { eraseMember } from './economy.js'; import { eraseInventory } from './economyShop.js'; @@ -65,6 +66,8 @@ import './welcomeSent.js'; import './applications.js'; import './economy.js'; import './economyShop.js'; +import { purgeBridgeGuild } from './bridge.js'; +import { purgeMirrorOf } from './mirror.js'; // Tables keyed directly by guild_id. A test in test/guildTables.test.js checks // this stays in sync with the schema so new guild data can't escape /forget or @@ -121,6 +124,9 @@ export const GUILD_TABLES = [ 'economy_items', 'economy_inventory', 'collectible_images', + // The requests nobody has accepted yet. The links name two servers (guild_a, guild_b), so they go + // through purgeBridgeGuild() in purgeGuild() below. + 'bridge_codes', ]; const simpleStmts = GUILD_TABLES.map((t) => prepare(`DELETE FROM ${t} WHERE guild_id = ?`)); @@ -141,14 +147,37 @@ const suggestionVoteStmt = prepare( 'DELETE FROM guild_suggestion_votes WHERE suggestion_id IN (SELECT id FROM guild_suggestions WHERE guild_id = ?)' ); +/** Work a module must do before the server's rows go (the bridge tells the server on its other end). */ +const beforePurge = []; + +/** + * Register something to run first when a server is purged, while its rows can still be read. A failing + * hook never stops the purge. + * @param {(guildId: string) => Promise} fn + */ +export function onBeforePurge(fn) { + beforePurge.push(fn); +} + /** Remove all of a guild's stored data. */ export async function purgeGuild(guildId) { + for (const fn of beforePurge) { + try { + await fn(guildId); + } catch (err) { + log.warn('purge', `a clean-up before the purge of ${guildId} failed:`, err); + } + } await ticketMsgStmt.run(guildId); // before `tickets`, which the subquery reads await giveawayEntryStmt.run(guildId); // before `giveaways`, which the subquery reads await giveawayPrizeStmt.run(guildId); await eventParticipantStmt.run(guildId); // before `guild_events`, which the subquery reads await suggestionVoteStmt.run(guildId); // before `guild_suggestions`, which the subquery reads for (const stmt of simpleStmts) await stmt.run(guildId); + // A mirror pair belongs to two servers, so it has no guild_id column: every pair this server is part of + // goes, with its id map and queue (the server on the other side keeps what Sylo made there). + await purgeMirrorOf(guildId); + await purgeBridgeGuild(guildId); // the links this server is in (on either side) and their message maps clearCommandPrefixCache(guildId); } diff --git a/src/db/retention.js b/src/db/retention.js index 13810f6c..5ca0dbc5 100644 --- a/src/db/retention.js +++ b/src/db/retention.js @@ -23,6 +23,7 @@ import { log } from '../lib/log.js'; // Owners of the tables swept below, for their Postgres bootstrap. import './tickets.js'; import './cache.js'; +import { sweepCodes, sweepMappings } from './bridge.js'; const DAY_MS = 86_400_000; const MAX_DAYS = 3650; // 10 years — matches the dashboard input cap @@ -88,6 +89,12 @@ export async function sweepRetention(now = Date.now()) { const orphanedTicketMessages = (await delOrphanedTicketMsgs.run()).changes; const staleStats = (await delStaleStats.run(now - STATS_KEEP_MS)).changes; + // The bridge's message map keeps a message for 30 days (edits, deletes and replies follow it that + // long), and a request nobody accepted is gone after its 15 minutes. Not part of the result: this + // is housekeeping of a table the bridge owns. + await sweepMappings(now); + await sweepCodes(now); + const result = { closedTickets, ticketMessages, inactiveCases, orphanedTicketMessages, staleStats }; if (result.closedTickets || result.inactiveCases) { log.info( diff --git a/src/lib/pigeon.js b/src/lib/pigeon.js index 100938a7..2b3a30c4 100644 --- a/src/lib/pigeon.js +++ b/src/lib/pigeon.js @@ -20,7 +20,8 @@ // dev-log notifications (src/lib/push.js notifyDevLogEvent), which are a dev-log sink, and a // warning or error from there would become a new dev-log entry and so another notification. // Only ensureRegistered(), which runs at start-up outside the sink, logs warnings and errors. -import { createCipheriv, createDecipheriv, hkdfSync, randomBytes, randomUUID } from 'node:crypto'; +import { randomUUID } from 'node:crypto'; +import { open, seal } from './seal.js'; import { log } from './log.js'; import { solvePowInWorker } from './pigeonPow.js'; @@ -88,35 +89,8 @@ async function readJson(res, cap) { } // --- sealing the credential ---------------------------------------------------------------- - -const sealKey = (secret) => Buffer.from(hkdfSync('sha256', String(secret), 'sylo', 'pigeon-credential', 32)); - -function seal(secret, plain) { - const iv = randomBytes(12); - const cipher = createCipheriv('aes-256-gcm', sealKey(secret), iv); - const data = Buffer.concat([cipher.update(plain, 'utf8'), cipher.final()]); - return [ - 'v1', - iv.toString('base64url'), - cipher.getAuthTag().toString('base64url'), - data.toString('base64url'), - ].join('.'); -} - -/** The text a sealed value holds, or null when it cannot be opened (another secret, damage). */ -function open(secret, sealed) { - try { - const [version, iv, tag, data] = String(sealed).split('.'); - if (version !== 'v1') return null; - const decipher = createDecipheriv('aes-256-gcm', sealKey(secret), Buffer.from(iv, 'base64url')); - decipher.setAuthTag(Buffer.from(tag, 'base64url')); - return Buffer.concat([decipher.update(Buffer.from(data, 'base64url')), decipher.final()]).toString( - 'utf8' - ); - } catch { - return null; - } -} +// (the shared helper in lib/seal.js; the purpose keeps the key the one this credential has always had) +const PURPOSE = 'pigeon-credential'; // --- the client ------------------------------------------------------------------------------ @@ -181,7 +155,7 @@ export function createPigeonClient({ try { const [sealed, forUrl] = await Promise.all([store.get(SETTING_CREDENTIAL), store.get(SETTING_URL)]); if (!sealed || forUrl !== base) return null; - const credential = open(secret, sealed); + const credential = open(secret, PURPOSE, sealed); if (!credential) report(loud, 'warn', 'The stored Pigeon credential cannot be opened; registering again.'); return credential; @@ -224,7 +198,7 @@ export function createPigeonClient({ throw new PigeonFailure('rejected', 'malformed_credential'); } try { - await store.set(SETTING_CREDENTIAL, seal(secret, credential)); + await store.set(SETTING_CREDENTIAL, seal(secret, PURPOSE, credential)); await store.set(SETTING_URL, base); } catch { throw new PigeonFailure('rejected', 'storage_error'); diff --git a/src/lib/seal.js b/src/lib/seal.js new file mode 100644 index 00000000..e261645c --- /dev/null +++ b/src/lib/seal.js @@ -0,0 +1,44 @@ +// Seal a stored credential with AES-256-GCM, keyed from the instance's session secret (HKDF-SHA256, the +// `purpose` as the key's context, so one purpose's key never opens another's value). A sealed value reads +// `v1...`. A value that cannot be opened (the secret changed, the text was damaged) reads as +// null, and the caller makes a new credential instead of failing. +// +// Used by the Postal Pigeon credential (lib/pigeon.js) and the bridge's webhook tokens (db/bridge.js). +import { createCipheriv, createDecipheriv, hkdfSync, randomBytes } from 'node:crypto'; + +const keyFor = (secret, purpose) => Buffer.from(hkdfSync('sha256', String(secret), 'sylo', purpose, 32)); + +/** Is this text a value this module sealed? */ +export const isSealed = (value) => typeof value === 'string' && value.startsWith('v1.'); + +/** + * @param {string} secret the session secret + * @param {string} purpose what the value is for ('pigeon-credential', 'bridge-webhook') + * @param {string} plain + */ +export function seal(secret, purpose, plain) { + const iv = randomBytes(12); + const cipher = createCipheriv('aes-256-gcm', keyFor(secret, purpose), iv); + const data = Buffer.concat([cipher.update(plain, 'utf8'), cipher.final()]); + return [ + 'v1', + iv.toString('base64url'), + cipher.getAuthTag().toString('base64url'), + data.toString('base64url'), + ].join('.'); +} + +/** The text a sealed value holds, or null when it cannot be opened (another secret or purpose, damage). */ +export function open(secret, purpose, sealed) { + try { + const [version, iv, tag, data] = String(sealed).split('.'); + if (version !== 'v1') return null; + const decipher = createDecipheriv('aes-256-gcm', keyFor(secret, purpose), Buffer.from(iv, 'base64url')); + decipher.setAuthTag(Buffer.from(tag, 'base64url')); + return Buffer.concat([decipher.update(Buffer.from(data, 'base64url')), decipher.final()]).toString( + 'utf8' + ); + } catch { + return null; + } +} diff --git a/src/mirror/bridgeLink.js b/src/mirror/bridgeLink.js new file mode 100644 index 00000000..43cd8c07 --- /dev/null +++ b/src/mirror/bridgeLink.js @@ -0,0 +1,257 @@ +// The last step of a copy: link every mirrored text channel through the bridge, so a message written in a +// Discord channel appears in its Fluxer copy and the other way round. +// +// What this does, and what it leaves to the bridge (src/modules/bridge.js): +// - It looks at the channel map of a pair (the id map says which Discord channel is which Fluxer channel) and +// links every pair of text and announcement channels, through the bridge's own link model and relay. +// - No link code is used. Both server owners already agreed to the mirror, and the pair keeps who and when; +// the link records those two people (as " (server mirror)") as the ones who agreed to it. +// - Every rule of the bridge holds for such a link (see createMirrorLink): age restriction, the two channels +// being as private as each other, the owners' and Sylo's own rights in the channels, a visible notice in +// both channels, bots off, a webhook per channel. A channel pair that fails a rule is skipped and reported, +// and the rest go on. The Bridge module is switched on in both servers (it has to be on for a link to carry +// anything) and both servers' audit logs say so. +// - It is safe to run again: a pair that already has its link (made by the mirror, or found already linked +// as this very pair) is left alone, and only missing pairs are linked. A pair that was unlinked on purpose +// afterwards (on the Bridge page) is not linked again; that decision belongs to the owners. +// - The links are held with the mirror: a mirror that is paused, stopped or removed pauses them, and they run +// again when the mirror does (syncMirrorLinks, releaseMirrorLinks; the engine calls both). +// +// Linking is done in batches of LINKS_PER_RUN; `remaining` says more are waiting, and the engine queues another +// run. It throws only for a failure the copy should retry (the queue retries it with a delay). +import { ChannelType } from 'discord.js'; +import { config } from '../config.js'; +import { log } from '../lib/log.js'; +import { runtime } from '../runtime.js'; +import { guildByKey } from '../platform/platformOf.js'; +import { + PAIR_STATUS, + getMapping, + getMirrorLink, + getPair, + listMappings, + listMirrorLinks, + putMapping, + putMirrorLink, +} from '../db/mirror.js'; +import { linkForChannel } from '../db/bridge.js'; +import { isModuleEnabled } from '../db/modules.js'; +import { + BridgeError, + createMirrorLink, + enableBridgeForMirror, + holdMirrorLink, + releaseMirrorLink, + resumeMirrorLink, +} from '../modules/bridge.js'; + +/** How many pairs one run links (each makes two webhooks and posts two notices). */ +export const LINKS_PER_RUN = 25; +const PAUSE_BETWEEN_MS = 500; + +const TEXT_TYPES = new Set([ChannelType.GuildText, ChannelType.GuildAnnouncement]); +const isText = (channel) => Boolean(channel) && TEXT_TYPES.has(channel.type); +const channelOf = (guild, id) => guild?.channels?.cache?.get?.(String(id)) ?? null; + +/** The states in which the mirror runs, and so do the links it made. */ +const RUNNING = new Set([PAIR_STATUS.COPYING, PAIR_STATUS.LIVE, PAIR_STATUS.DONE]); +export const mirrorRuns = (pair) => Boolean(pair) && RUNNING.has(pair.status); + +/** The first linking of a pair is the one that may switch the Bridge module on; this id map entry says it happened. */ +const MODULE_MARKER = ['bridge', 'module']; + +const sleepReal = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + +/** An owner's member in their own server (null when it cannot be had). */ +async function memberOf(guild, userId) { + if (!guild || !userId) return null; + try { + return ( + guild.members?.cache?.get?.(String(userId)) ?? (await guild.members?.fetch?.(String(userId))) ?? null + ); + } catch { + return null; + } +} + +/** + * Put a pair's links in step with the mirror: while it runs, the links it holds run again; otherwise they are + * paused. Only links the mirror made are touched, and only those it paused itself are resumed. + * @returns {Promise<{ held: number, resumed: number, waiting: string[] }>} + */ +export async function syncMirrorLinks(pairOrId) { + const pair = typeof pairOrId === 'object' ? pairOrId : await getPair(Number(pairOrId)); + const out = { held: 0, resumed: 0, waiting: [] }; + if (!pair) return out; + for (const row of await listMirrorLinks(pair.id)) { + if (mirrorRuns(pair)) { + const res = await resumeMirrorLink(row.linkId); + if (res.resumed) out.resumed += 1; + else if (res.reason) out.waiting.push(res.reason); + } else if (await holdMirrorLink(row.linkId)) { + out.held += 1; + } + } + return out; +} + +/** + * The mirror is being removed by `side` ('source' the Discord server, 'target' the Fluxer server): its links stay, + * paused by that server, so only its owner can resume them on the Bridge page and the other server's owner can + * unlink them. + */ +export async function releaseMirrorLinks(pairId, side = '') { + const linkSide = side === 'source' ? 'a' : side === 'target' ? 'b' : ''; + let released = 0; + for (const row of await listMirrorLinks(pairId)) { + if (await releaseMirrorLink(row.linkId, linkSide)) released += 1; + } + return released; +} + +/** + * Link every mirrored text channel pair that is not linked yet. + * @param {number} pairId + * @param {{ sleep?: (ms: number) => Promise, limit?: number, skip?: Set, open?: boolean }} [opts] + * `skip`: channels (Discord ids) an earlier run of the same step already tried, so a batch does not try them again; + * `open`: sign-in is off (the default follows the configuration), so there are no owners to check + * @returns {Promise<{ linked: number, already: number, remaining: number, tried: string[], skipped: { sourceId: string, name: string, code: string, message: string }[], reason?: string }>} + */ +export async function linkMirroredChannels( + pairId, + { sleep = sleepReal, limit = LINKS_PER_RUN, skip = new Set(), open = !config.authEnabled } = {} +) { + const none = (reason) => ({ linked: 0, already: 0, remaining: 0, tried: [], skipped: [], reason }); + const pair = await getPair(Number(pairId)); + if (!pair) return none('no-such-mirror'); + if (!mirrorRuns(pair)) return none('mirror-not-running'); + if (!runtime.clients.discord || !runtime.clients.fluxer) return none('bridge-not-available'); + + const sg = guildByKey(pair.sourceGuild); + const tg = guildByKey(pair.targetGuild); + if (!sg || !tg) return none('server-missing'); + + // The links the mirror holds go on again first (this is also how a "copy again" wakes them after a pause). + await syncMirrorLinks(pair); + + const todo = []; + const skipped = []; + let already = 0; + for (const m of await listMappings(pair.id, 'channel')) { + if (m.removedAt > 0) continue; + const src = channelOf(sg, m.sourceId); + const dst = channelOf(tg, m.targetId); + if (!isText(src) || !isText(dst)) continue; // voice, category, forum and the like are not bridged + if (await getMirrorLink(pair.id, m.sourceId)) { + already += 1; + continue; + } + if (skip.has(m.sourceId)) continue; + const found = await linkForChannel(pair.sourceGuild, m.sourceId); + if (found) { + const same = + found.side === 'a' && found.link.guildB === pair.targetGuild && found.link.channelB === m.targetId; + if (same && String(found.link.requestedBy).endsWith('(server mirror)')) { + // A link this mirror made whose record was lost (a restart between the two writes): take it over. + await putMirrorLink(pair.id, m.sourceId, m.targetId, found.link.id); + already += 1; + } else { + // A link the owners made themselves is theirs: it is never taken over, and never changed. + skipped.push({ + sourceId: m.sourceId, + name: m.label || src.name, + code: 'exists', + message: same + ? 'this pair is already bridged by hand, and the mirror leaves it as it is' + : 'this channel is already bridged to another channel', + }); + } + continue; + } + todo.push({ sourceId: m.sourceId, targetId: m.targetId, name: m.label || src.name }); + } + if (todo.length === 0) { + const out = { linked: 0, already, remaining: 0, tried: [], skipped }; + return already || skipped.length ? out : { ...out, reason: 'nothing-to-link' }; + } + + // Only the first linking of a pair switches the Bridge module on. After that, a server that switched it off has + // decided so (it is the switch for every bridge in that server), and the mirror does not switch it on again. + if (await getMapping(pair.id, ...MODULE_MARKER)) { + const off = []; + for (const key of [pair.sourceGuild, pair.targetGuild]) { + if (!(await isModuleEnabled(key, 'bridge'))) off.push(key); + } + if (off.length > 0) { + for (const item of todo) { + skipped.push({ + sourceId: item.sourceId, + name: item.name, + code: 'bridge-off', + message: + 'the Bridge module is switched off in one of the servers, and the mirror does not switch it on again', + }); + } + return { linked: 0, already, remaining: 0, tried: [], skipped, reason: 'bridge-off' }; + } + } else { + await enableBridgeForMirror(pair.sourceGuild); + await enableBridgeForMirror(pair.targetGuild); + await putMapping({ + pairId: pair.id, + kind: MODULE_MARKER[0], + sourceId: MODULE_MARKER[1], + targetId: '', + owned: false, + specHash: '', + }); + } + + // The owners who agreed must be able to see, read and send in each channel themselves (as for a link made by + // hand). With sign-in off there is no one to check. + const members = { + source: open ? null : await memberOf(sg, pair.sourceConsentUser), + target: open ? null : await memberOf(tg, pair.targetConsentUser), + }; + const consent = { + sourceBy: pair.sourceConsentBy, + targetBy: pair.targetConsentBy, + // The people who agreed to the mirror (the bridge's own owner check then pauses the link if a server changes hands). + sourceOwner: pair.sourceConsentUser || String(sg.ownerId ?? ''), + targetOwner: pair.targetConsentUser || String(tg.ownerId ?? ''), + }; + + let linked = 0; + let attempted = 0; + const tried = []; + for (const item of todo) { + if (attempted >= limit) break; + attempted += 1; + tried.push(item.sourceId); + try { + const link = await createMirrorLink({ + source: { guildKey: pair.sourceGuild, channelId: item.sourceId }, + target: { guildKey: pair.targetGuild, channelId: item.targetId }, + members, + open, + consent, + }); + await putMirrorLink(pair.id, item.sourceId, item.targetId, link.id); + linked += 1; + } catch (err) { + if (!(err instanceof BridgeError)) throw err; + log.info( + 'mirror', + `pair ${pair.id}: ${item.name} not linked (${err.code || err.status}): ${err.message}` + ); + skipped.push({ + sourceId: item.sourceId, + name: item.name, + code: err.code || 'refused', + message: err.message, + }); + } + if (attempted < limit && attempted < todo.length) await sleep(PAUSE_BETWEEN_MS); + } + return { linked, already, remaining: Math.max(0, todo.length - attempted), tried, skipped }; +} diff --git a/src/mirror/consent.js b/src/mirror/consent.js new file mode 100644 index 00000000..f2b06471 --- /dev/null +++ b/src/mirror/consent.js @@ -0,0 +1,251 @@ +// Consent for a mirror: nothing is created on the Fluxer server until the owner of each server has agreed. Only a +// server's owner can (src/web/lib/guildOwner.js): not an Administrator, not a dashboard Edit holder, not a bot master. +// The Discord owner agrees by asking. The Fluxer owner agrees in one of two ways: +// - the same person owns both servers (their account has both logins): both sides agree at once; +// - otherwise the request carries a one-time code. The Fluxer server's owner opens the dashboard there and enters +// it. The code expires after 15 minutes, is stored only as a hash, and five wrong tries cancel the request. +// The owners' platform user ids are kept with the agreement, and the engine checks them again before every piece of +// work (ownership.js): a server that changes hands pauses the mirror until the new owner confirms. +// +// Everything that wants to know whether a mirror may go ahead asks requirePairConsent(). It asks the current +// consent provider, so the check can later be replaced by the bridge's own link codes (setConsentProvider) +// without touching the engine or the routes. +import { createHash, randomBytes, timingSafeEqual } from 'node:crypto'; +import { + PAIR_STATUS, + createPair, + deletePair, + getPair, + getPairBySource, + getPairByTarget, + countMappings, + updatePair, +} from '../db/mirror.js'; +import { normaliseOptions } from './spec.js'; + +export const CODE_TTL_MS = 15 * 60_000; +export const MAX_CODE_ATTEMPTS = 5; + +export class ConsentError extends Error { + /** @param {string} message @param {string} code a short machine-readable reason */ + constructor(message, code) { + super(message); + this.name = 'ConsentError'; + this.code = code; + } +} + +// No 0, O, 1, I or L: a code is read aloud or typed from a screenshot. +const ALPHABET = 'ABCDEFGHJKMNPQRSTUVWXYZ23456789'; + +/** A new code like "K7QW-H2NP". */ +export function newCode() { + const bytes = randomBytes(8); + const chars = [...bytes].map((b) => ALPHABET[b % ALPHABET.length]); + return `${chars.slice(0, 4).join('')}-${chars.slice(4).join('')}`; +} + +const normaliseCode = (code) => + String(code ?? '') + .toUpperCase() + .replace(/[^A-Z0-9]/g, ''); +const hashCode = (pairKey, code) => + createHash('sha256') + .update(`${pairKey}|${normaliseCode(code)}`) + .digest('hex'); +const pairKey = (source, target) => `${source}>${target}`; + +/** + * The consent provider: given a pair, may a mirror run? The default checks the two agreements the pair + * records. A replacement (the bridge's link codes) has the same shape. + * @type {{ name: string, check: (pair: any) => Promise<{ ok: boolean, reason?: string }> | { ok: boolean, reason?: string } }} + */ +const codeProvider = { + name: 'mirror-code', + check(pair) { + if (!pair) return { ok: false, reason: 'There is no such mirror.' }; + if (!pair.sourceConsentBy) return { ok: false, reason: 'Nobody has agreed on the Discord server yet.' }; + if (!pair.targetConsentBy) return { ok: false, reason: 'Nobody has agreed on the Fluxer server yet.' }; + return { ok: true }; + }, +}; + +let provider = codeProvider; + +/** Swap the consent check (the bridge's link codes can reuse the same shape). Returns the one replaced. */ +export function setConsentProvider(next) { + const previous = provider; + provider = next ?? codeProvider; + return previous; +} + +/** + * Throws a ConsentError unless both servers have agreed to this mirror. Returns the pair. + * @param {number | any} pairOrId + */ +export async function requirePairConsent(pairOrId) { + const pair = typeof pairOrId === 'object' ? pairOrId : await getPair(Number(pairOrId)); + const verdict = await provider.check(pair); + if (!verdict.ok) + throw new ConsentError(verdict.reason ?? 'Both servers have to agree first.', 'no-consent'); + return pair; +} + +/** + * The options of a request. Real deletion on Fluxer only takes effect when the same person owns both servers; for + * anyone else it is kept as a request that the owner of the Fluxer server has to turn on with their own switch. + */ +function askedOptions(options, targetOwner) { + const out = normaliseOptions(options); + if (out.mirrorDeletes || out.mirrorDeletesRequested) { + out.mirrorDeletesRequested = true; + out.mirrorDeletes = Boolean(targetOwner) && out.mirrorDeletes; + } + return out; +} + +/** + * The options a new request keeps when it brings none of its own. Real deletion on Fluxer is the Fluxer owner's own + * switch, so it is kept only when the same person who switched it on (the Fluxer owner recorded on the pair) now owns + * both servers; a new Fluxer owner, or a request from someone who is not the Fluxer owner, starts with it off (the + * request for it stays, so the owner can switch it on themselves). + */ +function inheritedOptions(pair, targetOwner, targetUser) { + const out = normaliseOptions(pair.options); + out.mirrorDeletes = + out.mirrorDeletes && + Boolean(targetOwner) && + Boolean(pair.targetConsentUser) && + pair.targetConsentUser === targetUser; + return out; +} + +/** A pair that has never copied anything may be replaced by a new request; one that has, may not. */ +const replaceable = (pair) => + pair.status === PAIR_STATUS.PENDING || + pair.status === PAIR_STATUS.READY || + pair.status === PAIR_STATUS.STOPPED; + +/** + * Ask for a mirror of a Discord server onto a Fluxer server. The caller has checked that the actor is the owner of + * the Discord server and that Sylo is in both servers; `targetOwner` says whether the same person also owns the + * Fluxer server. `sourceUser` and `targetUser` are the owners' platform user ids, kept so the engine can tell when + * a server changes hands. + * @param {{ sourceGuild: string, targetGuild: string, actor: string, sourceUser?: string, targetUser?: string, targetOwner: boolean, options?: object, now?: number }} p + * @returns {Promise<{ pair: any, code: string | null }>} `code` is shown once, only when the Fluxer side still has to agree + */ +export async function requestMirror({ + sourceGuild, + targetGuild, + actor, + sourceUser = '', + targetUser = '', + targetOwner, + options, + now = Date.now(), +}) { + if (sourceGuild === targetGuild) + throw new ConsentError('A server cannot be mirrored onto itself.', 'same-server'); + + const bySource = await getPairBySource(sourceGuild); + if (bySource && (bySource.targetGuild !== targetGuild || !replaceable(bySource))) { + if (!replaceable(bySource)) { + throw new ConsentError('This server is already mirrored. Stop that mirror first.', 'already-mirrored'); + } + await deletePair(bySource.id); + } + const byTarget = await getPairByTarget(targetGuild); + if (byTarget && byTarget.sourceGuild !== sourceGuild) { + const stale = byTarget.status === PAIR_STATUS.PENDING && byTarget.codeExpiresAt < now; + if (!stale && !replaceable(byTarget)) { + throw new ConsentError('That Fluxer server is already the target of another mirror.', 'target-busy'); + } + if (!stale && byTarget.status !== PAIR_STATUS.STOPPED) { + throw new ConsentError('Another server has a pending request for that Fluxer server.', 'target-busy'); + } + // A stopped mirror that made something keeps its id map: the items it made stay known as Sylo's own. + if (!stale && (await countMappings(byTarget.id)) > 0) { + throw new ConsentError( + 'That Fluxer server was the target of a mirror that made things there. Remove that mirror first.', + 'target-busy' + ); + } + await deletePair(byTarget.id); + } + + let pair = await getPairBySource(sourceGuild); + const code = targetOwner ? null : newCode(); + const fields = { + status: targetOwner ? PAIR_STATUS.READY : PAIR_STATUS.PENDING, + requestedBy: actor, + sourceConsentBy: actor, + sourceConsentAt: now, + sourceConsentUser: sourceUser, + targetConsentBy: targetOwner ? actor : '', + targetConsentAt: targetOwner ? now : 0, + targetConsentUser: targetOwner ? targetUser : '', + pauseKind: '', + pausedBy: '', + codeHash: code ? hashCode(pairKey(sourceGuild, targetGuild), code) : '', + codeExpiresAt: code ? now + CODE_TTL_MS : 0, + codeAttempts: 0, + lastError: '', + failures: 0, + ...(options + ? { options: askedOptions(options, targetOwner) } + : pair + ? { options: inheritedOptions(pair, targetOwner, targetUser) } + : {}), + }; + if (pair) { + pair = await updatePair(pair.id, fields, now); + } else { + pair = await createPair({ sourceGuild, targetGuild, options: askedOptions(options, targetOwner), now }); + pair = await updatePair(pair.id, fields, now); + } + return { pair, code }; +} + +/** + * The owner of the Fluxer server agrees, with the code the request showed. Looks only at the pair + * whose target is `targetGuild`, so a code cannot be used for another server. + * @param {{ targetGuild: string, code: string, actor: string, userId?: string, now?: number }} p + */ +export async function acceptMirror({ targetGuild, code, actor, userId = '', now = Date.now() }) { + const pair = await getPairByTarget(targetGuild); + if (!pair || pair.status !== PAIR_STATUS.PENDING || !pair.codeHash) { + throw new ConsentError('There is no request waiting for this server.', 'no-request'); + } + if (pair.codeExpiresAt < now) { + await deletePair(pair.id); + throw new ConsentError('The code has expired. Ask for a new one on the Discord server.', 'expired'); + } + const given = Buffer.from(hashCode(pairKey(pair.sourceGuild, pair.targetGuild), code), 'hex'); + const want = Buffer.from(pair.codeHash, 'hex'); + if (given.length !== want.length || !timingSafeEqual(given, want)) { + const attempts = pair.codeAttempts + 1; + if (attempts >= MAX_CODE_ATTEMPTS) { + await deletePair(pair.id); + throw new ConsentError( + 'Too many wrong codes. The request is cancelled; ask for a new one.', + 'cancelled' + ); + } + await updatePair(pair.id, { codeAttempts: attempts }, now); + throw new ConsentError('That code is not right.', 'wrong-code'); + } + const next = await updatePair( + pair.id, + { + status: PAIR_STATUS.READY, + targetConsentBy: actor, + targetConsentAt: now, + targetConsentUser: userId, + codeHash: '', + codeExpiresAt: 0, + codeAttempts: 0, + }, + now + ); + return next; +} diff --git a/src/mirror/engine.js b/src/mirror/engine.js new file mode 100644 index 00000000..4816a141 --- /dev/null +++ b/src/mirror/engine.js @@ -0,0 +1,1538 @@ +// The mirror engine: previews a copy, runs it from a queue and keeps the Fluxer server in step afterwards. +// +// previewPair(pair) the dry run: the plan and everything that cannot be carried over, nothing created +// startCopy(pair) turns the plan into queue jobs (src/db/mirror.js) and starts working them +// runPair(pairId) works one pair's queue: one job at a time, spaced out, backed off when Fluxer says +// so, and paused (with the reason) when it keeps failing +// noteSourceChange(...) a change on Discord (a channel renamed, a role deleted ...) becomes a job +// tick() what the timer calls: starts a worker for every pair with work to do +// +// Rules the code below keeps: +// - Discord is the source and is only ever read. Fluxer is written to, and only through the id map: a +// Fluxer item is deleted only when Sylo created it (`owned`) and its Discord original is gone. +// - A job names an item, not a payload. It reads the item as it is when it runs, so a stale job never +// writes old data, and a burst of events about one item is one job. +// - Each pair has its own worker, its own spacing and its own failure count: one pair going wrong +// never slows or stops another. +// - The queue is in the database. After a restart the unfinished jobs are still there; a job that was running +// when the process stopped is marked "recovering", and before it creates anything it looks for what the +// dead run may already have made (and adopts it, never as something Sylo may delete). +import { config } from '../config.js'; +import * as store from '../db/mirror.js'; +import { getGuildModule, getGuildModules, setGuildModule } from '../db/modules.js'; +import { recordAudit } from '../db/audit.js'; +import { getModule } from '../modules/registry.js'; +import { invalidateHealth } from '../lib/healthCache.js'; +import { log } from '../lib/log.js'; +import { allClientsReady, guildByKey } from '../platform/platformOf.js'; +import { safeFetch } from '../lib/safeFetch.js'; +import { requirePairConsent } from './consent.js'; +import { + linkMirroredChannels as defaultLink, + releaseMirrorLinks as defaultRelease, + syncMirrorLinks as defaultSync, +} from './bridgeLink.js'; +import { MirrorError, MirrorPause, classifyError } from './errors.js'; +import { checkOwnership } from './ownership.js'; +import { NOT_COPIED_MODULES, buildModuleSpec, buildPlan, channelOrder, copiedSets } from './plan.js'; +import { snapshotDiscordGuild, snapshotFluxerGuild } from './snapshot.js'; +import { createFluxerTarget } from './target.js'; +import { + FLUXER_LIMITS, + buildChannelSpec, + buildEmojiSpec, + buildIdentitySpec, + buildRoleSpec, + buildStickerSpec, + normaliseOptions, + refsIn, + resolveSpec, + roleSkipReason, + specHash, +} from './spec.js'; +import { ELEVATED_FLUXER_MASK, keepFluxerOnly } from './permissions.js'; + +const { PAIR_STATUS } = store; +const KINDS_WITH_MAP = ['role', 'channel', 'emoji', 'sticker', 'guild', 'order', 'module']; +const SOURCE_LIST = { role: 'roles', channel: 'channels', emoji: 'emojis', sticker: 'stickers' }; +/** After a reorder-worthy change, wait a little so one drag of many channels is one reorder. */ +const REORDER_DELAY_MS = 3000; +const PRUNE_EVERY_MS = 60 * 60_000; +const LINK_SWEEP_MS = 60_000; +const KEEP_FINISHED_MS = 7 * 86_400_000; +const ADOPT_SKEW_MS = 5000; +// Fluxer ids are snowflakes: the time they were made is in the high bits. +const FLUXER_EPOCH = 1420070400000n; +const idTime = (id) => Number((BigInt(id) >> 22n) + FLUXER_EPOCH); + +const sleepReal = (ms) => + new Promise((resolve) => { + const t = setTimeout(resolve, ms); + t.unref?.(); + }); + +/** The default image download: only http(s), never a private address, never more than `maxBytes`. */ +async function fetchImageReal(url, { maxBytes }) { + const res = await safeFetch(url, { maxBytes, timeoutMs: 15_000, allowPrivate: false }); + if (!res.ok) throw new MirrorError(`The image could not be downloaded (HTTP ${res.status}).`, 'item'); + const type = String(res.headers.get('content-type') ?? '') + .split(';')[0] + .trim(); + const mime = /^image\/(png|gif|jpeg|webp)$/.test(type) ? type : sniffMime(res.body); + return { bytes: res.body, mime }; +} + +function sniffMime(buf) { + if (buf.length > 3 && buf[0] === 0x89 && buf[1] === 0x50) return 'image/png'; + if (buf.length > 2 && buf[0] === 0x47 && buf[1] === 0x49) return 'image/gif'; + if (buf.length > 2 && buf[0] === 0xff && buf[1] === 0xd8) return 'image/jpeg'; + if (buf.length > 11 && buf.toString('latin1', 8, 12) === 'WEBP') return 'image/webp'; + return 'image/png'; +} + +/** + * @param {Partial<{ + * now: () => number, sleep: (ms: number) => Promise, minIntervalMs: number, backoffBaseMs: number, + * maxBackoffMs: number, maxAttempts: number, pauseAfterFailures: number, jobBudgetMs: number, + * sourceGuild: (key: string) => any, targetGuild: (key: string) => any, makeTarget: (guild: any) => any, + * fetchImage: (url: string, opts: { maxBytes: number }) => Promise<{ bytes: Buffer, mime: string }>, + * clientsReady: () => boolean, linkMirroredChannels: (pairId: number) => Promise<{ linked: number, reason?: string }>, + * }>} [overrides] + */ +export function createEngine(overrides = {}) { + const deps = { + now: () => Date.now(), + sleep: sleepReal, + minIntervalMs: 600, + backoffBaseMs: 2000, + maxBackoffMs: 5 * 60_000, + maxAttempts: 8, + pauseAfterFailures: 3, + jobBudgetMs: 25_000, + sourceGuild: (key) => guildByKey(key), + targetGuild: (key) => guildByKey(key), + makeTarget: createFluxerTarget, + fetchImage: fetchImageReal, + clientsReady: allClientsReady, + linkMirroredChannels: defaultLink, + // The bridge links a mirror made follow its state: paused with it, running again with it, released when it goes. + syncLinks: defaultSync, + releaseLinks: defaultRelease, + // Only a server's owner may run a mirror: with sign-in on, the owners who agreed must still own the servers. + enforceOwnership: () => config.authEnabled, + // More than this many deletes on Discord in this many ms pause the mirror until its owner confirms them. + deleteBurst: { max: 5, windowMs: 10 * 60_000 }, + maxWorkers: 3, + ...overrides, + }; + + /** Pairs with a worker running right now: pair id -> promise. */ + const active = new Map(); + /** Per pair: kinds Fluxer refused as over a limit this run, and when it asked to wait. */ + const state = new Map(); + let lastPrune = 0; + let lastLinkSweep = 0; + let stopped = false; + + const stateOf = (pairId) => { + if (!state.has(pairId)) state.set(pairId, { limitReached: new Set(), blockedUntil: 0 }); + return state.get(pairId); + }; + + const touchHealth = (pair) => { + invalidateHealth(pair.sourceGuild); + invalidateHealth(pair.targetGuild); + }; + + // --- loading what a pair works on ------------------------------------------------------------------ + + /** The two servers of a pair as the platforms know them, or why they are not there. */ + function guildsOf(pair) { + const sg = deps.sourceGuild(pair.sourceGuild); + const tg = deps.targetGuild(pair.targetGuild); + return { sg, tg }; + } + + async function loadMappings(pairId) { + const out = {}; + for (const kind of KINDS_WITH_MAP) { + out[kind] = new Map((await store.listMappings(pairId, kind)).map((m) => [m.sourceId, m])); + } + return out; + } + + async function loadModules(pair) { + const rows = await getGuildModules(pair.sourceGuild); + const target = await getGuildModules(pair.targetGuild); + return { + source: rows.map((m) => ({ + id: m.id, + name: getModule(m.id)?.name ?? m.id, + enabled: m.enabled, + defaultEnabled: Boolean(getModule(m.id)?.defaultEnabled), + config: m.config, + ownerOnly: Boolean(getModule(m.id)?.ownerOnly), + })), + targetConfigured: new Set( + target.filter((m) => Object.keys(m.config ?? {}).length > 0).map((m) => m.id) + ), + }; + } + + /** + * What to tell the person when Sylo's role is not above the roles it has to order. Sylo cannot move its own role: the + * owner of the Fluxer server drags it (Server Settings, Roles). + * @param {Array<{ name: string }> | null} above the roles that rank above or level with Sylo's, or null when unknown + */ + function hierarchyAdvice(above) { + if (above === null) { + return "Sylo's own role on the Fluxer server could not be read. Check that Sylo is in the server with a role, then look again."; + } + const names = above.slice(0, 5).map((r) => r.name || r.id); + const more = above.length > names.length ? ` and ${above.length - names.length} more` : ''; + return `Sylo's role has to be the highest role on the Fluxer server (above every role except @everyone), or it cannot order the copied roles. These roles rank above or level with it: ${names.join(', ')}${more}. On Fluxer, the server owner opens Server Settings, then Roles, and drags Sylo's role to the very top (Sylo cannot move its own role); then look again here.`; + } + + /** The roles on Fluxer that rank above Sylo's (ignoring the roles the mirror made, which are ordered under it). */ + async function rolesAboveSylo(pairId, target) { + const own = (await store.listMappings(pairId, 'role')).map((m) => m.targetId); + return target.rolesAboveSylo(own); + } + + /** The copied roles in Discord's order, lowest first: the order they are sent to Fluxer in. */ + const roleOrderOf = (ctx) => + ctx.source.roles + .filter((r) => ctx.sets.copiedRoles.has(r.id)) + .sort((a, b) => a.position - b.position || (a.id < b.id ? -1 : 1)) + .map((r) => r.id); + + /** Has the current order of the copied roles been set on Fluxer? */ + async function rolesOrdered(ctx) { + const m = await store.getMapping(ctx.pair.id, 'order', 'roles'); + return Boolean(m && m.specHash === specHash(roleOrderOf(ctx))); + } + + // --- the dry run ----------------------------------------------------------------------------------- + + /** + * What a copy would do right now. Reads both servers and the id map; creates and changes nothing. + * @returns {Promise<{ ok: boolean, blockers: object[], warnings: object[], steps: object[], notes: object[], summary: object, target: object | null }>} + */ + async function previewPair(pair) { + const blockers = []; + const warnings = []; + const { sg, tg } = guildsOf(pair); + if (!sg) + blockers.push({ + code: 'source-missing', + message: 'Sylo is not in the Discord server (or it is not connected right now).', + }); + if (!tg) + blockers.push({ + code: 'target-missing', + message: 'Sylo is not in the Fluxer server (or it is not connected right now).', + }); + if (!sg || !tg) return { ok: false, blockers, warnings, steps: [], notes: [], summary: {}, target: null }; + + const target = deps.makeTarget(tg); + const missing = target.missingPermissions(); + if (missing.length) { + blockers.push({ + code: 'target-permissions', + message: `Sylo needs ${missing.join(', ')} on the Fluxer server. Give Sylo's role Administrator there, or those permissions.`, + }); + } + const owners = checkOwnership(pair, sg, tg, deps.enforceOwnership()); + if (!owners.ok) blockers.push({ code: 'ownership', message: owners.reason }); + const options = normaliseOptions(pair.options); + const source = snapshotDiscordGuild(sg); + const targetSnap = snapshotFluxerGuild(tg); + const sets = copiedSets(source, options); + if (options.roles && sets.copiedRoles.size > 0) { + // Fluxer ranks roles by position and then by age, and a new role starts at the bottom. Sylo can only order the + // roles below its own, so its role has to outrank every other role there. A position number is not a rank (a + // fresh server has Sylo at position 1 and is fine), so the rank is compared, not the number. + const above = await rolesAboveSylo(pair.id, target); + if (above === null || above.length > 0) { + blockers.push({ code: 'role-hierarchy', message: hierarchyAdvice(above) }); + } + } + const plan = buildPlan({ + source, + target: targetSnap, + mappings: await loadMappings(pair.id), + options, + modules: options.modules ? await loadModules(pair) : undefined, + limits: target.limits(), + }); + const existing = [ + [targetSnap.channels.length, 'channel(s)'], + [targetSnap.roles.filter((r) => !r.isEveryone && !r.managed).length, 'role(s)'], + [targetSnap.emojiCount, 'emoji'], + [targetSnap.stickerCount, 'sticker(s)'], + ].filter(([n]) => n > 0); + if (existing.length > 0 && pair.copyRun === 0) { + warnings.push({ + code: 'target-not-empty', + message: `The Fluxer server already has ${existing.map(([n, what]) => `${n} ${what}`).join(', ')}. Sylo leaves them as they are and never deletes anything it did not create.`, + }); + } + return { + ok: blockers.length === 0, + blockers, + warnings, + steps: plan.steps, + notes: plan.notes, + summary: plan.summary, + target: { id: tg.id, name: targetSnap.name }, + }; + } + + // --- turning a plan into work ---------------------------------------------------------------------- + + const JOB_OF_KIND = { + guild: { op: 'sync', kind: 'guild' }, + role: { op: 'sync', kind: 'role' }, + 'order-roles': { op: 'reorder', kind: 'roles' }, + channel: { op: 'sync', kind: 'channel' }, + 'order-channels': { op: 'reorder', kind: 'channels' }, + emoji: { op: 'sync', kind: 'emoji' }, + sticker: { op: 'sync', kind: 'sticker' }, + module: { op: 'sync', kind: 'module' }, + bridge: { op: 'link', kind: 'bridge' }, + }; + + /** + * Start (or repeat) the copy of a pair that both servers have agreed to. The plan becomes jobs; the worker + * does them. Throws a ConsentError when a side has not agreed, and a MirrorError when nothing can be done. + */ + async function startCopy(pairId) { + let pair = await requirePairConsent(pairId); + if (![PAIR_STATUS.READY, PAIR_STATUS.LIVE, PAIR_STATUS.DONE].includes(pair.status)) { + throw new MirrorError(`A mirror that is ${pair.status} cannot be copied now.`, 'item'); + } + const preview = await previewPair(pair); + if (!preview.ok) throw new MirrorError(preview.blockers.map((b) => b.message).join(' '), 'item'); + + const run = pair.copyRun + 1; + for (const step of preview.steps) { + if (step.op === 'unchanged') continue; + const job = JOB_OF_KIND[step.kind]; + await store.enqueueJob({ + pairId: pair.id, + run, + phase: 'copy', + op: job.op, + kind: job.kind, + sourceId: step.sourceId, + stepKey: `copy:${run}:${step.key}`, + payload: { + name: step.name, + planned: step.op, + ...(step.op === 'skip' ? { skip: step.reason ?? 'skipped', notes: step.notes ?? [] } : {}), + }, + now: deps.now(), + }); + } + await store.pruneMirrorJobs({ pairId: pair.id, keepRun: run - 1 }); + state.delete(pair.id); + pair = await store.updatePair( + pair.id, + { status: PAIR_STATUS.COPYING, copyRun: run, lastError: '', failures: 0, resumeStatus: '' }, + deps.now() + ); + touchHealth(pair); + return pair; + } + + // --- changes on Discord ---------------------------------------------------------------------------- + + const KIND_OPTION = { + role: 'roles', + channel: 'channels', + emoji: 'emoji', + sticker: 'stickers', + guild: 'name', + }; + + /** + * A thing changed on the Discord server. Becomes a job when a mirror is running for that server and + * continuous mirroring is on; otherwise nothing happens. + * @param {string} sourceKey + * @param {{ kind: 'role'|'channel'|'emoji'|'sticker'|'guild', op: 'sync'|'delete', id: string, reorder?: 'roles'|'channels' }} change + * @returns {Promise} whether work was queued + */ + async function noteSourceChange(sourceKey, change) { + const pair = await store.getPairBySource(sourceKey); + if (!pair) return false; + if (![PAIR_STATUS.COPYING, PAIR_STATUS.LIVE, PAIR_STATUS.PAUSED].includes(pair.status)) return false; + const options = normaliseOptions(pair.options); + if (!options.continuous) return false; + if ( + change.kind === 'guild' + ? !(options.name || options.icon || options.banner) + : !options[KIND_OPTION[change.kind]] + ) { + return false; + } + const now = deps.now(); + let queued; + if (change.op === 'delete') { + // A delete replaces any change still waiting for the same item. It is always queued: the job looks the + // copy up when it runs, so a delete that arrives while the copy is still being made is not lost. + await store.skipWaitingJobs(pair.id, change.kind, change.id, 'sync', 'deleted on Discord', now); + queued = await store.enqueueJob({ + pairId: pair.id, + phase: 'sync', + op: 'delete', + kind: change.kind, + sourceId: change.id, + stepKey: `delete:${change.kind}:${change.id}`, + now, + }); + // A burst of deletes (a nuke, a mistaken clean-up) must not run on its own: the mirror pauses, the deletes + // wait in the queue, and the owner has to confirm them in the dashboard before they are applied. + if (queued && pair.status !== PAIR_STATUS.PAUSED) { + const { max, windowMs } = deps.deleteBurst; + const recent = await store.countRecentDeletes(pair.id, now - windowMs); + if (recent > max) { + await pausePair( + pair, + `${recent} things were deleted on Discord within ${Math.round(windowMs / 60_000)} minutes. The mirror is paused so that a mistake or an attack does not spread to Fluxer. The owner has to confirm in the dashboard before these changes are applied.`, + { kind: 'deletes' } + ); + } + } + } else { + queued = await store.enqueueJob({ + pairId: pair.id, + phase: 'sync', + op: 'sync', + kind: change.kind, + sourceId: change.id, + stepKey: `sync:${change.kind}:${change.id}`, + now, + }); + } + if (change.reorder) { + const orderOk = change.reorder === 'roles' ? options.roles : options.channels; + if (orderOk) { + queued = + (await store.enqueueJob({ + pairId: pair.id, + phase: 'sync', + op: 'reorder', + kind: change.reorder, + sourceId: change.reorder, + stepKey: `reorder:${change.reorder}`, + now: now + REORDER_DELAY_MS, + })) || queued; + } + } + return queued; + } + + // --- the work of one job --------------------------------------------------------------------------- + + /** What a job found or did. `remote`: it called Fluxer, so the worker waits before the next one. */ + const done = (detail, extra = {}) => ({ status: 'done', detail, notes: [], remote: false, ...extra }); + const skipped = (detail, extra = {}) => ({ status: 'skipped', detail, notes: [], remote: false, ...extra }); + + const findSource = (ctx, kind, id) => ctx.source[SOURCE_LIST[kind]].find((x) => x.id === id) ?? null; + + /** Fluxer ids for the placeholders of a spec (making what is missing first), and the spec with them swapped in. */ + async function resolveFor(ctx, spec, depth) { + for (const ref of refsIn(spec)) { + const [kind, id] = ref.split(':'); + if (await store.getMapping(ctx.pair.id, kind, id)) continue; + if (depth >= 3) continue; + if (kind === 'role' && ctx.sets.copiedRoles.has(id)) await syncRole(ctx, id); + if (kind === 'channel' && ctx.sets.copiedChannels.has(id)) await syncChannel(ctx, id, depth + 1); + } + const ids = new Map(); + for (const ref of refsIn(spec)) { + const [kind, id] = ref.split(':'); + const m = await store.getMapping(ctx.pair.id, kind, id); + if (m) ids.set(ref, m.targetId); + } + const everyone = await store.getMapping(ctx.pair.id, 'role', ctx.source.everyoneId); + return resolveSpec(spec, (kind, id) => + kind === 'role' && id === 'everyone' + ? (everyone?.targetId ?? ctx.target.id) + : (ids.get(`${kind}:${id}`) ?? null) + ); + } + + /** + * Something on Fluxer that an interrupted job may have made just before it stopped: unmapped, matching, and made + * since the job first started (Fluxer ids carry their creation time). Anything older is not this job's work. + */ + async function adoptable(ctx, kind, match) { + const since = (ctx.job?.firstClaimAt || ctx.job?.updatedAt || 0) - ADOPT_SKEW_MS; + const taken = new Set((await store.listMappings(ctx.pair.id, kind)).map((m) => m.targetId)); + const pool = { + channel: ctx.target.listChannels(), + role: ctx.target.listRoles(), + emoji: ctx.target.listEmojis(), + sticker: ctx.target.listStickers(), + }[kind]; + return pool.find((x) => !taken.has(x.id) && idTime(x.id) >= since && match(x)) ?? null; + } + + /** @everyone exists on both sides: Sylo adopts it (never owns it) and only sets its permissions. */ + async function syncEveryone(ctx, role) { + const targetRole = snapshotFluxerGuild(ctx.targetGuild).roles.find((r) => r.isEveryone); + const { spec, notes } = buildRoleSpec(role, { keep: keepFluxerOnly(targetRole?.permissions ?? 0n) }); + const hash = specHash(spec); + const m = await store.getMapping(ctx.pair.id, 'role', role.id); + if (m && m.specHash === hash) return done('unchanged', { notes }); + await ctx.target.editRole(ctx.target.id, spec); + await store.putMapping({ + pairId: ctx.pair.id, + kind: 'role', + sourceId: role.id, + targetId: ctx.target.id, + owned: false, + specHash: hash, + now: deps.now(), + }); + return done('@everyone permissions set', { notes, remote: true }); + } + + async function syncRole(ctx, roleId) { + const role = findSource(ctx, 'role', roleId); + if (!role) return skipped('the role is gone on Discord'); + if (role.isEveryone) return syncEveryone(ctx, role); + if (!ctx.sets.copiedRoles.has(roleId)) return skipped(roleSkipReason(role) ?? 'roles are not copied'); + const { spec, notes } = buildRoleSpec(role); + const hash = specHash(spec); + // A role is made without its powerful permissions and gets them only once its place in the order is set: a new + // role starts at the bottom of the list, where a member with Manage Roles could hand it out. `specHash ''` marks + // a role that still waits for them (the role-order job asks for them again once the order is set). + const stripped = { ...spec, permissions: String(BigInt(spec.permissions) & ~ELEVATED_FLUXER_MASK) }; + const elevated = stripped.permissions !== spec.permissions; + const m = await store.getMapping(ctx.pair.id, 'role', roleId); + const waiting = elevated && (!m || m.specHash === '') && !(await rolesOrdered(ctx)); + const sent = waiting ? stripped : spec; + const put = (targetId, owned) => + store.putMapping({ + pairId: ctx.pair.id, + kind: 'role', + sourceId: roleId, + targetId, + owned, + specHash: waiting ? '' : hash, + label: role.name, + now: deps.now(), + }); + if (m) { + if (m.specHash === hash) return done('unchanged', { notes }); + try { + await ctx.target.editRole(m.targetId, sent); + await put(m.targetId, m.owned); + return done(waiting ? 'updated, powerful permissions wait for the role order' : 'updated', { + notes, + remote: true, + }); + } catch (err) { + if (classifyError(err).kind !== 'gone') throw err; + await store.deleteMapping(ctx.pair.id, 'role', roleId); // deleted on Fluxer: make it again + } + } + // Every creation or adoption, in the first copy and in continuous mode alike (also a role that was deleted on Fluxer + // and is made again), starts without the powerful permissions. The order step then runs, and only after it do the + // grant jobs give them back. `newRole` makes sure the order is set again and asked for. + const fresh = elevated; + const createSpec = fresh ? stripped : spec; + const newRole = () => roleMadeOnFluxer(ctx); + if (ctx.job?.recovering) { + const found = await adoptable( + ctx, + 'role', + (r) => r.name === spec.name && !r.managed && r.id !== ctx.target.id + ); + if (found) { + await ctx.target.editRole(found.id, createSpec); + await store.putMapping({ + pairId: ctx.pair.id, + kind: 'role', + sourceId: roleId, + targetId: found.id, + owned: false, + specHash: fresh ? '' : hash, + label: role.name, + now: deps.now(), + }); + await newRole(); + return done('adopted what an interrupted run made', { notes, remote: true }); + } + } + try { + const id = await ctx.target.createRole(createSpec); + await store.putMapping({ + pairId: ctx.pair.id, + kind: 'role', + sourceId: roleId, + targetId: id, + owned: true, + specHash: fresh ? '' : hash, + label: role.name, + now: deps.now(), + }); + await newRole(); + } catch (err) { + // The role was made but a second call failed: track it (with no fingerprint, so the next run updates it) + // instead of leaving it untracked and making another. + if (err?.createdId) { + await store.putMapping({ + pairId: ctx.pair.id, + kind: 'role', + sourceId: roleId, + targetId: err.createdId, + owned: true, + specHash: '', + label: role.name, + now: deps.now(), + }); + await newRole(); + } + throw err; + } + return done(fresh ? 'created, powerful permissions wait for the role order' : 'created', { + notes, + remote: true, + }); + } + + /** + * A role was made (or adopted, or made again) on Fluxer: its place in the order is not set yet. Forget that the order + * was set, so the order step really runs, and ask for it now. Inside a copy run the request is a copy job with the key + * of the plan's own order step, so it joins that step when it is still waiting; "copy again" leaves the step out when + * the order did not change on Discord, and a role that was deleted on Fluxer and is made again would otherwise stay at + * the bottom without its powerful permissions. An order that is already set is a no-op. + */ + async function roleMadeOnFluxer(ctx) { + await store.deleteMapping(ctx.pair.id, 'order', 'roles'); + const inCopy = ctx.job?.phase === 'copy'; + await store.enqueueJob({ + pairId: ctx.pair.id, + run: inCopy ? ctx.job.run : 0, + phase: inCopy ? 'copy' : 'sync', + op: 'reorder', + kind: 'roles', + sourceId: 'roles', + stepKey: inCopy ? `copy:${ctx.job.run}:order-roles:roles` : 'reorder:roles', + payload: inCopy ? { name: 'Role order', planned: 'update' } : {}, + now: deps.now(), + }); + } + + /** After the role order is set: ask again for every role that was made without its powerful permissions. */ + async function enqueueGrants(ctx) { + // Adopted roles wait too (they are not Sylo's to delete, but an interrupted run made them and they are ordered). + const pending = (await store.listMappings(ctx.pair.id, 'role')).filter((m) => m.specHash === ''); + for (const m of pending) { + await store.enqueueJob({ + pairId: ctx.pair.id, + run: ctx.job?.run ?? 0, + phase: ctx.job?.phase ?? 'sync', + op: 'sync', + kind: 'role', + sourceId: m.sourceId, + stepKey: `grant:role:${m.sourceId}`, + payload: { name: m.label, planned: 'update' }, + now: deps.now(), + }); + } + } + + async function syncChannel(ctx, channelId, depth = 0) { + const ch = findSource(ctx, 'channel', channelId); + if (!ch) return skipped('the channel is gone on Discord'); + const built = buildChannelSpec(ch, { source: ctx.source, copiedRoles: ctx.sets.copiedRoles }); + if (!built.spec) return skipped('no such channel type on Fluxer', { notes: built.notes }); + if (!ctx.sets.copiedChannels.has(channelId)) + return skipped('channels are not copied', { notes: built.notes }); + const hash = specHash(built.spec); + const m = await store.getMapping(ctx.pair.id, 'channel', channelId); + if (m && m.specHash === hash) return done('unchanged', { notes: built.notes }); + + const { value, unresolved } = await resolveFor(ctx, built.spec, depth); + value.permissionOverwrites = (value.permissionOverwrites ?? []).filter((o) => o.id); + const notes = [...built.notes]; + for (const ref of unresolved) { + notes.push({ + code: 'dependency-missing', + scope: 'channel', + ref: { kind: 'channel', id: ch.id, name: ch.name }, + message: `${ch.name}: ${ref.replace(':', ' ')} could not be made on Fluxer, so a link to it is left out.`, + }); + } + const put = (targetId, owned) => + store.putMapping({ + pairId: ctx.pair.id, + kind: 'channel', + sourceId: channelId, + targetId, + owned, + specHash: hash, + label: ch.name, + now: deps.now(), + }); + if (m) { + try { + await ctx.target.editChannel(m.targetId, value); + await put(m.targetId, m.owned); + return done('updated', { notes, remote: true }); + } catch (err) { + if (classifyError(err).kind !== 'gone') throw err; + await store.deleteMapping(ctx.pair.id, 'channel', channelId); + } + } + if (ctx.job?.recovering) { + const found = await adoptable( + ctx, + 'channel', + (c) => + c.name === value.name && c.type === value.type && (c.parentId ?? null) === (value.parentId ?? null) + ); + if (found) { + await ctx.target.editChannel(found.id, value); + await put(found.id, false); + return done('adopted what an interrupted run made', { notes, remote: true }); + } + } + await put(await ctx.target.createChannel(value), true); + // A channel made while the mirror runs is linked through the bridge too (the step links only what is missing). + if (ctx.job?.phase !== 'copy') await enqueueLinkStep(ctx.pair.id); + return done('created', { notes, remote: true }); + } + + /** Ask for the bridge step: link the mirrored channels that have no link yet. One open request at a time. */ + async function enqueueLinkStep(pairId) { + await store.enqueueJob({ + pairId, + phase: 'sync', + op: 'link', + kind: 'bridge', + sourceId: 'link', + stepKey: 'bridge:link', + payload: { name: 'Link new channels through the bridge', planned: 'create' }, + now: deps.now(), + }); + } + + /** Download an image for an emoji or sticker; null (with a note) when it is too big. */ + async function download(ctx, item, maxBytes, code, kind) { + if (!item.url) return { note: noteFor(code, kind, item, 'it has no image to copy') }; + try { + return { image: await deps.fetchImage(item.url, { maxBytes }) }; + } catch (err) { + if (/too large|over \d+ bytes/i.test(String(err?.message))) { + return { + note: noteFor( + code, + kind, + item, + `it is bigger than ${Math.round(maxBytes / 1024)} KB, the most Fluxer takes` + ), + }; + } + throw err; + } + } + const noteFor = (code, kind, item, why) => ({ + code, + scope: kind, + ref: { kind, id: item.id, name: item.name }, + message: `${item.name}: ${why}, so it is not copied.`, + }); + + async function syncExpression(ctx, kind, id) { + const item = findSource(ctx, kind, id); + if (!item) return skipped(`the ${kind} is gone on Discord`); + const built = kind === 'emoji' ? buildEmojiSpec(item) : buildStickerSpec(item); + if (!built.spec) return skipped('not supported on Fluxer', { notes: built.notes }); + if (kind === 'emoji' && item.managed) return skipped('managed by an integration'); + const hash = specHash(built.spec); + const m = await store.getMapping(ctx.pair.id, kind, id); + if (m && m.specHash === hash) return done('unchanged', { notes: built.notes }); + const put = (targetId, owned) => + store.putMapping({ + pairId: ctx.pair.id, + kind, + sourceId: id, + targetId, + owned, + specHash: hash, + label: item.name, + now: deps.now(), + }); + const limits = ctx.target.limits(); + if (m) { + try { + if (kind === 'emoji') await ctx.target.editEmoji(m.targetId, built.spec); + else await ctx.target.editSticker(m.targetId, built.spec); + await put(m.targetId, m.owned); + return done('updated', { notes: built.notes, remote: true }); + } catch (err) { + if (classifyError(err).kind !== 'gone') throw err; + await store.deleteMapping(ctx.pair.id, kind, id); + } + } + if (stateOf(ctx.pair.id).limitReached.has(kind)) { + return skipped('Fluxer is full', { + notes: [noteFor(`${kind}-limit`, kind, item, 'Fluxer has no room for more')], + }); + } + if (ctx.job?.recovering) { + const found = await adoptable(ctx, kind, (x) => x.name === built.spec.name); + if (found) { + await put(found.id, false); + return done('adopted what an interrupted run made', { notes: built.notes }); + } + } + const maxBytes = kind === 'emoji' ? limits.emojiBytes : limits.stickerBytes; + const got = await download(ctx, item, maxBytes, `${kind}-too-large`, kind); + if (got.note) return skipped('image not copied', { notes: [got.note] }); + try { + const targetId = + kind === 'emoji' + ? await ctx.target.createEmoji({ name: built.spec.name, image: got.image }) + : await ctx.target.createSticker({ ...built.spec, image: got.image }); + await put(targetId, true); + return done('created', { notes: built.notes, remote: true }); + } catch (err) { + if (classifyError(err).kind !== 'limit') throw err; + stateOf(ctx.pair.id).limitReached.add(kind); + return skipped('Fluxer is full', { + notes: [noteFor(`${kind}-limit`, kind, item, 'Fluxer refused it as over its limit')], + remote: true, + }); + } + } + + async function syncIdentity(ctx) { + const options = ctx.options; + const { spec } = buildIdentitySpec(ctx.source, options); + const hash = specHash(spec); + const m = await store.getMapping(ctx.pair.id, 'guild', 'identity'); + if (m && m.specHash === hash) return done('unchanged'); + const body = {}; + const notes = []; + if (spec.name) body.name = spec.name; + for (const key of ['icon', 'banner']) { + // Discord has none: leave whatever the Fluxer server has. + if (!spec[key]) continue; + const got = await download( + ctx, + { id: key, name: key, url: spec[key] }, + 8 * 1024 * 1024, + `${key}-too-large`, + 'guild' + ); + if (got.note) notes.push(got.note); + else body[key] = `data:${got.image.mime};base64,${got.image.bytes.toString('base64')}`; + } + if (Object.keys(body).length) await ctx.target.editGuild(body); + await store.putMapping({ + pairId: ctx.pair.id, + kind: 'guild', + sourceId: 'identity', + targetId: ctx.target.id, + owned: false, + specHash: hash, + now: deps.now(), + }); + return done('updated', { notes, remote: Object.keys(body).length > 0 }); + } + + async function syncModule(ctx, moduleId) { + const def = getModule(moduleId); + if (!def || def.ownerOnly || NOT_COPIED_MODULES.includes(moduleId)) return skipped('not copied'); + const source = await getGuildModule(ctx.pair.sourceGuild, moduleId); + const mapped = await store.getMapping(ctx.pair.id, 'module', moduleId); + const there = await getGuildModule(ctx.pair.targetGuild, moduleId); + if (!mapped && Object.keys(there.config ?? {}).length > 0) + return skipped('already set up on the Fluxer server'); + // The same checks as saving these settings in the dashboard: a role with powerful permissions is not handed + // out by the copy, and a giveaway does not start pinging everyone (module-grant-dropped, module-ping-lowered). + const built = buildModuleSpec({ id: moduleId, name: def.name, ...source }, ctx.sets, ctx.source); + const hash = specHash(built.spec); + if (mapped && mapped.specHash === hash) return done('unchanged', { notes: built.notes }); + const { value } = await resolveFor(ctx, built.spec, 0); + await setGuildModule(ctx.pair.targetGuild, moduleId, { + enabled: Boolean(value.enabled), + config: value.config ?? {}, + }); + await recordAudit(ctx.pair.targetGuild, { + actor: 'Sylo (mirror)', + action: 'mirror:module', + detail: `${def.name} copied from the Discord server`, + }); + await store.putMapping({ + pairId: ctx.pair.id, + kind: 'module', + sourceId: moduleId, + targetId: moduleId, + owned: false, + specHash: hash, + now: deps.now(), + }); + return done('copied', { notes: built.notes }); + } + + async function reorder(ctx, which) { + if (which === 'roles') { + const order = roleOrderOf(ctx); + const hash = specHash(order); + const m = await store.getMapping(ctx.pair.id, 'order', 'roles'); + if (m && m.specHash === hash) { + await enqueueGrants(ctx); + return done('unchanged'); + } + // Sylo has to outrank every other role to order the copied ones (see hierarchyAdvice). If it does not, nothing + // is sent and the mirror pauses until the Fluxer owner has moved Sylo's role up. + const above = await rolesAboveSylo(ctx.pair.id, ctx.target); + if (above === null || above.length > 0) throw new MirrorPause(hierarchyAdvice(above), 'hierarchy'); + const updates = []; + for (const [i, id] of order.entries()) { + const mapping = await store.getMapping(ctx.pair.id, 'role', id); + if (!mapping) continue; + updates.push({ id: mapping.targetId, position: i + 1 }); + } + const notes = []; + if (updates.length > 0) await ctx.target.setRolePositions(updates); + await store.putMapping({ + pairId: ctx.pair.id, + kind: 'order', + sourceId: 'roles', + targetId: '', + owned: false, + specHash: hash, + now: deps.now(), + }); + await enqueueGrants(ctx); + return done(`${updates.length} roles ordered`, { notes, remote: updates.length > 0 }); + } + const layout = channelOrder(ctx.source, ctx.sets.copiedChannels); + const hash = specHash(layout); + const m = await store.getMapping(ctx.pair.id, 'order', 'channels'); + if (m && m.specHash === hash) return done('unchanged'); + const updates = []; + for (const item of layout) { + const mapping = await store.getMapping(ctx.pair.id, 'channel', item.id); + if (!mapping) continue; + const parent = item.parentId ? await store.getMapping(ctx.pair.id, 'channel', item.parentId) : null; + updates.push({ id: mapping.targetId, position: item.position, parentId: parent?.targetId ?? null }); + } + if (updates.length > 1) await ctx.target.setChannelPositions(updates); + await store.putMapping({ + pairId: ctx.pair.id, + kind: 'order', + sourceId: 'channels', + targetId: '', + owned: false, + specHash: hash, + now: deps.now(), + }); + return done(`${updates.length} channels ordered`, { remote: updates.length > 1 }); + } + + /** Delete the Fluxer copy of something deleted on Discord: only what Sylo created, never anything else. */ + async function deleteCopy(ctx, kind, sourceId) { + const m = await store.getMapping(ctx.pair.id, kind, sourceId); + if (!m) return skipped('Sylo has no copy of it'); + if (!m.owned) { + await store.deleteMapping(ctx.pair.id, kind, sourceId); + return skipped('Sylo did not create it on Fluxer, so it is left alone'); + } + const method = { + role: 'deleteRole', + channel: 'deleteChannel', + emoji: 'deleteEmoji', + sticker: 'deleteSticker', + }[kind]; + if (!method) return skipped('nothing to delete'); + try { + await ctx.target[method](m.targetId); + } catch (err) { + if (classifyError(err).kind !== 'gone') throw err; + } + await store.deleteMapping(ctx.pair.id, kind, sourceId); + return done('deleted', { remote: true }); + } + + /** + * The default for something deleted on Discord: the Fluxer copy stays, locked, and is marked "removed on Discord" so + * the Fluxer server's owner can look at it and delete it. A channel is hidden from @everyone and made read-only (its + * messages are not lost: deleting a Fluxer channel deletes its messages too); a role loses its permissions; an + * emoji or sticker is only marked. Only what Sylo made is touched. + */ + async function lockCopy(ctx, kind, sourceId) { + const m = await store.getMapping(ctx.pair.id, kind, sourceId); + if (!m) return skipped('Sylo has no copy of it'); + if (!m.owned) { + await store.deleteMapping(ctx.pair.id, kind, sourceId); + return skipped('Sylo did not create it on Fluxer, so it is left alone'); + } + if (m.removedAt) return skipped('already marked as removed'); + let remote = false; + if (kind === 'channel') { + // View Channel, Send Messages, Add Reactions and Connect, denied to @everyone; every other overwrite goes. + const deny = String((1n << 10n) | (1n << 11n) | (1n << 6n) | (1n << 20n)); + await ctx.target.editChannel(m.targetId, { + name: m.label || undefined, + permissionOverwrites: [{ id: ctx.target.id, type: 0, allow: '0', deny }], + }); + remote = true; + } else if (kind === 'role') { + await ctx.target.editRole(m.targetId, { + name: m.label || undefined, + color: 0, + hoist: false, + mentionable: false, + permissions: '0', + }); + remote = true; + } + await store.markRemoved(ctx.pair.id, kind, sourceId, deps.now()); + return done(remote ? 'locked on Fluxer, marked as removed on Discord' : 'marked as removed on Discord', { + remote, + }); + } + + /** + * The last step of a copy: link the mirrored text channels through the bridge (src/mirror/bridgeLink.js). A pair the + * bridge refuses is skipped and listed in the notes; when more pairs wait than one run links, another run is queued. + */ + async function linkChannels(ctx, job) { + const tried = new Set(job.payload?.tried ?? []); + const res = await deps.linkMirroredChannels(ctx.pair.id, { skip: tried }); + const skippedPairs = res.skipped ?? []; + const notes = skippedPairs.map((s) => ({ + code: `bridge-${s.code}`, + scope: 'channel', + ref: { kind: 'channel', id: s.sourceId, name: s.name }, + message: `${s.name}: not linked through the bridge, ${s.message}`, + })); + if ((res.remaining ?? 0) > 0) { + await store.enqueueJob({ + pairId: ctx.pair.id, + run: job.run, + phase: job.phase, + op: 'link', + kind: 'bridge', + sourceId: job.sourceId, + stepKey: job.stepKey, + payload: { ...job.payload, tried: [...tried, ...(res.tried ?? [])] }, + now: deps.now(), + }); + } + const parts = []; + if (res.linked > 0) parts.push(`${res.linked} channels linked`); + if ((res.already ?? 0) > 0) parts.push(`${res.already} already linked`); + if (skippedPairs.length > 0) parts.push(`${skippedPairs.length} not linked (see the notes)`); + if ((res.remaining ?? 0) > 0) parts.push(`${res.remaining} more follow`); + return done(parts.length ? parts.join(', ') : `not linked: ${res.reason ?? 'nothing to link'}`, { + notes, + remote: res.linked > 0, + }); + } + + async function perform(ctx, job) { + if (job.payload?.skip) return skipped(String(job.payload.skip), { notes: job.payload.notes ?? [] }); + if (job.op === 'delete') { + return ctx.options.mirrorDeletes + ? deleteCopy(ctx, job.kind, job.sourceId) + : lockCopy(ctx, job.kind, job.sourceId); + } + if (job.op === 'reorder') return reorder(ctx, job.kind); + if (job.op === 'link') return linkChannels(ctx, job); + switch (job.kind) { + case 'guild': + return syncIdentity(ctx); + case 'role': + return syncRole(ctx, job.sourceId); + case 'channel': + return syncChannel(ctx, job.sourceId); + case 'emoji': + case 'sticker': + return syncExpression(ctx, job.kind, job.sourceId); + case 'module': + return syncModule(ctx, job.sourceId); + default: + return skipped('unknown step'); + } + } + + // --- the worker -------------------------------------------------------------------------------------- + + /** + * Pause a pair with the reason. `kind` says what resuming has to deal with ('failure', 'deletes', 'ownership', + * 'hierarchy', 'owner'); `by` is who paused it ('system', 'source' or 'target'): only that side can resume it. + */ + async function pausePair(pair, message, { kind = 'failure', by = 'system' } = {}) { + const next = await store.updatePair( + pair.id, + { + status: PAIR_STATUS.PAUSED, + resumeStatus: pair.status === PAIR_STATUS.PAUSED ? pair.resumeStatus : pair.status, + lastError: String(message).slice(0, 500), + pauseKind: kind, + pausedBy: by, + }, + deps.now() + ); + touchHealth(next); + log.warn('mirror', `pair ${pair.id} paused: ${message}`); + await linksFollow(next); + return next; + } + + /** The bridge links of a pair follow its state. A failure is logged and put right by the next sweep of the timer. */ + async function linksFollow(pair) { + try { + await deps.syncLinks(pair); + } catch (err) { + log.warn('mirror', `could not update the bridge links of pair ${pair.id}: ${err?.message ?? err}`); + } + } + + const backoffMs = (attempts) => + Math.min(deps.maxBackoffMs, deps.backoffBaseMs * 2 ** Math.max(0, attempts - 1)); + + /** Do one job and record what happened. Returns whether Fluxer was called. */ + async function runJob(pair, ctx, job) { + const s = stateOf(pair.id); + try { + const jobCtx = { ...ctx, job, source: snapshotDiscordGuild(ctx.sg) }; + jobCtx.sets = copiedSets(jobCtx.source, ctx.options); + const res = await perform(jobCtx, job); + await store.finishJob(job.id, res.status, res.detail, res.notes, deps.now()); + if (pair.failures > 0) pair = await store.updatePair(pair.id, { failures: 0 }, deps.now()); + return { pair, remote: res.remote }; + } catch (err) { + const c = classifyError(err); + const now = deps.now(); + if (err instanceof MirrorPause) { + await store.retryJob(job.id, job.attempts, now, err.message, now); + pair = await pausePair(pair, err.message, { kind: err.pauseKind }); + return { pair, remote: false }; + } + if (c.kind === 'rate-limit') { + s.blockedUntil = now + c.retryAfterMs; + await store.retryJob(job.id, job.attempts, now + c.retryAfterMs, 'Fluxer asked to slow down', now); + return { pair, remote: false }; + } + if (c.kind === 'transient') { + const attempts = job.attempts + 1; + if (attempts >= deps.maxAttempts) { + await store.finishJob(job.id, 'failed', c.message, [], now); + } else { + // The call may have gone through before it timed out: the next run looks for what it made first. + await store.retryJob(job.id, attempts, now + backoffMs(attempts), c.message, now, { + maybeDone: true, + }); + } + } else { + await store.finishJob(job.id, 'failed', c.message, [], now); + } + // Only trouble that is not about one item counts toward pausing the pair. + if (c.kind === 'transient' || c.kind === 'permission') { + const failures = pair.failures + 1; + pair = await store.updatePair(pair.id, { failures, lastError: c.message.slice(0, 300) }, now); + if (failures >= deps.pauseAfterFailures) { + pair = await pausePair( + pair, + c.kind === 'permission' + ? `Fluxer refused Sylo: ${c.message}. Check Sylo's permissions on the Fluxer server, then resume.` + : `Fluxer could not be reached: ${c.message}` + ); + } + } + return { pair, remote: true }; + } + } + + /** The copy is complete when no copy job is left waiting or running. */ + async function maybeFinishCopy(pair) { + if (pair.status !== PAIR_STATUS.COPYING) return pair; + if ((await store.openCopyCount(pair.id, pair.copyRun)) > 0) return pair; + const options = normaliseOptions(pair.options); + const next = await store.updatePair( + pair.id, + { status: options.continuous ? PAIR_STATUS.LIVE : PAIR_STATUS.DONE, copiedAt: deps.now() }, + deps.now() + ); + touchHealth(next); + return next; + } + + /** + * Work one pair's queue until nothing is due, the pair stops being active, or `maxJobs` is reached. + * Only one worker runs per pair at a time. + * @returns {Promise<{ ran: number }>} + */ + async function runPair(pairId, { maxJobs = Infinity } = {}) { + if (active.has(pairId)) return { ran: 0 }; + const worker = (async () => { + let ran = 0; + while (ran < maxJobs && !stopped) { + let pair = await store.getPair(pairId); + if (!pair || ![PAIR_STATUS.COPYING, PAIR_STATUS.LIVE].includes(pair.status)) break; + try { + await requirePairConsent(pair); + } catch (err) { + await pausePair(pair, err.message); + break; + } + const now = deps.now(); + if (stateOf(pairId).blockedUntil > now) break; + + const { sg, tg } = guildsOf(pair); + if (!sg || !tg) { + // A platform still connecting is not a reason to stop: wait. Once everything is up, a missing server is. + if (!deps.clientsReady()) break; + await pausePair(pair, `Sylo is no longer in the ${sg ? 'Fluxer' : 'Discord'} server.`); + break; + } + const owners = checkOwnership(pair, sg, tg, deps.enforceOwnership()); + if (!owners.ok) { + await pausePair(pair, owners.reason, { kind: 'ownership' }); + break; + } + const job = await store.nextDueJob(pairId, now); + if (!job) { + await maybeFinishCopy(pair); + break; + } + if (!(await store.claimJob(job.id, now))) continue; + const ctx = { + pair, + sg, + targetGuild: tg, + target: deps.makeTarget(tg), + options: normaliseOptions(pair.options), + }; + const out = await runJob(pair, ctx, job); + ran += 1; + if (out.remote) await deps.sleep(deps.minIntervalMs); + } + return { ran }; + })(); + active.set(pairId, worker); + try { + return await worker; + } finally { + active.delete(pairId); + } + } + + // --- the timer ------------------------------------------------------------------------------------- + + /** Start a worker for every pair that has work. Does not wait for them (see stop()). */ + async function tick() { + if (stopped) return; + const now = deps.now(); + if (now - lastPrune > PRUNE_EVERY_MS) { + lastPrune = now; + await store.pruneMirrorJobs({ before: now - KEEP_FINISHED_MS }); + } + // Keep the bridge links in step with their mirrors: a pause or resume that could not reach them is put right here. + if (now - lastLinkSweep > LINK_SWEEP_MS) { + lastLinkSweep = now; + for (const pair of await store.listAllPairs()) await linksFollow(pair); + } + for (const status of [PAIR_STATUS.COPYING, PAIR_STATUS.LIVE]) { + for (const pair of await store.listPairsIn(status)) { + if (active.has(pair.id)) continue; + // A few mirrors at a time, so mirror work cannot starve the bridge and the modules on the same Fluxer client. + if (active.size >= deps.maxWorkers) return; + const due = await store.nextDueAt(pair.id); + if (due === null && pair.status === PAIR_STATUS.LIVE) continue; + runPair(pair.id).catch((err) => log.error('mirror', `worker for pair ${pair.id} failed:`, err)); + } + } + } + + /** After a restart: work that was in flight waits again, marked so it looks for what it may have made. */ + async function resumeAfterRestart() { + await store.recoverRunningJobs(deps.now()); + } + + /** Stop taking new work and wait (briefly) for the workers that are running. */ + async function stop() { + stopped = true; + await Promise.race([Promise.allSettled([...active.values()]), deps.sleep(3000)]); + } + + // --- what an admin can do --------------------------------------------------------------------------- + + /** `side` is whose owner paused it ('source' for Discord, 'target' for Fluxer): only that side can resume it. */ + async function pause(pairId, side = 'system') { + const pair = await store.getPair(pairId); + if (!pair || ![PAIR_STATUS.COPYING, PAIR_STATUS.LIVE].includes(pair.status)) { + throw new MirrorError('Only a running mirror can be paused.', 'item'); + } + const next = await store.updatePair( + pairId, + { + status: PAIR_STATUS.PAUSED, + resumeStatus: pair.status, + lastError: '', + pauseKind: 'owner', + pausedBy: side, + }, + deps.now() + ); + touchHealth(next); + await linksFollow(next); + return next; + } + + /** + * Resume a paused mirror. The caller has checked that the person is the owner of the server `side` ('source' or + * 'target') they are on. + * - A mirror its owner paused can be resumed only from the side that paused it. + * - After a burst of deletes `confirm: true` is needed: the deletes that waited are then applied. + * - After a server changed hands, the new owner's confirmation records them as the person who agreed; the mirror + * goes on once both servers' owners have confirmed. + */ + async function resume(pairId, { side = 'system', confirm = false } = {}) { + let pair = await store.getPair(pairId); + if (!pair || pair.status !== PAIR_STATUS.PAUSED) + throw new MirrorError('Only a paused mirror can be resumed.', 'item'); + if (['source', 'target'].includes(pair.pausedBy) && side !== 'system' && pair.pausedBy !== side) { + throw new MirrorError( + `The owner of the ${pair.pausedBy === 'source' ? 'Discord' : 'Fluxer'} server paused this mirror, so only that owner can resume it.`, + 'item' + ); + } + // Deletes that wait (after a burst, or while the mirror was paused) are applied only after a confirmation that shows + // what they are. With real deletion on, only the owner of the Fluxer server can confirm: it destroys messages that + // exist only there. + const waiting = await pendingDeletes(pair); + if (waiting.count > 0 && (pair.pauseKind === 'deletes' || waiting.count > deps.deleteBurst.max)) { + if (normaliseOptions(pair.options).mirrorDeletes && side === 'source') { + throw Object.assign( + new MirrorError( + 'Deleting on Fluxer is switched on, so only the owner of the Fluxer server can confirm these deletions.', + 'item' + ), + { code: 'fluxer-owner-only', deletes: waiting.items } + ); + } + if (!confirm) { + throw Object.assign(new MirrorError('Confirm the deletions first.', 'item'), { + code: 'confirm-needed', + deletes: waiting.items, + deleteCount: waiting.count, + }); + } + } else if (pair.pauseKind === 'deletes' && !confirm) { + throw Object.assign(new MirrorError('Confirm the deletions first.', 'item'), { + code: 'confirm-needed', + deletes: [], + }); + } + const { sg, tg } = guildsOf(pair); + if (pair.pauseKind === 'ownership' && ['source', 'target'].includes(side)) { + const guild = side === 'source' ? sg : tg; + if (guild?.ownerId) { + const patch = + side === 'source' + ? { sourceConsentUser: String(guild.ownerId) } + : { targetConsentUser: String(guild.ownerId) }; + // Real deletion on Fluxer is the Fluxer owner's own switch: a new owner starts with it off (the request stays, + // so they can switch it on themselves). + if (side === 'target' && String(guild.ownerId) !== pair.targetConsentUser) { + patch.options = { ...normaliseOptions(pair.options), mirrorDeletes: false }; + } + pair = await store.updatePair(pairId, patch, deps.now()); + } + } + await requirePairConsent(pair); + const owners = checkOwnership(pair, sg, tg, deps.enforceOwnership()); + if (!owners.ok) { + const waiting = owners.sides.map((s) => (s === 'source' ? 'Discord' : 'Fluxer')); + const next = await store.updatePair( + pairId, + { + lastError: + `Waiting for the owner of the ${waiting.join(' and ')} server to confirm the mirror. ${owners.reason}`.slice( + 0, + 500 + ), + pauseKind: 'ownership', + }, + deps.now() + ); + touchHealth(next); + return next; + } + state.delete(pairId); + const next = await store.updatePair( + pairId, + { + status: pair.resumeStatus || PAIR_STATUS.LIVE, + resumeStatus: '', + lastError: '', + failures: 0, + pauseKind: '', + pausedBy: '', + }, + deps.now() + ); + touchHealth(next); + await linksFollow(next); + return next; + } + + /** End the mirror from either side. The id map stays, so asking again picks up where it left off. */ + async function stopPair(pairId) { + const pair = await store.getPair(pairId); + if (!pair) throw new MirrorError('There is no such mirror.', 'item'); + await store.dropOpenJobs(pairId, 'the mirror was stopped', deps.now()); + const next = await store.updatePair( + pairId, + { + status: PAIR_STATUS.STOPPED, + resumeStatus: '', + codeHash: '', + codeExpiresAt: 0, + sourceConsentBy: '', + sourceConsentAt: 0, + targetConsentBy: '', + targetConsentAt: 0, + sourceConsentUser: '', + targetConsentUser: '', + pauseKind: '', + pausedBy: '', + }, + deps.now() + ); + state.delete(pairId); + touchHealth(next); + await linksFollow(next); + return next; + } + + /** Forget the mirror altogether: the pair, its id map and its queue. Nothing on either server is touched. */ + async function removePair(pairId, side = '') { + const pair = await store.getPair(pairId); + if (!pair) return false; + if ([PAIR_STATUS.COPYING, PAIR_STATUS.LIVE].includes(pair.status)) { + throw new MirrorError('Stop the mirror before removing it.', 'item'); + } + // The bridge links it made stay, paused: the owners decide on the Bridge page whether to resume or unlink them. + await deps.releaseLinks(pairId, side); + await store.deletePair(pairId); + state.delete(pairId); + touchHealth(pair); + return true; + } + + /** The deletes waiting in a pair's queue, as what the person is asked to confirm: the item and what will happen to it. */ + async function pendingDeletes(pair) { + const { count, jobs } = await store.listPendingDeletes(pair.id, 50); + const really = normaliseOptions(pair.options).mirrorDeletes; + const items = []; + for (const j of jobs) { + const m = await store.getMapping(pair.id, j.kind, j.sourceId); + if (!m || !m.owned) continue; // nothing of Sylo's to touch + items.push({ kind: j.kind, name: m.label || j.sourceId, action: really ? 'delete' : 'lock' }); + } + return { count, items }; + } + + // --- what the dashboard shows ------------------------------------------------------------------------ + + /** How far a pair is: the copy's jobs by state, and the changes still waiting. */ + async function progress(pair) { + const counts = await store.jobCounts(pair.id, pair.copyRun, 'copy'); + const total = Object.values(counts).reduce((a, b) => a + b, 0); + const finished = counts.done + counts.skipped + counts.failed; + return { + run: pair.copyRun, + total, + done: counts.done, + skipped: counts.skipped, + failed: counts.failed, + waiting: counts.pending + counts.running, + percent: total === 0 ? 0 : Math.round((finished / total) * 100), + waitingChanges: await store.openSyncCount(pair.id), + pendingDeletes: (await pendingDeletes(pair)).items, + // Copies left on Fluxer whose Discord original was deleted (locked, not deleted): for the Fluxer owner to look at. + removed: (await store.listRemoved(pair.id)) + .slice(0, 100) + .map((m) => ({ kind: m.kind, label: m.label || m.targetId, at: m.removedAt })), + }; + } + + /** The report of the latest copy: every step and what became of it, and what could not be carried over. */ + async function report(pair, { limit = 2000 } = {}) { + const jobs = await store.listRunJobs(pair.id, pair.copyRun, { limit }); + const items = jobs.map((j) => ({ + kind: j.kind, + op: j.op, + sourceId: j.sourceId, + name: j.payload?.name ?? j.sourceId, + planned: j.payload?.planned ?? null, + status: j.status, + detail: j.detail, + })); + const notes = jobs.flatMap((j) => j.notes); + // The bridge step can take several runs (a batch of channels each): show them as one. + const links = jobs.filter((j) => j.op === 'link'); + const open = links.find((j) => ['pending', 'running', 'failed'].includes(j.status)); + const bridge = links.length + ? { + status: open ? open.status : links[links.length - 1].status, + detail: links + .map((j) => j.detail) + .filter(Boolean) + .join('; '), + } + : null; + return { + run: pair.copyRun, + items, + notes, + failed: items.filter((i) => i.status === 'failed'), + bridge, + summary: summariseJobs(jobs), + }; + } + + return { + deps, + previewPair, + startCopy, + noteSourceChange, + runPair, + tick, + resumeAfterRestart, + stop, + pause, + resume, + stopPair, + removePair, + progress, + report, + isRunning: (pairId) => active.has(pairId), + /** For tests: forget the in-memory state (limits reached, backoff). */ + resetState: () => state.clear(), + limits: FLUXER_LIMITS, + }; +} + +/** Counts of a run's jobs per kind and result. */ +function summariseJobs(jobs) { + const out = {}; + for (const j of jobs) { + out[j.kind] ??= { done: 0, skipped: 0, failed: 0, pending: 0, running: 0 }; + out[j.kind][j.status] += 1; + } + return out; +} diff --git a/src/mirror/errors.js b/src/mirror/errors.js new file mode 100644 index 00000000..9ecccca5 --- /dev/null +++ b/src/mirror/errors.js @@ -0,0 +1,97 @@ +// What a failed call to Fluxer means for the mirror. The engine does not look at the SDK's error classes: it +// asks classifyError() what to do, so a test can throw a plain object with a statusCode. +// +// rate-limit Fluxer asked Sylo to slow down: wait `retryAfterMs`, then try the same job again. +// transient a network failure or a 5xx: try again later with a growing delay. +// permission Sylo is not allowed (401/403): this item fails, and enough of them in a row pause the pair. +// gone 404: the thing is already gone (for a delete that is success; for an edit, create it again). +// limit Fluxer refused because of one of its own limits (too many emoji ...): skip the rest of that kind. +// item any other 4xx: this one item cannot be made as asked; the copy goes on with the next. + +/** @typedef {'rate-limit' | 'transient' | 'permission' | 'gone' | 'limit' | 'item'} ErrorKind */ + +const NETWORK_CODES = new Set([ + 'ECONNRESET', + 'ECONNREFUSED', + 'ETIMEDOUT', + 'ENOTFOUND', + 'EAI_AGAIN', + 'EPIPE', + 'UND_ERR_SOCKET', + 'UND_ERR_CONNECT_TIMEOUT', + 'UND_ERR_HEADERS_TIMEOUT', + 'UND_ERR_BODY_TIMEOUT', +]); + +const LIMIT_TEXT = + /maximum number|limit (has been )?reached|too many (emoji|sticker|role|channel)|no (more )?slots|max(imum)? .*(emoji|sticker|role|channel)/i; + +// Fluxer's own codes for "you have reached a limit". +const LIMIT_CODES = new Set(['MAX_EMOJIS', 'MAX_STICKERS', 'MAX_GUILD_ROLES', 'MAX_GUILD_CHANNELS']); + +/** + * Is this error a real "no such thing" answer? Only a 404 (or the SDK's own *_NOT_FOUND code for one) is. A timeout, a + * 429, a 5xx or a cache miss is not: reading them as "gone" would make a second copy of something that is still there. + */ +export function isNotFound(err) { + if (!err) return false; + const status = Number(err.statusCode ?? err.status ?? err.cause?.statusCode ?? 0); + if (status === 404) return true; + return /^(ROLE|CHANNEL|EMOJI|STICKER)_NOT_FOUND$/.test(String(err.code ?? '')); +} + +/** + * @param {any} err + * @returns {{ kind: ErrorKind, retryAfterMs?: number, message: string }} + */ +export function classifyError(err) { + const message = String(err?.message ?? err ?? 'Unknown error').slice(0, 300); + if (err instanceof MirrorError) return { kind: err.kind, message }; + // The SDK wraps a failed look-up in an error that keeps the HTTP status only in `cause`. + const status = Number(err?.statusCode ?? err?.status ?? err?.cause?.statusCode ?? err?.cause?.status ?? 0); + if (status === 429 || typeof err?.retryAfter === 'number') { + const seconds = Number(err?.retryAfter ?? 1); + return { + kind: 'rate-limit', + retryAfterMs: Math.min(Math.max(Math.ceil(seconds * 1000), 1000), 10 * 60_000), + message, + }; + } + if (status === 401 || status === 403) return { kind: 'permission', message }; + if (isNotFound(err)) return { kind: 'gone', message }; + if (status >= 500) return { kind: 'transient', message }; + if (LIMIT_CODES.has(String(err?.code ?? '').toUpperCase())) return { kind: 'limit', message }; + if (status >= 400) return { kind: LIMIT_TEXT.test(message) ? 'limit' : 'item', message }; + if ( + NETWORK_CODES.has(err?.code) || + NETWORK_CODES.has(err?.cause?.code) || + /fetch failed|timed? ?out|socket/i.test(message) + ) { + return { kind: 'transient', message }; + } + // Not an API error at all (a bug, a missing guild): do not retry it forever, fail the item. + return { kind: 'item', message }; +} + +/** An error the engine raises itself, already classified. */ +export class MirrorError extends Error { + /** @param {ErrorKind} kind */ + constructor(message, kind = 'item') { + super(message); + this.name = 'MirrorError'; + this.kind = kind; + } +} + +/** + * Raised by a job that must not go on until a person has done something (move Sylo's role up on Fluxer): the + * engine puts the job back, pauses the mirror with this message, and waits for a resume. + */ +export class MirrorPause extends MirrorError { + /** @param {string} pauseKind what the pause is about (shown to the person, and kept on the pair) */ + constructor(message, pauseKind) { + super(message, 'item'); + this.name = 'MirrorPause'; + this.pauseKind = pauseKind; + } +} diff --git a/src/mirror/health.js b/src/mirror/health.js new file mode 100644 index 00000000..59445b50 --- /dev/null +++ b/src/mirror/health.js @@ -0,0 +1,46 @@ +// The mirror's entries in a server's health report (src/web/lib/permissionHealth.js): a mirror that paused +// itself because Fluxer kept refusing or failing, and a copy that finished with items that failed. Both +// servers of a pair show the entry, because either side's admins may be the one who can fix it. +import { PAIR_STATUS, listFailedJobs, pairsOfGuild } from '../db/mirror.js'; +import { v2CardHref } from '../web/lib/overviewSummary.js'; + +/** + * @param {string} guildKey + * @returns {Promise} checks in the shape of the health report + */ +export async function mirrorHealthChecks(guildKey) { + const checks = []; + for (const pair of await pairsOfGuild(guildKey)) { + const base = { + module: 'mirror', + moduleName: 'Mirror to Fluxer', + href: v2CardHref('mirror', guildKey), + }; + if (pair.status === PAIR_STATUS.PAUSED && pair.lastError) { + checks.push({ + ...base, + severity: 'error', + kind: 'mirror', + code: 'mirror-paused', + message: `The mirror to Fluxer is paused: ${pair.lastError}`, + fix: 'Fix what the message says, then press Resume on the Mirror page.', + }); + continue; + } + if ([PAIR_STATUS.LIVE, PAIR_STATUS.DONE, PAIR_STATUS.COPYING].includes(pair.status)) { + const failed = await listFailedJobs(pair.id, 50); + const current = failed.filter((j) => j.run === pair.copyRun || j.phase === 'sync'); + if (current.length > 0) { + checks.push({ + ...base, + severity: 'warning', + kind: 'mirror', + code: 'mirror-items-failed', + message: `${current.length} item(s) could not be copied to Fluxer, for example ${current[0].payload?.name ?? current[0].sourceId}: ${current[0].detail}`, + fix: 'Open the report on the Mirror page to see them all.', + }); + } + } + } + return checks; +} diff --git a/src/mirror/index.js b/src/mirror/index.js new file mode 100644 index 00000000..fb2683af --- /dev/null +++ b/src/mirror/index.js @@ -0,0 +1,5 @@ +// The server mirror of this process: one engine, started by src/bot/events/mirror.js and used by the dashboard +// routes (src/web/routes/v2Mirror.js). See engine.js for how a copy runs. +import { createEngine } from './engine.js'; + +export const mirror = createEngine(); diff --git a/src/mirror/ownership.js b/src/mirror/ownership.js new file mode 100644 index 00000000..1b0926b8 --- /dev/null +++ b/src/mirror/ownership.js @@ -0,0 +1,28 @@ +// A mirror is run by the two servers' owners, and by nobody else. The people who agreed are recorded by their +// platform user id (the owner of each server at that time); before every piece of work the engine asks whether the +// servers still have those owners. If a server changed hands, the mirror pauses until the new owner has confirmed it. +// +// Open mode (no sign-in, a private instance) has no accounts, so nothing is enforced there: `enforce` is false. + +/** + * @param {{ sourceConsentUser: string, targetConsentUser: string }} pair + * @param {{ ownerId?: string } | undefined} sourceGuild the Discord server + * @param {{ ownerId?: string } | undefined} targetGuild the Fluxer server + * @param {boolean} enforce + * @returns {{ ok: boolean, sides: Array<'source' | 'target'>, reason?: string }} `sides`: whose owner is not the one who agreed + */ +export function checkOwnership(pair, sourceGuild, targetGuild, enforce) { + if (!enforce) return { ok: true, sides: [] }; + const sides = []; + const same = (agreed, guild) => + Boolean(agreed) && Boolean(guild?.ownerId) && String(guild.ownerId) === agreed; + if (!same(pair.sourceConsentUser, sourceGuild)) sides.push('source'); + if (!same(pair.targetConsentUser, targetGuild)) sides.push('target'); + if (!sides.length) return { ok: true, sides }; + const names = sides.map((s) => (s === 'source' ? 'Discord' : 'Fluxer')); + return { + ok: false, + sides, + reason: `The owner of the ${names.join(' and ')} server is not the person who agreed to the mirror (the server changed hands). The mirror is paused until the new owner confirms it.`, + }; +} diff --git a/src/mirror/permissions.js b/src/mirror/permissions.js new file mode 100644 index 00000000..4ac73355 --- /dev/null +++ b/src/mirror/permissions.js @@ -0,0 +1,200 @@ +// Discord permissions -> Fluxer permissions, bit by bit. +// +// Fluxer's permission flags were laid out like Discord's, so most bits are the same number. A few Discord +// permissions have no Fluxer counterpart (application commands, events, soundboard, polls ...): +// those bits are dropped, and the report says which ones were dropped where. Two Fluxer permissions have no +// Discord counterpart (UpdateRtcRegion, ViewChannelMembers): a mirror never touches them, so a role or a +// channel that already holds them on the Fluxer side keeps them (see keepFluxerOnly). +// +// The table is data: test/mirrorPermissions.test.js checks every row against discord.js's and the Fluxer +// SDK's own flag tables, so a renamed or renumbered permission fails a test instead of silently mapping wrong. + +/** + * One row per Discord permission bit that exists: its Discord name, its bit, the Fluxer name it becomes + * (null when Fluxer has no such permission) and a short label for the report. + * @type {ReadonlyArray<{ bit: number, discord: string, fluxer: string | null, label: string }>} + */ +export const PERMISSION_TABLE = Object.freeze( + [ + [0, 'CreateInstantInvite', 'CreateInstantInvite', 'Create Invite'], + [1, 'KickMembers', 'KickMembers', 'Kick Members'], + [2, 'BanMembers', 'BanMembers', 'Ban Members'], + [3, 'Administrator', 'Administrator', 'Administrator'], + [4, 'ManageChannels', 'ManageChannels', 'Manage Channels'], + [5, 'ManageGuild', 'ManageGuild', 'Manage Server'], + [6, 'AddReactions', 'AddReactions', 'Add Reactions'], + [7, 'ViewAuditLog', 'ViewAuditLog', 'View Audit Log'], + [8, 'PrioritySpeaker', 'PrioritySpeaker', 'Priority Speaker'], + [9, 'Stream', 'Stream', 'Video'], + [10, 'ViewChannel', 'ViewChannel', 'View Channel'], + [11, 'SendMessages', 'SendMessages', 'Send Messages'], + [12, 'SendTTSMessages', 'SendTtsMessages', 'Send Text-to-Speech Messages'], + [13, 'ManageMessages', 'ManageMessages', 'Manage Messages'], + [14, 'EmbedLinks', 'EmbedLinks', 'Embed Links'], + [15, 'AttachFiles', 'AttachFiles', 'Attach Files'], + [16, 'ReadMessageHistory', 'ReadMessageHistory', 'Read Message History'], + [17, 'MentionEveryone', 'MentionEveryone', 'Mention Everyone'], + [18, 'UseExternalEmojis', 'UseExternalEmojis', 'Use External Emoji'], + [19, 'ViewGuildInsights', null, 'View Server Insights'], + [20, 'Connect', 'Connect', 'Connect'], + [21, 'Speak', 'Speak', 'Speak'], + [22, 'MuteMembers', 'MuteMembers', 'Mute Members'], + [23, 'DeafenMembers', 'DeafenMembers', 'Deafen Members'], + [24, 'MoveMembers', 'MoveMembers', 'Move Members'], + [25, 'UseVAD', 'UseVad', 'Use Voice Activity'], + [26, 'ChangeNickname', 'ChangeNickname', 'Change Nickname'], + [27, 'ManageNicknames', 'ManageNicknames', 'Manage Nicknames'], + [28, 'ManageRoles', 'ManageRoles', 'Manage Roles'], + [29, 'ManageWebhooks', 'ManageWebhooks', 'Manage Webhooks'], + [30, 'ManageGuildExpressions', 'ManageExpressions', 'Manage Expressions'], + [31, 'UseApplicationCommands', null, 'Use Application Commands'], + [32, 'RequestToSpeak', null, 'Request to Speak'], + [33, 'ManageEvents', null, 'Manage Events'], + [34, 'ManageThreads', 'ManageThreads', 'Manage Threads'], + [35, 'CreatePublicThreads', 'CreatePublicThreads', 'Create Public Threads'], + [36, 'CreatePrivateThreads', 'CreatePrivateThreads', 'Create Private Threads'], + [37, 'UseExternalStickers', 'UseExternalStickers', 'Use External Stickers'], + [38, 'SendMessagesInThreads', 'SendMessagesInThreads', 'Send Messages in Threads'], + [39, 'UseEmbeddedActivities', null, 'Use Activities'], + [40, 'ModerateMembers', 'ModerateMembers', 'Timeout Members'], + [41, 'ViewCreatorMonetizationAnalytics', null, 'View Creator Analytics'], + [42, 'UseSoundboard', null, 'Use Soundboard'], + [43, 'CreateGuildExpressions', 'CreateExpressions', 'Create Expressions'], + [44, 'CreateEvents', null, 'Create Events'], + [45, 'UseExternalSounds', null, 'Use External Sounds'], + [46, 'SendVoiceMessages', null, 'Send Voice Messages'], + [48, 'SetVoiceChannelStatus', null, 'Set Voice Channel Status'], + [49, 'SendPolls', null, 'Create Polls'], + [50, 'UseExternalApps', null, 'Use External Apps'], + [51, 'PinMessages', 'PinMessages', 'Pin Messages'], + [52, 'BypassSlowmode', 'BypassSlowmode', 'Bypass Slowmode'], + ].map(([bit, discord, fluxer, label]) => Object.freeze({ bit, discord, fluxer, label })) +); + +/** The Fluxer bit of each mapped Discord bit (Fluxer's bit numbers, from the SDK's flag table). */ +const FLUXER_BIT = Object.freeze({ + CreateInstantInvite: 0, + KickMembers: 1, + BanMembers: 2, + Administrator: 3, + ManageChannels: 4, + ManageGuild: 5, + AddReactions: 6, + ViewAuditLog: 7, + PrioritySpeaker: 8, + Stream: 9, + ViewChannel: 10, + SendMessages: 11, + SendTtsMessages: 12, + ManageMessages: 13, + EmbedLinks: 14, + AttachFiles: 15, + ReadMessageHistory: 16, + MentionEveryone: 17, + UseExternalEmojis: 18, + Connect: 20, + Speak: 21, + MuteMembers: 22, + DeafenMembers: 23, + MoveMembers: 24, + UseVad: 25, + ChangeNickname: 26, + ManageNicknames: 27, + ManageRoles: 28, + ManageWebhooks: 29, + ManageExpressions: 30, + // Fluxer's thread permissions (packages/constants ThreadPermissionUtils); the SDK's flag table does not list them yet. + ManageThreads: 34, + CreatePublicThreads: 35, + CreatePrivateThreads: 36, + SendMessagesInThreads: 38, + UseExternalStickers: 37, + ModerateMembers: 40, + CreateExpressions: 43, + PinMessages: 51, + BypassSlowmode: 52, + UpdateRtcRegion: 53, + ViewChannelMembers: 54, +}); + +/** + * Permissions that make a role a privilege rather than a label (the same list the dashboard's role-grant guard uses, + * src/web/middleware/roleGrants.js): handing one out, or giving it to everyone, needs more than a mirror may decide. + */ +export const ELEVATED_NAMES = Object.freeze([ + 'Administrator', + 'ManageGuild', + 'ManageRoles', + 'ManageChannels', + 'ManageWebhooks', + 'ManageMessages', + 'ManageNicknames', + 'ManageThreads', + 'ManageEvents', + 'ManageGuildExpressions', + 'KickMembers', + 'BanMembers', + 'ModerateMembers', + 'MentionEveryone', + 'MoveMembers', + 'MuteMembers', + 'DeafenMembers', + 'ViewAuditLog', +]); + +/** The Discord bits of ELEVATED_NAMES, and the Fluxer bits they become. */ +export const ELEVATED_DISCORD_MASK = PERMISSION_TABLE.filter((r) => + ELEVATED_NAMES.includes(r.discord) +).reduce((acc, r) => acc | (1n << BigInt(r.bit)), 0n); +export const ELEVATED_FLUXER_MASK = mapPermissionsRaw(ELEVATED_DISCORD_MASK); + +/** Does a Discord permission bitfield hold any elevated permission? */ +export const isElevated = (discordBits) => (BigInt(discordBits ?? 0) & ELEVATED_DISCORD_MASK) !== 0n; + +/** Fluxer permissions Discord does not have: the mirror leaves these as they are on the Fluxer side. */ +export const FLUXER_ONLY_BITS = (1n << 53n) | (1n << 54n); + +const rowsByBit = new Map(PERMISSION_TABLE.map((r) => [r.bit, r])); + +function mapPermissionsRaw(discordBits) { + let bits = 0n; + for (const row of PERMISSION_TABLE) { + if (row.fluxer !== null && (discordBits >> BigInt(row.bit)) & 1n) + bits |= 1n << BigInt(FLUXER_BIT[row.fluxer]); + } + return bits; +} + +/** The Fluxer bit number for a Fluxer permission name of the table (for the tests). */ +export const fluxerBitOf = (name) => FLUXER_BIT[name]; + +/** + * Map a Discord permission bitfield to Fluxer's. + * @param {bigint | number | string} discordBits + * @returns {{ bits: bigint, dropped: string[], unknown: number[] }} `dropped`: the labels of the Discord + * permissions Fluxer has no equivalent for; `unknown`: bit numbers this table does not know (a newer Discord). + */ +export function mapPermissions(discordBits) { + let rest = BigInt(discordBits ?? 0); + let bits = 0n; + const dropped = []; + const unknown = []; + for (let bit = 0; rest > 0n; bit += 1, rest >>= 1n) { + if ((rest & 1n) === 0n) continue; + const row = rowsByBit.get(bit); + if (!row) unknown.push(bit); + else if (row.fluxer === null) dropped.push(row.label); + else bits |= 1n << BigInt(FLUXER_BIT[row.fluxer]); + } + return { bits, dropped, unknown }; +} + +/** The Fluxer-only bits of `existing` (what a role or overwrite already has there), to carry over. */ +export const keepFluxerOnly = (existing) => BigInt(existing ?? 0) & FLUXER_ONLY_BITS; + +/** Are the unmapped bits of a mapping result worth a line in the report? */ +export const lostSomething = (m) => m.dropped.length > 0 || m.unknown.length > 0; + +/** A readable list of what a mapping dropped, e.g. "Manage Events, Create Public Threads, bit 60". */ +export const describeDropped = (m) => + [...m.dropped, ...m.unknown.map((b) => `unknown permission (bit ${b})`)].join(', '); diff --git a/src/mirror/plan.js b/src/mirror/plan.js new file mode 100644 index 00000000..fee223c2 --- /dev/null +++ b/src/mirror/plan.js @@ -0,0 +1,486 @@ +// The plan of a copy: every step the mirror would take to make the Fluxer server match the Discord one, in +// the order it takes them, with what each would send and everything that cannot be carried over. +// +// buildPlan() is pure: snapshots and the id map in, steps out. The dry run shows the plan; the real copy +// turns the same plan into queue jobs, and each job rebuilds its spec with the same builders from the server as +// it is at that moment. So the preview and the copy agree by construction, and a server that changed in +// between simply gets the newer spec. +import { + FLUXER_LIMITS, + buildChannelSpec, + buildEmojiSpec, + buildIdentitySpec, + buildRoleSpec, + buildStickerSpec, + placeholderConfig, + roleSkipReason, + specHash, +} from './spec.js'; +import { DISCORD_CHANNEL } from './snapshot.js'; +import { isElevated, keepFluxerOnly } from './permissions.js'; +import { moduleGrantSites } from '../web/middleware/roleGrants.js'; + +/** + * Modules whose settings are never copied: the ones that hold the operator's own keys (ownerOnly), and the bridge, + * which has link state of its own. Every other module of the registry is copied; a test fails when a new module + * is neither, so each new module is decided on purpose. + */ +export const NOT_COPIED_MODULES = Object.freeze(['apple-connect', 'bridge']); + +/** + * The settings of one module as they go to Fluxer, with the same checks the dashboard makes when someone saves them: + * a role that gives powerful permissions is not handed out by the copied settings (autoroles, reaction roles, level + * rewards, the verified or birthday role, command actions), and a giveaway that pings @here or @everyone is copied + * at the lowest level. Anything dropped is a note, so the preview and the report say so. + * @param {{ id: string, name: string, enabled: boolean, config: object }} mod + * @param {{ copiedRoles: Set, skippedRoles: Set, copiedChannels: Set, skippedChannels: Set }} sets + * @param {any} source the Discord server snapshot + */ +export function buildModuleSpec(mod, sets, source) { + const notes = []; + const ref = { kind: 'module', id: mod.id, name: mod.name }; + const grants = new Set( + moduleGrantSites(mod.id, mod.config ?? {}) + .map((g) => g.roleId) + .filter(Boolean) + ); + const elevated = new Set(); + for (const id of grants) { + const role = source.roles.find((r) => r.id === id); + if (role && isElevated(role.permissions)) elevated.add(id); + } + for (const id of elevated) { + const role = source.roles.find((r) => r.id === id); + notes.push({ + code: 'module-grant-dropped', + scope: 'module', + ref, + message: `${mod.name}: the role ${role.name} gives powerful permissions, so the copied settings do not hand it out. Set it up on the Fluxer server yourself if you want it.`, + }); + } + const out = placeholderConfig(mod.config ?? {}, { + channels: sets.copiedChannels, + roles: new Set([...sets.copiedRoles].filter((id) => !elevated.has(id))), + skippedChannels: sets.skippedChannels, + skippedRoles: new Set([...sets.skippedRoles, ...elevated]), + }); + const config = out.config; + if (mod.id === 'giveaways' && (config.ping === 'here' || config.ping === 'everyone')) { + config.ping = 'none'; + notes.push({ + code: 'module-ping-lowered', + scope: 'module', + ref, + message: `${mod.name}: pinging @here or @everyone is not copied; it is off on the Fluxer server.`, + }); + } + if (out.dropped || out.cleared) { + notes.push({ + code: 'module-ids-dropped', + scope: 'module', + ref, + message: `${mod.name}: ${out.dropped} channel or role setting(s) point at something that is not copied and ${out.cleared} other id(s) (messages, members) are cleared; set them again on the Fluxer server.`, + }); + } + // The dashboard's funding guard (a copied economy that raises income from nothing) is not reproduced, so the economy + // comes over switched off: its owner looks at the shop and the income settings, then turns it on. + let enabled = mod.enabled; + if (mod.id === 'economy' && enabled) { + enabled = false; + notes.push({ + code: 'module-economy-off', + scope: 'module', + ref, + message: `${mod.name}: it is copied switched off. Look at the income settings and the shop on the Fluxer server, then switch it on.`, + }); + } + return { spec: { enabled, config }, notes, remapped: out.remapped }; +} + +/** The kinds of step, in the order the copy runs them. */ +export const STEP_KINDS = Object.freeze([ + 'guild', + 'role', + 'order-roles', + 'channel', + 'order-channels', + 'emoji', + 'sticker', + 'module', + 'bridge', +]); + +/** + * @typedef {Object} Step + * @property {string} key ":", unique in a plan + * @property {string} kind one of STEP_KINDS + * @property {string} sourceId the Discord id (or a fixed name for the steps that are not one thing) + * @property {string} name what to show + * @property {'create'|'update'|'unchanged'|'skip'} op + * @property {string} [reason] why a step is skipped + * @property {object|null} spec what would be sent (placeholders for ids, see spec.js) + * @property {string} hash fingerprint of the spec + * @property {object[]} notes what cannot be carried over + */ + +const mappingOf = (mappings, kind, sourceId) => mappings?.[kind]?.get?.(sourceId); + +function decide(mappings, kind, sourceId, hash) { + const m = mappingOf(mappings, kind, sourceId); + if (!m) return 'create'; + return m.specHash === hash ? 'unchanged' : 'update'; +} + +/** + * Which Discord roles and channels a copy takes along, and which it leaves out (a managed role, a channel type + * Fluxer lacks, everything of a kind the admin turned off). + */ +export function copiedSets(source, options) { + const copiedRoles = new Set(); + const skippedRoles = new Set(); + for (const role of source.roles) { + if (role.isEveryone) continue; + if (options.roles && !roleSkipReason(role)) copiedRoles.add(role.id); + else skippedRoles.add(role.id); + } + const copiedChannels = new Set(); + const skippedChannels = new Set(); + for (const ch of source.channels) { + const { spec } = buildChannelSpec(ch, { source, copiedRoles }); + if (options.channels && spec) copiedChannels.add(ch.id); + else skippedChannels.add(ch.id); + } + return { copiedRoles, skippedRoles, copiedChannels, skippedChannels }; +} + +/** For something that always exists on Fluxer already (the server itself, @everyone): update, never create. */ +function existing(mappings, kind, sourceId, hash) { + const op = decide(mappings, kind, sourceId, hash); + return op === 'create' ? 'update' : op; +} + +/** Order roles and channels the way they sit on the source: by position, then id for a stable tie. */ +const byPosition = (a, b) => a.position - b.position || (a.id < b.id ? -1 : a.id > b.id ? 1 : 0); + +/** + * @param {{ + * source: ReturnType, + * target?: ReturnType | null, + * mappings?: Record>, + * options: import('./spec.js').DEFAULT_OPTIONS, + * modules?: { source: Array<{ id: string, name: string, enabled: boolean, defaultEnabled?: boolean, config: object, ownerOnly?: boolean }>, targetConfigured: Set }, + * limits?: typeof FLUXER_LIMITS, + * }} input + * @returns {{ steps: Step[], notes: object[], summary: Record> }} + */ +export function buildPlan({ + source, + target = null, + mappings = {}, + options, + modules, + limits = FLUXER_LIMITS, +}) { + /** @type {Step[]} */ + const steps = []; + const push = (step) => steps.push({ reason: undefined, notes: [], ...step }); + + const { copiedRoles, skippedRoles, copiedChannels, skippedChannels } = copiedSets(source, options); + + // 1. the server's own name, icon and banner + if (options.name || options.icon || options.banner) { + const { spec, notes } = buildIdentitySpec(source, options); + const hash = specHash(spec); + push({ + key: 'guild:identity', + kind: 'guild', + sourceId: 'identity', + name: 'Server name, icon and banner', + op: existing(mappings, 'guild', 'identity', hash), + spec, + hash, + notes, + }); + } + + // 2. roles, lowest first, then their order + if (options.roles) { + const everyone = source.roles.find((r) => r.isEveryone); + if (everyone) { + const keep = keepFluxerOnly(target?.roles?.find((r) => r.isEveryone)?.permissions ?? 0n); + const { spec, notes } = buildRoleSpec(everyone, { keep }); + const hash = specHash(spec); + push({ + key: `role:${everyone.id}`, + kind: 'role', + sourceId: everyone.id, + name: '@everyone', + op: existing(mappings, 'role', everyone.id, hash), + spec, + hash, + notes, + }); + } + for (const role of [...source.roles].filter((r) => !r.isEveryone).sort(byPosition)) { + if (!copiedRoles.has(role.id)) { + push({ + key: `role:${role.id}`, + kind: 'role', + sourceId: role.id, + name: role.name, + op: 'skip', + reason: roleSkipReason(role), + spec: null, + hash: '', + notes: [ + { + code: 'managed-role', + scope: 'role', + ref: { kind: 'role', id: role.id, name: role.name }, + message: `${role.name}: ${roleSkipReason(role)}, so it is not copied.`, + }, + ], + }); + continue; + } + const { spec, notes } = buildRoleSpec(role); + const hash = specHash(spec); + push({ + key: `role:${role.id}`, + kind: 'role', + sourceId: role.id, + name: role.name, + op: decide(mappings, 'role', role.id, hash), + spec, + hash, + notes, + }); + } + const roleOrder = [...copiedRoles].sort((a, b) => { + const ra = source.roles.find((r) => r.id === a); + const rb = source.roles.find((r) => r.id === b); + return byPosition(ra, rb); + }); + if (roleOrder.length > 0) { + const hash = specHash(roleOrder); + push({ + key: 'order-roles:roles', + kind: 'order-roles', + sourceId: 'roles', + name: 'Role order', + op: decide(mappings, 'order', 'roles', hash), + spec: { order: roleOrder }, + hash, + }); + } + } + + // 3. categories first, then the other channels, then their order + if (options.channels) { + const categories = source.channels.filter((c) => c.type === DISCORD_CHANNEL.CATEGORY).sort(byPosition); + const others = source.channels.filter((c) => c.type !== DISCORD_CHANNEL.CATEGORY).sort(byPosition); + for (const ch of [...categories, ...others]) { + const { spec, notes } = buildChannelSpec(ch, { source, copiedRoles }); + if (!spec) { + push({ + key: `channel:${ch.id}`, + kind: 'channel', + sourceId: ch.id, + name: ch.name, + op: 'skip', + reason: 'no such channel type on Fluxer', + spec: null, + hash: '', + notes, + }); + continue; + } + const hash = specHash(spec); + push({ + key: `channel:${ch.id}`, + kind: 'channel', + sourceId: ch.id, + name: ch.name, + op: decide(mappings, 'channel', ch.id, hash), + spec, + hash, + notes, + }); + } + const layout = channelOrder(source, copiedChannels); + if (layout.length > 1) { + const hash = specHash(layout); + push({ + key: 'order-channels:channels', + kind: 'order-channels', + sourceId: 'channels', + name: 'Channel order', + op: decide(mappings, 'order', 'channels', hash), + spec: { order: layout }, + hash, + }); + } + } + + // 4. emoji and stickers, as many as fit + if (options.emoji) { + const emojis = source.emojis.filter((e) => !e.managed).sort((a, b) => (a.id < b.id ? -1 : 1)); + addExpressions(steps, { + kind: 'emoji', + items: emojis, + build: buildEmojiSpec, + mappings, + slots: slotsLeft(limits.emoji, target?.emojiCount ?? 0, mappings.emoji), + limitCode: 'emoji-limit', + noun: 'emoji', + }); + } + if (options.stickers) { + const stickers = [...source.stickers].sort((a, b) => (a.id < b.id ? -1 : 1)); + addExpressions(steps, { + kind: 'sticker', + items: stickers, + build: buildStickerSpec, + mappings, + slots: slotsLeft(limits.stickers, target?.stickerCount ?? 0, mappings.sticker), + limitCode: 'sticker-limit', + noun: 'sticker', + }); + } + + // 5. Sylo's own settings + if (options.modules && modules) { + for (const mod of modules.source) { + if (mod.ownerOnly || NOT_COPIED_MODULES.includes(mod.id)) continue; + // Only a module somebody set up: switched on or off against its default, or with settings saved. + const hasSettings = + mod.enabled !== Boolean(mod.defaultEnabled) || Object.keys(mod.config ?? {}).length > 0; + if (!hasSettings) continue; + if (modules.targetConfigured.has(mod.id)) { + push({ + key: `module:${mod.id}`, + kind: 'module', + sourceId: mod.id, + name: mod.name, + op: 'skip', + reason: 'already set up on the Fluxer server', + spec: null, + hash: '', + notes: [], + }); + continue; + } + const { spec, notes, remapped } = buildModuleSpec( + mod, + { copiedRoles, skippedRoles, copiedChannels, skippedChannels }, + source + ); + const hash = specHash(spec); + push({ + key: `module:${mod.id}`, + kind: 'module', + sourceId: mod.id, + name: mod.name, + op: decide(mappings, 'module', mod.id, hash), + spec, + hash, + notes, + remapped, + }); + } + } + + // 6. the last step: link every copied channel through the bridge (src/mirror/bridgeLink.js) + push({ + key: 'bridge:link', + kind: 'bridge', + sourceId: 'link', + name: 'Link every copied channel through the bridge', + op: 'create', + spec: null, + hash: '', + }); + + const notes = steps.flatMap((s) => s.notes ?? []); + return { steps, notes, summary: summarise(steps) }; +} + +/** The copied channels in the order they sit: for each parent, its children by position. */ +export function channelOrder(source, copied) { + const list = source.channels.filter((c) => copied.has(c.id)); + const out = []; + const groups = new Map(); + for (const c of list) { + const parent = c.parentId && copied.has(c.parentId) ? c.parentId : ''; + if (!groups.has(parent)) groups.set(parent, []); + groups.get(parent).push(c); + } + for (const [parent, children] of groups) { + children + .sort(byPosition) + .forEach((c, i) => out.push({ id: c.id, parentId: parent || null, position: i })); + } + // Categories are ordered among themselves like any top-level channel. + return out.sort((a, b) => (a.parentId ?? '').localeCompare(b.parentId ?? '') || a.position - b.position); +} + +function slotsLeft(cap, targetCount, mapped) { + const ours = mapped?.size ?? 0; + const foreign = Math.max(0, targetCount - ours); + return Math.max(0, cap - foreign); +} + +function addExpressions(steps, { kind, items, build, mappings, slots, limitCode, noun }) { + let used = 0; + // Items that already exist on Fluxer keep their slot; new ones take what is left. + const ordered = [ + ...items.filter((i) => mappings[kind]?.has(i.id)), + ...items.filter((i) => !mappings[kind]?.has(i.id)), + ]; + for (const item of ordered) { + const { spec, notes } = build(item); + const base = { key: `${kind}:${item.id}`, kind, sourceId: item.id, name: item.name }; + if (!spec) { + steps.push({ + ...base, + op: 'skip', + reason: notes[0]?.message ?? 'not supported', + spec: null, + hash: '', + notes, + }); + continue; + } + if (used >= slots) { + steps.push({ + ...base, + op: 'skip', + reason: `over Sylo's limit of ${noun}s for a Fluxer server`, + spec: null, + hash: '', + notes: [ + { + code: limitCode, + scope: kind, + ref: { kind, id: item.id, name: item.name }, + message: `${item.name}: there is no room for it on the Fluxer server, so it is not copied.`, + }, + ], + }); + continue; + } + used += 1; + const hash = specHash(spec); + steps.push({ ...base, op: decide(mappings, kind, item.id, hash), spec, hash, notes }); + } +} + +/** Counts per kind and operation: { role: { create: 3, update: 0, unchanged: 1, skip: 2 }, ... }. */ +export function summarise(steps) { + const out = {}; + for (const s of steps) { + out[s.kind] ??= { create: 0, update: 0, unchanged: 0, skip: 0 }; + out[s.kind][s.op] += 1; + } + return out; +} diff --git a/src/mirror/snapshot.js b/src/mirror/snapshot.js new file mode 100644 index 00000000..48984497 --- /dev/null +++ b/src/mirror/snapshot.js @@ -0,0 +1,200 @@ +// Reading a Discord server into plain data, and the same for the Fluxer server it is copied to. The engine +// and the preview work on these snapshots, never on the SDK objects, so what the preview shows is exactly +// what the copy then reads, and tests can build a server from plain objects. + +/** discord.js ChannelType numbers that a mirror understands. */ +export const DISCORD_CHANNEL = Object.freeze({ + TEXT: 0, + VOICE: 2, + CATEGORY: 4, + ANNOUNCEMENT: 5, + STAGE: 13, + FORUM: 15, + MEDIA: 16, +}); + +/** Fluxer ChannelType numbers. */ +export const FLUXER_CHANNEL = Object.freeze({ + TEXT: 0, + VOICE: 2, + CATEGORY: 4, + ANNOUNCEMENT: 5, + FORUM: 15, + MEDIA: 16, +}); + +/** + * The Fluxer channel type a Discord channel type becomes, or null when it has no counterpart. A stage + * channel becomes a voice channel (Fluxer has no stages); threads and directories are not copied. + */ +export function targetChannelType(discordType) { + switch (discordType) { + case DISCORD_CHANNEL.TEXT: + return FLUXER_CHANNEL.TEXT; + case DISCORD_CHANNEL.VOICE: + case DISCORD_CHANNEL.STAGE: + return FLUXER_CHANNEL.VOICE; + case DISCORD_CHANNEL.CATEGORY: + return FLUXER_CHANNEL.CATEGORY; + case DISCORD_CHANNEL.ANNOUNCEMENT: + return FLUXER_CHANNEL.ANNOUNCEMENT; + case DISCORD_CHANNEL.FORUM: + return FLUXER_CHANNEL.FORUM; + case DISCORD_CHANNEL.MEDIA: + return FLUXER_CHANNEL.MEDIA; + default: + return null; + } +} + +const values = (collection) => [...(collection?.values?.() ?? [])]; +const big = (v) => { + try { + return BigInt(v?.bitfield ?? v ?? 0); + } catch { + return 0n; + } +}; + +/** Whether a role-or-member overwrite is for a role. discord.js: type 0 or 'role' is a role. */ +const isRoleOverwrite = (o) => o.type === 0 || o.type === 'role' || o.type === 'Role'; + +function readChannel(c) { + const overwrites = values(c.permissionOverwrites?.cache).map((o) => ({ + id: String(o.id), + type: isRoleOverwrite(o) ? 'role' : 'member', + allow: big(o.allow), + deny: big(o.deny), + })); + return { + id: String(c.id), + name: String(c.name ?? ''), + type: c.type, + parentId: c.parentId ? String(c.parentId) : null, + position: Number(c.rawPosition ?? c.position ?? 0), + topic: c.topic ?? null, + nsfw: Boolean(c.nsfw), + rateLimitPerUser: Number(c.rateLimitPerUser ?? 0), + bitrate: c.bitrate == null ? null : Number(c.bitrate), + userLimit: c.userLimit == null ? null : Number(c.userLimit), + overwrites, + // A forum or media channel's own settings. + forum: + c.type === DISCORD_CHANNEL.FORUM || c.type === DISCORD_CHANNEL.MEDIA + ? { + tags: (c.availableTags ?? []).map((t) => ({ + name: String(t.name ?? ''), + moderated: Boolean(t.moderated), + emojiName: t.emoji?.name ?? null, + })), + sortOrder: c.defaultSortOrder ?? null, + layout: c.defaultForumLayout ?? null, + slowmode: c.defaultThreadRateLimitPerUser ?? null, + } + : null, + }; +} + +function isThread(c) { + return typeof c.isThread === 'function' ? c.isThread() : false; +} + +/** + * A Discord server as plain data. Channels that are threads are left out; managed roles are listed (with + * `managed: true`) so the report can say why they are not copied. + * @param {any} guild a discord.js Guild (or a test double of its shape) + */ +export function snapshotDiscordGuild(guild) { + const everyoneId = String(guild.roles?.everyone?.id ?? guild.id); + const base = big(guild.roles?.everyone?.permissions ?? guild.roles?.cache?.get?.(everyoneId)?.permissions); + const snap = { + id: String(guild.id), + name: String(guild.name ?? ''), + iconUrl: guild.iconURL?.({ size: 1024, extension: 'png' }) ?? null, + bannerUrl: guild.bannerURL?.({ size: 1024, extension: 'png' }) ?? null, + everyoneId, + // A server marked age-restricted: its channels carry no flag of their own, so the copies get one. + ageRestricted: Number(guild.nsfwLevel) === 3, + channels: values(guild.channels?.cache) + .filter((c) => !isThread(c)) + .map(readChannel), + roles: values(guild.roles?.cache).map((r) => ({ + id: String(r.id), + name: String(r.name ?? ''), + color: Number(r.color ?? r.colour ?? 0), + position: Number(r.rawPosition ?? r.position ?? 0), + hoist: Boolean(r.hoist), + mentionable: Boolean(r.mentionable), + permissions: big(r.permissions), + managed: Boolean(r.managed), + isEveryone: String(r.id) === everyoneId, + })), + emojis: values(guild.emojis?.cache).map((e) => ({ + id: String(e.id), + name: String(e.name ?? ''), + animated: Boolean(e.animated), + managed: Boolean(e.managed), + url: e.imageURL?.({ extension: e.animated ? 'gif' : 'png', size: 128 }) ?? e.url ?? null, + })), + stickers: values(guild.stickers?.cache).map((st) => ({ + id: String(st.id), + name: String(st.name ?? ''), + description: st.description ?? '', + tags: String(st.tags ?? '') + .split(',') + .map((x) => x.trim()) + .filter(Boolean), + // discord.js StickerFormatType: 1 PNG, 2 APNG, 3 Lottie, 4 GIF + format: Number(st.format ?? 1), + url: st.url ?? null, + })), + }; + // What @everyone can do in each channel on Discord, so a copy is never more open than the original. + for (const ch of snap.channels) Object.assign(ch, everyoneAccess(base, ch.overwrites, everyoneId)); + return snap; +} + +const BIT_ADMIN = 1n << 3n; +const BIT_VIEW = 1n << 10n; +const BIT_CONNECT = 1n << 20n; + +/** + * Can @everyone see a channel, and join it if it is a voice channel? Discord's own order: the role's permissions, + * then the channel's @everyone overwrite (deny, then allow). Administrator sees everything. + */ +export function everyoneAccess(base, overwrites, everyoneId) { + const o = overwrites.find((x) => x.type === 'role' && x.id === everyoneId); + const perms = base & BIT_ADMIN ? ~0n : (base & ~(o?.deny ?? 0n)) | (o?.allow ?? 0n); + return { everyoneCanView: (perms & BIT_VIEW) !== 0n, everyoneCanConnect: (perms & BIT_CONNECT) !== 0n }; +} + +/** + * A Fluxer server as plain data: what is already there (a new server comes with a few channels and its + * @everyone role). The mirror leaves all of it alone unless it made it. + * @param {any} guild a Fluxer SDK Guild (or a test double of its shape) + */ +export function snapshotFluxerGuild(guild) { + const everyoneId = String(guild.id); + return { + id: String(guild.id), + name: String(guild.name ?? ''), + everyoneId, + channels: values(guild.channels).map((c) => ({ + id: String(c.id), + name: String(c.name ?? ''), + type: c.type, + parentId: c.parentId ? String(c.parentId) : null, + position: Number(c.position ?? 0), + })), + roles: values(guild.roles?.cache ?? guild.roles).map((r) => ({ + id: String(r.id), + name: String(r.name ?? ''), + position: Number(r.position ?? 0), + permissions: big(r._permissions ?? r.permissions), + managed: Boolean(r.managed), + isEveryone: String(r.id) === everyoneId, + })), + emojiCount: values(guild.emojis).length, + stickerCount: values(guild.stickers).length, + }; +} diff --git a/src/mirror/spec.js b/src/mirror/spec.js new file mode 100644 index 00000000..f32db24d --- /dev/null +++ b/src/mirror/spec.js @@ -0,0 +1,462 @@ +// What Sylo sends to Fluxer for each thing it copies, built from a Discord snapshot (snapshot.js). +// +// A spec is plain JSON. It never holds a Fluxer id: where it refers to another copied thing (a channel's +// category, a permission overwrite for a role) it holds a placeholder, "$channel:" or +// "$role:", and resolveSpec() swaps the placeholders for Fluxer ids at the moment of sending. +// That is what lets the preview show exactly what the copy will send (the same builders run for both), and +// it makes a spec's fingerprint independent of the ids Fluxer hands out. +// +// Every builder also returns `notes`: what could not be carried over (a permission Fluxer lacks, an overwrite +// for one member, a channel type Fluxer has no counterpart for ...). The preview lists the notes before +// anything is created; the copy stores the same notes with each item for the report. +import { createHash } from 'node:crypto'; +import { FLUXER_CHANNEL, targetChannelType } from './snapshot.js'; +import { ELEVATED_FLUXER_MASK, describeDropped, lostSomething, mapPermissions } from './permissions.js'; + +/** What the mirror copies, and whether it goes on after the first copy. All on unless the admin turns one off. */ +export const DEFAULT_OPTIONS = Object.freeze({ + name: true, + icon: true, + banner: true, + roles: true, + channels: true, + emoji: true, + stickers: true, + modules: true, + continuous: true, + // Off: a Discord delete locks the Fluxer copy instead of deleting it. On: it is deleted (only what Sylo made). + mirrorDeletes: false, + // The Discord owner asked for deletions to be mirrored; they only take effect (`mirrorDeletes`) once the owner of the + // Fluxer server has confirmed, because deleting a Fluxer channel deletes messages that exist only on Fluxer. + mirrorDeletesRequested: false, +}); + +/** The options as stored: only the known switches, each a boolean, the rest at their default. */ +export function normaliseOptions(raw) { + const out = { ...DEFAULT_OPTIONS }; + for (const key of Object.keys(DEFAULT_OPTIONS)) { + if (typeof raw?.[key] === 'boolean') out[key] = raw[key]; + } + return out; +} + +/** + * Sylo's planning limits for a Fluxer server. They decide what the preview offers; they are not read from + * Fluxer. If Fluxer refuses an item as over its own limit while copying, the copy stops that kind and + * the report lists the rest. + */ +export const FLUXER_LIMITS = Object.freeze({ + emoji: 500, + stickers: 500, + emojiBytes: 512 * 1024, + stickerBytes: 512 * 1024, + channelName: 100, + roleName: 100, + topic: 1024, + bitrateMin: 8000, + bitrateMax: 96000, +}); + +export const channelRef = (id) => `$channel:${id}`; +export const roleRef = (id) => `$role:${id}`; +const REF = /\$(channel|role):(\d{17,20})/g; +const WHOLE_REF = /^\$(channel|role):(\d{17,20})$/; + +/** A fingerprint of a spec: stable for equal content whatever the key order. */ +export function specHash(spec) { + const canon = (v) => + Array.isArray(v) + ? v.map(canon) + : v && typeof v === 'object' + ? Object.fromEntries( + Object.keys(v) + .sort() + .map((k) => [k, canon(v[k])]) + ) + : v; + return createHash('sha1') + .update(JSON.stringify(canon(spec))) + .digest('hex') + .slice(0, 20); +} + +const clip = (text, max) => String(text ?? '').slice(0, max); + +const note = (code, scope, ref, message, detail = '') => ({ + code, + scope, + ref, + message, + ...(detail ? { detail } : {}), +}); +const refOf = (kind, item) => ({ kind, id: item.id, name: item.name }); + +// --- roles ------------------------------------------------------------------------------------------ + +/** + * The Fluxer role for a Discord role. The @everyone role is never created: it exists on both sides, so its + * spec holds only the permissions (carried over together with the Fluxer-only permissions it already has). + * @param {any} role a role of the snapshot + * @param {{ keep?: bigint }} [ctx] `keep`: Fluxer-only permission bits the Fluxer role already holds + */ +export function buildRoleSpec(role, { keep = 0n } = {}) { + const notes = []; + const mapped = mapPermissions(role.permissions); + if (lostSomething(mapped)) { + notes.push( + note( + 'permissions-dropped', + 'role', + refOf('role', role), + `${role.name}: Fluxer has no permission for ${describeDropped(mapped)}, so it is left off.`, + describeDropped(mapped) + ) + ); + } + let bits = mapped.bits; + if (role.isEveryone && bits & ELEVATED_FLUXER_MASK) { + // @everyone is every member: a powerful permission there would be handed to the whole Fluxer server. + bits &= ~ELEVATED_FLUXER_MASK; + notes.push( + note( + 'everyone-elevated-stripped', + 'role', + refOf('role', role), + '@everyone: powerful permissions (Administrator, Manage Roles and the like) are not copied onto @everyone, because they would apply to every member of the Fluxer server.' + ) + ); + } + const permissions = String(bits | keep); + if (role.isEveryone) return { spec: { permissions }, notes }; + if (mapped.bits & (1n << 3n)) { + notes.push( + note( + 'administrator-role', + 'role', + refOf('role', role), + `${role.name} has Administrator, so on Fluxer it can do everything too.` + ) + ); + } + return { + spec: { + name: clip(role.name, FLUXER_LIMITS.roleName) || 'role', + color: role.color, + hoist: role.hoist, + mentionable: role.mentionable, + permissions, + }, + notes, + }; +} + +/** Why a role is not copied, or null. */ +export function roleSkipReason(role) { + if (role.isEveryone) return null; + if (role.managed) return 'managed by an integration or a bot'; + return null; +} + +// --- channels --------------------------------------------------------------------------------------- + +/** + * The Fluxer channel for a Discord channel. Returns spec null (with a note) for a type Fluxer has no + * counterpart for. + * @param {any} ch a channel of the snapshot + * @param {{ source: any, copiedRoles: Set }} ctx `copiedRoles`: the Discord role ids that are + * copied (an overwrite for any other role is dropped) + */ +export function buildChannelSpec(ch, { source, copiedRoles }) { + const notes = []; + const type = targetChannelType(ch.type); + if (type === null) { + notes.push( + note( + 'channel-type-unsupported', + 'channel', + refOf('channel', ch), + `${ch.name}: Fluxer has no channel of this kind, so it is not copied.` + ) + ); + return { spec: null, notes }; + } + if (ch.type === 13) { + notes.push( + note( + 'stage-as-voice', + 'channel', + refOf('channel', ch), + `${ch.name}: Fluxer has no stage channels, so it becomes a voice channel.` + ) + ); + } + + const overwrites = []; + for (const o of ch.overwrites) { + if (o.type === 'member') { + notes.push( + note( + 'member-overwrite-dropped', + 'channel', + refOf('channel', ch), + `${ch.name}: a permission overwrite for one member is not copied, because members' accounts are not linked between the servers.` + ) + ); + continue; + } + const isEveryone = o.id === source.everyoneId; + if (!isEveryone && !copiedRoles.has(o.id)) { + const role = source.roles.find((r) => r.id === o.id); + notes.push( + note( + 'overwrite-role-not-copied', + 'channel', + refOf('channel', ch), + `${ch.name}: an overwrite for the role ${role?.name ?? o.id} is dropped, because that role is not copied.` + ) + ); + continue; + } + const allow = mapPermissions(o.allow); + const deny = mapPermissions(o.deny); + const dropped = [...new Set([...allow.dropped, ...deny.dropped])]; + const unknown = [...new Set([...allow.unknown, ...deny.unknown])]; + if (dropped.length || unknown.length) { + const text = describeDropped({ dropped, unknown }); + notes.push( + note( + 'permissions-dropped', + 'channel', + refOf('channel', ch), + `${ch.name}: Fluxer has no permission for ${text}, so it is left off the overwrite.`, + text + ) + ); + } + if (allow.bits === 0n && deny.bits === 0n) continue; + overwrites.push({ + id: isEveryone ? '$role:everyone' : roleRef(o.id), + type: 0, + allow: String(allow.bits), + deny: String(deny.bits), + }); + } + // Never more open than on Discord. A channel @everyone cannot see there is hidden from @everyone here, by an + // overwrite of its own, whatever else is copied: with roles off, or while the @everyone role itself has not been + // set yet, the channel is still private on Fluxer (where @everyone can see channels by default). + let hide = 0n; + if (ch.everyoneCanView === false) hide |= 1n << 10n; + if (type === FLUXER_CHANNEL.VOICE && ch.everyoneCanConnect === false) hide |= 1n << 20n; + if (hide) { + const own = overwrites.find((o) => o.id === '$role:everyone'); + if (own) { + own.deny = String(BigInt(own.deny) | hide); + own.allow = String(BigInt(own.allow) & ~hide); + } else { + overwrites.push({ id: '$role:everyone', type: 0, allow: '0', deny: String(hide) }); + } + } + overwrites.sort((a, b) => (a.id < b.id ? -1 : a.id > b.id ? 1 : 0)); + + const spec = { + name: clip(ch.name, FLUXER_LIMITS.channelName) || 'channel', + type, + parentId: ch.parentId ? channelRef(ch.parentId) : null, + permissionOverwrites: overwrites, + }; + if (type === FLUXER_CHANNEL.TEXT || type === FLUXER_CHANNEL.ANNOUNCEMENT) { + spec.topic = ch.topic ? clip(ch.topic, FLUXER_LIMITS.topic) : null; + spec.nsfw = ch.nsfw || Boolean(source.ageRestricted); + if (type === FLUXER_CHANNEL.TEXT) spec.rateLimitPerUser = ch.rateLimitPerUser; + } else if (type === FLUXER_CHANNEL.VOICE) { + const bitrate = + ch.bitrate == null + ? null + : Math.min(Math.max(ch.bitrate, FLUXER_LIMITS.bitrateMin), FLUXER_LIMITS.bitrateMax); + spec.bitrate = bitrate; + spec.userLimit = ch.userLimit ?? 0; + } else if (type === FLUXER_CHANNEL.FORUM || type === FLUXER_CHANNEL.MEDIA) { + spec.topic = ch.topic ? clip(ch.topic, FLUXER_LIMITS.topic) : null; + spec.nsfw = ch.nsfw || Boolean(source.ageRestricted); + if (ch.forum) { + spec.availableTags = ch.forum.tags.slice(0, 20).map((t) => ({ + name: clip(t.name, 50), + moderated: t.moderated, + ...(t.emojiName ? { emojiName: t.emojiName } : {}), + })); + if (ch.forum.sortOrder != null) spec.defaultSortOrder = ch.forum.sortOrder; + if (ch.forum.layout != null && type === FLUXER_CHANNEL.FORUM) spec.defaultForumLayout = ch.forum.layout; + if (ch.forum.slowmode != null) spec.defaultThreadRateLimitPerUser = ch.forum.slowmode; + } + } + return { spec, notes }; +} + +// --- emoji and stickers ------------------------------------------------------------------------------- + +/** An emoji name Fluxer accepts: letters, digits and underscores, at least two characters. */ +export function emojiName(name) { + const cleaned = String(name ?? '') + .replace(/[^A-Za-z0-9_]/g, '_') + .slice(0, 32); + return cleaned.length >= 2 ? cleaned : `${cleaned}_`.padEnd(2, '_'); +} + +export const buildEmojiSpec = (emoji) => ({ + spec: { name: emojiName(emoji.name), animated: emoji.animated }, + notes: [], +}); + +/** discord.js StickerFormatType: 3 is Lottie, an animation format Fluxer does not take. */ +export function buildStickerSpec(sticker) { + if (sticker.format === 3) { + return { + spec: null, + notes: [ + note( + 'sticker-lottie', + 'sticker', + refOf('sticker', sticker), + `${sticker.name}: Lottie stickers cannot be copied to Fluxer.` + ), + ], + }; + } + return { + spec: { + name: clip(sticker.name, 30).padEnd(2, '_'), + // Fluxer wants 1 to 500 characters or nothing. + description: clip(sticker.description, 500) || null, + tags: sticker.tags.slice(0, 5).map((t) => clip(t, 30)), + }, + notes: [], + }; +} + +// --- the server's own name, icon and banner ---------------------------------------------------------- + +export function buildIdentitySpec(source, options) { + const spec = {}; + if (options.name) spec.name = clip(source.name, 100); + // The URL carries the asset's hash, so a changed icon changes the fingerprint without a download. + if (options.icon) spec.icon = source.iconUrl ?? null; + if (options.banner) spec.banner = source.bannerUrl ?? null; + return { spec, notes: [] }; +} + +// --- Sylo's own settings (module configs) ------------------------------------------------------------ + +const MESSAGE_ID_KEY = /message_?id$/i; +const ID = /^\d{17,20}$/; + +/** + * A module's settings with the ids of this server turned into placeholders: a channel or role id that is + * copied becomes "$channel:" / "$role:" (also inside `<#id>` and `<@&id>` mentions in text); an id + * of a channel or role that is not copied is dropped; the id of a message is cleared, because the message + * is not copied; any other id of 17 to 20 digits (a user, a deleted item) is cleared. Short numbers stay. + * @param {any} config + * @param {{ channels: Set, roles: Set, skippedChannels: Set, skippedRoles: Set }} ids + * @returns {{ config: any, remapped: number, dropped: number, cleared: number }} + */ +export function placeholderConfig(config, ids) { + const stats = { remapped: 0, dropped: 0, cleared: 0 }; + const swap = (id) => { + if (ids.channels.has(id)) return channelRef(id); + if (ids.roles.has(id)) return roleRef(id); + return null; + }; + const isLost = (id) => ids.skippedChannels.has(id) || ids.skippedRoles.has(id); + const walk = (value, key) => { + if (typeof value === 'string') { + if (MESSAGE_ID_KEY.test(key ?? '') && ID.test(value)) { + stats.cleared += 1; + return ''; + } + if (ID.test(value)) { + const ref = swap(value); + if (ref) { + stats.remapped += 1; + return ref; + } + if (isLost(value)) { + stats.dropped += 1; + return undefined; + } + // A user id, the id of something deleted, a number only Discord knows: meaningless on Fluxer. + stats.cleared += 1; + return undefined; + } + return value.replace(/<(#|@&)(\d{17,20})>/g, (all, sigil, id) => { + const ref = swap(id); + if (ref) { + stats.remapped += 1; + return `<${sigil}${ref}>`; + } + if (isLost(id)) { + stats.dropped += 1; + return ''; + } + return all; + }); + } + if (Array.isArray(value)) return value.map((v) => walk(v, key)).filter((v) => v !== undefined); + if (value && typeof value === 'object') { + const out = {}; + for (const [k, v] of Object.entries(value)) { + const next = walk(v, k); + out[k] = next === undefined ? '' : next; + } + return out; + } + return value; + }; + return { config: walk(config, ''), ...stats }; +} + +// --- placeholders ------------------------------------------------------------------------------------- + +/** The placeholders a value contains, as "channel:" / "role:". */ +export function refsIn(value) { + const found = new Set(); + const walk = (v) => { + if (typeof v === 'string') for (const m of v.matchAll(REF)) found.add(`${m[1]}:${m[2]}`); + else if (Array.isArray(v)) v.forEach(walk); + else if (v && typeof v === 'object') Object.values(v).forEach(walk); + }; + walk(value); + return [...found]; +} + +/** + * Swap the placeholders of a spec for Fluxer ids. `lookup('channel' | 'role', discordId)` gives the Fluxer id or + * null. A placeholder that stands alone and cannot be resolved becomes null (or leaves its array); one inside + * a text is removed. + * @returns {{ value: any, unresolved: string[] }} + */ +export function resolveSpec(spec, lookup) { + const unresolved = []; + const find = (kind, id) => { + if (kind === 'role' && id === 'everyone') return lookup('role', 'everyone'); + const found = lookup(kind, id); + if (!found) unresolved.push(`${kind}:${id}`); + return found; + }; + const walk = (v) => { + if (typeof v === 'string') { + const whole = WHOLE_REF.exec(v) ?? /^\$(role):(everyone)$/.exec(v); + if (whole) return find(whole[1], whole[2]) ?? undefined; + return v.replace(/\$(channel|role):(\d{17,20})/g, (_all, kind, id) => find(kind, id) ?? ''); + } + if (Array.isArray(v)) return v.map(walk).filter((x) => x !== undefined); + if (v && typeof v === 'object') { + const out = {}; + for (const [k, x] of Object.entries(v)) { + const next = walk(x); + out[k] = next === undefined ? null : next; + } + return out; + } + return v; + }; + return { value: walk(spec), unresolved }; +} diff --git a/src/mirror/target.js b/src/mirror/target.js new file mode 100644 index 00000000..f4d7b307 --- /dev/null +++ b/src/mirror/target.js @@ -0,0 +1,238 @@ +// The Fluxer side of the mirror: everything the engine does to the Fluxer server goes through this one +// object, so the engine never touches the SDK itself and a test can hand it a fake with the same methods. +// +// Each method takes a spec from spec.js with its placeholders already resolved to Fluxer ids, and returns +// the id of what it made. An SDK error is thrown as is; the engine classifies it (errors.js). +// +// `me` is the Fluxer server's own Sylo member: the engine checks its permissions before anything is created +// and clamps role positions under Sylo's own highest role. +import { FLUXER_LIMITS } from './spec.js'; +import { isNotFound } from './errors.js'; + +const get = (collection, id) => collection?.get?.(id) ?? collection?.cache?.get?.(id) ?? null; +const list = (collection) => [...(collection?.values?.() ?? collection?.cache?.values?.() ?? [])]; + +const bits = (value) => BigInt(value ?? 0); + +/** The permissions Sylo needs on the Fluxer server for a mirror. Administrator covers all of them. */ +export const NEEDED_PERMISSIONS = Object.freeze([ + 'ManageChannels', + 'ManageRoles', + 'ManageExpressions', + 'ManageGuild', +]); + +/** + * Fluxer ranks roles by position, highest first, and by id (the older, smaller one first) when positions are equal; + * every role starts at position 1 and positions are only renumbered when someone reorders. So a position number says + * nothing by itself: what counts is the rank. The roles of `roles` that rank above `sylo` (not `ignore`d, not Sylo's + * own), in rank order. + * @param {{ id: string, position: number }} sylo Sylo's highest role + * @param {Array<{ id: string, name: string, position: number }>} roles + * @param {Set} ignore + */ +export function rolesAbove(sylo, roles, ignore = new Set()) { + const outranks = (a, b) => + a.position > b.position || (a.position === b.position && BigInt(a.id) < BigInt(b.id)); + return roles + .filter((r) => r.id !== sylo.id && !ignore.has(r.id) && outranks(r, sylo)) + .sort((a, b) => b.position - a.position || (BigInt(a.id) < BigInt(b.id) ? -1 : 1)); +} + +/** @param {{ mime: string, bytes: Buffer }} image */ +export const toDataUri = (image) => `data:${image.mime};base64,${image.bytes.toString('base64')}`; + +/** + * @param {any} guild a Fluxer SDK Guild + */ +export function createFluxerTarget(guild) { + const overwrites = (spec) => + (spec.permissionOverwrites ?? []).map((o) => ({ + id: o.id, + type: o.type, + allow: bits(o.allow), + deny: bits(o.deny), + })); + + const channelBody = (spec) => ({ + ...spec, + ...(spec.permissionOverwrites ? { permissionOverwrites: overwrites(spec) } : {}), + }); + + // The cached item, else a look-up on Fluxer. Only a real "not found" answer is null (and so "gone"): a failed + // look-up (a timeout, a 429, a 5xx) is thrown, so the engine retries it instead of making a second copy. + const find = async (cached, lookup, what) => { + if (cached) return cached; + if (typeof lookup !== 'function') throw new Error(`The ${what} could not be looked up.`); + try { + return (await lookup()) ?? null; + } catch (err) { + if (isNotFound(err)) return null; + throw err; + } + }; + const channelOf = (id) => + find( + get(guild.channels, id), + guild.client?.channels?.fetch && (() => guild.client.channels.fetch(id)), + 'channel' + ); + const roleOf = (id) => find(get(guild.roles, id), guild.fetchRole && (() => guild.fetchRole(id)), 'role'); + const emojiOf = (id) => + find(get(guild.emojis, id), guild.fetchEmoji && (() => guild.fetchEmoji(id)), 'emoji'); + const stickerOf = (id) => + find(get(guild.stickers, id), guild.fetchSticker && (() => guild.fetchSticker(id)), 'sticker'); + + const gone = () => Object.assign(new Error('Unknown channel or role'), { statusCode: 404 }); + + return { + kind: 'fluxer', + id: String(guild.id), + name: String(guild.name ?? ''), + limits: () => FLUXER_LIMITS, + + /** The permissions of Sylo's role on the server that the mirror needs and Sylo lacks. */ + missingPermissions() { + const perms = guild.members?.me?.permissions; + if (!perms || typeof perms.has !== 'function') return []; + if (perms.has('Administrator')) return []; + return NEEDED_PERMISSIONS.filter((p) => !perms.has(p)); + }, + /** + * The roles on the Fluxer server that rank above Sylo's own (see rolesAbove), apart from `ignore` (the ids of roles + * the mirror made itself: they are ordered under Sylo). Null when Sylo's own role is not known. + */ + rolesAboveSylo(ignore = []) { + const highest = guild.members?.me?.roles?.highest; + if (!highest?.id) return null; + const roles = list(guild.roles).map((r) => ({ + id: String(r.id), + name: String(r.name ?? ''), + position: Number(r.position ?? 0), + })); + return rolesAbove( + { id: String(highest.id), position: Number(highest.position ?? 0) }, + roles, + new Set([guild.id, ...ignore]) + ); + }, + + // --- what is there now (for adoption after a crash, and the preview's counts) --- + listChannels: () => + list(guild.channels).map((c) => ({ + id: String(c.id), + name: String(c.name ?? ''), + type: c.type, + parentId: c.parentId ? String(c.parentId) : null, + })), + listRoles: () => + list(guild.roles).map((r) => ({ + id: String(r.id), + name: String(r.name ?? ''), + managed: Boolean(r.managed), + position: Number(r.position ?? 0), + })), + listEmojis: () => list(guild.emojis).map((e) => ({ id: String(e.id), name: String(e.name ?? '') })), + listStickers: () => list(guild.stickers).map((e) => ({ id: String(e.id), name: String(e.name ?? '') })), + + // --- the server itself --- + async editGuild({ name, icon, banner }) { + const body = {}; + if (name !== undefined) body.name = name; + if (icon !== undefined) body.icon = icon; + if (banner !== undefined) body.banner = banner; + await guild.edit(body); + }, + + // --- roles --- + async createRole(spec) { + const role = await guild.createRole({ + name: spec.name, + permissions: bits(spec.permissions), + color: spec.color, + }); + // Hoisting and mentionability cannot be set when a role is made. If this second call fails the role exists: + // the error carries its id, so the engine tracks it and the retry updates it instead of making another. + if (spec.hoist || spec.mentionable) { + try { + await role.edit({ hoist: spec.hoist, mentionable: spec.mentionable }); + } catch (err) { + err.createdId = String(role.id); + throw err; + } + } + return String(role.id); + }, + async editRole(id, spec) { + const role = await roleOf(id); + if (!role) throw gone(); + const body = { permissions: bits(spec.permissions) }; + if (spec.name !== undefined) + Object.assign(body, { + name: spec.name, + color: spec.color, + hoist: spec.hoist, + mentionable: spec.mentionable, + }); + await role.edit(body); + }, + async deleteRole(id) { + const role = await roleOf(id); + if (!role) throw gone(); + await role.delete('Mirrored from the Discord server'); + }, + async setRolePositions(updates) { + await guild.setRolePositions(updates); + }, + + // --- channels --- + async createChannel(spec) { + const channel = await guild.createChannel(channelBody(spec)); + return String(channel.id); + }, + async editChannel(id, spec) { + const channel = await channelOf(id); + if (!channel) throw gone(); + const { type, ...body } = channelBody(spec); + // Only a text channel and an announcement channel can change into each other. + const textual = (t) => t === 0 || t === 5; + await channel.edit(textual(type) && textual(channel.type) ? { type, ...body } : body); + }, + async deleteChannel(id) { + const channel = await channelOf(id); + if (!channel) throw gone(); + await channel.delete({ reason: 'Mirrored from the Discord server' }); + }, + async setChannelPositions(updates) { + await guild.setChannelPositions(updates); + }, + + // --- emoji and stickers --- + async createEmoji({ name, image }) { + return String((await guild.createEmoji({ name, image: toDataUri(image) })).id); + }, + async editEmoji(id, { name }) { + const emoji = await emojiOf(id); + if (!emoji) throw gone(); + await emoji.edit({ name }); + }, + async deleteEmoji(id) { + const emoji = await emojiOf(id); + if (!emoji) throw gone(); + await emoji.delete('Mirrored from the Discord server'); + }, + async createSticker({ name, description, tags, image }) { + return String((await guild.createSticker({ name, description, tags, image: toDataUri(image) })).id); + }, + async editSticker(id, { name, description, tags }) { + const sticker = await stickerOf(id); + if (!sticker) throw gone(); + await sticker.edit({ name, description, tags }); + }, + async deleteSticker(id) { + const sticker = await stickerOf(id); + if (!sticker) throw gone(); + await sticker.delete('Mirrored from the Discord server'); + }, + }; +} diff --git a/src/modules/bridge.js b/src/modules/bridge.js new file mode 100644 index 00000000..8f4f3215 --- /dev/null +++ b/src/modules/bridge.js @@ -0,0 +1,1679 @@ +// Bridge: one conversation between a Discord channel and a Fluxer channel. +// +// A link joins two channels, one on each platform. It starts with a request from one server (a one-time +// code that expires in 15 minutes) and is accepted, with that code, by someone with Edit access in the +// other server. Both servers therefore agreed, Sylo must be in both, and either can unlink at any time. +// Each side has its own options (what leaves THAT server), so neither can make the other share more. +// +// What crosses, through a webhook Sylo makes in each channel: messages (with the author's name and picture), +// replies (a quote of the original), files (re-uploaded within limits, otherwise a link), custom emoji with the +// same name, stickers, rich embeds, polls (as text) and, if a server turns it on, join messages. Edits and +// deletes follow through the message map (db/bridge.js), which keeps a message 30 days. +// +// Safety, in the order a message meets it: +// - no loop: a message sent by any Sylo bridge webhook, or by Sylo itself, is never relayed again, and a +// channel is in at most one link, so a copy cannot come back +// - no ping: every copy is sent with allowedMentions none, and the text has its mentions turned into names +// - bots are off by default (per side), and a message that is not a plain chat message is never relayed +// - an age-restricted channel is only ever linked to an age-restricted channel, checked at link time and again +// on every message (a change pauses the link) +// - the webhook is rate limited: one queue per webhook, spaced sends, back-off on 429, a cap on the queue +// - a link that fails (no permission, webhook cannot be remade, delivery fails five times in a row) is +// stopped, shown as a problem in Health, and announced in both channels; it is not retried forever +// - each server is isolated: queues, links and failures are per link, never shared between servers +import { ChannelType } from 'discord.js'; +import { on } from './dispatch.js'; +import { every } from './schedule.js'; +import { getGuildModule, isModuleEnabled, setGuildModule } from '../db/modules.js'; +import { recordAudit } from '../db/audit.js'; +import { onBeforePurge } from '../db/purge.js'; +import { PAIR_RUNNING, getPair as getMirrorPair, mirrorPairOfLink } from '../db/mirror.js'; +import * as store from '../db/bridge.js'; +import { runtime } from '../runtime.js'; +import { splitPlatformGuildId } from '../lib/platformId.js'; +import { guildByKey } from '../platform/platformOf.js'; +import { createWebhookTransport } from '../platform/webhooks.js'; +import { isAgeRestricted as ageOf, jumpUrl } from './quoteBoard.js'; +import { safeFetch } from '../lib/safeFetch.js'; +import { log } from '../lib/log.js'; +import { sweepMap } from '../lib/sweepMap.js'; +import { BridgeQueueError, classifyError, createRelayQueue } from './lib/bridgeQueue.js'; +import { + FILE_BYTES_LIMIT, + escapeMarkdown, + fileLinkLines, + hashContent, + ZWSP, + inertAt, + joinParts, + mapEmoji, + neutraliseMentions, + normaliseOptions, + pollText, + planAttachments, + plainEmbeds, + plainTimestamps, + replyHead, + senderName, + stickerLines, +} from './lib/bridgeText.js'; + +// The module has no settings of its own (modules/limits.js lists none); these are its ceilings, sent to the +// page with the overview. +/** The most links a server can have, and the most requests it can have waiting. */ +export const MAX_LINKS = 10; +export const MAX_REQUESTS = 5; +/** The ceiling for a server that is mirrored (src/mirror/bridgeLink.js links every copied text channel). */ +export const MAX_MIRROR_LINKS = 50; + +/** How many failed deliveries in a row stop a link. */ +export const FAIL_THRESHOLD = 5; +const HOOK_NAME = 'Sylo bridge'; +const CACHE_MS = 60_000; + +/** A refusal the dashboard shows as it is. `status` is the HTTP answer. */ +export class BridgeError extends Error { + /** `code` names the refusal for code that has to tell them apart (the server mirror's report). */ + constructor(message, status = 400, code = '') { + super(message); + this.name = 'BridgeError'; + this.status = status; + this.code = code; + } +} + +// --- wiring that tests replace ----------------------------------------------------------------------- + +let transport = createWebhookTransport(); +let queue = createRelayQueue(); +let download = (url, maxBytes) => safeFetch(url, { maxBytes, timeoutMs: 15_000 }); + +// One budget for every file download of the process, across all servers: files are held in memory until +// they are sent, so a few busy bridges must not add up to hundreds of megabytes. A file that does not fit in +// the budget crosses as a link instead of waiting for room. +const DEFAULT_BUDGET = Object.freeze({ concurrent: 4, bytes: 32 * 1024 * 1024 }); +let budget = { ...DEFAULT_BUDGET }; +const inUse = { concurrent: 0, bytes: 0 }; +function reserveDownload(size) { + if (inUse.concurrent >= budget.concurrent || inUse.bytes + size > budget.bytes) return false; + inUse.concurrent += 1; + inUse.bytes += size; + return true; +} +function releaseDownload(size) { + inUse.concurrent -= 1; + inUse.bytes -= size; +} + +/** Use other parts (tests): a transport, a queue, a downloader, a download budget. No arguments: the real ones. */ +export function useBridgeParts({ transport: t, queue: q, download: d, budget: b } = {}) { + transport = t ?? createWebhookTransport(); + queue = q ?? createRelayQueue(); + download = d ?? ((url, maxBytes) => safeFetch(url, { maxBytes, timeoutMs: 15_000 })); + budget = { ...DEFAULT_BUDGET, ...b }; +} + +// Relays that are still on their way, so a test (and a shutdown) can wait for them. +const inFlight = new Set(); +const track = (promise) => { + inFlight.add(promise); + promise.finally(() => inFlight.delete(promise)).catch(() => {}); + return promise; +}; +/** Resolves when every relay started so far has been delivered or given up. */ +export const bridgeIdle = async () => { + while (inFlight.size) await Promise.allSettled([...inFlight]); +}; + +// --- small helpers ----------------------------------------------------------------------------------- + +const other = (side) => (side === 'a' ? 'b' : 'a'); +const guildOf = (link, side) => (side === 'a' ? link.guildA : link.guildB); +const channelOf = (link, side) => (side === 'a' ? link.channelA : link.channelB); +const optionsOf = (link, side) => normaliseOptions(side === 'a' ? link.optionsA : link.optionsB); +const hookOf = (link, side) => + side === 'a' + ? { id: link.hookAId, token: link.hookAToken, channelId: link.channelA } + : { id: link.hookBId, token: link.hookBToken, channelId: link.channelB }; +const sideOf = (link, key) => (link.guildA === key ? 'a' : link.guildB === key ? 'b' : null); +const platformOf = (key) => splitPlatformGuildId(key)?.platform ?? 'discord'; +const laneKey = (link, side) => `${link.id}|${side}`; +const TEXT_TYPES = new Set([ChannelType.GuildText, ChannelType.GuildAnnouncement]); + +const channelIn = (key, channelId) => guildByKey(key)?.channels?.cache?.get?.(channelId) ?? null; +const isTextChannel = (channel) => Boolean(channel) && TEXT_TYPES.has(channel.type); +/** + * Is this channel age-restricted, the way the platform decides it? On Fluxer that is the channel's own + * setting, else its category's, else the server's (and what cannot be read counts as restricted: + * quoteBoard.js, fluxerAgeRestricted). A Discord server that is itself marked age-restricted counts for + * every channel in it. + */ +function ageRestricted(key, channel) { + const guild = guildByKey(key); + if (platformOf(key) === 'discord' && guild?.nsfwLevel === 3) return true; + return ageOf(guild, channel); +} + +// Names of servers and channels in Sylo's own notices: no markdown, so a name cannot make a link. +const serverLabel = (key) => escapeMarkdown(inertAt(guildByKey(key)?.name ?? 'another server')); +const channelLabel = (key, channelId) => escapeMarkdown(channelIn(key, channelId)?.name ?? 'a channel'); + +// Messages that were deleted: a delete can win the race against the relay of the same message (moderation +// runs right after the relay starts), so the relay asks before it posts and again after. +const deletedSoon = new Map(); +const TOMBSTONE_MS = 2 * 60_000; +const markDeleted = (key, messageId) => { + if (deletedSoon.size > 20_000) deletedSoon.clear(); + deletedSoon.set(`${key}|${messageId}`, Date.now()); +}; +const wasDeleted = (key, messageId) => + Date.now() - (deletedSoon.get(`${key}|${messageId}`) ?? 0) < TOMBSTONE_MS; +sweepMap(deletedSoon, (at, now) => now - at > TOMBSTONE_MS); + +// The bridge is for the owners of the two servers only (the routes check who is calling). The owner of each +// server is recorded when the link is made and when a new owner confirms it; if the platform reports another +// owner later, the link pauses until that owner presses Resume. +const ownerOf = (key) => String(guildByKey(key)?.ownerId ?? ''); + +/** The side whose server has another owner than the one recorded, or null. An unknown owner is not a change. */ +function ownerChangedSide(link) { + for (const s of ['a', 'b']) { + const recorded = s === 'a' ? link.ownerA : link.ownerB; + const now = ownerOf(guildOf(link, s)); + if (recorded && now && recorded !== now) return s; + } + return null; +} + +/** Pause a running link because the owner of `side`'s server changed, and say so on both sides. */ +async function pauseForOwner(link, side) { + if (link.status !== 'active') return; + await store.setLinkStatus(link.id, { status: 'paused', pausedSide: side, reason: 'owner-changed' }); + dropBridgeCache(); + queue.clear(laneKey(link, 'a')); + queue.clear(laneKey(link, 'b')); + log.info('bridge', `link ${link.id} paused: the owner of a server changed`); + for (const s of ['a', 'b']) { + await recordAudit(guildOf(link, s), { + actor: 'Sylo', + action: 'bridge:pause', + detail: + `the bridge of #${channelName(guildOf(link, s), channelOf(link, s))} paused: the owner of ${serverName(guildOf(link, side))} changed`.slice( + 0, + 500 + ), + change: null, + }); + await say( + guildOf(link, s), + channelOf(link, s), + `🌉 **Bridge paused:** the owner of ${serverLabel(guildOf(link, side))} changed. Nothing is shared until the new owner confirms it.` + ); + } +} + +/** Cached lookups of "which link is this channel in", dropped whenever a link changes. */ +const cache = new Map(); +export const dropBridgeCache = () => cache.clear(); +async function findLink(key, channelId) { + const id = `${key}|${channelId}`; + const hit = cache.get(id); + if (hit && Date.now() - hit.at < CACHE_MS) return hit.value; + const value = await store.linkForChannel(key, channelId); + if (cache.size > 5000) cache.clear(); + cache.set(id, { at: Date.now(), value }); + return value; +} + +/** Say something in a channel (the visible notices). Best effort: a channel Sylo cannot write in is skipped. */ +async function say(key, channelId, content) { + try { + await channelIn(key, channelId)?.send?.({ content, allowedMentions: { parse: [] } }); + } catch (err) { + log.debug('bridge', `could not post a notice in ${channelId}: ${err?.message ?? err}`); + } +} + +const platformName = (key) => (platformOf(key) === 'fluxer' ? 'Fluxer' : 'Discord'); +const serverName = (key) => inertAt(guildByKey(key)?.name ?? 'another server'); +const channelName = (key, channelId) => channelIn(key, channelId)?.name ?? 'a channel'; + +/** The notice both channels show while a bridge is on. */ +const onNotice = (mineKey, theirKey, theirChannelId) => + `🌉 **Bridge on.** Messages in this channel are shared with #${channelLabel(theirKey, theirChannelId)} in ` + + `**${serverLabel(theirKey)}** on ${platformName(theirKey)}, and theirs appear here. Nobody is pinged across ` + + `the bridge. Server admins can pause or unlink it at any time in the Sylo dashboard.`; + +const REASONS = { + permissions: + 'Sylo is missing a permission it needs here (Manage Webhooks, or to send in the channel). Fix the permission, then resume the bridge in the dashboard.', + webhook: 'Sylo could not make a new webhook after the old one was deleted.', + unreachable: 'Messages could not be delivered several times in a row.', + 'channel-deleted': 'A bridged channel was deleted.', + 'channel-missing': 'Sylo cannot see one of the bridged channels.', + 'owner-changed': + 'The owner of one of the servers changed. The bridge is paused until the new owner confirms it: as the owner, open the Bridge page and press Resume.', + 'age-restriction': + 'One channel changed its age restriction, so the two no longer match. An age-restricted channel can only be bridged with an age-restricted one.', + mirror: + 'This bridge was made by the server mirror and is paused with it. It runs again when the mirror does.', + 'mirror-ended': + 'The server mirror that made this bridge was removed, and the bridge was paused. Only the server that removed the mirror can resume it; either server can unlink it.', + visibility: + 'The two channels are no longer open to the same people (one became private). A channel everyone can read is never bridged to a private one.', +}; +/** The text a link's stored failure reason stands for. */ +export const reasonText = (code) => REASONS[code] ?? 'The bridge stopped working.'; + +// --- the pieces of a link, looked up on the platforms ------------------------------------------------ + +/** Does `perms` lack `name`? No answer at all counts as lacking; a permission name the platform does not know does not. */ +function lacks(perms, name) { + if (!perms || typeof perms.has !== 'function') return true; + try { + return perms.has(name) !== true; + } catch { + return false; + } +} + +/** + * The person linking must be able to view, read and send in the channel themselves: Sylo does not let + * anyone make it do what they could not do (the bridge would carry a hidden channel to a server they run). + * `member` is their member in this server. Open mode (no logins) passes `open: true`. + */ +export function assertActorMayUse(key, channelId, member, { open = false } = {}) { + if (open) return; + const channel = channelIn(key, channelId); + const perms = member && channel?.permissionsFor?.(member); + const missing = ['ViewChannel', 'ReadMessageHistory', 'SendMessages'].filter((p) => lacks(perms, p)); + if (!perms || missing.length) { + throw new BridgeError( + `You need to be able to view, read and send in #${channel?.name ?? 'that channel'} yourself to bridge it.`, + 403, + 'visibility' + ); + } +} + +/** + * Check a channel of a server can be bridged: Sylo sees it, it is a normal text channel, and Sylo may + * view it, send in it and manage webhooks there. + * @returns {{ guild: any, channel: any }} + */ +function checkChannel(key, channelId) { + const guild = guildByKey(key); + if (!guild) throw new BridgeError('Sylo is not in that server.', 404, 'no-guild'); + const channel = guild.channels?.cache?.get?.(channelId); + if (!isTextChannel(channel)) { + throw new BridgeError('Pick a normal text channel of this server.', 400, 'not-text'); + } + const perms = channel.permissionsFor?.(guild.members?.me); + // Sylo's own rights: when they cannot be read, they count as missing. + const missing = ['ViewChannel', 'SendMessages', 'ManageWebhooks'].filter((p) => lacks(perms, p)); + if (missing.length) { + throw new BridgeError( + `Sylo needs ${missing.join(', ')} in #${channel.name}. Give Sylo's role those permissions there first.`, + 400, + 'permissions' + ); + } + return { guild, channel }; +} + +async function requireModule(key) { + if (!(await isModuleEnabled(key, 'bridge'))) { + throw new BridgeError('Turn the Bridge module on for this server first.', 409); + } +} + +function requireBothPlatforms() { + if (!runtime.clients.discord || !runtime.clients.fluxer) { + throw new BridgeError( + 'A bridge needs this Sylo to be connected to both Discord and Fluxer, and it is not.', + 409, + 'platform' + ); + } +} + +/** The age rule: an age-restricted channel only with an age-restricted channel. */ +function checkAge(keyA, a, keyB, b) { + if (ageRestricted(keyA, a) !== ageRestricted(keyB, b)) { + throw new BridgeError( + 'An age-restricted channel can only be bridged with another age-restricted channel.', + 400, + 'age-restriction' + ); + } +} + +// --- linking: request, accept ------------------------------------------------------------------------ + +/** + * Ask for a link from one of this server's channels. Returns the one-time code to give the other server. + * @param {{ guildKey: string, channelId: string, options?: object, actor: string }} req + */ +export async function requestLink({ guildKey, channelId, options, actor }) { + requireBothPlatforms(); + await requireModule(guildKey); + checkChannel(guildKey, channelId); + if (await store.linkForChannel(guildKey, channelId)) + throw new BridgeError('That channel is already bridged.', 409); + if (await store.channelHasPendingCode(guildKey, channelId)) { + throw new BridgeError('That channel already has a request waiting. Cancel it to make a new one.', 409); + } + if ((await store.linkCount(guildKey)) >= MAX_LINKS) { + throw new BridgeError(`A server can have at most ${MAX_LINKS} bridged channels.`, 409); + } + if ((await store.pendingCodeCount(guildKey)) >= MAX_REQUESTS) { + throw new BridgeError(`At most ${MAX_REQUESTS} requests can wait at once. Cancel one first.`, 409); + } + const made = await store.createCode({ + guildId: guildKey, + channelId, + options: normaliseOptions(options), + requestedBy: actor, + guildOwner: ownerOf(guildKey), + }); + await recordAudit(guildKey, { + actor, + action: 'bridge:request', + detail: `asked to bridge #${channelName(guildKey, channelId)} (valid 15 minutes)`, + change: null, + }); + return made; +} + +export async function cancelRequest({ guildKey, id, actor }) { + if (!(await store.cancelCode(guildKey, id))) throw new BridgeError('That request is gone already.', 404); + await recordAudit(guildKey, { + actor, + action: 'bridge:cancel', + detail: 'cancelled a bridge request', + change: null, + }); +} + +const INVALID_CODE = 'That code is not valid, has expired or was already used.'; + +/** + * Accept a request with its code, linking one of THIS server's channels to the channel the code was made for. + * Everything that can be refused is checked before the code is used up, so a wrong channel does not burn it. + * @param {{ guildKey: string, channelId: string, code: string, options?: object, actor: string }} req + */ +export async function acceptLink({ guildKey, channelId, code, options, actor }) { + requireBothPlatforms(); + await requireModule(guildKey); + const mine = checkChannel(guildKey, channelId); + if (await store.linkForChannel(guildKey, channelId)) + throw new BridgeError('That channel is already bridged.', 409); + if ((await store.linkCount(guildKey)) >= MAX_LINKS) { + throw new BridgeError(`A server can have at most ${MAX_LINKS} bridged channels.`, 409); + } + + const request = await store.peekCode(code); + if (!request) throw new BridgeError(INVALID_CODE, 404); + if (request.guildId === guildKey) throw new BridgeError('That request was made in this server.', 409); + const ownerA = ownerOf(request.guildId); + if (request.guildOwner && ownerA && request.guildOwner !== ownerA) { + throw new BridgeError('The server that made this request has a new owner. Ask for a new code.', 409); + } + if (platformOf(request.guildId) === platformOf(guildKey)) { + throw new BridgeError( + 'A bridge joins a Discord channel to a Fluxer channel, not two on the same platform.' + ); + } + // The other server must still have Sylo, with the module on and the channel as it was asked for. + if (!(await isModuleEnabled(request.guildId, 'bridge'))) { + throw new BridgeError('The server that made this request turned its Bridge module off.', 409); + } + let theirs; + try { + theirs = checkChannel(request.guildId, request.channelId); + } catch (err) { + if (err instanceof BridgeError) { + throw new BridgeError(`The other server's channel can no longer be bridged: ${err.message}`, 409); + } + throw err; + } + if (await store.linkForChannel(request.guildId, request.channelId)) { + throw new BridgeError('The other channel was bridged in the meantime.', 409); + } + checkAge(guildKey, mine.channel, request.guildId, theirs.channel); + + // A webhook in each channel, made before the code is claimed (the claim is the atomic step: if two servers + // race, one loses and its webhooks are removed again). + const made = []; + const dropMade = async () => { + for (const [key, hook] of made) await transport.deleteHook(key, hook).catch(() => {}); + }; + let hookA; + let hookB; + try { + hookA = await transport.createHook(request.guildId, request.channelId, HOOK_NAME); + made.push([request.guildId, hookA]); + hookB = await transport.createHook(guildKey, channelId, HOOK_NAME); + made.push([guildKey, hookB]); + } catch (err) { + await dropMade(); + log.warn('bridge', `could not make the webhooks for a link: ${err?.message ?? err}`); + throw new BridgeError( + 'Sylo could not make a webhook in one of the channels. Check that it has Manage Webhooks in both.', + 502 + ); + } + + const claimed = await store.claimCode(code); + if (!claimed) { + await dropMade(); + throw new BridgeError(INVALID_CODE, 404); + } + const id = await store.createLink({ + guildA: request.guildId, + channelA: request.channelId, + guildB: guildKey, + channelB: channelId, + optionsA: normaliseOptions(claimed.options), + optionsB: normaliseOptions(options), + hookA, + hookB, + requestedBy: claimed.requestedBy, + acceptedBy: actor, + ownerA, + ownerB: ownerOf(guildKey), + }); + if (!id) { + await dropMade(); + throw new BridgeError('One of the channels was bridged in the meantime.', 409); + } + dropBridgeCache(); + const link = await store.getLink(id); + await recordAudit(guildKey, { + actor, + action: 'bridge:link', + detail: `bridged #${channelName(guildKey, channelId)} with #${channelName(request.guildId, request.channelId)} in ${serverName(request.guildId)} (${platformName(request.guildId)})`, + change: null, + }); + await recordAudit(request.guildId, { + actor: claimed.requestedBy ? `${claimed.requestedBy}` : 'Dashboard', + action: 'bridge:link', + detail: `bridged #${channelName(request.guildId, request.channelId)} with #${channelName(guildKey, channelId)} in ${serverName(guildKey)} (${platformName(guildKey)}); accepted there`, + change: null, + }); + await say(guildKey, channelId, onNotice(guildKey, request.guildId, request.channelId)); + await say(request.guildId, request.channelId, onNotice(request.guildId, guildKey, channelId)); + return link; +} + +// --- link lifecycle: options, pause, resume, unlink -------------------------------------------------- + +/** A link of this server (either side), or a 404: a server never sees or changes another server's links. */ +async function ownLink(guildKey, id) { + const link = await store.getLink(id); + const side = link && sideOf(link, guildKey); + if (!link || !side) throw new BridgeError('No such bridged channel in this server.', 404); + return { link, side }; +} + +export async function setSideOptions({ guildKey, id, options, actor }) { + const { link, side } = await ownLink(guildKey, id); + const next = normaliseOptions({ ...optionsOf(link, side), ...options }); + await store.setLinkOptions(link.id, side, next); + dropBridgeCache(); + await recordAudit(guildKey, { + actor, + action: 'bridge:options', + detail: `changed what #${channelName(guildKey, channelOf(link, side))} shares (bots ${next.bots ? 'on' : 'off'}, joins ${next.joinMessages ? 'on' : 'off'}, files ${next.uploadFiles ? 'uploaded' : 'as links'})`, + change: null, + }); + return next; +} + +export async function pauseLink({ guildKey, id, actor }) { + const { link, side } = await ownLink(guildKey, id); + if (link.status !== 'active') throw new BridgeError('That bridge is not running.', 409); + await store.setLinkStatus(link.id, { status: 'paused', pausedSide: side }); + dropBridgeCache(); + queue.clear(laneKey(link, 'a')); + queue.clear(laneKey(link, 'b')); + await recordAudit(guildKey, { + actor, + action: 'bridge:pause', + detail: `paused #${channelName(guildKey, channelOf(link, side))}`, + change: null, + }); + const theirKey = guildOf(link, other(side)); + await say( + guildOf(link, side), + channelOf(link, side), + '🌉 **Bridge paused** by this server. Nothing is shared until it is resumed.' + ); + await say( + theirKey, + channelOf(link, other(side)), + `🌉 **Bridge paused** by ${serverLabel(guildKey)}. Nothing is shared until they resume it.` + ); +} + +/** + * Everything that has to hold before a stopped or paused link runs again: both platforms connected, the module on + * on both servers, both channels usable by Sylo, the age rule, and a working webhook on each side. + */ +async function revalidate(link) { + requireBothPlatforms(); + await requireModule(link.guildA); + await requireModule(link.guildB); + const a = checkChannel(link.guildA, link.channelA); + const b = checkChannel(link.guildB, link.channelB); + checkAge(link.guildA, a.channel, link.guildB, b.channel); + for (const s of ['a', 'b']) { + const key = guildOf(link, s); + const hook = hookOf(link, s); + if (!hook.id || !hook.token) { + await remakeHook(link, s); + } else { + try { + // A webhook can be deleted while the link is stopped; the first message would then fail. Make a new one now. + const probe = channelIn(key, channelOf(link, s))?.fetchWebhooks; + if (probe) { + const list = await channelIn(key, channelOf(link, s)).fetchWebhooks(); + const alive = [...(list?.values?.() ?? list ?? [])].some((h) => String(h.id) === hook.id); + if (!alive) await remakeHook(link, s); + } + } catch (err) { + log.debug('bridge', `could not check the webhook of link ${link.id}: ${err?.message ?? err}`); + } + } + } + failures.delete(link.id); +} + +/** + * Resume a link. A link a server paused can only be resumed by that server; one Sylo stopped (a failure) + * can be resumed by either, after the cause is fixed: the channels, permissions, webhooks and the age rule + * are checked again first. + */ +export async function resumeLink({ guildKey, id, actor }) { + const { link, side } = await ownLink(guildKey, id); + if (link.status === 'active') throw new BridgeError('That bridge is already running.', 409); + if ((await mirrorHold(link)) === 'held') { + throw new BridgeError( + 'The server mirror made this bridge and holds it while the mirror is not running. It runs again when the mirror does.', + 409 + ); + } + // A link nobody owns the pause of (its mirror was removed by nobody in particular) can be resumed by either server. + if (link.status === 'paused' && link.pausedSide && link.pausedSide !== side) { + throw new BridgeError('The other server paused this bridge, so only they can resume it.', 403); + } + // The one pressing Resume is the owner of this server (the route checks): this is the confirmation a new owner gives. + await store.setLinkOwner(link.id, side, ownerOf(guildKey)); + link[side === 'a' ? 'ownerA' : 'ownerB'] = ownerOf(guildKey); + await revalidate(link); + // The other server's owner may have changed too: it has to confirm as well before anything is shared. + const stillChanged = ownerChangedSide(link); + if (stillChanged) { + await store.setLinkStatus(link.id, { + status: 'paused', + pausedSide: stillChanged, + reason: 'owner-changed', + }); + dropBridgeCache(); + await recordAudit(guildKey, { + actor, + action: 'bridge:resume', + detail: `confirmed #${channelName(guildKey, channelOf(link, side))}; waiting for the other server's new owner`, + change: null, + }); + return { active: false }; + } + await store.setLinkStatus(link.id, { status: 'active' }); + dropBridgeCache(); + await recordAudit(guildKey, { + actor, + action: 'bridge:resume', + detail: `resumed #${channelName(guildKey, channelOf(link, side))}`, + change: null, + }); + await say(link.guildA, link.channelA, onNotice(link.guildA, link.guildB, link.channelB)); + await say(link.guildB, link.channelB, onNotice(link.guildB, link.guildA, link.channelA)); + return { active: true }; +} + +// --- links made by the server mirror ------------------------------------------------------------------ +// +// The server mirror (src/mirror/) copies a Discord server's channels onto a Fluxer server and then links each +// pair of text channels through the bridge, so one conversation runs through both. Both server owners agreed +// to the mirror, which is the consent a link code carries for a link made by hand, so these links are made +// without a code. Everything else of the bridge holds for them: the age rule, the visibility rules, Sylo's and +// the owners' own rights in the channels, a visible notice in both channels, bots off by default, the owners +// recorded for the ownership check, and either owner can unlink a channel at any time. A link the mirror made +// is held with the mirror: while the mirror is paused or stopped, its links are paused (reason `mirror`), and +// they run again when the mirror does. + +/** What a mirror-made link records as the people who agreed to it. */ +const mirrorBy = (name) => `${String(name || 'server owner').slice(0, 60)} (server mirror)`.slice(0, 100); + +/** Can @everyone see this channel? null when that cannot be worked out (it then counts as not matching). */ +function everyoneSees(key, channel) { + const guild = guildByKey(key); + try { + // eslint-disable-next-line no-restricted-syntax -- the @everyone role id is the platform guild id + const perms = channel?.permissionsFor?.(String(guild?.id ?? '')); + if (!perms || typeof perms.has !== 'function') return null; + return perms.has('ViewChannel') === true; + } catch { + return null; + } +} + +/** Are the two channels equally open: both visible to @everyone or both not? Unknown counts as different. */ +function samePrivacy(keyA, channelA, keyB, channelB) { + const a = everyoneSees(keyA, channelA); + const b = everyoneSees(keyB, channelB); + return a !== null && b !== null && a === b; +} + +/** A link the server mirror made records its two owners as " (server mirror)". */ +const MIRROR_TAG = '(server mirror)'; +const madeByMirror = (link) => String(link?.requestedBy ?? '').endsWith(MIRROR_TAG); + +/** + * Who holds a link: 'held' while its server mirror is not running (or the link is paused by it), 'ended' when it + * says the mirror holds it but that mirror is gone (the pair was replaced), otherwise 'none'. A held link cannot + * be resumed on the Bridge page: it comes back with the mirror. + */ +export async function mirrorHold(link) { + const pairId = await mirrorPairOfLink(link.id); + if (!pairId) return link.status === 'paused' && link.failReason === 'mirror' ? 'ended' : 'none'; + const pair = await getMirrorPair(pairId); + if (!pair) return link.status === 'paused' && link.failReason === 'mirror' ? 'ended' : 'none'; + if (!PAIR_RUNNING.includes(pair.status)) return 'held'; + return link.status === 'paused' && link.failReason === 'mirror' ? 'held' : 'none'; +} + +/** + * Turn the Bridge module on in a server the mirror links channels in. The owner agreed to the mirror, which + * links every copied channel, and the module has to be on for the links to carry anything. + * @returns {Promise} whether it was switched on now + */ +export async function enableBridgeForMirror(key, actor = 'Server mirror') { + if (await isModuleEnabled(key, 'bridge')) return false; + await setGuildModule(key, 'bridge', { enabled: true }); + await recordAudit(key, { + actor, + action: 'bridge:module', + detail: 'turned the Bridge module on because the server mirror links the channels of this server', + change: null, + }); + return true; +} + +/** + * Link a mirrored channel pair (the Discord channel is side a, the Fluxer copy side b). Throws a BridgeError + * with a `code` when the pair cannot be linked (`not-text`, `permissions`, `age-restriction`, `visibility`, + * `exists`, `limit`, `webhook`, `no-guild`, `platform`); the caller reports it and goes on with the next pair. + * `members` are the two owners' members in their own servers, to check that each can see, read and send in + * the channel; `open` says sign-in is off, so there is no one to check. + * @param {{ + * source: { guildKey: string, channelId: string }, target: { guildKey: string, channelId: string }, + * members?: { source?: any, target?: any }, open?: boolean, + * consent?: { sourceBy?: string, targetBy?: string, sourceOwner?: string, targetOwner?: string } + * }} req + */ +export async function createMirrorLink({ source, target, members = {}, open = false, consent = {} }) { + requireBothPlatforms(); + const a = checkChannel(source.guildKey, source.channelId); + const b = checkChannel(target.guildKey, target.channelId); + if ( + (await store.linkForChannel(source.guildKey, source.channelId)) || + (await store.linkForChannel(target.guildKey, target.channelId)) + ) { + throw new BridgeError('One of the channels is already bridged.', 409, 'exists'); + } + for (const key of [source.guildKey, target.guildKey]) { + if ((await store.linkCount(key)) >= MAX_MIRROR_LINKS) { + throw new BridgeError(`A server can have at most ${MAX_MIRROR_LINKS} bridged channels.`, 409, 'limit'); + } + } + checkAge(source.guildKey, a.channel, target.guildKey, b.channel); + // A channel only some members can see is linked only to a channel that is as closed: nothing private is + // shared with a channel everyone can read, in either direction. + if (!samePrivacy(source.guildKey, a.channel, target.guildKey, b.channel)) { + throw new BridgeError( + 'The two channels are not open to the same people (one is private, or it cannot be told), so they are not bridged.', + 409, + 'visibility' + ); + } + assertActorMayUse(source.guildKey, source.channelId, members.source, { open }); + assertActorMayUse(target.guildKey, target.channelId, members.target, { open }); + + const made = []; + const dropMade = async () => { + for (const [key, hook] of made) await transport.deleteHook(key, hook).catch(() => {}); + }; + let hookA; + let hookB; + try { + hookA = await transport.createHook(source.guildKey, source.channelId, HOOK_NAME); + made.push([source.guildKey, hookA]); + hookB = await transport.createHook(target.guildKey, target.channelId, HOOK_NAME); + made.push([target.guildKey, hookB]); + } catch (err) { + await dropMade(); + log.warn('bridge', `could not make the webhooks for a mirrored link: ${err?.message ?? err}`); + throw new BridgeError('Sylo could not make a webhook in one of the channels.', 502, 'webhook'); + } + const id = await store.createLink({ + guildA: source.guildKey, + channelA: source.channelId, + guildB: target.guildKey, + channelB: target.channelId, + optionsA: normaliseOptions({}), + optionsB: normaliseOptions({}), + hookA, + hookB, + requestedBy: mirrorBy(consent.sourceBy), + acceptedBy: mirrorBy(consent.targetBy), + ownerA: consent.sourceOwner || ownerOf(source.guildKey), + ownerB: consent.targetOwner || ownerOf(target.guildKey), + }); + if (!id) { + await dropMade(); + throw new BridgeError('One of the channels was bridged in the meantime.', 409, 'exists'); + } + dropBridgeCache(); + const link = await store.getLink(id); + for (const [mine, theirs, by] of [ + [source, target, consent.sourceBy], + [target, source, consent.targetBy], + ]) { + await recordAudit(mine.guildKey, { + actor: by || 'Server mirror', + action: 'bridge:link', + detail: `bridged #${channelName(mine.guildKey, mine.channelId)} with #${channelName(theirs.guildKey, theirs.channelId)} in ${serverName(theirs.guildKey)} (${platformName(theirs.guildKey)}) through the server mirror`, + change: null, + }); + } + const note = ' This link was made by the server mirror that both servers’ owners agreed to.'; + await say( + source.guildKey, + source.channelId, + onNotice(source.guildKey, target.guildKey, target.channelId) + note + ); + await say( + target.guildKey, + target.channelId, + onNotice(target.guildKey, source.guildKey, source.channelId) + note + ); + return link; +} + +/** Pause a running link because its mirror is not running. No notice per channel: nothing is shared while it is paused. */ +export async function holdMirrorLink(linkId) { + const link = await store.getLink(linkId); + if (!link || link.status !== 'active') return false; + await store.setLinkStatus(link.id, { status: 'paused', pausedSide: '', reason: 'mirror' }); + dropBridgeCache(); + queue.clear(laneKey(link, 'a')); + queue.clear(laneKey(link, 'b')); + return true; +} + +/** + * Run a link again that is paused with its mirror. The checks of a resume are made again (channels, rights, age + * rule, webhooks), and a server that changed hands since keeps the link paused until its new owner confirms. + * @returns {Promise<{ resumed: boolean, reason?: string }>} + */ +export async function resumeMirrorLink(linkId) { + const link = await store.getLink(linkId); + if (!link || link.status !== 'paused' || link.failReason !== 'mirror') return { resumed: false }; + try { + await revalidate(link); + } catch (err) { + if (err instanceof BridgeError) return { resumed: false, reason: err.message }; + throw err; + } + // The channels must still be equally open: a channel can have turned private since the link was made. + if ( + !samePrivacy( + link.guildA, + channelIn(link.guildA, link.channelA), + link.guildB, + channelIn(link.guildB, link.channelB) + ) + ) { + return { resumed: false, reason: reasonText('visibility') }; + } + const changed = ownerChangedSide(link); + if (changed) { + await store.setLinkStatus(link.id, { status: 'paused', pausedSide: changed, reason: 'owner-changed' }); + dropBridgeCache(); + return { resumed: false, reason: 'owner-changed' }; + } + await store.setLinkStatus(link.id, { status: 'active' }); + dropBridgeCache(); + return { resumed: true }; +} + +/** + * The mirror that made a link was removed by `side` ('a' the Discord server, 'b' the Fluxer server): the link stays, + * paused by that server, so only its owner can resume it and the other server's owner can unlink it. Without a side + * (a removal nobody owns) either owner can resume it. + */ +export async function releaseMirrorLink(linkId, side = '') { + const link = await store.getLink(linkId); + if (!link) return false; + const mine = link.status === 'active' || (link.status === 'paused' && link.failReason === 'mirror'); + if (!mine) return false; + await store.setLinkStatus(link.id, { + status: 'paused', + pausedSide: side === 'a' || side === 'b' ? side : '', + reason: 'mirror-ended', + }); + dropBridgeCache(); + queue.clear(laneKey(link, 'a')); + queue.clear(laneKey(link, 'b')); + return true; +} + +/** Unlink: either side may, at any time. The webhooks are deleted and the message map with the link. */ +export async function unlinkLink({ guildKey, id, actor }) { + const { link, side } = await ownLink(guildKey, id); + await dismantle(link, `unlinked by ${serverLabel(guildKey)}`); + await recordAudit(guildKey, { + actor, + action: 'bridge:unlink', + detail: `unlinked #${channelName(guildKey, channelOf(link, side))} from #${channelName(guildOf(link, other(side)), channelOf(link, other(side)))} in ${serverName(guildOf(link, other(side)))}`, + change: null, + }); + await recordAudit(guildOf(link, other(side)), { + actor: `${serverName(guildKey)}`, + action: 'bridge:unlink', + detail: `${serverName(guildKey)} unlinked #${channelName(guildOf(link, other(side)), channelOf(link, other(side)))}`, + change: null, + }); +} + +/** Remove a link everywhere: queues, webhooks, rows, and tell both channels. */ +async function dismantle(link, why) { + queue.clear(laneKey(link, 'a')); + queue.clear(laneKey(link, 'b')); + for (const s of ['a', 'b']) { + const hook = hookOf(link, s); + if (hook.id) await transport.deleteHook(guildOf(link, s), hook).catch(() => {}); + } + await store.deleteLink(link.id); + failures.delete(link.id); + dropBridgeCache(); + for (const s of ['a', 'b']) { + await say( + guildOf(link, s), + channelOf(link, s), + `🌉 **Bridge ended:** ${why}. This channel is no longer shared with the other platform.` + ); + } +} + +/** + * Sylo was removed from a server: the server on the other end loses its webhook and is told, before the + * rows go (the purge deletes them). Registered with the purge below. + */ +export async function bridgeServerLeft(key) { + const links = await store.linksOfGuild(key); + for (const link of links) { + const side = sideOf(link, key); + const theirs = other(side); + const hook = hookOf(link, theirs); + queue.clear(laneKey(link, 'a')); + queue.clear(laneKey(link, 'b')); + if (hook.id) await transport.deleteHook(guildOf(link, theirs), hook).catch(() => {}); + // The leaving server's own webhook goes too (a webhook can be deleted by its token without the bot). + const own = hookOf(link, side); + if (own.id) await transport.deleteHook(key, own).catch(() => {}); + await say( + guildOf(link, theirs), + channelOf(link, theirs), + '🌉 **Bridge ended:** the other server removed Sylo. This channel is no longer shared with the other platform.' + ); + await recordAudit(guildOf(link, theirs), { + actor: 'Sylo', + action: 'bridge:unlink', + detail: 'the server on the other end removed Sylo, so the bridge ended', + change: null, + }); + } + dropBridgeCache(); +} +onBeforePurge(bridgeServerLeft); + +// --- failures ---------------------------------------------------------------------------------------- + +/** In a row, how many deliveries of a link failed (reset by a success). In memory: it only gates a pause. */ +const failures = new Map(); + +async function remakeHook(link, side) { + const key = guildOf(link, side); + const channelId = channelOf(link, side); + const old = hookOf(link, side); + const made = await transport.createHook(key, channelId, HOOK_NAME); + await store.setLinkHook(link.id, side, made); + dropBridgeCache(); + if (old.id) await transport.deleteHook(key, old).catch(() => {}); + return { ...made, channelId }; +} +// One recreation at a time per webhook, so a burst of failing sends makes one new webhook, not many. +const remaking = new Map(); +function recreateHook(link, side) { + const id = laneKey(link, side); + if (!remaking.has(id)) { + remaking.set( + id, + remakeHook(link, side).finally(() => remaking.delete(id)) + ); + } + return remaking.get(id); +} + +/** Stop a link that cannot work, show why in Health, and say so in both channels. */ +export async function failLink(linkId, reason) { + const link = await store.getLink(linkId); + // Only a running link is stopped: a link a server paused stays paused (and only that server resumes it). + if (!link || link.status !== 'active') return; + await store.setLinkStatus(link.id, { status: 'failed', reason, failures: FAIL_THRESHOLD }); + dropBridgeCache(); + queue.clear(laneKey(link, 'a')); + queue.clear(laneKey(link, 'b')); + failures.delete(link.id); + log.warn('bridge', `link ${link.id} stopped: ${reason}`); + for (const s of ['a', 'b']) { + await recordAudit(guildOf(link, s), { + actor: 'Sylo', + action: 'bridge:stopped', + detail: + `the bridge of #${channelName(guildOf(link, s), channelOf(link, s))} stopped: ${reasonText(reason)}`.slice( + 0, + 500 + ), + change: null, + }); + await say( + guildOf(link, s), + channelOf(link, s), + `🌉 **Bridge stopped.** Messages are no longer shared. Server admins can see why under Health in the Sylo dashboard and resume it there.` + ); + } +} + +/** A delivery to `side`'s channel failed for good (after the queue's own retries). */ +async function noteFailure(link, side, err) { + if (err instanceof BridgeQueueError) { + log.debug('bridge', `link ${link.id}: a message was dropped (${err.reason})`); + return; + } + const kind = err?.failure?.kind ?? classifyError(err).kind; + if (kind === 'bad-request' || kind === 'message-gone') { + log.debug('bridge', `link ${link.id}: the platform refused one message (${kind})`); + return; + } + if (err?.code === 'NO_GUILD' || err?.code === 'NO_CHANNEL') return failLink(link.id, 'channel-missing'); + if (kind === 'forbidden') return failLink(link.id, 'permissions'); + if (kind === 'hook-gone') return failLink(link.id, 'webhook'); + const n = (failures.get(link.id) ?? 0) + 1; + failures.set(link.id, n); + log.warn('bridge', `link ${link.id}: delivery to ${side} failed (${n} in a row): ${err?.message ?? err}`); + if (n >= FAIL_THRESHOLD) return failLink(link.id, 'unreachable'); + return undefined; +} + +// --- reading a message ------------------------------------------------------------------------------- + +const toList = (collection) => + collection + ? typeof collection.values === 'function' + ? [...collection.values()] + : Array.isArray(collection) + ? collection + : [] + : []; + +const displayNameOf = (message) => + message.member?.displayName ?? + message.author?.globalName ?? + message.author?.displayName ?? + message.author?.username ?? + 'someone'; + +function avatarOf(message) { + try { + const url = + message.member?.displayAvatarURL?.({ size: 128 }) ?? + message.author?.displayAvatarURL?.({ size: 128, extension: 'png' }) ?? + message.author?.avatarURL?.(); + return typeof url === 'string' && /^https?:\/\//.test(url) ? url : undefined; + } catch { + return undefined; + } +} + +/** Names for the mentions in a message, from what the platform sent with it and the server's caches. */ +function nameResolvers(message, guild) { + const m = message.mentions; + const users = Array.isArray(m) ? m : toList(m?.users); + return { + user: (id) => + m?.members?.get?.(id)?.displayName ?? + users.find((u) => u.id === id)?.displayName ?? + users.find((u) => u.id === id)?.username ?? + guild?.members?.cache?.get?.(id)?.displayName ?? + guild?.members?.cache?.get?.(id)?.user?.username, + role: (id) => m?.roles?.get?.(id)?.name ?? guild?.roles?.cache?.get?.(id)?.name, + channel: (id) => guild?.channels?.cache?.get?.(id)?.name, + }; +} + +/** A custom emoji of the same name in the server the copy goes to. */ +function emojiFinder(guild) { + return (name) => { + const wanted = String(name).toLowerCase(); + for (const e of toList(guild?.emojis?.cache)) if (String(e.name).toLowerCase() === wanted) return e; + return undefined; + }; +} + +/** Is this a message a person (or an allowed bot) wrote, not a join notice, pin notice or other system message? */ +function isChatMessage(message) { + if (message.system) return false; + const type = message.type; + return ( + type === undefined || type === null || type === 0 || type === 19 || type === 'Default' || type === 'Reply' + ); +} + +const SUFFIX = /\s\[(?:Discord|Fluxer)\]$/; + +async function replyInfo(message, link, side) { + const refId = message.reference?.messageId ?? message.messageReference?.messageId; + if (!refId) return null; + const mapped = await store.mappingOf(link.id, side, refId); + let ref = message.referencedMessage ?? null; + if (!ref && typeof message.fetchReference === 'function') { + try { + ref = await message.fetchReference(); + } catch { + ref = null; + } + } + const theirs = other(side); + return { + name: ref + ? String(ref.member?.displayName ?? ref.author?.username ?? 'someone').replace(SUFFIX, '') + : 'someone', + raw: ref?.content ?? '', + ref, + url: mapped ? jumpUrl(guildOf(link, theirs), channelOf(link, theirs), mapped.counterpartId) : undefined, + }; +} + +/** + * Everything a copy of `message` needs: the sender's name and picture, the text with its mentions made + * harmless, the files (downloaded within limits, the rest as links), embeds, stickers and a poll as text. + * Null when there is nothing to show. + */ +async function buildCopy(message, link, side, { guildKey }) { + const srcGuild = guildByKey(guildKey); + const dstKey = guildOf(link, other(side)); + const dstGuild = guildByKey(dstKey); + const options = optionsOf(link, side); + const names = nameResolvers(message, srcGuild); + const find = emojiFinder(dstGuild); + const convert = (text) => mapEmoji(neutraliseMentions(text, names), find); + + const body = convert(String(message.content ?? '')); + const embeds = plainEmbeds(message.embeds, (t) => plainTimestamps(convert(t))); + + const reply = await replyInfo(message, link, side); + const head = reply + ? replyHead({ + name: reply.name, + snippet: convert(reply.raw).replace(/\[([^\]]*)\]\([^)]*\)/g, '$1'), + url: reply.url, + }) + : ''; + + const attachments = toList(message.attachments).map((a) => ({ + name: a.name ?? a.filename ?? 'file', + url: a.url, + size: a.size, + })); + const plan = planAttachments(attachments, { upload: options.uploadFiles }); + const files = []; + const links = [...plan.links]; + // The files are held in memory until the copy is sent (and its retries are over), so the process-wide budget + // is held that long too: `release` gives it back, once. + const held = []; + let released = false; + const release = () => { + if (released) return; + released = true; + for (const n of held) releaseDownload(n); + }; + for (const a of plan.upload) { + // No room in the process-wide budget: a link, never a wait. + if (!reserveDownload(a.size)) { + links.push(a); + continue; + } + try { + const res = await download(a.url, FILE_BYTES_LIMIT); + if (!res.ok || !res.body?.length) throw new Error(`HTTP ${res.status}`); + files.push({ name: a.name, data: res.body }); + held.push(a.size); + } catch (err) { + releaseDownload(a.size); + log.debug('bridge', `file left as a link (${err?.message ?? err})`); + links.push(a); + } + } + + const poll = message.poll + ? pollText({ + question: message.poll.question?.text ?? message.poll.question, + answers: toList(message.poll.answers).map((a) => ({ text: a.text, emoji: a.emoji?.name })), + }) + : ''; + // The sticker, poll and file names are text from the other side too: the same mention rules apply. + const tail = convert( + [...stickerLines(toList(message.stickers)), poll, ...fileLinkLines(links)].filter(Boolean).join('\n') + ); + + const content = joinParts([head, body, tail]); + if (!content && !embeds.length && !files.length) { + release(); + return null; + } + return { + release, + content, + head, + body, + tail, + embeds, + files, + username: senderName(displayNameOf(message), platformOf(guildKey), { + forDiscord: platformOf(dstKey) === 'discord', + }), + avatarUrl: avatarOf(message), + }; +} + +// --- relaying ---------------------------------------------------------------------------------------- + +/** + * A webhook send that gives way when the platform refuses it (a 400): first with a plain sender name (some + * names are refused), then without the files and embeds (the text still crosses). + */ +async function sendCopy(dstKey, hook, payload) { + try { + return await transport.send(dstKey, hook, payload); + } catch (err) { + if (classifyError(err).kind !== 'bad-request') throw err; + const plain = { + ...payload, + username: `Bridge [${platformName(dstKey) === 'Discord' ? 'Fluxer' : 'Discord'}]`, + }; + try { + return await transport.send(dstKey, hook, plain); + } catch (err2) { + if (classifyError(err2).kind !== 'bad-request' || (!payload.files && !payload.embeds)) throw err2; + log.info('bridge', 'a message crossed without its files and embeds: the platform refused them'); + return transport.send(dstKey, hook, { + content: payload.content || '(a file or embed could not be copied)', + username: plain.username, + avatarUrl: payload.avatarUrl, + }); + } + } +} + +/** + * Run a transport call with the link's webhook, making a new webhook once if the platform says it is gone + * (or if the stored token cannot be read any more). Resolves `{ value, hook }`: the webhook that was used. + */ +async function withHook(link, side, call) { + let hook = hookOf(link, side); + if (!hook.id || !hook.token) hook = await recreateHook(link, side); + try { + return { value: await call(hook), hook }; + } catch (err) { + if (classifyError(err).kind !== 'hook-gone') throw err; + hook = await recreateHook(link, side); + return { value: await call(hook), hook }; + } +} + +/** + * Remove a copy from the target channel: through the webhook that posted it, and when that cannot (the + * webhook was replaced or is gone) as the bot, which needs Manage Messages there. True when the copy is gone. + */ +async function removeCopy(link, target, copyId, copyHookId) { + const dstKey = guildOf(link, target); + const hook = hookOf(link, target); + if (hook.id && hook.token && (!copyHookId || copyHookId === hook.id)) { + try { + await transport.remove(dstKey, hook, copyId); + return true; + } catch (err) { + const kind = classifyError(err, 'delete').kind; + if (kind === 'message-gone') return true; + // A temporary error: the delete job is tried again (the queue backs off), the map entry stays. + if (kind === 'rate' || kind === 'transient') throw err; + } + } + try { + await transport.botDelete(dstKey, channelOf(link, target), copyId); + return true; + } catch (err) { + const kind = classifyError(err, 'delete').kind; + if (kind === 'message-gone') return true; + if (kind === 'rate' || kind === 'transient') throw err; + log.debug('bridge', `a copy could not be removed (link ${link.id}): ${err?.message ?? err}`); + return false; + } +} + +/** Can a message in this channel be relayed at all right now? Loop, bot and age rules. */ +async function allowed(message, link, side, key) { + if (!isChatMessage(message)) return false; + // Webhook messages never cross, whatever the bots option says: they are what bridges (this one and any + // other) post, and relaying them is how two bridges on the same channels would feed each other forever. + if (message.webhookId) return false; + if (message.author?.id && message.author.id === message.client?.user?.id) return false; + if (message.author?.bot) { + if (!optionsOf(link, side).bots) return false; + // A bot that already carries a platform tag in its name is a bridge's relay account. + if (SUFFIX.test(String(message.author.username ?? ''))) return false; + } + const changed = ownerChangedSide(link); + if (changed) { + await pauseForOwner(link, changed); + return false; + } + const theirKey = guildOf(link, other(side)); + if (!(await isModuleEnabled(theirKey, 'bridge'))) return false; + // The age rule on every message: a channel can change after the link was made. + const mine = channelIn(key, message.channelId); + const theirs = channelIn(theirKey, channelOf(link, other(side))); + if (!mine || !theirs) { + await failLink(link.id, 'channel-missing'); + return false; + } + if (ageRestricted(key, mine) !== ageRestricted(theirKey, theirs)) { + await failLink(link.id, 'age-restriction'); + return false; + } + // A link the server mirror made also keeps the two channels equally open (a channel can turn private later). + if (madeByMirror(link) && !samePrivacy(key, mine, theirKey, theirs)) { + await failLink(link.id, 'visibility'); + return false; + } + return true; +} + +/** `messageCreateAny`: relay a new message to the other end of its channel's link. */ +export async function handleMessage(message, _config, key) { + const found = await findLink(key, message.channelId); + if (!found || found.link.status !== 'active') return; + const { link, side } = found; + if (wasDeleted(key, message.id)) return; + if (!(await allowed(message, link, side, key))) return; + + const target = other(side); + const dstKey = guildOf(link, target); + let built; + const job = async () => { + const current = await store.getLink(link.id); + if (!current || current.status !== 'active') return null; + if (wasDeleted(key, message.id)) return null; + built ??= await buildCopy(message, current, side, { guildKey: key }); + if (!built || wasDeleted(key, message.id)) return null; + const payload = { + content: built.content || undefined, + username: built.username, + avatarUrl: built.avatarUrl, + embeds: built.embeds.length ? built.embeds : undefined, + files: built.files.length ? built.files : undefined, + }; + const { value: copyId, hook } = await withHook(current, target, (h) => sendCopy(dstKey, h, payload)); + if (failures.has(current.id)) failures.delete(current.id); + // Deleted while it was being sent (moderation can be quicker than the relay): take the copy down again. + try { + await store.saveMapping(current.id, side, { + sourceId: message.id, + copyId, + hookId: hook.id, + hash: hashContent(`${built.body}\n${JSON.stringify(built.embeds)}`), + extra: { head: built.head.slice(0, 900), tail: built.tail.slice(0, 900) }, + }); + } catch (err) { + log.warn('bridge', `could not remember a copy: ${err?.message ?? err}`); + } + if (wasDeleted(key, message.id)) { + // Deleted while it was being sent: take the copy down again. If that cannot be done right now, the delete + // job (which keeps trying, and keeps the map entry) does it. + const gone = await removeCopy(current, target, copyId, hook.id).catch(() => false); + if (gone) await store.deleteMapping(current.id, side, message.id).catch(() => {}); + else await deleteCopies(current, side, [message.id]); + } + return copyId; + }; + track( + queue + .enqueue(laneKey(link, target), job, { op: 'send' }) + .catch((err) => noteFailure(link, target, err)) + .finally(() => built?.release?.()) + ); +} + +/** `messageUpdate`: a message that was relayed is edited on the other side. */ +export async function handleUpdate({ new: updated }, _config, key) { + let message = updated; + if (message?.partial && typeof message.fetch === 'function') { + try { + message = await message.fetch(); + } catch { + return; + } + } + if (!message?.channelId) return; + const found = await findLink(key, message.channelId); + if (!found || found.link.status !== 'active') return; + const { link, side } = found; + if (!(await allowed(message, link, side, key))) return; + + const target = other(side); + const dstKey = guildOf(link, target); + // The map is read inside the job: the job waits behind the send of the same message in the same lane, so + // an edit that comes quickly still finds the copy. + const job = async () => { + const current = await store.getLink(link.id); + if (!current || current.status !== 'active') return null; + const map = await store.mappingOf(current.id, side, message.id); + if (!map?.isOriginal) return null; + const names = nameResolvers(message, guildByKey(key)); + const find = emojiFinder(guildByKey(dstKey)); + const convert = (text) => mapEmoji(neutraliseMentions(text, names), find); + const body = convert(String(message.content ?? '')); + const embeds = plainEmbeds(message.embeds, (t) => plainTimestamps(convert(t))); + const hash = hashContent(`${body}\n${JSON.stringify(embeds)}`); + // An edit event also fires when only a link preview appeared; nothing visible changed then. + if (hash === map.contentHash) return null; + const content = joinParts([map.extra?.head, body, map.extra?.tail]); + const hook = hookOf(current, target); + // A token that cannot be read any more (the session secret changed): make the webhook again for what comes + // next. The copy was posted by the old one and cannot be edited, which is not a failure of the link. + if (hook.id && !hook.token) { + await recreateHook(current, target); + return null; + } + // The copy can only be edited by the webhook that posted it; if that one was replaced, leave it as it was. + if (!hook.id || (map.copyHookId && map.copyHookId !== hook.id)) return null; + await transport.edit(dstKey, hook, map.counterpartId, { content: content || ZWSP, embeds }); + await store.setMappingHash(current.id, map.aMessageId, map.bMessageId, hash); + return map.counterpartId; + }; + track( + queue.enqueue(laneKey(link, target), job, { op: 'edit' }).catch((err) => noteFailure(link, target, err)) + ); +} + +/** + * Take the copies of deleted messages down. A removal by a moderator must reach the other side, so this + * never depends on the state the link has right now: it runs while the link is paused or stopped, it looks + * the map up inside the job (behind the send of the same message in the same lane), it is not dropped for a + * full or slow lane, and when the webhook cannot do it the bot does. A purge of many messages is one job. + */ +async function deleteCopies(link, side, ids) { + const target = other(side); + const sourceKey = guildOf(link, side); + for (const id of ids) markDeleted(sourceKey, id); + const job = async () => { + const current = await store.getLink(link.id); + if (!current) return null; + for (const id of ids) { + const map = await store.mappingOf(current.id, side, id); + if (!map) continue; + // Someone removed a copy: forget it, and leave the original alone. The map entry stays until the copy + // is really gone, so a delete that failed for good can still be tried again later. + const gone = map.isOriginal + ? await removeCopy(current, target, map.counterpartId, map.copyHookId) + : true; + if (gone) await store.deleteMapping(current.id, side, id); + } + return ids.length; + }; + track( + queue + .enqueue(laneKey(link, target), job, { op: 'delete' }) + .catch((err) => + log.debug('bridge', `a delete was not carried across (link ${link.id}): ${err?.message ?? err}`) + ) + ); +} + +/** `messageDelete`: the copy of a deleted message is deleted too. */ +export async function handleDelete(message, _config, key) { + if (!message?.channelId || !message.id) return; + // Not only on a running link: a removal while the link is paused or stopped still has to reach the copy. + const found = await findLink(key, message.channelId); + if (!found) return; + await deleteCopies(found.link, found.side, [String(message.id)]); +} + +/** `messageDeleteBulk`: the same for a purge of many messages. */ +export async function handleBulkDelete({ messages, channel }, _config, key) { + const ids = typeof messages?.keys === 'function' ? [...messages.keys()].map(String) : []; + // Fluxer may not have the channel cached: the deleted messages carry its id as well. + const channelId = channel?.id ?? [...(messages?.values?.() ?? [])][0]?.channelId; + if (!channelId || !ids.length) return; + const found = await findLink(key, channelId); + if (!found) return; + await deleteCopies(found.link, found.side, ids); +} + +/** `guildMemberAdd`: a join notice on the other side, for the links whose server turned it on. */ +export async function handleJoin(member, _config, key) { + const links = (await store.linksOfGuild(key)).filter((l) => l.status === 'active'); + for (const link of links) { + const side = sideOf(link, key); + const options = optionsOf(link, side); + if (!options.joinMessages) continue; + if (member.user?.bot && !options.bots) continue; + const changed = ownerChangedSide(link); + if (changed) { + await pauseForOwner(link, changed); + continue; + } + const target = other(side); + const dstKey = guildOf(link, target); + if (!(await isModuleEnabled(dstKey, 'bridge'))) continue; + const who = escapeMarkdown( + inertAt(String(member.displayName ?? member.user?.username ?? 'Someone')).slice(0, 80) + ); + const payload = { + content: `👋 **${who}** joined **${serverLabel(key)}**`, + username: senderName(guildByKey(key)?.name ?? 'Server', platformOf(key), { + forDiscord: platformOf(dstKey) === 'discord', + }), + }; + track( + queue + .enqueue( + laneKey(link, target), + async () => { + const current = await store.getLink(link.id); + if (!current || current.status !== 'active') return null; + return (await withHook(current, target, (hook) => sendCopy(dstKey, hook, payload))).value; + }, + { op: 'send' } + ) + .catch((err) => noteFailure(link, target, err)) + ); + } +} + +/** `channelDelete`: a bridged channel that is gone stops its link. */ +export async function handleChannelDelete(channel, _config, key) { + const found = await findLink(key, channel.id); + if (!found) return; + const { link } = found; + if (link.status === 'active') { + await failLink(link.id, 'channel-deleted'); + return; + } + // Paused or already stopped: it stays as it is, but Health must say the channel is gone (resume would refuse). + await store.setLinkStatus(link.id, { + status: link.status, + pausedSide: link.pausedSide, + reason: 'channel-deleted', + failures: link.failureCount, + }); + dropBridgeCache(); + await recordAudit(key, { + actor: 'Sylo', + action: 'bridge:stopped', + detail: `a bridged channel was deleted while the bridge of #${channel.name ?? 'it'} was not running`, + change: null, + }); +} + +/** + * Check the owners of every running link (a change of owner pauses it). Messages check as they come; this + * covers a quiet link, so a new owner is asked to confirm without waiting for traffic. + */ +export async function sweepOwners() { + let paused = 0; + for (const link of await store.activeLinks()) { + const changed = ownerChangedSide(link); + if (changed) { + await pauseForOwner(link, changed); + paused += 1; + } + } + return paused; +} +every('bridge', 5 * 60_000, () => sweepOwners()); + +on('bridge', 'messageCreateAny', handleMessage); +on('bridge', 'messageUpdate', handleUpdate); +on('bridge', 'messageDelete', handleDelete); +on('bridge', 'messageDeleteBulk', handleBulkDelete); +on('bridge', 'guildMemberAdd', handleJoin); +on('bridge', 'channelDelete', handleChannelDelete); + +// --- what the dashboard shows ------------------------------------------------------------------------ + +/** The end of a link on the other server, as much as both servers agreed to show each other. */ +function endView(key, channelId) { + const channel = channelIn(key, channelId); + return { + platform: platformOf(key), + server: guildByKey(key)?.name ?? 'Unknown server', + channel: channel?.name ?? 'unknown channel', + ageRestricted: ageRestricted(key, channel), + }; +} + +/** One link as the server `key` sees it (never a webhook token). */ +export function linkView(link, key, hold = 'none') { + const side = sideOf(link, key); + const theirs = other(side); + return { + id: link.id, + status: link.status, + pausedByMe: link.status === 'paused' && link.pausedSide === side, + pausedByThem: link.status === 'paused' && link.pausedSide === theirs, + // Whether this server's owner can press Resume: a stopped link, one this server paused, or one nobody holds + // (its server mirror ended). A link held by a running-state mirror comes back with the mirror. + resumable: + hold !== 'held' && + (link.status === 'failed' || + (link.status === 'paused' && (link.pausedSide === side || !link.pausedSide))), + problem: link.status === 'failed' ? { code: link.failReason, text: reasonText(link.failReason) } : null, + // Why a paused link is paused beyond "a server paused it": its owner changed, or its channel is gone. + pauseNote: + hold === 'held' && link.status !== 'active' + ? reasonText('mirror') + : link.status === 'paused' && link.failReason + ? reasonText(hold === 'ended' ? 'mirror-ended' : link.failReason) + : null, + mine: { channelId: channelOf(link, side), ...endView(key, channelOf(link, side)) }, + theirs: endView(guildOf(link, theirs), channelOf(link, theirs)), + options: optionsOf(link, side), + theirOptions: optionsOf(link, theirs), + createdAt: link.createdAt, + // Only this server's own dashboard name: the other server's staff are not shown to it. + by: side === 'a' ? link.requestedBy : link.acceptedBy, + queued: queue.stats(laneKey(link, theirs)).queued, + dropped: queue.stats(laneKey(link, theirs)).dropped, + }; +} + +/** Everything the Bridge page needs for a server. */ +export async function bridgeOverview(key) { + const guild = guildByKey(key); + const links = await store.linksOfGuild(key); + const requests = await store.pendingCodes(key); + const taken = new Set([ + ...links.map((l) => channelOf(l, sideOf(l, key))), + ...requests.map((r) => r.channelId), + ]); + return { + platform: platformOf(key), + bothPlatforms: Boolean(runtime.clients.discord && runtime.clients.fluxer), + enabled: (await getGuildModule(key, 'bridge')).enabled, + limits: { links: MAX_LINKS, requests: MAX_REQUESTS }, + links: await Promise.all(links.map(async (l) => linkView(l, key, await mirrorHold(l)))), + requests: requests.map((r) => ({ + id: r.id, + channelId: r.channelId, + channel: channelIn(key, r.channelId)?.name ?? 'unknown channel', + expiresAt: r.expiresAt, + requestedBy: r.requestedBy, + })), + channels: toList(guild?.channels?.cache) + .filter(isTextChannel) + .sort((x, y) => (x.rawPosition ?? 0) - (y.rawPosition ?? 0)) + .map((c) => ({ id: c.id, name: c.name, ageRestricted: ageRestricted(key, c), taken: taken.has(c.id) })), + }; +} + +/** + * The links of a server that need attention, for the Health page: a stopped link, and the permissions + * the bridge needs in its channels are checked by the generic channel check there. + */ +export async function bridgeProblems(key) { + return ( + (await store.linksOfGuild(key)) + // A link the server mirror holds is covered by the mirror's own Health entry. + .filter( + (l) => l.status === 'failed' || (l.status === 'paused' && l.failReason && l.failReason !== 'mirror') + ) + .map((l) => { + const side = sideOf(l, key); + return { + linkId: l.id, + channelId: channelOf(l, side), + code: l.failReason, + text: reasonText(l.failReason), + }; + }) + ); +} + +/** The bridged channels of a server, with the permissions the bridge needs in each. */ +export async function bridgedChannels(key) { + return (await store.linksOfGuild(key)).map((l) => channelOf(l, sideOf(l, key))); +} diff --git a/src/modules/index.js b/src/modules/index.js index d1e481df..c42ee5e2 100644 --- a/src/modules/index.js +++ b/src/modules/index.js @@ -36,6 +36,7 @@ import './autoThreads.js'; import './quoteBoard.js'; import './introductions.js'; import './applications.js'; +import './bridge.js'; import './economy.js'; import './messageCreator.js'; // registers the role-button / role-select component handlers import './moderation.js'; // no gateway handlers, but runs the temporary-ban expiry loop diff --git a/src/modules/lib/bridgeQueue.js b/src/modules/lib/bridgeQueue.js new file mode 100644 index 00000000..3a8c76da --- /dev/null +++ b/src/modules/lib/bridgeQueue.js @@ -0,0 +1,171 @@ +// The send queue of the bridge: one queue per target webhook, so a busy or failing channel never slows +// another server's bridge, and messages reach the other side in the order they were written. +// +// A webhook may only be called a few times a second. Each queue spaces its sends (`minGapMs`), waits out +// a rate limit the platform reports (`retry after`), retries a transient failure with a growing pause, and +// gives up on a job after `maxAttempts`. A queue holds at most `maxQueued` jobs: when a channel is flooded +// the newest message is dropped (and counted) instead of the bridge growing without bound, and a job that +// has waited longer than `maxAgeMs` is dropped as stale instead of arriving minutes late. +// +// A delete is the exception to all three limits: a moderator's removal must reach the copy however busy the +// lane is, however long it waited, and even if the link was paused or stopped meanwhile (clear() keeps it). +// +// What a failure MEANS for the link (pause it, recreate the webhook) is the engine's decision, in +// modules/bridge.js; this file only classifies errors and decides whether to try again. + +/** @typedef {'rate' | 'hook-gone' | 'message-gone' | 'forbidden' | 'bad-request' | 'transient'} FailureKind */ + +export class BridgeQueueError extends Error { + /** @param {'overflow' | 'expired' | 'cleared'} reason */ + constructor(reason) { + super(`bridge queue: ${reason}`); + this.name = 'BridgeQueueError'; + this.reason = reason; + } +} + +const statusOf = (err, depth = 0) => { + if (!err || typeof err !== 'object') return undefined; + return err.status ?? err.statusCode ?? (depth < 2 ? statusOf(err.cause, depth + 1) : undefined); +}; +const codeOf = (err, depth = 0) => { + if (!err || typeof err !== 'object') return undefined; + return err.code ?? err.rawError?.code ?? (depth < 2 ? codeOf(err.cause, depth + 1) : undefined); +}; + +/** + * How long the platform said to wait, in ms. The API's own `retry_after` field is always seconds. The SDKs' + * `retryAfter` property is not the same unit everywhere (discord.js reports milliseconds), so a small number + * there is read as seconds and a large one as milliseconds. + */ +function retryAfterOf(err, depth = 0) { + if (!err || typeof err !== 'object') return undefined; + const seconds = Number(err.retry_after ?? err.data?.retry_after ?? err.rawError?.retry_after); + if (Number.isFinite(seconds) && seconds > 0) return seconds * 1000; + const n = Number(err.retryAfter); + if (Number.isFinite(n) && n > 0) return n <= 120 ? n * 1000 : n; + return depth < 2 ? retryAfterOf(err.cause, depth + 1) : undefined; +} + +const HOOK_GONE_CODES = [10015, 'UNKNOWN_WEBHOOK', 'WEBHOOK_NOT_FOUND', 50027, 'INVALID_WEBHOOK_TOKEN']; +const MESSAGE_GONE_CODES = [10008, 'UNKNOWN_MESSAGE', 'MESSAGE_NOT_FOUND']; +const FORBIDDEN_CODES = [50001, 50013, 'MISSING_ACCESS', 'MISSING_PERMISSIONS']; + +/** + * What a failed webhook call means. `op` matters for a 404: on a send it is the webhook that is gone, on + * an edit or a delete it is the message (someone removed the copy) unless the platform names the webhook. + * @param {unknown} err + * @param {'send' | 'edit' | 'delete'} [op] + * @returns {{ kind: FailureKind, retryAfterMs?: number }} + */ +export function classifyError(err, op = 'send') { + const status = statusOf(err); + const code = codeOf(err); + if (status === 429 || code === 'RATE_LIMITED' || err?.name === 'RateLimitError') { + return { kind: 'rate', retryAfterMs: Math.min(60_000, Math.max(250, retryAfterOf(err) ?? 2000)) }; + } + if (HOOK_GONE_CODES.includes(code) || status === 401) return { kind: 'hook-gone' }; + if (MESSAGE_GONE_CODES.includes(code)) return { kind: 'message-gone' }; + if (status === 404) return { kind: op === 'send' ? 'hook-gone' : 'message-gone' }; + if (status === 403 || FORBIDDEN_CODES.includes(code)) return { kind: 'forbidden' }; + if (status === 400 || status === 413 || status === 422) return { kind: 'bad-request' }; + return { kind: 'transient' }; +} + +const defaultSleep = (ms) => + new Promise((resolve) => { + setTimeout(resolve, ms).unref?.(); + }); + +/** + * @param {{ minGapMs?: number, maxQueued?: number, maxAgeMs?: number, maxAttempts?: number, + * sleep?: (ms: number) => Promise, now?: () => number }} [opts] + */ +export function createRelayQueue({ + minGapMs = 350, + maxQueued = 100, + maxAgeMs = 5 * 60_000, + maxAttempts = 4, + sleep = defaultSleep, + now = Date.now, +} = {}) { + /** @type {Map} */ + const lanes = new Map(); + + async function drain(key, lane) { + lane.running = true; + try { + while (lane.tasks.length) { + const task = lane.tasks[0]; + if (!task.keep && now() - task.at > maxAgeMs) { + lane.tasks.shift(); + lane.dropped += 1; + task.reject(new BridgeQueueError('expired')); + continue; + } + try { + const value = await task.run(); + lane.tasks.shift(); + task.resolve(value); + } catch (err) { + const failure = classifyError(err, task.op); + if ((failure.kind === 'rate' || failure.kind === 'transient') && task.attempts < maxAttempts - 1) { + task.attempts += 1; + await sleep(failure.kind === 'rate' ? failure.retryAfterMs : 1000 * 2 ** (task.attempts - 1)); + continue; + } + lane.tasks.shift(); + task.reject(Object.assign(err instanceof Error ? err : new Error(String(err)), { failure })); + } + if (lane.tasks.length) await sleep(minGapMs); + } + } finally { + lane.running = false; + if (!lane.tasks.length) lanes.delete(key); + } + } + + return { + /** + * Queue a job for a target webhook. The returned promise settles when the job has run (or been dropped). + * @template T + * @param {string} key the target webhook (one lane per webhook) + * @param {() => Promise} run + * @param {{ op?: 'send' | 'edit' | 'delete' }} [opts] + * @returns {Promise} + */ + enqueue(key, run, { op = 'send' } = {}) { + let lane = lanes.get(key); + if (!lane) { + lane = { tasks: [], running: false, dropped: 0 }; + lanes.set(key, lane); + } + if (op !== 'delete' && lane.tasks.length >= maxQueued) { + lane.dropped += 1; + return Promise.reject(new BridgeQueueError('overflow')); + } + const result = new Promise((resolve, reject) => { + lane.tasks.push({ run, op, keep: op === 'delete', at: now(), attempts: 0, resolve, reject }); + }); + if (!lane.running) void drain(key, lane); + return result; + }, + /** How many jobs wait for a webhook, and how many were dropped since the lane was last empty. */ + stats(key) { + const lane = lanes.get(key); + return { queued: lane?.tasks.length ?? 0, dropped: lane?.dropped ?? 0 }; + }, + /** Drop everything waiting for a webhook (the link was paused, unlinked or failed). */ + clear(key) { + const lane = lanes.get(key); + if (!lane) return 0; + // The job at the head may be running, and a delete is never dropped: leave those, drop the rest. + const dropped = lane.tasks.filter((t, i) => !t.keep && !(lane.running && i === 0)); + lane.tasks = lane.tasks.filter((t) => !dropped.includes(t)); + for (const t of dropped) t.reject(new BridgeQueueError('cleared')); + return dropped.length; + }, + /** How many webhooks have a queue right now (for tests and the health page). */ + lanes: () => lanes.size, + }; +} diff --git a/src/modules/lib/bridgeText.js b/src/modules/lib/bridgeText.js new file mode 100644 index 00000000..14e292a1 --- /dev/null +++ b/src/modules/lib/bridgeText.js @@ -0,0 +1,291 @@ +// The pure part of the bridge: turning one platform's message into text and attachments the other +// platform can show, with every ping made harmless on the way. Nothing here touches a platform, the +// database or the network, so the safety rules (no ping crosses, nothing breaks the other side's limits) +// are tested directly in test/bridgeText.test.js. +// +// Mentions: the relay posts with allowedMentions set to none, so nothing is pinged whatever the text says. +// This module also changes the text, so a person reads "@Alice" and "@moderators" on the other side +// instead of a raw id that means nothing there, and "@everyone" can never be turned into a ping by a +// later change to how the copy is sent. +import { createHash } from 'node:crypto'; + +/** What is left of the message limit of the platforms: Discord and Fluxer both allow 2000 characters. */ +export const MESSAGE_LIMIT = 2000; +/** The most embeds one message carries, and the most files. */ +export const EMBED_LIMIT = 10; +export const FILE_COUNT_LIMIT = 10; +/** A file is re-uploaded only up to this size, and only this much per message; bigger ones become a link. */ +export const FILE_BYTES_LIMIT = 8 * 1024 * 1024; +export const TOTAL_BYTES_LIMIT = 8 * 1024 * 1024; + +/** U+200B, the zero-width space: invisible, and enough to break a mention. */ +export const ZWSP = String.fromCharCode(0x200b); + +/** What each side of a link shares about itself. `bots` and `joinMessages` are off until a server asks. */ +export const DEFAULT_OPTIONS = Object.freeze({ bots: false, joinMessages: false, uploadFiles: true }); + +/** The options of one side as stored or submitted, always the full shape with booleans. */ +export function normaliseOptions(raw) { + const o = raw && typeof raw === 'object' ? raw : {}; + return { + bots: o.bots === true, + joinMessages: o.joinMessages === true, + uploadFiles: o.uploadFiles === undefined ? DEFAULT_OPTIONS.uploadFiles : o.uploadFiles === true, + }; +} + +/** A short stable hash of what was sent, so an edit event that changed nothing visible is skipped. */ +export const hashContent = (text) => + createHash('sha256') + .update(String(text ?? '')) + .digest('hex') + .slice(0, 24); + +// --- text helpers ------------------------------------------------------------------------------------ + +/** Cut to `max` UTF-16 units without a half emoji at the end, ending in "…". */ +export function clip(value, max) { + const text = String(value ?? ''); + if (text.length <= max) return text; + let out = text.slice(0, Math.max(0, max - 1)); + if (/[\uD800-\uDBFF]$/.test(out)) out = out.slice(0, -1); + return `${out}…`; +} + +/** + * Control characters and invisible format characters (direction overrides and the like) out of a display + * name or a quote. The zero-width joiner stays, because emoji sequences need it, and so does the + * zero-width space: it is what keeps a mention broken once neutraliseMentions has put it there. + */ +const ZWJ = String.fromCharCode(0x200d); +const stripControl = (text) => + String(text ?? '') + .replace(/[\p{Cc}\p{Cf}]/gu, (c) => (c === ZWJ || c === ZWSP ? c : '')) + .trim(); + +/** A name that shows an @ but cannot ping: the zero-width space after the @ breaks the mention. */ +export const inertAt = (text) => String(text ?? '').replace(/@/g, `@${ZWSP}`); + +/** + * Markdown out of a name that Sylo puts into its own text (a server, a channel, a member in a notice or a + * quote): a name like "x](https://evil.example)[y" must not turn into a link under Sylo's identity. + */ +export const escapeMarkdown = (text) => String(text ?? '').replace(/([\\*_~`|[\]()>#<])/g, '\\$1'); + +/** + * Replace the mentions in a message with plain text and break @everyone / @here. `names` looks up what + * the other side should read for a user, a role or a channel; it may return nothing (then a neutral word). + * @param {string} text + * @param {{ user?: (id: string) => string | undefined, role?: (id: string) => string | undefined, + * channel?: (id: string) => string | undefined }} [names] + */ +export function neutraliseMentions(text, names = {}) { + return String(text ?? '') + .replace(/<@!?(\d{5,25})>/g, (_, id) => inertAt(`@${stripControl(names.user?.(id)) || 'user'}`)) + .replace(/<@&(\d{5,25})>/g, (_, id) => inertAt(`@${stripControl(names.role?.(id)) || 'role'}`)) + .replace(/<#(\d{5,25})>/g, (_, id) => `#${stripControl(names.channel?.(id)) || 'channel'}`) + .replace(/<\/([^:>\s]{1,32}(?: [^:>\s]{1,32}){0,2}):\d{5,25}>/g, (_, name) => `/${name}`) + .replace(/@(everyone|here)\b/gi, (_, word) => `@${ZWSP}${word}`); +} + +/** + * Custom emoji: the same name in the other server when it has one, otherwise ":name:". Unicode emoji + * are plain text and cross as they are. + * @param {string} text + * @param {(name: string) => { id: string, name: string, animated?: boolean } | undefined} find + */ +export function mapEmoji(text, find) { + return String(text ?? '').replace(/<(a?):([\w]{2,32}):(\d{5,25})>/g, (_, anim, name) => { + const match = find?.(name); + if (match?.id) return `<${match.animated ? 'a' : ''}:${match.name ?? name}:${match.id}>`; + return `:${name}:`; + }); +} + +/** `` is a Discord timestamp: the other platform may not render it, so say it in UTC. */ +const MONTHS = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']; +const pad = (n) => String(n).padStart(2, '0'); +export function plainTimestamps(text) { + return String(text ?? '').replace(//g, (_, seconds) => { + const d = new Date(Number(seconds) * 1000); + if (Number.isNaN(d.getTime())) return ''; + return `${d.getUTCDate()} ${MONTHS[d.getUTCMonth()]} ${d.getUTCFullYear()}, ${pad(d.getUTCHours())}:${pad(d.getUTCMinutes())} UTC`; + }); +} + +// --- the name and the picture of the sender ---------------------------------------------------------- + +export const PLATFORM_TAG = Object.freeze({ discord: 'Discord', fluxer: 'Fluxer' }); + +// Discord refuses a webhook name that contains these words, so a name from the other platform that has +// one gets a zero-width space inside it. +const FORBIDDEN_NAME = /(clyde|discord)/gi; + +/** + * The name a relayed message is posted under: the author's display name and where they are, so the + * conversation says who is on the other platform ("Alice [Fluxer]"). At most 80 characters, never empty. + */ +export function senderName(displayName, platform, { forDiscord = false } = {}) { + const tag = ` [${PLATFORM_TAG[platform] ?? platform}]`; + let name = inertAt(stripControl(displayName)).replace(/\s+/g, ' '); + if (forDiscord) name = name.replace(FORBIDDEN_NAME, (w) => `${w[0]}${ZWSP}${w.slice(1)}`); + name = clip(name, 80 - tag.length); + if (!name || name === ZWSP) name = 'Someone'; + return `${name}${tag}`; +} + +// --- replies ----------------------------------------------------------------------------------------- + +/** The first line of a copy that answers another message: who, a short quote, and a link when it is mapped. */ +export function replyHead({ name, snippet, url }) { + const who = escapeMarkdown(inertAt(clip(stripControl(name) || 'someone', 40))); + const text = clip(stripControl(String(snippet ?? '').replace(/\s+/g, ' ')).replace(/^>+\s*/, ''), 100); + const label = url ? `[**${who}**](<${url}>)` : `**${who}**`; + return text ? `> ↪ ${label}: ${text}` : `> ↪ ${label}`; +} + +// --- stickers, polls, files -------------------------------------------------------------------------- + +/** One line per sticker: the name and its picture link, which both platforms show as an image. */ +export function stickerLines(stickers) { + return [...(stickers ?? [])] + .filter((s) => s && (s.url || s.name)) + .slice(0, 3) + .map((s) => + s.url + ? `*sticker: ${clip(stripControl(s.name) || 'sticker', 40)}* ${s.url}` + : `*sticker: ${clip(s.name, 40)}*` + ); +} + +/** + * A poll as text: Fluxer has no polls, and a bridge cannot make a vote on one side count on the other, + * so the question and the answers are shown and the vote stays where the poll is. + * @param {{ question?: string, answers?: Array, multiselect?: boolean }} poll + */ +export function pollText(poll) { + if (!poll) return ''; + const answers = [...(poll.answers ?? [])] + .map((a) => (typeof a === 'string' ? { text: a } : a)) + .filter((a) => a?.text) + .slice(0, 10); + const lines = [`📊 **Poll:** ${clip(stripControl(poll.question) || 'Poll', 300)}`]; + for (const a of answers) lines.push(`• ${a.emoji ? `${a.emoji} ` : ''}${clip(stripControl(a.text), 100)}`); + lines.push('*Vote on the original server: polls do not cross the bridge.*'); + return lines.join('\n'); +} + +/** + * Decide which attachments are re-uploaded and which become a link. A file is uploaded when uploading is + * on, it is not larger than the per-file limit, and the message stays within the count and total limits; + * everything else is a link line. + * @param {Array<{ name: string, url: string, size?: number }>} attachments + * @param {{ upload?: boolean, perFile?: number, total?: number, count?: number }} [limits] + */ +export function planAttachments( + attachments, + { upload = true, perFile = FILE_BYTES_LIMIT, total = TOTAL_BYTES_LIMIT, count = FILE_COUNT_LIMIT } = {} +) { + const toUpload = []; + const links = []; + let used = 0; + for (const a of attachments ?? []) { + if (!a?.url) continue; + const size = Number(a.size); + const fits = + upload && + Number.isFinite(size) && + size > 0 && + size <= perFile && + used + size <= total && + toUpload.length < count; + if (fits) { + toUpload.push(a); + used += size; + } else { + links.push(a); + } + } + return { upload: toUpload, links }; +} + +/** The link lines for attachments that were not uploaded. */ +export const fileLinkLines = (links) => + links.slice(0, 10).map((a) => `📎 ${clip(stripControl(a.name) || 'file', 60)}: ${a.url}`); + +// --- embeds ------------------------------------------------------------------------------------------ + +/** + * A rich embed from either SDK as the plain API shape both accept, with its text neutralised. Embeds that + * are only a link preview are left out: the other platform makes its own preview of the link. + * @param {any} embed a discord.js Embed or a Fluxer MessageEmbed + * @param {(text: string) => string} text what to do to every piece of text (mentions, emoji) + */ +export function plainEmbed(embed, text = (t) => t) { + const e = embed?.toJSON?.() ?? embed; + if (!e) return null; + const type = e.type ?? 'rich'; + if (type !== 'rich') return null; + const pick = (v) => (typeof v === 'string' && v ? v : undefined); + const clean = (o) => (o ? Object.fromEntries(Object.entries(o).filter(([, v]) => v !== undefined)) : o); + const url = (v) => (typeof v === 'string' && /^https?:\/\//i.test(v) ? v : undefined); + const out = { + title: pick(text(clip(e.title ?? '', 256))), + description: pick(text(clip(e.description ?? '', 4000))), + url: url(e.url), + color: Number.isFinite(e.color) ? e.color : undefined, + timestamp: pick( + typeof e.timestamp === 'string' + ? e.timestamp + : e.timestamp instanceof Date + ? e.timestamp.toISOString() + : undefined + ), + footer: e.footer?.text + ? clean({ + text: text(clip(e.footer.text, 2048)), + icon_url: url(e.footer.icon_url ?? e.footer.iconURL ?? e.footer.iconUrl), + }) + : undefined, + image: url(e.image?.url) ? { url: e.image.url } : undefined, + thumbnail: url(e.thumbnail?.url) ? { url: e.thumbnail.url } : undefined, + author: e.author?.name + ? clean({ + name: text(clip(e.author.name, 256)), + url: url(e.author.url), + icon_url: url(e.author.icon_url ?? e.author.iconURL ?? e.author.iconUrl), + }) + : undefined, + fields: Array.isArray(e.fields) + ? e.fields + .filter((f) => f?.name && f?.value) + .slice(0, 25) + .map((f) => ({ + name: text(clip(f.name, 256)), + value: text(clip(f.value, 1024)), + inline: f.inline === true, + })) + : undefined, + }; + for (const key of Object.keys(out)) if (out[key] === undefined) delete out[key]; + return Object.keys(out).length ? out : null; +} + +/** The embeds of a message as plain API objects, at most ten, link previews left out. */ +export function plainEmbeds(embeds, text) { + return [...(embeds ?? [])] + .map((e) => plainEmbed(e, text)) + .filter(Boolean) + .slice(0, EMBED_LIMIT); +} + +/** Joins the parts of a copy with a blank-free line break and keeps the whole within the message limit. */ +export function joinParts(parts, limit = MESSAGE_LIMIT) { + return clip( + parts + .map((p) => String(p ?? '').trim()) + .filter(Boolean) + .join('\n'), + limit + ); +} diff --git a/src/modules/registry.js b/src/modules/registry.js index 0212448f..670e18cc 100644 --- a/src/modules/registry.js +++ b/src/modules/registry.js @@ -393,6 +393,17 @@ export const MODULES = [ configurable: true, beta: true, }, + { + id: 'bridge', + name: 'Bridge', + description: + 'Share one conversation between a Discord channel and a Fluxer channel. Both servers agree to a link with a one-time code, and either can unlink at any time.', + icon: '🌉', + requiredIntents: ['MessageContent'], + defaultEnabled: false, + configurable: true, + beta: true, + }, { id: 'economy', name: 'Economy', diff --git a/src/platform/modulePlatformNotes.js b/src/platform/modulePlatformNotes.js index 40ffa210..189d487a 100644 --- a/src/platform/modulePlatformNotes.js +++ b/src/platform/modulePlatformNotes.js @@ -31,6 +31,8 @@ const FLUXER_NOTES = { `Fluxer has no slash commands, no private replies, no modals, no buttons and no threads yet. Members start with a chat command: ${prefix}apply start (and ${prefix}apply withdraw). Sylo answers "Check your DMs" in the channel and removes that message after a few seconds, then asks the questions in DM. A choice or role question is asked as numbered options, and the member answers with the numbers (at most 10 options a question). The panel message, the Answer in DMs and Open on the web choice, the Submit and Start over choices under the preview, and the Accept and Deny buttons on each application are shown as reactions. Staff who want to give a reason use ${prefix}application accept or ${prefix}application deny , or the dashboard. No staff thread is opened. The web form and the dashboard work as on Discord.`, economy: ({ prefix }) => `Fluxer has no slash commands or buttons yet: members use chat commands, for example ${prefix}balance, ${prefix}daily, ${prefix}work, ${prefix}pay @member 100, ${prefix}deposit all and ${prefix}economy-leaderboard (turn the pages with the ⬅️ and ➡️ reactions). Staff use ${prefix}eco add @member 500 reason:"Event prize". A rob result is posted in the channel as on Discord, and the victim gets the same DM. The shop is ${prefix}shop (a numbered list that pages with the ⬅️ and ➡️ reactions), ${prefix}buy , ${prefix}inventory and ${prefix}collection (collectibles with their pictures as embeds; its pages turn with the same reactions; Fluxer has no private replies, so with "Members can show their collection" off Sylo sends your own collection to your DMs, and tells you when your DMs are closed). A giveaway with an entry fee or a currency prize works as on Discord: members press the reaction to enter, and a second press leaves and gets the fee back. The dashboard and the web shop work as on Discord.`, + bridge: ({ prefix }) => + `A bridge joins this Fluxer channel to a Discord channel: set it up in the dashboard (no ${prefix}command is needed). Messages cross through webhooks with the author's name and picture; a poll on Discord shows on Fluxer as text, and a sticker as a picture link. Nothing pings across the bridge: mentions are shown as names.`, 'auto-threads': () => 'Fluxer has no threads yet, so this module does nothing on a Fluxer server: posts stay as they are. The settings can be saved here and start working on a Discord server. Fluxer lists threads for 2026; Sylo will support them then.', welcome: () => diff --git a/src/platform/webhooks.js b/src/platform/webhooks.js new file mode 100644 index 00000000..21185619 --- /dev/null +++ b/src/platform/webhooks.js @@ -0,0 +1,156 @@ +// Webhooks on both platforms behind one small interface, for the bridge (modules/bridge.js). +// +// A server key ("123..." or "fluxer:123...") says which platform a call is for, so the bridge never +// branches on the platform itself: it asks for a webhook in a channel, then sends, edits and deletes +// through it. The two SDKs differ in small ways (discord.js has a WebhookClient and names the avatar +// `avatarURL`; Fluxer builds a Webhook from an id and a token and names it `avatarUrl`; a file is +// `{ attachment, name }` on one and `{ name, data }` on the other), and those differences live here. +// +// Every message is sent with allowedMentions set to none: whatever the text says, a webhook copy never +// pings anyone. Callers pass a platform-neutral payload: +// { content?, username?, avatarUrl?, embeds?, files?: [{ name, data: Buffer }] } +import { WebhookClient } from 'discord.js'; +import { splitPlatformGuildId } from '../lib/platformId.js'; +import { guildByKey } from './platformOf.js'; + +/** No mention of anything: the copy shows names as text and pings nobody. */ +export const NO_MENTIONS = Object.freeze({ parse: [] }); + +/** @typedef {{ id: string, token: string, channelId?: string }} Hook (the channel is only for the Fluxer SDK's own bookkeeping) */ + +/** + * @typedef {object} BridgeTransport + * @property {(guildKey: string, channelId: string, name: string) => Promise} createHook + * @property {(guildKey: string, hook: Hook, payload: object) => Promise} send resolves the new message id + * @property {(guildKey: string, hook: Hook, messageId: string, payload: object) => Promise} edit + * @property {(guildKey: string, hook: Hook, messageId: string) => Promise} remove + * @property {(guildKey: string, hook: Hook) => Promise} deleteHook best effort + * @property {(guildKey: string, channelId: string, messageId: string) => Promise} botDelete delete a message as the + * bot (needs Manage Messages): the way to remove a copy whose webhook is gone + */ + +const platformOfKey = (key) => splitPlatformGuildId(key)?.platform ?? 'discord'; + +// A webhook client per webhook, kept so each send does not build one. Bounded: a long-running instance +// with many links still holds only a few hundred. +const MAX_CACHED = 500; +const clients = new Map(); +const remember = (cacheKey, make) => { + let client = clients.get(cacheKey); + if (!client) { + if (clients.size >= MAX_CACHED) clients.delete(clients.keys().next().value); + client = make(); + clients.set(cacheKey, client); + } + return client; +}; + +/** Forget the cached client of a webhook (it was deleted or replaced). */ +export function forgetHook(hookId) { + for (const key of [...clients.keys()]) if (key.endsWith(`|${hookId}`)) clients.delete(key); +} + +async function discordHook(hook) { + return remember(`discord|${hook.id}`, () => new WebhookClient({ id: hook.id, token: hook.token })); +} + +async function fluxerHook(guild, hook) { + const { Webhook } = await import('@fluxerjs/core'); + return remember(`fluxer|${hook.id}`, () => + Webhook.fromToken(guild.client, hook.id, hook.token, { channelId: hook.channelId, guildId: guild.id }) + ); +} + +const guildOf = (key) => { + const guild = guildByKey(key); + if (!guild) throw Object.assign(new Error('Sylo is not in that server'), { code: 'NO_GUILD' }); + return guild; +}; + +/** The payload in the shape one platform's send() takes. Always with allowedMentions none. */ +export function webhookBody(platform, payload, { forEdit = false } = {}) { + const out = { allowedMentions: NO_MENTIONS }; + if (payload.content !== undefined) out.content = payload.content; + if (payload.embeds) out.embeds = payload.embeds; + if (!forEdit) { + if (payload.username) out.username = payload.username; + if (payload.avatarUrl) out[platform === 'fluxer' ? 'avatarUrl' : 'avatarURL'] = payload.avatarUrl; + if (payload.files?.length) { + out.files = payload.files.map((f) => + platform === 'fluxer' ? { name: f.name, data: f.data } : { attachment: f.data, name: f.name } + ); + } + } + return out; +} + +/** @returns {BridgeTransport} */ +export function createWebhookTransport() { + return { + async createHook(guildKey, channelId, name) { + const guild = guildOf(guildKey); + const channel = guild.channels?.cache?.get(channelId); + if (!channel?.createWebhook) + throw Object.assign(new Error('That channel cannot have a webhook'), { code: 'NO_CHANNEL' }); + const created = await channel.createWebhook({ name, reason: 'Sylo bridge' }); + if (!created?.id || !created?.token) throw new Error('The platform did not return a webhook token'); + return { id: String(created.id), token: String(created.token) }; + }, + + async send(guildKey, hook, payload) { + const platform = platformOfKey(guildKey); + if (platform === 'fluxer') { + const client = await fluxerHook(guildOf(guildKey), hook); + const sent = await client.send(webhookBody(platform, payload), true); + if (!sent?.id) throw new Error('Fluxer did not return the sent message'); + return String(sent.id); + } + const client = await discordHook(hook); + const sent = await client.send(webhookBody(platform, payload)); + return String(sent.id); + }, + + async edit(guildKey, hook, messageId, payload) { + const platform = platformOfKey(guildKey); + const body = webhookBody(platform, payload, { forEdit: true }); + if (platform === 'fluxer') { + const client = await fluxerHook(guildOf(guildKey), hook); + await client.editMessage(messageId, body); + return; + } + await (await discordHook(hook)).editMessage(messageId, body); + }, + + async remove(guildKey, hook, messageId) { + const platform = platformOfKey(guildKey); + if (platform === 'fluxer') { + const client = await fluxerHook(guildOf(guildKey), hook); + await client.deleteMessage(messageId); + return; + } + await (await discordHook(hook)).deleteMessage(messageId); + }, + + async botDelete(guildKey, channelId, messageId) { + const channel = guildOf(guildKey).channels?.cache?.get(channelId); + if (!channel?.messages?.fetch) + throw Object.assign(new Error('That channel cannot be read'), { code: 'NO_CHANNEL' }); + const message = await channel.messages.fetch(messageId); + await message.delete(); + }, + + async deleteHook(guildKey, hook) { + const platform = platformOfKey(guildKey); + try { + if (platform === 'fluxer') { + const client = await fluxerHook(guildOf(guildKey), hook); + await client.delete(); + } else { + await (await discordHook(hook)).delete('Sylo bridge unlinked'); + } + } finally { + forgetHook(hook.id); + } + }, + }; +} diff --git a/src/web/lib/cardSummary.js b/src/web/lib/cardSummary.js index 1da061ed..5cc0a91f 100644 --- a/src/web/lib/cardSummary.js +++ b/src/web/lib/cardSummary.js @@ -86,6 +86,10 @@ const FORMAT = { noun(num(l, 'Saved quotes'), 'saved quote'), ...(isOn(l, 'Public page') ? ['public page'] : []), ], + bridge: (l) => [ + ...count(l, 'Bridged channels', 'bridged channel'), + ...(num(l, 'Stopped') ? [`${num(l, 'Stopped')} stopped`] : []), + ], introductions: (l) => [ ...(isOn(l, 'Channel') ? [value(l, 'Channel')] : ['No channel yet']), ...count(l, 'Introductions', 'introduction'), @@ -187,6 +191,7 @@ const NEEDS_OWNER_KEYS = new Set(['twitch-alerts', 'kick-alerts', 'apple-connect /** What an empty module says it is missing, instead of the general "Not set up yet". */ const EMPTY = { sticky: 'No sticky messages yet', + bridge: 'No bridged channels yet', 'auto-threads': 'No channels yet', 'auto-react': 'No rules yet', 'custom-commands': 'No commands yet', diff --git a/src/web/lib/configImport.js b/src/web/lib/configImport.js index 7411868e..e13056ab 100644 --- a/src/web/lib/configImport.js +++ b/src/web/lib/configImport.js @@ -446,6 +446,8 @@ const SKIPPED = { 'Custom commands are registered with the platform one by one, so they are in the file but not imported; build them in the module.', reminders: 'Reminders are in the file but not imported yet; create them in the module.', 'youtube-alerts': 'YouTube channels are looked up when you add them; add them in the module.', + bridge: + 'A bridge joins two servers and needs both to agree with a code, so it cannot be imported; set it up in the module.', insights: 'Server insights has nothing to configure.', 'game-stats': 'Game stats has nothing to configure.', 'channel-cleanup': diff --git a/src/web/lib/guildOwner.js b/src/web/lib/guildOwner.js new file mode 100644 index 00000000..36042325 --- /dev/null +++ b/src/web/lib/guildOwner.js @@ -0,0 +1,43 @@ +// "Only the server owner": the check some actions of the dashboard need, beyond the dashboard access levels +// (src/web/middleware/access.js). Edit access, Administrator and the bot-master roles do not make anyone the +// owner. The owner is read from the platform's own guild object (`ownerId`, kept current by the gateway, see +// bot/lib/adminSnapshot.js), and the answer is worked out on every call, never remembered. +// +// In open mode (no logins: the dashboard is a LAN tool and everyone is "local admin") there is no one to +// compare, so the check passes, as every other access check does there. +import { config } from '../../config.js'; +import { guildByKey } from '../../platform/platformOf.js'; +import { userIdForGuild } from '../middleware/auth.js'; + +/** The refusal: an HTTP 403 with a message the dashboard shows as it is. */ +export class GuildOwnerError extends Error { + constructor(message = 'Only the server owner can do this.') { + super(message); + this.name = 'GuildOwnerError'; + this.status = 403; + } +} + +/** + * Is the signed-in person the owner of this server? Their id on the server's own platform is compared with + * the owner the platform reports; no identity there, an unknown server or an owner that is not known all mean no. + * @param {import('express').Request} req + * @param {string} guildKey the server key + * @returns {boolean} + */ +export function isGuildOwner(req, guildKey) { + if (!config.authEnabled) return true; + const userId = userIdForGuild(req, guildKey); + const ownerId = guildByKey(guildKey)?.ownerId; + return Boolean(userId && ownerId && String(userId) === String(ownerId)); +} + +/** + * Refuse (GuildOwnerError, 403) unless the signed-in person owns this server. + * @param {import('express').Request} req + * @param {string} guildKey the server key + * @returns {void} + */ +export function requireGuildOwner(req, guildKey) { + if (!isGuildOwner(req, guildKey)) throw new GuildOwnerError(); +} diff --git a/src/web/lib/moduleIcons.js b/src/web/lib/moduleIcons.js index 09af8332..75214574 100644 --- a/src/web/lib/moduleIcons.js +++ b/src/web/lib/moduleIcons.js @@ -34,6 +34,7 @@ export const MODULE_ICONS = { 'temp-roles': 'hourglass', 'auto-threads': 'threads', 'quote-board': 'quote', + bridge: 'threads', introductions: 'id-card', applications: 'clipboard', economy: 'coins', diff --git a/src/web/lib/overviewSummary.js b/src/web/lib/overviewSummary.js index 0a7a610d..f0511596 100644 --- a/src/web/lib/overviewSummary.js +++ b/src/web/lib/overviewSummary.js @@ -21,6 +21,7 @@ import { openSuggestionCount, suggestionCount } from '../../db/suggestions.js'; import { nextTempRoleExpiry, tempRoleCount } from '../../db/tempRoles.js'; import { quoteCount } from '../../db/quotes.js'; import { introductionCount } from '../../db/introductions.js'; +import { linksOfGuild } from '../../db/bridge.js'; import { formCount, pendingCount } from '../../db/applications.js'; import { recentLookups } from '../../db/cache.js'; import { dailySeries } from '../../db/insights.js'; @@ -84,6 +85,7 @@ const LAYOUT = [ 'autoresponder', 'auto-react', 'auto-threads', + 'bridge', 'afk', 'server-stats', 'insights', @@ -158,6 +160,7 @@ export const V2_CARD_PAGES = { commands: 'moderation/commands', messages: 'messages', insights: 'insights', + mirror: 'mirror', }; export function v2CardHref(id, guildId) { @@ -326,6 +329,14 @@ async function moduleLines(id, guild, cfg) { const n = Array.isArray(cfg.stickies) ? cfg.stickies.length : 0; return [n ? on('Active sticky messages', String(n)) : off('Active sticky messages', 'none')]; } + case 'bridge': { + const links = await linksOfGuild(guild.id); + const stopped = links.filter((l) => l.status === 'failed').length; + return [ + links.length ? on('Bridged channels', String(links.length)) : off('Bridged channels', 'none'), + stopped ? off('Stopped', String(stopped)) : neutral('Stopped', '0'), + ]; + } case 'quote-board': { const saved = await quoteCount(guild.id); return [ diff --git a/src/web/lib/permissionHealth.js b/src/web/lib/permissionHealth.js index 685ed148..d6ee9519 100644 --- a/src/web/lib/permissionHealth.js +++ b/src/web/lib/permissionHealth.js @@ -19,11 +19,13 @@ import { listCleanupSchedules } from '../../db/channelCleanup.js'; import { listGithubWatches } from '../../db/githubWatches.js'; import { listItems } from '../../db/economyShop.js'; import { listForms } from '../../db/applications.js'; +import { bridgeProblems, bridgedChannels } from '../../modules/bridge.js'; import { guildSupports, platformOfGuild, rawGuildId } from '../../platform/platformOf.js'; import { healthFor } from '../../lib/healthCache.js'; import { sourceLabel } from '../../modules/rss.js'; import { v2CardHref } from './overviewSummary.js'; import { log } from '../../lib/log.js'; +import { mirrorHealthChecks } from '../../mirror/health.js'; const ERROR = 'error'; const WARNING = 'warning'; @@ -44,6 +46,7 @@ const LABELS = { ManageRoles: 'Manage Roles', ManageChannels: 'Manage Channels', ManageNicknames: 'Manage Nicknames', + ManageWebhooks: 'Manage Webhooks', ManageGuild: 'Manage Server', ModerateMembers: 'Timeout Members', KickMembers: 'Kick Members', @@ -240,6 +243,16 @@ export const NEEDS = { arr(cfg.boards).map((b) => post(b?.channelId, `The channel of ${b?.name || 'a starboard'}`, { field: 'channelId' }) ), + // The bridge makes and uses a webhook in each bridged channel, and says things in it. + // Manage Messages is optional: it lets Sylo remove a copy as the bot when the webhook that posted it is gone. + bridge: (_cfg, x) => + arr(x.channels).flatMap((id) => [ + post(id, 'A bridged channel', { embed: false, extra: ['ManageWebhooks'] }), + inChannel(id, 'A bridged channel (to remove copies when their webhook is gone)', ['ManageMessages'], { + severity: WARNING, + part: 'delete', + }), + ]), introductions: (cfg) => [ post(cfg.channelId, 'The introductions channel', { extra: ['ReadMessageHistory'], field: 'channelId' }), post(cfg.promptChannelId, 'The prompt channel', { optional: true, field: 'promptChannelId' }), @@ -423,6 +436,10 @@ const EXTRAS = { 'channel-cleanup': async (guildId) => ({ cleanup: await listCleanupSchedules(guildId) }), reminders: async (guildId) => ({ reminders: await listScheduled(guildId) }), github: async (guildId) => ({ githubWatches: await listGithubWatches(guildId) }), + bridge: async (guildId) => ({ + channels: await bridgedChannels(guildId), + problems: await bridgeProblems(guildId), + }), }; // --- comparing a need with what Sylo can do --------------------------------------------------- @@ -629,8 +646,9 @@ export async function buildHealthReport(guild, { modules } = {}) { const mod = getModule(def.id); const cfg = row?.config && typeof row.config === 'object' ? row.config : {}; let needs; + let x; try { - const x = EXTRAS[def.id] ? await EXTRAS[def.id](guild.id) : {}; + x = EXTRAS[def.id] ? await EXTRAS[def.id](guild.id) : {}; needs = needsOf(cfg, x, guild).filter(Boolean); } catch (err) { log.warn('health', `could not read the settings of ${def.id} (${guild.id})`, err); @@ -654,6 +672,21 @@ export async function buildHealthReport(guild, { modules } = {}) { checks.push(check); } } + // A bridged channel Sylo stopped (a lost permission, a webhook it could not remake, ...) is a problem to fix. + for (const problem of def.id === 'bridge' ? arr(x.problems) : []) { + const channel = guild.channels?.cache?.get(problem.channelId); + checks.push({ + ...moduleBase, + severity: ERROR, + kind: 'bridge-stopped', + code: 'bridge-stopped', + targetKind: 'channel', + targetId: problem.channelId, + targetName: channel?.name ?? null, + message: `The bridge of ${channel ? channelName(channel) : 'a channel'} stopped: ${problem.text}`, + fix: 'Fix the cause, then press Resume for that channel on the Bridge page.', + }); + } // A role to give or take away also needs Manage Roles, once per module, not once per role. if (needs.some((n) => n.type === 'role' && n.action !== 'ping')) { const missing = guildCheck(moduleBase, guildPerm('ManageRoles', 'give and take away roles'), me); @@ -661,6 +694,13 @@ export async function buildHealthReport(guild, { modules } = {}) { checks.push(missing); } } + // The server mirror is not a module, so it is read from its own tables: a mirror that paused itself, or a copy + // with items that failed, on either side of the pair. + try { + checks.push(...(await mirrorHealthChecks(guild.id))); + } catch (err) { + log.warn('health', `could not read the mirror state of ${guild.id}`, err); + } checks.sort((a, b) => (SEVERITY_ORDER[a.severity] ?? 9) - (SEVERITY_ORDER[b.severity] ?? 9)); return { ...base, available: true, checks, targets: pickerTargets(guild, me) }; } diff --git a/src/web/lib/routeAccess.js b/src/web/lib/routeAccess.js index fb09e393..df75b7c8 100644 --- a/src/web/lib/routeAccess.js +++ b/src/web/lib/routeAccess.js @@ -199,6 +199,16 @@ export const ROUTE_ACCESS = Object.freeze({ 'POST /apply/:guildId/:formId/submit': 'member', 'POST /apply/:guildId/:formId/withdraw': 'member', + // v2Bridge.js + 'GET /guilds/:guildId/modules/bridge/config': 'view', + 'POST /guilds/:guildId/modules/bridge/request': 'edit', + 'POST /guilds/:guildId/modules/bridge/requests/:id/cancel': 'edit', + 'POST /guilds/:guildId/modules/bridge/accept': 'edit', + 'POST /guilds/:guildId/modules/bridge/links/:id/options': 'edit', + 'POST /guilds/:guildId/modules/bridge/links/:id/pause': 'edit', + 'POST /guilds/:guildId/modules/bridge/links/:id/resume': 'edit', + 'POST /guilds/:guildId/modules/bridge/links/:id/unlink': 'edit', + // v2Economy.js 'GET /guilds/:guildId/modules/economy/config': 'view', 'POST /guilds/:guildId/modules/economy/config': 'edit', @@ -314,6 +324,21 @@ export const ROUTE_ACCESS = Object.freeze({ 'GET /guilds/:guildId/setup/suggest': 'view', 'POST /guilds/:guildId/setup/apply': 'edit', + // v2Mirror.js: the page and its preview and report are for looking; asking, agreeing, starting, pausing and + // stopping change what Sylo does to another server. They need Edit on the server the call is made on. + 'GET /guilds/:guildId/mirror': 'view', + 'GET /guilds/:guildId/mirror/preview': 'view', + 'GET /guilds/:guildId/mirror/report': 'view', + 'POST /guilds/:guildId/mirror/request': 'edit', + 'POST /guilds/:guildId/mirror/accept': 'edit', + 'POST /guilds/:guildId/mirror/options': 'edit', + 'POST /guilds/:guildId/mirror/deletes': 'edit', + 'POST /guilds/:guildId/mirror/start': 'edit', + 'POST /guilds/:guildId/mirror/pause': 'edit', + 'POST /guilds/:guildId/mirror/resume': 'edit', + 'POST /guilds/:guildId/mirror/stop': 'edit', + 'POST /guilds/:guildId/mirror/remove': 'edit', + // v2Health.js (the re-check with ?refresh=1 has its own per-server limit) 'GET /guilds/:guildId/health': 'view', }); diff --git a/src/web/routes/mobile.js b/src/web/routes/mobile.js index 1d5575c4..337a4bc4 100644 --- a/src/web/routes/mobile.js +++ b/src/web/routes/mobile.js @@ -312,6 +312,8 @@ export const mobileApiRouter = Router(); // sign-in is on and both Discord and Fluxer logins are set up. `access-levels`: View / Inbox / Edit // dashboard access, with `access` on each GET /api/v2/guilds entry and GET/POST // /api/v2/guilds/:id/access. `message-move`: a posted message moves to its new channel on publish. +// `bridge`: the Bridge module (GET /api/v2/guilds/:id/modules/bridge/config and the request, accept, options, +// pause, resume and unlink calls under /modules/bridge/), which joins a Discord and a Fluxer channel. const FEATURES = Object.freeze([ 'app-login', 'live-access', @@ -328,6 +330,8 @@ const FEATURES = Object.freeze([ 'duplicate', 'message-templates', 'message-move', + 'bridge', + 'mirror', ]); // Public and ahead of the open-mode guard: the app calls it before anyone signs in, and an open-mode diff --git a/src/web/routes/v2Api.js b/src/web/routes/v2Api.js index 1db30f44..03804701 100644 --- a/src/web/routes/v2Api.js +++ b/src/web/routes/v2Api.js @@ -64,8 +64,10 @@ import tempRolesRouter from './v2TempRoles.js'; import quotesRouter from './v2Quotes.js'; import searchRouter from './v2Search.js'; import introRouter from './v2Intro.js'; +import bridgeRouter from './v2Bridge.js'; import applicationsRouter from './v2Applications.js'; import setupRouter from './v2Setup.js'; +import mirrorRouter from './v2Mirror.js'; import applyMemberRouter from './v2Apply.js'; import economyRouter from './v2Economy.js'; import shopRouter from './v2Shop.js'; @@ -545,6 +547,8 @@ router.use('/guilds/:guildId', tempRolesRouter); router.use('/guilds/:guildId', quotesRouter); // Introductions: settings, publish the prompt message (v2Intro.js). router.use('/guilds/:guildId', introRouter); +// Bridge: links to a channel on the other platform, requests and codes, options (v2Bridge.js). +router.use('/guilds/:guildId', bridgeRouter); // Economy: settings, members and their ledger, staff changes, reset (v2Economy.js). router.use('/guilds/:guildId', economyRouter); // Economy shop: items, a member's inventory, the activity log (v2Shop.js). @@ -553,6 +557,8 @@ router.use('/guilds/:guildId', shopRouter); router.use('/guilds/:guildId', applicationsRouter); // Setup wizard: the proposal for the server and applying it (v2Setup.js). router.use('/guilds/:guildId', setupRouter); +// Mirror to Fluxer: ask, agree, preview, copy, pause and stop (v2Mirror.js). +router.use('/guilds/:guildId', mirrorRouter); // The command palette's lookup of an inbox item by number (v2Search.js). router.use('/guilds/:guildId', searchRouter); diff --git a/src/web/routes/v2Bridge.js b/src/web/routes/v2Bridge.js new file mode 100644 index 00000000..5a145364 --- /dev/null +++ b/src/web/routes/v2Bridge.js @@ -0,0 +1,199 @@ +// JSON API for the Bridge page of the dashboard: the links of a server, asking for a link, accepting one +// with a code, the options of this server's side, and pause, resume and unlink. The rules live in +// modules/bridge.js. +// +// Mounted at /guilds/:guildId AFTER the access guard in v2Api.js, so req.guild is set and the caller has +// the level routeAccess.js gives each route: View reads the page, Edit changes anything. Every link is +// looked up through the server in the URL (a link names two servers, and a server may only see and change +// its own side), so another server's link id is a 404 here. +import { accessRouter } from '../middleware/access.js'; +import { currentUser, userIdForGuild } from '../middleware/auth.js'; +import { clientKey, platformActionLimit, rateLimit } from '../middleware/rateLimit.js'; +import { asyncHandler } from '../lib/asyncHandler.js'; +import { GuildOwnerError, isGuildOwner, requireGuildOwner } from '../lib/guildOwner.js'; +import { sweepMap } from '../../lib/sweepMap.js'; +import { + BridgeError, + acceptLink, + assertActorMayUse, + bridgeOverview, + cancelRequest, + pauseLink, + requestLink, + resumeLink, + setSideOptions, + unlinkLink, +} from '../../modules/bridge.js'; + +const router = accessRouter('/guilds/:guildId', { mergeParams: true }); + +const actor = (req) => (currentUser(req)?.open ? 'Dashboard' : `${currentUser(req).name} (dashboard)`); +const isId = (v) => /^\d{17,20}$/.test(String(v ?? '')); +const linkId = (req) => { + const n = Number(req.params.id); + if (!Number.isSafeInteger(n) || n < 1) + throw new BridgeError('No such bridged channel in this server.', 404); + return n; +}; + +/** + * Run a changing handler. Every change to a bridge is the SERVER OWNER's alone (the owner the platform reports + * for this server, worked out again on every call): Edit access, Administrator and the bot-master roles do + * not reach it. View access reads the page (the GET below). A refusal becomes its own status and message. + */ +const guarded = (fn) => + asyncHandler(async (req, res) => { + try { + requireGuildOwner(req, req.guild.id); + await fn(req, res); + } catch (err) { + if (err instanceof BridgeError || err instanceof GuildOwnerError) { + return res.status(err.status).json({ error: err.message }); + } + throw err; + } + }); + +// A code is about 59 bits, so guessing one is hopeless; the limits are about not letting the endpoint be a +// way to hammer the platforms (every accept makes webhooks) or to try codes at speed. One budget per +// person, whichever server they are in, and a lock after several wrong codes. +const acceptLimit = rateLimit({ + windowMs: 60_000, + max: 8, + message: 'Too many attempts. Wait a minute and try again.', + keyFn: (req) => `bridge-accept|${req.account?.id ?? clientKey(req)}`, +}); +const WRONG_LIMIT = 6; +const WRONG_WINDOW_MS = 10 * 60_000; +const wrong = new Map(); // `${person}|${server}` -> { n, at } +sweepMap(wrong, (v, now) => now - v.at > WRONG_WINDOW_MS); +const who = (req) => `${req.account?.id ?? clientKey(req)}|${req.params.guildId}`; +const locked = (req) => { + const hit = wrong.get(who(req)); + return Boolean(hit && Date.now() - hit.at < WRONG_WINDOW_MS && hit.n >= WRONG_LIMIT); +}; +const noteWrong = (req) => { + const id = who(req); + const hit = wrong.get(id); + wrong.set(id, { n: (hit && Date.now() - hit.at < WRONG_WINDOW_MS ? hit.n : 0) + 1, at: Date.now() }); +}; + +// `owner`: may this person change the bridge? The page shows the status to everyone with View access and +// enables the controls only for the owner. +const overview = async (req) => ({ + ...(await bridgeOverview(req.guild.id)), + owner: isGuildOwner(req, req.guild.id), +}); + +// The person must be able to view, read and send in the channel themselves (Edit access alone is not enough: +// a bot-master may not see every channel, and the bridge would carry it to a server they run). +async function requireActorAccess(req, channelId) { + const user = currentUser(req); + if (user?.open) return; + const userId = userIdForGuild(req, req.guild.id); + const guild = req.guild; + const member = userId + ? (guild.members.cache.get(userId) ?? (await guild.members.fetch(userId).catch(() => null))) + : null; + assertActorMayUse(guild.id, channelId, member); +} + +router.get( + '/modules/bridge/config', + asyncHandler(async (req, res) => { + res.json(await overview(req)); + }) +); + +router.post( + '/modules/bridge/request', + platformActionLimit, + guarded(async (req, res) => { + if (!isId(req.body?.channelId)) throw new BridgeError('Pick a channel.'); + await requireActorAccess(req, String(req.body.channelId)); + const made = await requestLink({ + guildKey: req.guild.id, + channelId: String(req.body.channelId), + options: req.body?.options, + actor: actor(req), + }); + // The code is shown once: only its hash is stored. + res.json({ code: made.code, expiresAt: made.expiresAt, overview: await overview(req) }); + }) +); + +router.post( + '/modules/bridge/requests/:id/cancel', + guarded(async (req, res) => { + await cancelRequest({ guildKey: req.guild.id, id: linkId(req), actor: actor(req) }); + res.json({ overview: await overview(req) }); + }) +); + +router.post( + '/modules/bridge/accept', + acceptLimit, + platformActionLimit, + guarded(async (req, res) => { + if (locked(req)) { + throw new BridgeError('Too many wrong codes. Wait a few minutes before trying again.', 429); + } + if (!isId(req.body?.channelId)) throw new BridgeError('Pick a channel.'); + await requireActorAccess(req, String(req.body.channelId)); + try { + await acceptLink({ + guildKey: req.guild.id, + channelId: String(req.body.channelId), + code: String(req.body?.code ?? ''), + options: req.body?.options, + actor: actor(req), + }); + } catch (err) { + // Only a code that did not work counts as wrong; a channel problem is the person's to fix. + if (err instanceof BridgeError && err.status === 404) noteWrong(req); + throw err; + } + res.json({ overview: await overview(req) }); + }) +); + +router.post( + '/modules/bridge/links/:id/options', + guarded(async (req, res) => { + await setSideOptions({ + guildKey: req.guild.id, + id: linkId(req), + options: req.body?.options ?? {}, + actor: actor(req), + }); + res.json({ overview: await overview(req) }); + }) +); + +router.post( + '/modules/bridge/links/:id/pause', + platformActionLimit, + guarded(async (req, res) => { + await pauseLink({ guildKey: req.guild.id, id: linkId(req), actor: actor(req) }); + res.json({ overview: await overview(req) }); + }) +); + +router.post( + '/modules/bridge/links/:id/resume', + platformActionLimit, + guarded(async (req, res) => { + await resumeLink({ guildKey: req.guild.id, id: linkId(req), actor: actor(req) }); + res.json({ overview: await overview(req) }); + }) +); + +router.post( + '/modules/bridge/links/:id/unlink', + guarded(async (req, res) => { + await unlinkLink({ guildKey: req.guild.id, id: linkId(req), actor: actor(req) }); + res.json({ overview: await overview(req) }); + }) +); + +export default router; diff --git a/src/web/routes/v2Mirror.js b/src/web/routes/v2Mirror.js new file mode 100644 index 00000000..defe7b93 --- /dev/null +++ b/src/web/routes/v2Mirror.js @@ -0,0 +1,446 @@ +// JSON API for the "Mirror to Fluxer" page: copy a Discord server's structure to a Fluxer server and keep it +// in step (src/mirror/). +// +// Mounted at /guilds/:guildId in v2Api.js behind the access guard, so req.guild is set and the caller has a +// dashboard level in THIS server. The levels are in src/web/lib/routeAccess.js: seeing the page and its +// preview and report needs View. Everything that changes anything (asking, agreeing, the options, starting, +// pausing, resuming, stopping, removing) is Edit in the table and, on top of that, only the owner of the server +// the call is made on (isGuildOwner in src/web/lib/guildOwner.js): an Administrator, an Edit holder or a bot master is not enough. +// +// The page is used on both servers of a pair. On the Discord server (the source) its owner asks for a mirror, +// looks at the preview and starts the copy. On the Fluxer server (the target) its owner agrees with the code the +// request showed, and may pause or stop the mirror at any time. Every call looks only at the pair this server is +// part of, so a code, a pair id or a server id of another server reaches nothing. +import { config } from '../../config.js'; +import { splitPlatformGuildId, makePlatformGuildId } from '../../lib/platformId.js'; +import { PAIR_STATUS, pairsOfGuild, updatePair } from '../../db/mirror.js'; +import { recordAudit } from '../../db/audit.js'; +import { accessRouter } from '../middleware/access.js'; +import { adminGuildIds, currentUser, userIdForGuild } from '../middleware/auth.js'; +import { isGuildOwner } from '../lib/guildOwner.js'; +import { platformActionKey, rateLimit } from '../middleware/rateLimit.js'; +import { asyncHandler } from '../lib/asyncHandler.js'; +import { guildByKey, platformOfGuild } from '../../platform/platformOf.js'; +import { mirror } from '../../mirror/index.js'; +import { ConsentError, acceptMirror, requestMirror } from '../../mirror/consent.js'; +import { MirrorError } from '../../mirror/errors.js'; +import { DEFAULT_OPTIONS, FLUXER_LIMITS, normaliseOptions } from '../../mirror/spec.js'; +import { invalidateHealth } from '../../lib/healthCache.js'; + +const router = accessRouter('/guilds/:guildId', { mergeParams: true }); + +const actor = (req) => (currentUser(req)?.open ? 'Dashboard' : `${currentUser(req).name} (dashboard)`); +const fail = (res, message, status = 400, extra = {}) => + res.status(status).json({ error: message, ...extra }); + +// Asking and agreeing change who may write to a server: a few tries in ten minutes per server and person. +const askLimit = rateLimit({ + windowMs: 10 * 60_000, + max: 12, + message: 'Too many mirror requests just now. Wait a few minutes and try again.', + keyFn: (req) => `mirror-ask|${platformActionKey(req)}`, +}); +// The preview reads both servers whole. +const previewLimit = rateLimit({ + windowMs: 60_000, + max: 10, + message: 'The preview was asked for too often. Wait a minute.', + keyFn: (req) => `mirror-preview|${platformActionKey(req)}`, +}); +// Starting a copy queues work on another platform. +const startLimit = rateLimit({ + windowMs: 10 * 60_000, + max: 4, + message: 'The copy was started several times just now. Wait a few minutes.', + keyFn: (req) => `mirror-start|${platformActionKey(req)}`, +}); +const controlLimit = rateLimit({ + windowMs: 60_000, + max: 20, + keyFn: (req) => `mirror-control|${platformActionKey(req)}`, +}); + +/** The pair this server is part of, with which side it is, or null. */ +async function pairOf(req) { + const key = req.guild.id; + const pair = (await pairsOfGuild(key))[0] ?? null; + if (!pair) return null; + return { pair, side: pair.sourceGuild === key ? 'source' : 'target' }; +} + +/** The person's id on a server's platform ('' in open mode, where there are no accounts). */ +const userIdOn = (req, key) => userIdForGuild(req, key) ?? ''; + +/** Answer 403 unless the signed-in person is the owner of this server. Returns their id there, or null. */ +function needOwner(req, res, what) { + if (!isGuildOwner(req, req.guild.id)) { + fail(res, `Only the server owner can ${what}.`, 403, { code: 'owner-only' }); + return null; + } + return { userId: userIdOn(req, req.guild.id) }; +} + +/** What the page needs to know about a pair, for the side that is looking. Never a code, a hash or a secret. */ +function pairView(pair, side) { + const otherKey = side === 'source' ? pair.targetGuild : pair.sourceGuild; + const other = guildByKey(otherKey); + // The Discord side learns the Fluxer server's name only once someone there has agreed. + const showName = side === 'target' || Boolean(pair.targetConsentBy); + return { + id: pair.id, + side, + status: pair.status, + resumeStatus: pair.resumeStatus, + options: normaliseOptions(pair.options), + otherId: otherKey, + otherName: showName ? (other?.name ?? null) : null, + otherPresent: Boolean(other), + // A dashboard user's name is shown only on their own server, never to the other server's viewers. + requestedBy: side === 'source' ? pair.requestedBy : '', + consent: { + source: pair.sourceConsentBy + ? { by: side === 'source' ? pair.sourceConsentBy : '', at: pair.sourceConsentAt } + : null, + target: pair.targetConsentBy + ? { by: side === 'target' ? pair.targetConsentBy : '', at: pair.targetConsentAt } + : null, + }, + pauseKind: pair.status === PAIR_STATUS.PAUSED ? pair.pauseKind : '', + pausedBy: pair.status === PAIR_STATUS.PAUSED ? pair.pausedBy : '', + codeExpiresAt: pair.status === PAIR_STATUS.PENDING ? pair.codeExpiresAt : 0, + lastError: pair.lastError, + copiedAt: pair.copiedAt, + copyRun: pair.copyRun, + createdAt: pair.createdAt, + }; +} + +async function stateOf(req) { + const platform = platformOfGuild(req.guild); + const found = await pairOf(req); + const candidates = + platform === 'discord' && isGuildOwner(req, req.guild.id) + ? [...adminGuildIds(req)] + .filter((k) => splitPlatformGuildId(k)?.platform === 'fluxer') + .filter((k) => isGuildOwner(req, k)) + .map((k) => ({ id: k, name: guildByKey(k)?.name ?? k })) + : []; + return { + platform, + canMirror: platform === 'discord', + fluxerConnected: config.fluxerEnabled, + // Only the server owner can change anything here; everyone with access can look. + isOwner: isGuildOwner(req, req.guild.id), + pair: found ? pairView(found.pair, found.side) : null, + progress: found ? await mirror.progress(found.pair) : null, + candidates, + defaults: DEFAULT_OPTIONS, + limits: { emoji: FLUXER_LIMITS.emoji, stickers: FLUXER_LIMITS.stickers }, + }; +} + +router.get( + '/mirror', + asyncHandler(async (req, res) => { + res.set('Cache-Control', 'no-store'); + res.json(await stateOf(req)); + }) +); + +/** A raw Fluxer id, or "fluxer:", as a key. Null when it is neither. */ +function fluxerKeyOf(raw) { + const text = String(raw ?? '').trim(); + const key = /^\d{17,20}$/.test(text) ? makePlatformGuildId('fluxer', text) : text; + return splitPlatformGuildId(key)?.platform === 'fluxer' ? key : null; +} + +const consentStatus = (err) => + ({ 'no-request': 404, expired: 410, 'wrong-code': 403, cancelled: 403, 'no-consent': 409 })[err.code] ?? + 409; + +// Ask for a mirror: the Discord server is the source, and only its owner can ask. When the same person also owns the +// Fluxer server, both sides agree at once; otherwise the answer carries a one-time code to give to its owner. +router.post( + '/mirror/request', + askLimit, + asyncHandler(async (req, res) => { + if (platformOfGuild(req.guild) !== 'discord') { + return fail(res, 'Only a Discord server can be mirrored. Ask for the mirror on the Discord server.'); + } + const owner = needOwner(req, res, 'ask for a mirror'); + if (!owner) return; + const targetKey = fluxerKeyOf(req.body?.targetGuildId); + if (!targetKey) return fail(res, 'Enter the id of the Fluxer server.'); + if (!guildByKey(targetKey)) { + return fail( + res, + 'Sylo is not in that Fluxer server. Invite Sylo there with the Administrator permission first.', + 404 + ); + } + try { + // The same person owns the Fluxer server too (their account has that login): both sides agree at once. + const targetOwner = isGuildOwner(req, targetKey); + const { pair, code } = await requestMirror({ + sourceGuild: req.guild.id, + targetGuild: targetKey, + actor: actor(req), + sourceUser: owner.userId, + targetUser: userIdOn(req, targetKey), + targetOwner, + options: req.body?.options, + }); + await recordAudit(req.guild.id, { + actor: actor(req), + action: 'mirror:request', + detail: targetOwner + ? 'asked for a mirror to a Fluxer server (agreed on both sides)' + : 'asked for a mirror to a Fluxer server', + }); + if (targetOwner) { + await recordAudit(targetKey, { + actor: actor(req), + action: 'mirror:agree', + detail: 'agreed to be the mirror target', + }); + } + invalidateHealth(req.guild.id); + res.json({ ...(await stateOf(req)), code, agreed: Boolean(pair.targetConsentBy) }); + } catch (err) { + if (err instanceof ConsentError) return fail(res, err.message, 409, { code: err.code }); + throw err; + } + }) +); + +// The Fluxer side agrees with the code. +router.post( + '/mirror/accept', + askLimit, + asyncHandler(async (req, res) => { + if (platformOfGuild(req.guild) !== 'fluxer') { + return fail(res, 'The code is entered on the Fluxer server that is asked to be the mirror.'); + } + const owner = needOwner(req, res, 'agree to a mirror'); + if (!owner) return; + try { + await acceptMirror({ + targetGuild: req.guild.id, + code: req.body?.code, + actor: actor(req), + userId: owner.userId, + }); + } catch (err) { + if (err instanceof ConsentError) return fail(res, err.message, consentStatus(err), { code: err.code }); + throw err; + } + const found = await pairOf(req); + await recordAudit(req.guild.id, { + actor: actor(req), + action: 'mirror:agree', + detail: 'agreed to be the mirror target', + }); + if (found) { + await recordAudit(found.pair.sourceGuild, { + actor: actor(req), + action: 'mirror:agree', + detail: 'the Fluxer server agreed to the mirror', + }); + invalidateHealth(found.pair.sourceGuild); + } + invalidateHealth(req.guild.id); + res.json(await stateOf(req)); + }) +); + +/** The pair of this (source) server, or an answer that says why not. */ +async function sourcePair(req, res, { needConsent = false } = {}) { + const found = await pairOf(req); + if (!found || found.side !== 'source') { + fail(res, 'This server has no mirror to a Fluxer server. Ask for one first.', 404); + return null; + } + if (needConsent && !(found.pair.sourceConsentBy && found.pair.targetConsentBy)) { + fail(res, 'Both servers have to agree first.', 409, { code: 'no-consent' }); + return null; + } + return found.pair; +} + +router.post( + '/mirror/options', + controlLimit, + asyncHandler(async (req, res) => { + if (!needOwner(req, res, 'change what is mirrored')) return; + const pair = await sourcePair(req, res); + if (!pair) return; + const options = normaliseOptions({ ...pair.options, ...req.body }); + // Real deletion on Fluxer takes the owner of the Fluxer server: from here it can only be asked for, never switched + // on (POST /mirror/deletes on the Fluxer server does that). Switching it off is always allowed. + if (req.body?.mirrorDeletes === true) { + options.mirrorDeletes = pair.options?.mirrorDeletes === true; + options.mirrorDeletesRequested = true; + } else if (req.body?.mirrorDeletes === false) { + options.mirrorDeletes = false; + options.mirrorDeletesRequested = false; + } else { + options.mirrorDeletes = pair.options?.mirrorDeletes === true; + } + if (req.body?.mirrorDeletesRequested === undefined && req.body?.mirrorDeletes === undefined) { + options.mirrorDeletesRequested = pair.options?.mirrorDeletesRequested === true; + } + await updatePair(pair.id, { options }); + await recordAudit(req.guild.id, { + actor: actor(req), + action: 'mirror:options', + detail: 'mirror options changed', + }); + res.json(await stateOf(req)); + }) +); + +// The owner of the Fluxer server allows (or stops) real deletion: only after the Discord side asked for it, and +// only from the Fluxer server's own page. +router.post( + '/mirror/deletes', + controlLimit, + asyncHandler(async (req, res) => { + if (!needOwner(req, res, 'allow deleting on this server')) return; + const found = await pairOf(req); + if (!found || found.side !== 'target') { + return fail(res, 'This server is not the target of a mirror.', 404); + } + const options = normaliseOptions(found.pair.options); + if (req.body?.allow === true) { + if (!options.mirrorDeletesRequested) { + return fail(res, 'The Discord server has not asked for deletions to be mirrored.', 409); + } + options.mirrorDeletes = true; + } else { + options.mirrorDeletes = false; + options.mirrorDeletesRequested = false; + } + await updatePair(found.pair.id, { options }); + for (const key of [found.pair.sourceGuild, found.pair.targetGuild]) { + await recordAudit(key, { + actor: actor(req), + action: 'mirror:deletes', + detail: options.mirrorDeletes + ? 'deleting on Fluxer was allowed' + : 'deleting on Fluxer was switched off', + }); + } + res.json(await stateOf(req)); + }) +); + +// The dry run: what a copy would do now, and everything that cannot be carried over. Creates nothing. +router.get( + '/mirror/preview', + previewLimit, + asyncHandler(async (req, res) => { + const pair = await sourcePair(req, res, { needConsent: true }); + if (!pair) return; + const preview = await mirror.previewPair(pair); + res.set('Cache-Control', 'no-store'); + res.json({ + ok: preview.ok, + blockers: preview.blockers, + warnings: preview.warnings, + summary: preview.summary, + notes: preview.notes, + target: preview.target, + steps: preview.steps.map((s) => ({ + key: s.key, + kind: s.kind, + name: s.name, + op: s.op, + reason: s.reason ?? null, + notes: (s.notes ?? []).length, + })), + }); + }) +); + +// What became of the latest copy, step by step. +router.get( + '/mirror/report', + asyncHandler(async (req, res) => { + const pair = await sourcePair(req, res); + if (!pair) return; + res.set('Cache-Control', 'no-store'); + res.json(await mirror.report(pair)); + }) +); + +router.post( + '/mirror/start', + startLimit, + asyncHandler(async (req, res) => { + if (!needOwner(req, res, 'start a copy')) return; + const pair = await sourcePair(req, res, { needConsent: true }); + if (!pair) return; + if (req.body?.confirm !== true) return fail(res, 'Confirm the copy first.', 400); + try { + await mirror.startCopy(pair.id); + } catch (err) { + if (err instanceof ConsentError) return fail(res, err.message, 409, { code: err.code }); + if (err instanceof MirrorError) return fail(res, err.message, 409); + throw err; + } + await recordAudit(req.guild.id, { + actor: actor(req), + action: 'mirror:start', + detail: 'started copying to the Fluxer server', + }); + await recordAudit(pair.targetGuild, { + actor: actor(req), + action: 'mirror:start', + detail: 'a copy from the Discord server started', + }); + res.json(await stateOf(req)); + }) +); + +/** + * Pause, resume, stop and remove, from either server of the pair, by that server's owner. A mirror its owner paused + * can be resumed only from the same side; after a burst of deletes the owner has to confirm them (`confirm: true`). + */ +function control(name, run, detail) { + router.post( + `/mirror/${name}`, + controlLimit, + asyncHandler(async (req, res) => { + if (!needOwner(req, res, `${name} a mirror`)) return; + const found = await pairOf(req); + if (!found) return fail(res, 'This server has no mirror.', 404); + try { + await run(found.pair, found.side, req.body); + } catch (err) { + if (err instanceof MirrorError || err instanceof ConsentError) { + return fail(res, err.message, 409, { + ...(err.code ? { code: err.code } : {}), + ...(err.deletes + ? { deletes: err.deletes, deleteCount: err.deleteCount ?? err.deletes.length } + : {}), + }); + } + throw err; + } + for (const key of [found.pair.sourceGuild, found.pair.targetGuild]) { + await recordAudit(key, { actor: actor(req), action: `mirror:${name}`, detail }); + } + res.json(await stateOf(req)); + }) + ); +} +control('pause', (pair, side) => mirror.pause(pair.id, side), 'mirror paused'); +control( + 'resume', + (pair, side, body) => mirror.resume(pair.id, { side, confirm: body?.confirm === true }), + 'mirror resumed' +); +control('stop', (pair) => mirror.stopPair(pair.id), 'mirror stopped'); +control('remove', (pair, side) => mirror.removePair(pair.id, side), 'mirror removed'); + +export default router; diff --git a/test/bridge.db.test.js b/test/bridge.db.test.js new file mode 100644 index 00000000..12d6bb66 --- /dev/null +++ b/test/bridge.db.test.js @@ -0,0 +1,470 @@ +// Bridge storage: the three tables (migration 83), one-time codes (only the hash is stored, they expire +// and are used up by the first claim), links (a channel is in at most one), the message map (a lookup +// from either side, swept after 30 days by the retention job) and the guild-leave purge. +import './helpers/tmpDb.js'; +import test, { beforeEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { db, MIGRATIONS, SCHEMA_VERSION } from '../src/db/index.js'; +import { + CODE_TTL_MS, + MESSAGE_MAP_TTL_MS, + cancelCode, + claimCode, + createCode, + createLink, + deleteLink, + deleteMapping, + getLink, + linkCount, + linkForChannel, + linkForHook, + linksOfGuild, + looksLikeCode, + mappingOf, + newCode, + normaliseCode, + peekCode, + pendingCodeCount, + pendingCodes, + purgeBridgeGuild, + saveMapping, + setLinkHook, + setLinkOptions, + setLinkStatus, + sweepCodes, + sweepMappings, +} from '../src/db/bridge.js'; +import { GUILD_TABLES, purgeGuild } from '../src/db/purge.js'; +import { sweepRetention } from '../src/db/retention.js'; + +const D = '710000000000000001'; +const F = 'fluxer:710000000000000002'; +const D2 = '710000000000000003'; +const CH = { + d1: '720000000000000001', + f1: '720000000000000002', + d2: '720000000000000003', + f2: '720000000000000004', +}; + +const link = (over = {}) => ({ + guildA: D, + channelA: CH.d1, + guildB: F, + channelB: CH.f1, + hookA: { id: 'hook-a', token: 'secret-a' }, + hookB: { id: 'hook-b', token: 'secret-b' }, + requestedBy: 'Alice (dashboard)', + acceptedBy: 'Bob (dashboard)', + ...over, +}); + +beforeEach(async () => { + for (const g of [D, F, D2]) await purgeGuild(g); + db.prepare('DELETE FROM bridge_codes').run(); +}); + +test('migration 83 creates the three tables, the unique channel indexes and the 30-day index', () => { + assert.equal(SCHEMA_VERSION, MIGRATIONS.length); + assert.ok(MIGRATIONS.length >= 83, 'the Bridge keeps version 83 (81 and 82 belong to other changes)'); + const names = (t) => + db + .prepare(`PRAGMA table_info(${t})`) + .all() + .map((c) => c.name); + assert.deepEqual(names('bridge_links'), [ + 'id', + 'guild_a', + 'channel_a', + 'guild_b', + 'channel_b', + 'status', + 'paused_side', + 'fail_reason', + 'failure_count', + 'options_a', + 'options_b', + 'hook_a_id', + 'hook_a_token', + 'hook_b_id', + 'hook_b_token', + 'requested_by_name', + 'accepted_by_name', + 'owner_a', + 'owner_b', + 'created_at', + 'updated_at', + ]); + assert.deepEqual(names('bridge_codes'), [ + 'id', + 'guild_id', + 'channel_id', + 'code_hash', + 'options', + 'requested_by_name', + 'guild_owner', + 'expires_at', + 'created_at', + ]); + assert.deepEqual(names('bridge_messages'), [ + 'link_id', + 'origin', + 'a_message_id', + 'b_message_id', + 'copy_hook_id', + 'content_hash', + 'extra', + 'created_at', + ]); + const indexes = db + .prepare("SELECT name FROM sqlite_master WHERE type = 'index' AND name LIKE 'idx_bridge_%'") + .all() + .map((r) => r.name); + for (const name of [ + 'idx_bridge_links_a', + 'idx_bridge_links_b', + 'idx_bridge_codes_hash', + 'idx_bridge_messages_b', + 'idx_bridge_messages_created', + ]) { + assert.ok(indexes.includes(name), name); + } + // 81 and 82 are placeholders until their own changes land: they must not have made any table of theirs here. + assert.equal(typeof MIGRATIONS[80], 'function'); + assert.equal(typeof MIGRATIONS[81], 'function'); +}); + +test('bridge_codes is in GUILD_TABLES; the links go through purgeBridgeGuild (they name two servers)', () => { + assert.ok(GUILD_TABLES.includes('bridge_codes')); + assert.ok(!GUILD_TABLES.includes('bridge_links')); +}); + +test('the table columns hold names, not user ids (so /forget has nothing to find)', () => { + const columns = db + .prepare('PRAGMA table_info(bridge_links)') + .all() + .map((c) => c.name); + assert.ok(columns.includes('requested_by_name') && columns.includes('accepted_by_name')); + assert.ok( + !columns.some((c) => /(user|author|owner|decided|moderator|actor)_(id|by)$|^created_by$/.test(c)) + ); +}); + +// --- codes --------------------------------------------------------------------------------------------- + +test('a code is 12 characters of an unambiguous alphabet, shown as XXXX-XXXX-XXXX', () => { + for (let i = 0; i < 50; i += 1) { + const code = newCode(); + assert.match(code, /^[A-HJKMNP-Z2-9]{4}-[A-HJKMNP-Z2-9]{4}-[A-HJKMNP-Z2-9]{4}$/); + assert.ok(looksLikeCode(code)); + assert.ok(looksLikeCode(code.toLowerCase().replace(/-/g, ' ')), 'typed in lower case, with spaces'); + } + assert.equal(normaliseCode(' ab-cd ef '), 'ABCDEF'); + assert.ok(!looksLikeCode('0123-4567-89AB'), '0, 1, O, I and L are not in the alphabet'); + assert.ok(!looksLikeCode('')); + assert.ok(!looksLikeCode(null)); +}); + +test('only the hash of a code is stored, and it is used up by the first claim', async () => { + const made = await createCode({ + guildId: D, + channelId: CH.d1, + options: { bots: true }, + requestedBy: 'Alice', + }); + const stored = JSON.stringify(db.prepare('SELECT * FROM bridge_codes').all()); + assert.ok(!stored.includes(made.code), 'the code is not stored'); + assert.ok(!stored.includes(normaliseCode(made.code)), 'nor its plain form'); + assert.equal(db.prepare('SELECT code_hash FROM bridge_codes').get().code_hash.length, 64); + + assert.equal((await peekCode(made.code)).guildId, D, 'peek reads without using it up'); + assert.equal(await pendingCodeCount(D), 1); + const claimed = await claimCode(made.code.toLowerCase()); + assert.deepEqual( + { + guildId: claimed.guildId, + channelId: claimed.channelId, + options: claimed.options, + by: claimed.requestedBy, + }, + { guildId: D, channelId: CH.d1, options: { bots: true }, by: 'Alice' } + ); + assert.equal(await claimCode(made.code), null, 'a second server cannot use it'); + assert.equal(await peekCode(made.code), null); +}); + +test('a code expires after its time, and a wrong or malformed code finds nothing', async () => { + const made = await createCode({ guildId: D, channelId: CH.d1 }); + assert.equal(made.expiresAt - Date.now() <= CODE_TTL_MS, true); + assert.equal(await claimCode('ABCD-EFGH-JKMN'), null); + assert.equal(await claimCode('not a code'), null); + assert.equal(await claimCode(made.code, made.expiresAt + 1), null, 'past its time'); + assert.ok(await claimCode(made.code, made.expiresAt - 1), 'still inside its time'); + + const old = await createCode({ guildId: D, channelId: CH.d2, ttlMs: -1000 }); + assert.equal(await pendingCodeCount(D), 0, 'an expired request is not pending'); + assert.equal(await sweepCodes(), 1); + assert.equal(await claimCode(old.code), null); +}); + +test('pending requests list without their codes, and only the server that made one can cancel it', async () => { + const a = await createCode({ guildId: D, channelId: CH.d1, requestedBy: 'Alice' }); + await createCode({ guildId: D2, channelId: CH.d2 }); + const list = await pendingCodes(D); + assert.equal(list.length, 1); + assert.deepEqual(Object.keys(list[0]).sort(), ['channelId', 'createdAt', 'expiresAt', 'id', 'requestedBy']); + assert.equal(await cancelCode(D2, a.id), false, 'another server cannot cancel it'); + assert.equal(await cancelCode(D, a.id), true); + assert.equal(await pendingCodeCount(D), 0); + assert.equal(await claimCode(a.code), null); +}); + +// --- links --------------------------------------------------------------------------------------------- + +test('create, read and find a link from either side, from the channel or from a webhook', async () => { + const id = await createLink(link()); + assert.ok(id > 0); + const row = await getLink(id); + assert.equal(row.status, 'active'); + assert.equal(row.guildA, D); + assert.equal(row.channelB, CH.f1); + assert.deepEqual(row.optionsA, {}); + assert.equal(row.requestedBy, 'Alice (dashboard)'); + assert.equal((await linkForChannel(D, CH.d1)).side, 'a'); + assert.equal((await linkForChannel(F, CH.f1)).side, 'b'); + assert.equal(await linkForChannel(D, CH.f1), null, 'a channel id of another server is not a match'); + assert.equal((await linkForHook('hook-b')).id, id); + assert.equal(await linkForHook('nope'), null); + assert.equal(await linkForHook(''), null); + assert.deepEqual( + (await linksOfGuild(F)).map((l) => l.id), + [id] + ); + assert.equal(await linkCount(D), 1); + assert.equal(await linkCount(D2), 0); +}); + +test('a channel is in at most one link, from either side', async () => { + assert.ok(await createLink(link())); + assert.equal(await createLink(link({ channelB: CH.f2 })), null, 'the same channel a second time'); + assert.equal(await createLink(link({ channelA: CH.d2 })), null, 'the same channel b second time'); + assert.ok(await createLink(link({ channelA: CH.d2, channelB: CH.f2 })), 'other channels are free'); +}); + +test('status, options and the webhook of a side change independently', async () => { + const id = await createLink(link()); + await setLinkStatus(id, { status: 'paused', pausedSide: 'b' }); + let row = await getLink(id); + assert.deepEqual([row.status, row.pausedSide], ['paused', 'b']); + await setLinkStatus(id, { status: 'failed', pausedSide: 'a', reason: 'permissions', failures: 5 }); + row = await getLink(id); + assert.deepEqual( + [row.status, row.pausedSide, row.failReason, row.failureCount], + ['failed', '', 'permissions', 5] + ); + await setLinkStatus(id, { status: 'active' }); + row = await getLink(id); + assert.deepEqual([row.status, row.failReason, row.failureCount], ['active', '', 0]); + + await setLinkOptions(id, 'b', { bots: true }); + row = await getLink(id); + assert.deepEqual([row.optionsA, row.optionsB], [{}, { bots: true }]); + + await setLinkHook(id, 'a', { id: 'hook-new', token: 'tok-new' }); + row = await getLink(id); + assert.deepEqual([row.hookAId, row.hookAToken, row.hookBId], ['hook-new', 'tok-new', 'hook-b']); + assert.equal(await linkForHook('hook-a'), null, 'the old webhook no longer belongs to the link'); +}); + +// --- the message map ------------------------------------------------------------------------------------- + +test('the map finds a message from either side, with the counterpart and who was the original', async () => { + const id = await createLink(link()); + await saveMapping(id, 'a', { + sourceId: 'src-1', + copyId: 'copy-1', + hookId: 'hook-b', + hash: 'h1', + extra: { head: '> hi' }, + }); + await saveMapping(id, 'b', { sourceId: 'src-2', copyId: 'copy-2', hookId: 'hook-a' }); + + const fromA = await mappingOf(id, 'a', 'src-1'); + assert.deepEqual( + [fromA.counterpartId, fromA.isOriginal, fromA.copyHookId, fromA.contentHash, fromA.extra], + ['copy-1', true, 'hook-b', 'h1', { head: '> hi' }] + ); + const fromB = await mappingOf(id, 'b', 'copy-1'); + assert.deepEqual( + [fromB.counterpartId, fromB.isOriginal], + ['src-1', false], + 'the copy points back to the original' + ); + const second = await mappingOf(id, 'b', 'src-2'); + assert.deepEqual([second.counterpartId, second.isOriginal], ['copy-2', true]); + assert.equal(await mappingOf(id, 'a', 'nope'), null); + assert.equal(await mappingOf(id, 'b', 'src-1'), null, 'an id is looked up on its own side only'); + + await deleteMapping(id, 'a', 'src-1'); + assert.equal(await mappingOf(id, 'a', 'src-1'), null); + assert.equal(await mappingOf(id, 'b', 'copy-1'), null); +}); + +test('saving a mapping again for the same message replaces it, and the map is per link', async () => { + const one = await createLink(link()); + const two = await createLink( + link({ channelA: CH.d2, channelB: CH.f2, hookA: { id: 'x', token: 'x' }, hookB: { id: 'y', token: 'y' } }) + ); + await saveMapping(one, 'a', { sourceId: 'm', copyId: 'c1' }); + await saveMapping(one, 'a', { sourceId: 'm', copyId: 'c2', hash: 'new' }); + assert.equal((await mappingOf(one, 'a', 'm')).counterpartId, 'c2'); + assert.equal(await mappingOf(two, 'a', 'm'), null, 'another link has its own map'); +}); + +test('a mapping expires after 30 days: the sweep, and the daily retention job, remove it', async () => { + const id = await createLink(link()); + const day = 86_400_000; + await saveMapping(id, 'a', { sourceId: 'old', copyId: 'old-copy' }); + await saveMapping(id, 'a', { sourceId: 'new', copyId: 'new-copy' }); + db.prepare('UPDATE bridge_messages SET created_at = ? WHERE a_message_id = ?').run( + Date.now() - 31 * day, + 'old' + ); + assert.equal(MESSAGE_MAP_TTL_MS, 30 * day); + assert.equal(await sweepMappings(), 1); + assert.equal(await mappingOf(id, 'a', 'old'), null); + assert.ok(await mappingOf(id, 'a', 'new')); + + db.prepare('UPDATE bridge_messages SET created_at = ? WHERE a_message_id = ?').run( + Date.now() - 40 * day, + 'new' + ); + await createCode({ guildId: D, channelId: CH.d1, ttlMs: -1 }); + await sweepRetention(); + assert.equal(await mappingOf(id, 'a', 'new'), null, 'the retention job sweeps the map'); + assert.equal(db.prepare('SELECT COUNT(*) AS n FROM bridge_codes').get().n, 0, 'and the expired requests'); +}); + +// --- purge ----------------------------------------------------------------------------------------------- + +test('deleting a link removes its map; a server leaving removes every link it is in, on either side', async () => { + const one = await createLink(link()); + const two = await createLink( + link({ + guildA: D2, + channelA: CH.d2, + channelB: CH.f2, + hookA: { id: 'p', token: 'p' }, + hookB: { id: 'q', token: 'q' }, + }) + ); + await saveMapping(one, 'a', { sourceId: 'm1', copyId: 'c1' }); + await saveMapping(two, 'a', { sourceId: 'm2', copyId: 'c2' }); + await createCode({ guildId: F, channelId: CH.f1 }); + + assert.equal(await deleteLink(one), 1); + assert.equal(await mappingOf(one, 'a', 'm1'), null); + assert.ok(await mappingOf(two, 'a', 'm2'), 'the other link keeps its map'); + + const one2 = await createLink(link()); + await saveMapping(one2, 'a', { sourceId: 'm3', copyId: 'c3' }); + assert.equal(await purgeBridgeGuild(F), 2, 'F is on the b side of both links'); + assert.equal(await getLink(one2), null); + assert.equal(await getLink(two), null); + assert.equal(await mappingOf(one2, 'a', 'm3'), null); + assert.equal(db.prepare('SELECT COUNT(*) AS n FROM bridge_messages').get().n, 0); + + // purgeGuild does the same, and clears the unaccepted requests of the server too + const three = await createLink(link()); + await purgeGuild(D); + assert.equal(await getLink(three), null); + await createCode({ guildId: D, channelId: CH.d1 }); + await purgeGuild(D); + assert.equal(db.prepare('SELECT COUNT(*) AS n FROM bridge_codes WHERE guild_id = ?').get(D).n, 0); +}); + +test('a webhook token is stored sealed and read back whole; a legacy plain value and an unreadable one are handled', async () => { + const { seal } = await import('../src/lib/seal.js'); + const id = await createLink(link()); + const row = db.prepare('SELECT hook_a_token, hook_b_token FROM bridge_links WHERE id = ?').get(id); + for (const [stored, token] of [ + [row.hook_a_token, 'secret-a'], + [row.hook_b_token, 'secret-b'], + ]) { + assert.ok(stored.startsWith('v1.')); + assert.ok(!stored.includes(token), 'the plain token is not in the column'); + } + const read = await getLink(id); + assert.deepEqual([read.hookAToken, read.hookBToken], ['secret-a', 'secret-b']); + + await setLinkHook(id, 'a', { id: 'hook-new', token: 'rotated' }); + assert.ok( + db.prepare('SELECT hook_a_token FROM bridge_links WHERE id = ?').get(id).hook_a_token.startsWith('v1.') + ); + assert.equal((await getLink(id)).hookAToken, 'rotated'); + + db.prepare('UPDATE bridge_links SET hook_a_token = ? WHERE id = ?').run('legacy-plain-token', id); + assert.equal( + (await getLink(id)).hookAToken, + 'legacy-plain-token', + 'a value from before sealing reads as it is' + ); + assert.ok( + db.prepare('SELECT hook_a_token FROM bridge_links WHERE id = ?').get(id).hook_a_token.startsWith('v1.'), + 'and is sealed in place by that first read' + ); + assert.equal((await getLink(id)).hookAToken, 'legacy-plain-token', 'and still reads whole once sealed'); + db.prepare('UPDATE bridge_links SET hook_a_token = ? WHERE id = ?').run( + seal('some other secret', 'bridge-webhook', 'x'), + id + ); + assert.equal((await getLink(id)).hookAToken, '', 'sealed with another secret: empty, not a crash'); + db.prepare('UPDATE bridge_links SET hook_a_token = ? WHERE id = ?').run( + seal(process.env.SESSION_SECRET ?? 'x', 'pigeon-credential', 'x'), + id + ); + assert.equal((await getLink(id)).hookAToken, '', 'another purpose never opens it'); +}); + +test('the shared seal helper keeps the Pigeon credential format (v1.iv.tag.data) and rejects damage', async () => { + const { seal, open, isSealed } = await import('../src/lib/seal.js'); + const sealed = seal('secret', 'pigeon-credential', 'credential'); + assert.match(sealed, /^v1\.[\w-]+\.[\w-]+\.[\w-]+$/); + assert.ok(isSealed(sealed) && !isSealed('plain') && !isSealed(null)); + assert.equal(open('secret', 'pigeon-credential', sealed), 'credential'); + assert.equal(open('other', 'pigeon-credential', sealed), null); + assert.equal(open('secret', 'bridge-webhook', sealed), null); + assert.equal(open('secret', 'pigeon-credential', `${sealed}x`), null); + assert.equal(open('secret', 'pigeon-credential', 'v1.a.b'), null); +}); + +test('plain tokens are sealed on first read through every way a link is read', async () => { + const id = await createLink(link()); + const plain = (column, value) => + db.prepare(`UPDATE bridge_links SET ${column} = ? WHERE id = ?`).run(value, id); + const stored = (column) => db.prepare(`SELECT ${column} AS v FROM bridge_links WHERE id = ?`).get(id).v; + plain('hook_a_token', 'plain-a'); + plain('hook_b_token', 'plain-b'); + const [found] = await linksOfGuild(D); + assert.deepEqual([found.hookAToken, found.hookBToken], ['plain-a', 'plain-b']); + assert.ok(stored('hook_a_token').startsWith('v1.') && stored('hook_b_token').startsWith('v1.')); + plain('hook_a_token', 'plain-again'); + assert.equal((await linkForChannel(D, CH.d1)).link.hookAToken, 'plain-again'); + assert.ok(stored('hook_a_token').startsWith('v1.')); + plain('hook_b_token', 'plain-hook'); + assert.equal((await linkForHook('hook-b')).hookBToken, 'plain-hook'); + assert.ok(stored('hook_b_token').startsWith('v1.')); +}); + +test('the owners of both servers are recorded with the link, and a code remembers who asked', async () => { + const { setLinkOwner } = await import('../src/db/bridge.js'); + const id = await createLink(link({ ownerA: 'owner-1', ownerB: 'owner-2' })); + let row = await getLink(id); + assert.deepEqual([row.ownerA, row.ownerB], ['owner-1', 'owner-2']); + await setLinkOwner(id, 'b', 'owner-3'); + row = await getLink(id); + assert.deepEqual([row.ownerA, row.ownerB], ['owner-1', 'owner-3']); + const made = await createCode({ guildId: D, channelId: CH.d2, guildOwner: 'owner-1' }); + assert.equal((await peekCode(made.code)).guildOwner, 'owner-1'); + assert.equal((await claimCode(made.code)).guildOwner, 'owner-1'); +}); diff --git a/test/bridge.engine.test.js b/test/bridge.engine.test.js new file mode 100644 index 00000000..264c2116 --- /dev/null +++ b/test/bridge.engine.test.js @@ -0,0 +1,1833 @@ +// The Bridge engine (src/modules/bridge.js): the consent flow, and everything a message meets on its way +// across: loops, pings, bots, the age rule, replies, edits and deletes, files, emoji, stickers, embeds, +// polls, join notices, the rate limit and what a failing link does. Platforms and webhooks are fakes +// (helpers/fakeBridge.js); the HTTP routes are in test/v2Bridge.test.js. +import './helpers/tmpDb.js'; +import test, { afterEach, beforeEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { setGuildModule } from '../src/db/modules.js'; +import { purgeGuild } from '../src/db/purge.js'; +import { listAudit } from '../src/db/audit.js'; +import * as store from '../src/db/bridge.js'; +import { db } from '../src/db/index.js'; +import { config } from '../src/config.js'; +import { seal } from '../src/lib/seal.js'; +import { + BridgeError, + FAIL_THRESHOLD, + acceptLink, + assertActorMayUse, + failLink, + bridgeIdle, + bridgeProblems, + cancelRequest, + dropBridgeCache, + handleBulkDelete, + handleChannelDelete, + handleDelete, + handleJoin, + handleMessage, + handleUpdate, + sweepOwners, + linkView, + pauseLink, + requestLink, + resumeLink, + setSideOptions, + unlinkLink, + useBridgeParts, +} from '../src/modules/bridge.js'; +import { createRelayQueue } from '../src/modules/lib/bridgeQueue.js'; +import { jumpUrl } from '../src/modules/quoteBoard.js'; +import { + D, + F, + ME_D, + apiError, + installPlatforms, + message, + removePlatforms, + snowflake, + useFakes, +} from './helpers/fakeBridge.js'; + +let p; // the platforms +let fakes; // transport, sleeps, downloads + +beforeEach(async () => { + p = installPlatforms(); + // Links left by the previous test are purged first (that talks to the webhook transport), then the + // fakes start empty for this test. + useFakes(); + dropBridgeCache(); + for (const key of [D, F]) await purgeGuild(key); + fakes = useFakes(); + await setGuildModule(D, 'bridge', { enabled: true }); + await setGuildModule(F, 'bridge', { enabled: true }); +}); +afterEach(() => { + useBridgeParts(); + removePlatforms(); +}); + +/** A bridged pair of channels, made through the real request and accept calls. */ +async function bridged({ nsfw = false, optionsD, optionsF } = {}) { + const dCh = p.channel(p.dGuild, { name: 'discord-chat', nsfw }); + const fCh = p.channel(p.fGuild, { name: 'fluxer-chat', nsfw }); + const made = await requestLink({ + guildKey: D, + channelId: dCh.id, + options: optionsD, + actor: 'Alice (dashboard)', + }); + const link = await acceptLink({ + guildKey: F, + channelId: fCh.id, + code: made.code, + options: optionsF, + actor: 'Bob (dashboard)', + }); + return { dCh, fCh, link, code: made.code }; +} +const say = (channel) => channel.said.map((s) => s.content); +const fromD = (dCh, over = {}) => message({ channelId: dCh.id, client: p.discord, ...over }); +const fromF = (fCh, over = {}) => message({ channelId: fCh.id, client: p.fluxer, ...over }); +const relay = async (msg, key) => { + await handleMessage(msg, {}, key); + await bridgeIdle(); +}; +const refuses = async (promise, pattern, status) => { + await assert.rejects(promise, (err) => { + assert.ok(err instanceof BridgeError, `a BridgeError, got ${err}`); + assert.match(err.message, pattern); + if (status) assert.equal(err.status, status); + return true; + }); +}; + +// --- the consent flow ------------------------------------------------------------------------------------ + +test('a request gives a one-time code; accepting it in the other server links the two channels', async () => { + const dCh = p.channel(p.dGuild, { name: 'discord-chat' }); + const fCh = p.channel(p.fGuild, { name: 'fluxer-chat' }); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'Alice (dashboard)' }); + assert.match(made.code, /^[A-Z2-9]{4}-[A-Z2-9]{4}-[A-Z2-9]{4}$/); + assert.equal((await store.pendingCodes(D)).length, 1); + assert.equal( + await store.linkForChannel(D, dCh.id), + null, + 'nothing is linked until the other server accepts' + ); + assert.equal(fakes.transport.hooks.length, 0, 'and no webhook exists yet'); + + const link = await acceptLink({ + guildKey: F, + channelId: fCh.id, + code: made.code, + actor: 'Bob (dashboard)', + }); + assert.deepEqual( + [link.guildA, link.channelA, link.guildB, link.channelB, link.status], + [D, dCh.id, F, fCh.id, 'active'] + ); + assert.deepEqual([link.requestedBy, link.acceptedBy], ['Alice (dashboard)', 'Bob (dashboard)']); + assert.deepEqual( + fakes.transport.hooks.map((h) => [h.guildKey, h.channelId]), + [ + [D, dCh.id], + [F, fCh.id], + ], + 'one webhook in each channel' + ); + assert.ok(link.hookAId && link.hookAToken && link.hookBId && link.hookBToken); + assert.equal((await store.pendingCodes(D)).length, 0, 'the request is used up'); + await refuses( + acceptLink({ guildKey: F, channelId: p.channel(p.fGuild, {}).id, code: made.code, actor: 'Eve' }), + /not valid, has expired or was already used/, + 404 + ); + + // both channels show a visible notice, and both servers' audit logs record it + assert.match(say(dCh)[0], /Bridge on.*#fluxer-chat.*Fluxer Home.*Fluxer.*Nobody is pinged/s); + assert.match(say(fCh)[0], /Bridge on.*#discord-chat.*Discord Home.*Discord.*Nobody is pinged/s); + assert.deepEqual(dCh.said[0].allowedMentions, { parse: [] }, 'the notice pings nobody either'); + assert.ok((await listAudit(D)).some((e) => e.action === 'bridge:link')); + assert.ok((await listAudit(F)).some((e) => e.action === 'bridge:link')); + assert.ok((await listAudit(D)).some((e) => e.action === 'bridge:request')); +}); + +test('a server cannot accept its own request, nor link two servers of the same platform', async () => { + const dCh = p.channel(p.dGuild, {}); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'Alice' }); + await refuses( + acceptLink({ guildKey: D, channelId: p.channel(p.dGuild, {}).id, code: made.code, actor: 'Alice' }), + /made in this server/, + 409 + ); + const { dGuild: d2 } = p.extra('91'); + const d2Key = d2.id; + await setGuildModule(d2Key, 'bridge', { enabled: true }); + await refuses( + acceptLink({ guildKey: d2Key, channelId: p.channel(d2, {}).id, code: made.code, actor: 'Eve' }), + /Discord channel to a Fluxer channel/ + ); + assert.equal((await store.pendingCodes(D)).length, 1, 'neither attempt used the code up'); +}); + +test('a wrong, malformed, expired or cancelled code is refused the same way', async () => { + const fCh = p.channel(p.fGuild, {}); + const dCh = p.channel(p.dGuild, {}); + const accept = (code) => acceptLink({ guildKey: F, channelId: fCh.id, code, actor: 'Bob' }); + await refuses(accept('ABCD-EFGH-JKMN'), /not valid, has expired or was already used/, 404); + await refuses(accept('hello'), /not valid, has expired or was already used/, 404); + await refuses(accept(''), /not valid, has expired or was already used/, 404); + const expired = await store.createCode({ guildId: D, channelId: dCh.id, ttlMs: -1000 }); + await refuses(accept(expired.code), /not valid, has expired or was already used/, 404); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'Alice' }); + const [pending] = await store.pendingCodes(D); + await cancelRequest({ guildKey: D, id: pending.id, actor: 'Alice' }); + await refuses(accept(made.code), /not valid, has expired or was already used/, 404); + await refuses(cancelRequest({ guildKey: D, id: pending.id, actor: 'Alice' }), /gone already/, 404); + assert.equal(fakes.transport.hooks.length, 0, 'no wrong code made a webhook'); +}); + +test("only the requesting server may cancel a request, and no server sees another's", async () => { + const dCh = p.channel(p.dGuild, {}); + await requestLink({ guildKey: D, channelId: dCh.id, actor: 'Alice' }); + const [pending] = await store.pendingCodes(D); + await refuses(cancelRequest({ guildKey: F, id: pending.id, actor: 'Eve' }), /gone already/, 404); + assert.equal((await store.pendingCodes(D)).length, 1); +}); + +test('both servers need the module on, and Sylo needs the permissions in the channel', async () => { + const dCh = p.channel(p.dGuild, {}); + const fCh = p.channel(p.fGuild, {}); + await setGuildModule(D, 'bridge', { enabled: false }); + await refuses( + requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }), + /Turn the Bridge module on/, + 409 + ); + await setGuildModule(D, 'bridge', { enabled: true }); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + + await setGuildModule(F, 'bridge', { enabled: false }); + await refuses( + acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }), + /Turn the Bridge module on/, + 409 + ); + await setGuildModule(F, 'bridge', { enabled: true }); + + await setGuildModule(D, 'bridge', { enabled: false }); + await refuses( + acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }), + /turned its Bridge module off/, + 409 + ); + await setGuildModule(D, 'bridge', { enabled: true }); + + const locked = p.channel(p.fGuild, { name: 'locked', deny: ['ManageWebhooks'] }); + await refuses( + acceptLink({ guildKey: F, channelId: locked.id, code: made.code, actor: 'B' }), + /needs ManageWebhooks in #locked/ + ); + assert.equal(fakes.transport.hooks.length, 0); + assert.ok( + await acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }), + 'the code still worked' + ); +}); + +test('only a normal text channel of the same server can be bridged', async () => { + const fCh = p.channel(p.fGuild, {}); + await refuses( + requestLink({ guildKey: D, channelId: fCh.id, actor: 'A' }), + /normal text channel of this server/ + ); + await refuses( + requestLink({ guildKey: D, channelId: snowflake(), actor: 'A' }), + /normal text channel of this server/ + ); + const voice = p.channel(p.dGuild, { type: 2 }); + await refuses( + requestLink({ guildKey: D, channelId: voice.id, actor: 'A' }), + /normal text channel of this server/ + ); +}); + +test('a channel can be in one link or one request only', async () => { + const { dCh } = await bridged(); + await refuses(requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }), /already bridged/, 409); + const other = p.channel(p.dGuild, {}); + await requestLink({ guildKey: D, channelId: other.id, actor: 'A' }); + await refuses(requestLink({ guildKey: D, channelId: other.id, actor: 'A' }), /request waiting/, 409); + const fSecond = p.channel(p.fGuild, {}); + const made = await requestLink({ guildKey: D, channelId: p.channel(p.dGuild, {}).id, actor: 'A' }); + const fCh = [...p.fGuild.channels.cache.values()][0]; + await refuses( + acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }), + /already bridged/, + 409 + ); + assert.ok(await acceptLink({ guildKey: F, channelId: fSecond.id, code: made.code, actor: 'B' })); +}); + +test('an age-restricted channel only links to an age-restricted one, in both directions', async () => { + const mk = async (dNsfw, fNsfw) => { + const dCh = p.channel(p.dGuild, { nsfw: dNsfw }); + const fCh = p.channel(p.fGuild, { nsfw: fNsfw }); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + return () => acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }); + }; + await refuses( + (await mk(true, false))(), + /age-restricted channel can only be bridged with another age-restricted/ + ); + await refuses( + (await mk(false, true))(), + /age-restricted channel can only be bridged with another age-restricted/ + ); + assert.equal(fakes.transport.hooks.length, 0, 'refused before any webhook was made'); + assert.ok(await (await mk(true, true))(), 'two age-restricted channels link'); + assert.ok(await (await mk(false, false))(), 'two ordinary channels link'); +}); + +test('a webhook that cannot be made refuses the link without using the code up', async () => { + const dCh = p.channel(p.dGuild, {}); + const fCh = p.channel(p.fGuild, {}); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + fakes.transport.hookFailures.push(null, apiError(403)); + await refuses( + acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }), + /could not make a webhook/, + 502 + ); + assert.equal(fakes.transport.deleted.length, 1, 'the webhook that was made is removed again'); + assert.equal((await store.pendingCodes(D)).length, 1); + assert.ok(await acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' })); +}); + +test('two servers that accept the same code at once: one wins, the other leaves nothing behind', async () => { + const dCh = p.channel(p.dGuild, {}); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + const f1 = p.channel(p.fGuild, {}); + const f2 = p.channel(p.fGuild, {}); + const results = await Promise.allSettled([ + acceptLink({ guildKey: F, channelId: f1.id, code: made.code, actor: 'B' }), + acceptLink({ guildKey: F, channelId: f2.id, code: made.code, actor: 'C' }), + ]); + assert.equal(results.filter((r) => r.status === 'fulfilled').length, 1); + assert.equal((await store.linksOfGuild(D)).length, 1); + assert.equal( + fakes.transport.hooks.length - fakes.transport.deleted.length, + 2, + 'only the winner keeps its two webhooks' + ); +}); + +// --- relaying a message ---------------------------------------------------------------------------------- + +test('a message is posted on the other side under the author name and platform, never with a ping', async () => { + const { dCh, fCh, link } = await bridged(); + await relay(fromD(dCh, { content: 'Hello from Discord' }), D); + assert.equal(fakes.transport.sends.length, 1); + const sent = fakes.transport.sends[0]; + assert.equal(sent.guildKey, F, 'sent into the Fluxer server'); + assert.deepEqual(sent.hook, { id: link.hookBId, token: link.hookBToken, channelId: fCh.id }); + assert.equal(sent.payload.content, 'Hello from Discord'); + assert.equal(sent.payload.username, 'Alice [Discord]'); + + await relay(fromF(fCh, { content: 'Hello back', member: { displayName: 'Bob' } }), F); + const back = fakes.transport.sends[1]; + assert.equal(back.guildKey, D); + assert.equal(back.hook.id, link.hookAId); + assert.equal(back.payload.username, 'Bob [Fluxer]'); +}); + +test('mentions are shown as names and @everyone and @here cannot ping: users, roles, channels, commands', async () => { + const { dCh } = await bridged(); + const msg = fromD(dCh, { + content: + '@everyone @here <@111111111111111111> <@!111111111111111111> <@&222222222222222222> <#333333333333333333> hi', + mentions: { + users: new Map([ + ['111111111111111111', { id: '111111111111111111', username: 'bob', displayName: 'Bobby' }], + ]), + members: new Map(), + roles: new Map([['222222222222222222', { id: '222222222222222222', name: 'moderators' }]]), + }, + }); + p.dGuild.channels.cache.set('333333333333333333', { id: '333333333333333333', name: 'rules' }); + await relay(msg, D); + const text = fakes.transport.sends[0].payload.content; + assert.ok(!/<[@#&]/.test(text), `no raw mention tokens left: ${text}`); + assert.ok(!/@everyone|@here/.test(text), 'a plain @everyone or @here is broken by an invisible character'); + assert.match(text, /@.everyone @.here @.Bobby @.Bobby @.moderators #rules \/ban hi/); +}); + +test('custom emoji map to the same name on the other side, else become :name:; Unicode crosses as it is', async () => { + const { dCh } = await bridged(); + p.fGuild.emojis.cache.set('555555555555555555', { + id: '555555555555555555', + name: 'Party', + animated: true, + }); + await relay(fromD(dCh, { content: 'hi 🎉 <:party:123456789012345678> <:nope:123456789012345679>' }), D); + assert.equal(fakes.transport.sends[0].payload.content, 'hi 🎉 :nope:'); +}); + +test('a reply carries a short quote of the original and a link to its copy when it is mapped', async () => { + const { dCh, fCh } = await bridged(); + const original = fromF(fCh, { content: 'Who wants pizza?', member: { displayName: 'Bob' } }); + await relay(original, F); + const copyOnD = fakes.transport.sends[0].id; + + const reply = fromD(dCh, { + content: 'Me!', + reference: { messageId: copyOnD }, + referencedMessage: { content: 'Who wants pizza?', author: { username: 'Bob [Fluxer]' } }, + }); + await relay(reply, D); + const content = fakes.transport.sends[1].payload.content; + const url = jumpUrl(F, fCh.id, original.id); + assert.equal(content, `> ↪ [**Bob**](<${url}>): Who wants pizza?\nMe!`); + + const unmapped = fromD(dCh, { + content: 'Hm', + reference: { messageId: snowflake() }, + referencedMessage: { + content: 'old message', + author: { username: 'carol' }, + member: { displayName: 'Carol' }, + }, + }); + await relay(unmapped, D); + assert.equal( + fakes.transport.sends[2].payload.content, + '> ↪ **Carol**: old message\nHm', + 'quoted, with no link' + ); +}); + +test('a quote of a message with pings is neutralised too', async () => { + const { dCh, fCh } = await bridged(); + const reply = fromF(fCh, { + content: 'ok', + messageReference: { messageId: snowflake() }, + referencedMessage: { content: '@everyone look', author: { username: 'x' } }, + }); + await relay(reply, F); + assert.match(fakes.transport.sends[0].payload.content, /^> ↪ \*\*x\*\*: @.everyone look\nok$/); + assert.equal(fakes.transport.sends[0].guildKey, D); + assert.ok(dCh); +}); + +// --- loops, bots and what is not chat ------------------------------------------------------------------------ + +test('what a bridge webhook posted is never relayed again, so a copy cannot come back', async () => { + const { dCh, fCh, link } = await bridged(); + await relay(fromD(dCh, { content: 'ping' }), D); + assert.equal(fakes.transport.sends.length, 1); + // The copy arrives in the Fluxer channel as a message from the webhook Sylo made there. + const copy = fromF(fCh, { + content: 'ping', + webhookId: link.hookBId, + author: { bot: true, username: 'Alice [Discord]' }, + }); + await relay(copy, F); + // Even with bots switched on for that side, and even from the OTHER webhook of the link. + await setSideOptions({ guildKey: F, id: link.id, options: { bots: true }, actor: 'B' }); + await relay(copy, F); + await relay(fromD(dCh, { content: 'x', webhookId: link.hookAId, author: { bot: true } }), D); + assert.equal(fakes.transport.sends.length, 1, 'nothing went back'); +}); + +test("Sylo's own messages and messages from any Sylo bridge webhook are not relayed, even with bots on", async () => { + const { dCh, link } = await bridged({ optionsD: { bots: true } }); + await relay(fromD(dCh, { content: 'welcome!', author: { id: ME_D, bot: true } }), D); + // another link's webhook posting into this channel + const otherD = p.channel(p.dGuild, {}); + const otherF = p.channel(p.fGuild, {}); + const made = await requestLink({ guildKey: D, channelId: otherD.id, actor: 'A' }); + const second = await acceptLink({ guildKey: F, channelId: otherF.id, code: made.code, actor: 'B' }); + await relay(fromD(dCh, { content: 'stray', webhookId: second.hookAId, author: { bot: true } }), D); + assert.equal(fakes.transport.sends.length, 0); + assert.ok(link); +}); + +test('bots are off by default and only cross when that side turns them on', async () => { + const { dCh, link } = await bridged(); + const bot = () => fromD(dCh, { content: 'beep', member: null, author: { bot: true, username: 'RoleBot' } }); + const foreignHook = () => + fromD(dCh, { + content: 'ci done', + member: null, + webhookId: snowflake(), + author: { bot: true, username: 'CI' }, + }); + await relay(bot(), D); + await relay(foreignHook(), D); + assert.equal(fakes.transport.sends.length, 0, 'off by default'); + await setSideOptions({ guildKey: D, id: link.id, options: { bots: true }, actor: 'A' }); + await relay(bot(), D); + await relay(foreignHook(), D); + assert.equal(fakes.transport.sends.length, 1, 'a bot account crosses; a webhook message never does'); + assert.equal(fakes.transport.sends[0].payload.username, 'RoleBot [Discord]'); +}); + +test("each side's options only change what that side sends", async () => { + const { dCh, fCh, link } = await bridged({ optionsD: { bots: true } }); + await relay(fromD(dCh, { content: 'a', author: { bot: true } }), D); + await relay(fromF(fCh, { content: 'b', author: { bot: true } }), F); + assert.equal(fakes.transport.sends.length, 1, 'the Fluxer side never turned bots on'); + const view = linkView(await store.getLink(link.id), F); + assert.deepEqual([view.options.bots, view.theirOptions.bots], [false, true]); +}); + +test('system messages, empty messages and threads are not relayed', async () => { + const { dCh } = await bridged(); + await relay(fromD(dCh, { content: 'x joined', type: 7, system: true }), D); + await relay(fromD(dCh, { content: '' }), D); + await relay(message({ channelId: snowflake(), client: p.discord, content: 'in a thread' }), D); + assert.equal(fakes.transport.sends.length, 0); +}); + +test('nothing is relayed while the other server has the module off, or the link is paused', async () => { + const { dCh, link } = await bridged(); + await setGuildModule(F, 'bridge', { enabled: false }); + await relay(fromD(dCh, { content: 'one' }), D); + await setGuildModule(F, 'bridge', { enabled: true }); + await pauseLink({ guildKey: F, id: link.id, actor: 'B' }); + await relay(fromD(dCh, { content: 'two' }), D); + assert.equal(fakes.transport.sends.length, 0); + await resumeLink({ guildKey: F, id: link.id, actor: 'B' }); + await relay(fromD(dCh, { content: 'three' }), D); + assert.equal(fakes.transport.sends.length, 1); +}); + +test('the age rule is checked on every message: a channel that changes pauses the link', async () => { + const { dCh, link } = await bridged({ nsfw: true }); + await relay(fromD(dCh, { content: 'fine' }), D); + assert.equal(fakes.transport.sends.length, 1); + dCh.nsfw = false; // the Discord channel is made ordinary + await relay(fromD(dCh, { content: 'now what' }), D); + assert.equal(fakes.transport.sends.length, 1, 'not relayed'); + const row = await store.getLink(link.id); + assert.deepEqual([row.status, row.failReason], ['failed', 'age-restriction']); + assert.ok( + say(dCh).some((s) => /Bridge stopped/.test(s)), + 'and said so in the channel' + ); +}); + +// --- edits and deletes ------------------------------------------------------------------------------------- + +test('an edit is mirrored on the copy; an edit that changes nothing visible is not', async () => { + const { dCh } = await bridged(); + const original = fromD(dCh, { content: 'first' }); + await relay(original, D); + const copyId = fakes.transport.sends[0].id; + + await handleUpdate( + { old: original, new: { ...original, content: 'second <@123456789012345678> @everyone' } }, + {}, + D + ); + await bridgeIdle(); + assert.equal(fakes.transport.edits.length, 1); + const edit = fakes.transport.edits[0]; + assert.equal(edit.messageId, copyId); + assert.match(edit.payload.content, /^second @.user @.everyone$/); + + // a link preview appearing fires an update with the same text: nothing to do + await handleUpdate( + { + old: original, + new: { + ...original, + content: 'second <@123456789012345678> @everyone', + embeds: [{ type: 'link', url: 'https://x' }], + }, + }, + {}, + D + ); + await bridgeIdle(); + assert.equal(fakes.transport.edits.length, 1); +}); + +test('an edit keeps the quote and the file links of the original copy', async () => { + const { dCh, fCh } = await bridged(); + const original = fromF(fCh, { content: 'pizza?', member: { displayName: 'Bob' } }); + await relay(original, F); + const reply = fromD(dCh, { + content: 'yes', + reference: { messageId: fakes.transport.sends[0].id }, + referencedMessage: { content: 'pizza?', author: { username: 'Bob [Fluxer]' } }, + attachments: new Map([['a', { name: 'big.zip', url: 'https://cdn/big.zip', size: 50 * 1024 * 1024 }]]), + }); + await relay(reply, D); + const first = fakes.transport.sends[1].payload.content; + await handleUpdate({ old: reply, new: { ...reply, content: 'yes please' } }, {}, D); + await bridgeIdle(); + const edited = fakes.transport.edits[0].payload.content; + assert.match(first, /^> ↪ .*\nyes\n📎 big.zip: https:\/\/cdn\/big.zip$/s); + assert.match(edited, /^> ↪ .*\nyes please\n📎 big.zip: https:\/\/cdn\/big.zip$/s); +}); + +test('only the original is mirrored: an edit of a copy, or of a message that was never relayed, does nothing', async () => { + const { dCh, fCh } = await bridged(); + await relay(fromD(dCh, { content: 'one' }), D); + const copyOnF = { + id: fakes.transport.sends[0].id, + channelId: fCh.id, + content: 'tampered', + client: p.fluxer, + webhookId: 'x', + }; + await handleUpdate({ old: copyOnF, new: copyOnF }, {}, F); + await handleUpdate({ old: null, new: fromD(dCh, { content: 'never relayed' }) }, {}, D); + await bridgeIdle(); + assert.equal(fakes.transport.edits.length, 0); +}); + +test('a delete is mirrored through the map, also for a purge of many; deleting a copy leaves the original', async () => { + const { dCh, fCh } = await bridged(); + const m1 = fromD(dCh, { content: 'one' }); + const m2 = fromD(dCh, { content: 'two' }); + const m3 = fromD(dCh, { content: 'three' }); + for (const m of [m1, m2, m3]) await relay(m, D); + const [c1, c2, c3] = fakes.transport.sends.map((s) => s.id); + + await handleDelete({ id: m1.id, channelId: dCh.id }, {}, D); + await bridgeIdle(); + assert.deepEqual( + fakes.transport.removes.map((r) => r.messageId), + [c1] + ); + assert.equal(fakes.transport.removes[0].guildKey, F); + assert.equal( + await store.mappingOf((await store.linksOfGuild(D))[0].id, 'a', m1.id), + null, + 'the map row is gone' + ); + + await handleBulkDelete( + { + messages: new Map([ + [m2.id, {}], + [m3.id, {}], + ]), + channel: dCh, + }, + {}, + D + ); + await bridgeIdle(); + assert.deepEqual( + fakes.transport.removes.map((r) => r.messageId), + [c1, c2, c3] + ); + + // someone deletes a copy on the Fluxer side: the original on Discord stays + const m4 = fromD(dCh, { content: 'four' }); + await relay(m4, D); + await handleDelete({ id: fakes.transport.sends[3].id, channelId: fCh.id }, {}, F); + await bridgeIdle(); + assert.equal(fakes.transport.removes.length, 3, 'nothing else was deleted'); + await handleDelete({ id: snowflake(), channelId: dCh.id }, {}, D); + assert.equal(fakes.transport.removes.length, 3, 'an unknown message is ignored'); +}); + +test('a message older than the map (30 days) is no longer followed', async () => { + const { dCh, link } = await bridged(); + const m = fromD(dCh, { content: 'old' }); + await relay(m, D); + await store.sweepMappings(Date.now() + 31 * 86_400_000); + assert.equal(await store.mappingOf(link.id, 'a', m.id), null); + await handleDelete({ id: m.id, channelId: dCh.id }, {}, D); + await handleUpdate({ old: m, new: { ...m, content: 'edited' } }, {}, D); + await bridgeIdle(); + assert.equal(fakes.transport.removes.length + fakes.transport.edits.length, 0); +}); + +// --- files, stickers, embeds, polls, joins ----------------------------------------------------------------- + +test('a small file is re-uploaded, a big one or one that cannot be fetched becomes a link', async () => { + const MB = 1024 * 1024; + fakes = useFakes({ files: { 'https://cdn/cat.png': 'PNGDATA' } }); + const { dCh } = await bridged(); + const att = (name, size, url = `https://cdn/${name}`) => [name, { name, url, size }]; + await relay( + fromD(dCh, { + content: 'files', + attachments: new Map([att('cat.png', 7), att('movie.mp4', 20 * MB), att('gone.png', 100)]), + }), + D + ); + const { payload } = fakes.transport.sends[0]; + assert.deepEqual( + payload.files.map((f) => [f.name, f.data.toString()]), + [['cat.png', 'PNGDATA']] + ); + assert.equal( + payload.content, + 'files\n📎 movie.mp4: https://cdn/movie.mp4\n📎 gone.png: https://cdn/gone.png' + ); + assert.deepEqual( + fakes.downloads.map((d) => d.url), + ['https://cdn/cat.png', 'https://cdn/gone.png'], + 'the big one was never fetched' + ); +}); + +test('a server can send every file as a link instead of copying it', async () => { + fakes = useFakes({ files: { 'https://cdn/cat.png': 'PNGDATA' } }); + const { dCh } = await bridged({ optionsD: { uploadFiles: false } }); + await relay( + fromD(dCh, { + content: '', + attachments: new Map([['a', { name: 'cat.png', url: 'https://cdn/cat.png', size: 7 }]]), + }), + D + ); + assert.equal(fakes.downloads.length, 0); + assert.equal(fakes.transport.sends[0].payload.files, undefined); + assert.equal(fakes.transport.sends[0].payload.content, '📎 cat.png: https://cdn/cat.png'); +}); + +test('stickers become a picture link, a poll becomes readable text, and link previews are left out', async () => { + const { dCh } = await bridged({ optionsD: { bots: true } }); + await relay( + fromD(dCh, { + content: 'look', + stickers: new Map([['s', { name: 'wave', url: 'https://cdn/stickers/1.png' }]]), + poll: { + question: { text: 'Pizza or pasta?' }, + answers: new Map([ + [1, { text: 'Pizza', emoji: { name: '🍕' } }], + [2, { text: 'Pasta' }], + ]), + }, + embeds: [{ type: 'link', url: 'https://example.com', title: 'A preview' }], + }), + D + ); + const { content, embeds } = fakes.transport.sends[0].payload; + assert.equal( + content, + 'look\n*sticker: wave* https://cdn/stickers/1.png\n📊 **Poll:** Pizza or pasta?\n• 🍕 Pizza\n• Pasta\n*Vote on the original server: polls do not cross the bridge.*' + ); + assert.equal(embeds, undefined, 'a link preview is the other platform’s own job'); +}); + +test('a rich embed from a bot crosses with its text made harmless', async () => { + const { dCh } = await bridged({ optionsD: { bots: true } }); + await relay( + fromD(dCh, { + content: '', + author: { bot: true, username: 'Deploy' }, + embeds: [ + { + type: 'rich', + title: 'Build done @everyone', + description: 'by <@123456789012345678>', + color: 0x00ff00, + fields: [{ name: 'Env', value: 'prod', inline: true }], + footer: { text: 'at ' }, + }, + ], + }), + D + ); + const [embed] = fakes.transport.sends[0].payload.embeds; + assert.match(embed.title, /^Build done @.everyone$/); + assert.match(embed.description, /^by @.user$/); + assert.equal(embed.color, 0x00ff00); + assert.deepEqual(embed.fields, [{ name: 'Env', value: 'prod', inline: true }]); + assert.match(embed.footer.text, /^at 14 Nov 2023, 22:13 UTC$/); +}); + +test('join notices are off until a server turns them on, and go to the other side only', async () => { + const { dCh, fCh, link } = await bridged(); + const member = { displayName: 'Zed', user: { bot: false } }; + await handleJoin(member, {}, D); + await bridgeIdle(); + assert.equal(fakes.transport.sends.length, 0); + await setSideOptions({ guildKey: D, id: link.id, options: { joinMessages: true }, actor: 'A' }); + await handleJoin(member, {}, D); + await handleJoin({ displayName: 'AutoBot', user: { bot: true } }, {}, D); + await handleJoin(member, {}, F); + await bridgeIdle(); + assert.equal( + fakes.transport.sends.length, + 1, + 'the Fluxer side never turned it on, and bots need their option' + ); + const sent = fakes.transport.sends[0]; + assert.equal(sent.guildKey, F); + assert.equal(sent.payload.content, '👋 **Zed** joined **Discord Home**'); + assert.equal(sent.payload.username, 'Discord Home [Discord]'); + assert.ok(dCh && fCh); +}); + +// --- the rate limit, back-off and failing links -------------------------------------------------------------- + +test('a rate limit is waited out and the message still arrives, in order', async () => { + const { dCh } = await bridged(); + fakes.transport.sendFailures.push(apiError(429, { retryAfter: 1.5 })); + await handleMessage(fromD(dCh, { content: 'first' }), {}, D); + await handleMessage(fromD(dCh, { content: 'second' }), {}, D); + await bridgeIdle(); + assert.deepEqual( + fakes.transport.sends.map((s) => s.payload.content), + ['first', 'second'] + ); + assert.deepEqual(fakes.sleeps, [1500], 'it waited exactly what the platform asked'); + assert.equal(fakes.transport.attempts, 3); +}); + +test('a short outage backs off and recovers, and the failure count starts over', async () => { + const { dCh, link } = await bridged(); + fakes.transport.sendFailures.push(apiError(500), apiError(502)); + await relay(fromD(dCh, { content: 'try' }), D); + assert.equal(fakes.transport.sends.length, 1); + assert.deepEqual(fakes.sleeps, [1000, 2000], 'a growing pause'); + assert.equal((await store.getLink(link.id)).status, 'active'); +}); + +test(`a link whose deliveries fail ${FAIL_THRESHOLD} times in a row is stopped, announced and not retried`, async () => { + const { dCh, fCh, link } = await bridged(); + for (let i = 0; i < 4 * FAIL_THRESHOLD; i += 1) fakes.transport.sendFailures.push(apiError(500)); + for (let i = 0; i < FAIL_THRESHOLD; i += 1) await relay(fromD(dCh, { content: `m${i}` }), D); + const row = await store.getLink(link.id); + assert.deepEqual([row.status, row.failReason], ['failed', 'unreachable']); + assert.ok( + say(dCh).some((s) => /Bridge stopped/.test(s)) && say(fCh).some((s) => /Bridge stopped/.test(s)), + 'both channels say so' + ); + const before = fakes.transport.attempts; + await relay(fromD(dCh, { content: 'later' }), D); + assert.equal(fakes.transport.attempts, before, 'a stopped link is not tried again'); + assert.ok((await listAudit(D)).some((e) => e.action === 'bridge:stopped')); + assert.deepEqual( + (await bridgeProblems(D)).map((x) => [x.linkId, x.channelId, x.code]), + [[link.id, dCh.id, 'unreachable']], + 'and it is listed for the Health page' + ); + assert.equal(linkView(row, D).problem.code, 'unreachable'); +}); + +test('a success between failures starts the count again, so a flaky link keeps running', async () => { + const { dCh, link } = await bridged(); + for (let round = 0; round < 3; round += 1) { + for (let i = 0; i < 4 * (FAIL_THRESHOLD - 1); i += 1) fakes.transport.sendFailures.push(apiError(500)); + for (let i = 0; i < FAIL_THRESHOLD - 1; i += 1) await relay(fromD(dCh, { content: 'x' }), D); + await relay(fromD(dCh, { content: 'ok' }), D); + } + assert.equal((await store.getLink(link.id)).status, 'active', 'never five in a row'); + assert.equal(fakes.transport.sends.filter((s) => s.payload.content === 'ok').length, 3); +}); + +test('a refused webhook (permissions) stops the link at once', async () => { + const { dCh, link } = await bridged(); + fakes.transport.sendFailures.push(apiError(403)); + await relay(fromD(dCh, { content: 'x' }), D); + const row = await store.getLink(link.id); + assert.deepEqual([row.status, row.failReason], ['failed', 'permissions']); +}); + +test('a deleted webhook is made again, once, and the message still arrives', async () => { + const { dCh, link } = await bridged(); + fakes.transport.sendFailures.push(apiError(404, { code: 10015 })); + await relay(fromD(dCh, { content: 'after the webhook was deleted' }), D); + assert.equal(fakes.transport.sends.length, 1); + const row = await store.getLink(link.id); + assert.equal(row.status, 'active'); + assert.notEqual(row.hookBId, link.hookBId, 'a new webhook for the Fluxer channel'); + assert.equal(fakes.transport.sends[0].hook.id, row.hookBId); + assert.equal(fakes.transport.hooks.length, 3); + assert.deepEqual( + fakes.transport.deleted.map((d) => d.hook.id), + [link.hookBId], + 'the old one is cleaned up' + ); +}); + +test('a webhook that cannot be made again stops the link', async () => { + const { dCh, link } = await bridged(); + fakes.transport.sendFailures.push(apiError(404, { code: 10015 })); + fakes.transport.hookFailures.push(apiError(403)); + await relay(fromD(dCh, { content: 'x' }), D); + const row = await store.getLink(link.id); + assert.equal(row.status, 'failed'); + assert.equal(fakes.transport.sends.length, 0); +}); + +test('one flooded channel drops its newest messages instead of growing; other links are unaffected', async () => { + const gate = {}; + gate.open = new Promise((resolve) => { + gate.release = resolve; + }); + fakes = useFakes({ queue: createRelayQueue({ minGapMs: 0, maxQueued: 2, sleep: async () => {} }) }); + const realSend = fakes.transport.send; + let sendCalls = 0; + fakes.transport.send = async (...args) => { + sendCalls += 1; + await gate.open; + return realSend(...args); + }; + const { dCh, link } = await bridged(); + const other = p.extra('92'); + await setGuildModule(other.dGuild.id, 'bridge', { enabled: true }); + await setGuildModule(`fluxer:${other.fGuild.id}`, 'bridge', { enabled: true }); + const dCh2 = p.channel(other.dGuild, {}); + const fCh2 = p.channel(other.fGuild, {}); + const made = await requestLink({ guildKey: other.dGuild.id, channelId: dCh2.id, actor: 'A' }); + await acceptLink({ + guildKey: `fluxer:${other.fGuild.id}`, + channelId: fCh2.id, + code: made.code, + actor: 'B', + }); + + for (let i = 0; i < 6; i += 1) await handleMessage(fromD(dCh, { content: `flood ${i}` }), {}, D); + await handleMessage(fromD(dCh2, { content: 'quiet server' }), {}, other.dGuild.id); + const view = linkView(await store.getLink(link.id), D); + assert.equal(view.queued, 2, 'two wait'); + assert.equal(view.dropped, 4, 'four were dropped'); + gate.release(); + await bridgeIdle(); + const contents = fakes.transport.sends.map((s) => s.payload.content); + assert.deepEqual( + contents.filter((c) => c.startsWith('flood')), + ['flood 0', 'flood 1'] + ); + assert.ok(contents.includes('quiet server'), 'the other server was not held up by the flood'); + assert.ok(sendCalls >= 3); +}); + +test('a failing link does not touch the links of other servers', async () => { + const one = await bridged(); + const other = p.extra('93'); + const otherF = `fluxer:${other.fGuild.id}`; + await setGuildModule(other.dGuild.id, 'bridge', { enabled: true }); + await setGuildModule(otherF, 'bridge', { enabled: true }); + const dCh2 = p.channel(other.dGuild, {}); + const fCh2 = p.channel(other.fGuild, {}); + const made = await requestLink({ guildKey: other.dGuild.id, channelId: dCh2.id, actor: 'A' }); + const second = await acceptLink({ guildKey: otherF, channelId: fCh2.id, code: made.code, actor: 'B' }); + + fakes.transport.sendFailures.push(apiError(403)); + await relay(fromD(one.dCh, { content: 'breaks the first link' }), D); + assert.equal((await store.getLink(one.link.id)).status, 'failed'); + assert.equal((await store.getLink(second.id)).status, 'active'); + await relay(fromD(dCh2, { content: 'still fine' }), other.dGuild.id); + assert.equal(fakes.transport.sends.at(-1).payload.content, 'still fine'); + assert.ok(!say(dCh2).some((s) => /Bridge stopped/.test(s)), 'the other server was not told anything'); +}); + +test('a deleted bridged channel stops its link', async () => { + const { dCh, link } = await bridged(); + await handleChannelDelete(dCh, {}, D); + const row = await store.getLink(link.id); + assert.deepEqual([row.status, row.failReason], ['failed', 'channel-deleted']); +}); + +// --- pause, resume, unlink, leaving ----------------------------------------------------------------------------- + +test('a server that paused a link is the only one that can resume it; a stopped link can be resumed by either', async () => { + const { dCh, fCh, link } = await bridged(); + await pauseLink({ guildKey: D, id: link.id, actor: 'A' }); + assert.ok( + say(fCh).some((s) => /Bridge paused.*Discord Home/s.test(s)), + 'the other side is told who paused it' + ); + await refuses(resumeLink({ guildKey: F, id: link.id, actor: 'B' }), /only they can resume/, 403); + await refuses(pauseLink({ guildKey: D, id: link.id, actor: 'A' }), /not running/, 409); + const view = linkView(await store.getLink(link.id), F); + assert.deepEqual([view.status, view.pausedByMe, view.pausedByThem], ['paused', false, true]); + await resumeLink({ guildKey: D, id: link.id, actor: 'A' }); + assert.equal((await store.getLink(link.id)).status, 'active'); + assert.match(say(dCh).at(-1), /Bridge on/); + + await store.setLinkStatus(link.id, { status: 'failed', reason: 'permissions' }); + dropBridgeCache(); + await resumeLink({ guildKey: F, id: link.id, actor: 'B' }); + assert.equal((await store.getLink(link.id)).status, 'active'); + await refuses(resumeLink({ guildKey: F, id: link.id, actor: 'B' }), /already running/, 409); + assert.ok(fCh); +}); + +test('a stopped link is only resumed when its channels can be bridged again', async () => { + const { dCh, fCh, link } = await bridged(); + await store.setLinkStatus(link.id, { status: 'failed', reason: 'permissions' }); + dCh.permissionsFor = () => ({ has: (perm) => perm !== 'ManageWebhooks' }); + await refuses(resumeLink({ guildKey: D, id: link.id, actor: 'A' }), /needs ManageWebhooks/); + dCh.permissionsFor = () => ({ has: () => true }); + fCh.nsfw = true; + await refuses(resumeLink({ guildKey: D, id: link.id, actor: 'A' }), /age-restricted/); + fCh.nsfw = false; + await resumeLink({ guildKey: D, id: link.id, actor: 'A' }); + assert.equal((await store.getLink(link.id)).status, 'active'); +}); + +test('either side can unlink: both webhooks go, the rows go, both channels are told', async () => { + const { dCh, fCh, link } = await bridged(); + const m = fromD(dCh, { content: 'hi' }); + await relay(m, D); + await unlinkLink({ guildKey: F, id: link.id, actor: 'B' }); + assert.equal(await store.getLink(link.id), null); + assert.equal(await store.linkForChannel(D, dCh.id), null); + assert.deepEqual(fakes.transport.deleted.map((d) => d.hook.id).sort(), [link.hookAId, link.hookBId].sort()); + assert.match(say(dCh).at(-1), /Bridge ended.*Fluxer Home/s); + assert.match(say(fCh).at(-1), /Bridge ended/); + assert.equal(await store.mappingCount(link.id), 0, 'the message map of the link went with it'); + assert.ok((await listAudit(D)).some((e) => e.action === 'bridge:unlink')); + assert.ok((await listAudit(F)).some((e) => e.action === 'bridge:unlink')); + await relay(fromD(dCh, { content: 'after' }), D); + assert.equal(fakes.transport.sends.length, 1, 'nothing crosses any more'); + // the channels can be bridged again + const again = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + assert.ok(await acceptLink({ guildKey: F, channelId: fCh.id, code: again.code, actor: 'B' })); +}); + +test('a server never sees or changes a link it is not in', async () => { + const { link } = await bridged(); + const { dGuild: stranger } = p.extra('94'); + for (const act of [ + () => pauseLink({ guildKey: stranger.id, id: link.id, actor: 'X' }), + () => resumeLink({ guildKey: stranger.id, id: link.id, actor: 'X' }), + () => unlinkLink({ guildKey: stranger.id, id: link.id, actor: 'X' }), + () => setSideOptions({ guildKey: stranger.id, id: link.id, options: { bots: true }, actor: 'X' }), + ]) { + await refuses(act(), /No such bridged channel/, 404); + } + assert.equal((await store.getLink(link.id)).status, 'active'); + assert.deepEqual((await store.getLink(link.id)).optionsA, { + bots: false, + joinMessages: false, + uploadFiles: true, + }); +}); + +test('when Sylo is removed from a server, the server on the other end loses its webhook and is told', async () => { + const { dCh, fCh, link } = await bridged(); + await relay(fromD(dCh, { content: 'hi' }), D); + await purgeGuild(F); + assert.equal(await store.getLink(link.id), null); + assert.deepEqual( + fakes.transport.deleted.map((d) => [d.guildKey, d.hook.id]).sort(), + [ + [D, link.hookAId], + [F, link.hookBId], + ].sort(), + "the other side's webhook, and the leaving server's own" + ); + assert.match(say(dCh).at(-1), /the other server removed Sylo/); + assert.ok((await listAudit(D)).some((e) => e.action === 'bridge:unlink')); + assert.ok(fCh); +}); + +test('the dashboard view never carries a webhook token', async () => { + const { link } = await bridged(); + const text = JSON.stringify(linkView(await store.getLink(link.id), D)); + assert.ok(!text.includes(link.hookAToken) && !text.includes(link.hookBToken)); + assert.ok(!/hook/i.test(text)); +}); + +test('a bulk delete finds the channel from the messages when the platform did not cache it (Fluxer)', async () => { + const { dCh, fCh } = await bridged(); + const m = fromF(fCh, { content: 'gone soon' }); + await relay(m, F); + await handleBulkDelete( + { messages: new Map([[m.id, { id: m.id, channelId: fCh.id }]]), channel: null }, + {}, + F + ); + await bridgeIdle(); + assert.deepEqual( + fakes.transport.removes.map((r) => r.guildKey), + [D] + ); + assert.ok(dCh); +}); + +// --- round 2: age restriction on Fluxer, the person linking, deletes, loops, sealed tokens, budgets --------- + +const waitFor = () => { + const gate = {}; + gate.open = new Promise((resolve) => { + gate.release = resolve; + }); + return gate; +}; + +test('Fluxer age restriction counts the category, the server and what cannot be read, not only the own flag', async () => { + const dCh = p.channel(p.dGuild, { name: 'ordinary-discord' }); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + const accept = (channel) => () => + acceptLink({ guildKey: F, channelId: channel.id, code: made.code, actor: 'B' }); + const tooStrict = /age-restricted channel can only be bridged with another age-restricted/; + + const category = p.channel(p.fGuild, { type: 4, nsfwOverride: true }); + const inheriting = p.channel(p.fGuild, { parentId: category.id }); // nsfw false, nsfwOverride null: inherits + await refuses(accept(inheriting)(), tooStrict); + + p.fGuild.nsfw = true; // the whole Fluxer server is age-restricted + const inServer = p.channel(p.fGuild, {}); + await refuses(accept(inServer)(), tooStrict); + p.fGuild.nsfw = false; + + const unreadable = p.channel(p.fGuild, {}); + unreadable.nsfwOverride = undefined; // the field is missing: cannot tell, so restricted + await refuses(accept(unreadable)(), tooStrict); + const orphan = p.channel(p.fGuild, { parentId: snowflake() }); // its category is unknown + await refuses(accept(orphan)(), tooStrict); + assert.equal(fakes.transport.hooks.length, 0, 'refused before any webhook was made'); + + const plain = p.channel(p.fGuild, {}); + assert.ok(await accept(plain)(), 'a channel that is clearly ordinary links'); +}); + +test('an age-restricted Fluxer category is refused for an ordinary Discord channel, and the same the other way round', async () => { + const category = p.channel(p.fGuild, { type: 4, nsfwOverride: true }); + const fCh = p.channel(p.fGuild, { parentId: category.id }); + const dNsfw = p.channel(p.dGuild, { nsfw: true }); + const dPlain = p.channel(p.dGuild, {}); + const made = await requestLink({ guildKey: D, channelId: dPlain.id, actor: 'A' }); + await refuses( + acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }), + /age-restricted/ + ); + const made2 = await requestLink({ guildKey: D, channelId: dNsfw.id, actor: 'A' }); + assert.ok( + await acceptLink({ guildKey: F, channelId: fCh.id, code: made2.code, actor: 'B' }), + 'restricted with restricted' + ); +}); + +test('a Discord server that is itself age-restricted counts for all its channels', async () => { + p.dGuild.nsfwLevel = 3; + const dCh = p.channel(p.dGuild, {}); + const fCh = p.channel(p.fGuild, {}); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + await refuses( + acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }), + /age-restricted/ + ); +}); + +test('a category that turns age-restricted later stops the link on the next message, and resume re-checks', async () => { + const category = p.channel(p.fGuild, { type: 4, nsfwOverride: false }); + const dCh = p.channel(p.dGuild, {}); + const fCh = p.channel(p.fGuild, { parentId: category.id }); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + const link = await acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }); + await relay(fromD(dCh, { content: 'fine' }), D); + assert.equal(fakes.transport.sends.length, 1); + + category.nsfwOverride = true; + await relay(fromD(dCh, { content: 'now what' }), D); + await relay(fromF(fCh, { content: 'and back' }), F); + assert.equal(fakes.transport.sends.length, 1, 'nothing crossed after the change'); + const row = await store.getLink(link.id); + assert.deepEqual([row.status, row.failReason], ['failed', 'age-restriction']); + await refuses(resumeLink({ guildKey: D, id: link.id, actor: 'A' }), /age-restricted/); + category.nsfwOverride = false; + await resumeLink({ guildKey: D, id: link.id, actor: 'A' }); + assert.equal((await store.getLink(link.id)).status, 'active'); +}); + +test('the person linking must be able to view, read and send in the channel; open mode needs no check', async () => { + const hidden = p.channel(p.dGuild, { name: 'staff' }); + const lacking = (denied) => ({ id: 'someone', denied }); + hidden.permissionsFor = (member) => ({ has: (perm) => !(member?.denied ?? []).includes(perm) }); + assert.doesNotThrow(() => assertActorMayUse(D, hidden.id, lacking([]))); + for (const perm of ['ViewChannel', 'ReadMessageHistory', 'SendMessages']) { + assert.throws( + () => assertActorMayUse(D, hidden.id, lacking([perm])), + (e) => e instanceof BridgeError && e.status === 403 && /#staff/.test(e.message), + perm + ); + } + assert.throws(() => assertActorMayUse(D, hidden.id, null), BridgeError, 'no member found: refused'); + assert.throws(() => assertActorMayUse(D, snowflake(), lacking([])), BridgeError, 'no such channel'); + assert.doesNotThrow( + () => assertActorMayUse(D, hidden.id, null, { open: true }), + 'open mode: everyone is the admin' + ); +}); + +test("Sylo's own rights that cannot be read count as missing", async () => { + const dCh = p.channel(p.dGuild, {}); + dCh.permissionsFor = undefined; + await refuses( + requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }), + /Sylo needs ViewChannel, SendMessages, ManageWebhooks/ + ); + dCh.permissionsFor = () => null; + await refuses(requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }), /Sylo needs/); +}); + +// deletes + +/** A webhook send that waits at a gate, so a delete or an edit can arrive while the relay is in progress. */ +function slowDownloads(gate) { + useBridgeParts({ + transport: fakes.transport, + queue: fakes.queue, + download: async () => { + await gate.open; + return { ok: true, status: 200, body: Buffer.from('FILE') }; + }, + }); +} +const withFile = (over = {}) => ({ + attachments: new Map([['a', { name: 'cat.png', url: 'https://cdn/cat.png', size: 4 }]]), + ...over, +}); + +test('a delete that comes while the copy is still being made leaves no copy behind', async () => { + const { dCh } = await bridged(); + const gate = waitFor(); + slowDownloads(gate); + const m = fromD(dCh, withFile({ content: 'about to be removed by automod' })); + await handleMessage(m, {}, D); // the relay is queued and waits for the download + await handleDelete({ id: m.id, channelId: dCh.id }, {}, D); + gate.release(); + await bridgeIdle(); + const standing = + fakes.transport.sends.length - fakes.transport.removes.length - fakes.transport.botDeletes.length; + assert.equal(standing, 0, 'no copy of the deleted message is left on the other side'); +}); + +test('a delete that comes before the relay has even started stops the relay', async () => { + const { dCh } = await bridged(); + const m = fromD(dCh, { content: 'gone first' }); + await handleDelete({ id: m.id, channelId: dCh.id }, {}, D); + await relay(m, D); + assert.equal(fakes.transport.sends.length, 0); +}); + +test('a copy posted while its original is deleted is taken down again', async () => { + const { dCh } = await bridged(); + const gate = waitFor(); + const realSend = fakes.transport.send; + fakes.transport.send = async (...args) => { + await gate.open; // the delete lands while the send is in flight + return realSend(...args); + }; + const m = fromD(dCh, { content: 'raced' }); + await handleMessage(m, {}, D); + await new Promise((r) => setImmediate(r)); + await handleDelete({ id: m.id, channelId: dCh.id }, {}, D); + gate.release(); + await bridgeIdle(); + assert.equal(fakes.transport.sends.length, 1); + assert.equal(fakes.transport.removes.length, 1, 'removed as soon as it was posted'); + assert.equal(fakes.transport.removes[0].messageId, fakes.transport.sends[0].id); +}); + +test('a delete reaches the copy while the link is paused or stopped', async () => { + const { dCh, link } = await bridged(); + const one = fromD(dCh, { content: 'one' }); + const two = fromD(dCh, { content: 'two' }); + await relay(one, D); + await relay(two, D); + await pauseLink({ guildKey: D, id: link.id, actor: 'A' }); + await handleDelete({ id: one.id, channelId: dCh.id }, {}, D); + await bridgeIdle(); + assert.deepEqual( + fakes.transport.removes.map((r) => r.messageId), + [fakes.transport.sends[0].id], + 'paused' + ); + await resumeLink({ guildKey: D, id: link.id, actor: 'A' }); + await failLink(link.id, 'permissions'); + await handleDelete({ id: two.id, channelId: dCh.id }, {}, D); + await bridgeIdle(); + assert.equal(fakes.transport.removes.length, 2, 'stopped'); + assert.equal(await store.mappingOf(link.id, 'a', two.id), null); +}); + +test('the webhook that really posted a copy is the one remembered, so a delete after a recreation works', async () => { + const { dCh, link } = await bridged(); + fakes.transport.sendFailures.push(apiError(404, { code: 10015 })); // the webhook was deleted: remade + const m = fromD(dCh, { content: 'posted by the new webhook' }); + await relay(m, D); + const row = await store.getLink(link.id); + assert.notEqual(row.hookBId, link.hookBId); + assert.equal( + (await store.mappingOf(link.id, 'a', m.id)).copyHookId, + row.hookBId, + 'the id of the webhook that sent it' + ); + await handleDelete({ id: m.id, channelId: dCh.id }, {}, D); + await bridgeIdle(); + assert.equal(fakes.transport.removes.length, 1); + assert.equal(fakes.transport.botDeletes.length, 0, 'the webhook did it'); +}); + +test('when the webhook cannot delete a copy (it was replaced, or is gone) the bot does', async () => { + const { dCh, fCh, link } = await bridged(); + const m1 = fromD(dCh, { content: 'one' }); + const m2 = fromD(dCh, { content: 'two' }); + const m3 = fromD(dCh, { content: 'three' }); + for (const m of [m1, m2, m3]) await relay(m, D); + const copies = fakes.transport.sends.map((s) => s.id); + // m1: its copy was posted by a webhook that has since been replaced + db.prepare('UPDATE bridge_messages SET copy_hook_id = ? WHERE a_message_id = ?').run( + 'an-older-webhook', + m1.id + ); + // m2: the webhook is gone when the delete is tried + fakes.transport.removeFailures.push(apiError(404, { code: 10015 })); + await handleDelete({ id: m1.id, channelId: dCh.id }, {}, D); + await handleDelete({ id: m2.id, channelId: dCh.id }, {}, D); + await bridgeIdle(); + assert.deepEqual( + fakes.transport.botDeletes.map((d) => [d.guildKey, d.channelId, d.messageId]), + [ + [F, fCh.id, copies[0]], + [F, fCh.id, copies[1]], + ] + ); + // m3: nothing can delete it (no Manage Messages): the failure is quiet and the link keeps running + fakes.transport.removeFailures.push(apiError(404, { code: 10015 })); + fakes.transport.botDeleteFailures.push(apiError(403)); + await handleDelete({ id: m3.id, channelId: dCh.id }, {}, D); + await bridgeIdle(); + assert.equal((await store.getLink(link.id)).status, 'active', 'a delete that fails never stops the link'); +}); + +test('a purge of 100 messages on a busy lane removes every copy; deletes are not dropped for a full lane', async () => { + const gate = waitFor(); + fakes = useFakes({ queue: createRelayQueue({ minGapMs: 0, maxQueued: 2, sleep: async () => {} }) }); + const realSend = fakes.transport.send; + fakes.transport.send = async (...args) => { + await gate.open; + return realSend(...args); + }; + const { dCh, link } = await bridged(); + const ids = Array.from({ length: 100 }, (_, i) => `9000000000000${String(1000 + i)}`); + for (const [i, id] of ids.entries()) + await store.saveMapping(link.id, 'a', { sourceId: id, copyId: `c${i}`, hookId: link.hookBId }); + for (let i = 0; i < 4; i += 1) await handleMessage(fromD(dCh, { content: `busy ${i}` }), {}, D); // the lane is full (cap 2) + await handleBulkDelete({ messages: new Map(ids.map((id) => [id, { id }])), channel: dCh }, {}, D); + gate.release(); + await bridgeIdle(); + assert.equal(fakes.transport.removes.length, 100); + assert.equal( + await store.mappingCount(link.id), + 2, + 'only the two busy messages that fit are left in the map' + ); +}); + +test('a delete is not dropped for waiting too long', async () => { + let clock = 1_000_000; + fakes = useFakes({ + queue: createRelayQueue({ + minGapMs: 0, + maxAgeMs: 1000, + now: () => clock, + sleep: async () => { + clock += 5000; + }, + }), + }); + const gate = waitFor(); + const realSend = fakes.transport.send; + fakes.transport.send = async (...args) => { + await gate.open; + return realSend(...args); + }; + const { dCh, link } = await bridged(); + await store.saveMapping(link.id, 'a', { + sourceId: '900000000000009001', + copyId: 'old-copy', + hookId: link.hookBId, + }); + await handleMessage(fromD(dCh, { content: 'blocker' }), {}, D); + await handleDelete({ id: '900000000000009001', channelId: dCh.id }, {}, D); + clock += 60_000; // far past the age at which an ordinary job is dropped + gate.release(); + await bridgeIdle(); + assert.deepEqual( + fakes.transport.removes.map((r) => r.messageId), + ['old-copy'] + ); +}); + +test('an edit that comes while the copy is still being made reaches it', async () => { + const { dCh } = await bridged(); + const gate = waitFor(); + slowDownloads(gate); + const m = fromD(dCh, withFile({ content: 'first' })); + await handleMessage(m, {}, D); + await handleUpdate({ old: m, new: { ...m, content: 'second' } }, {}, D); + gate.release(); + await bridgeIdle(); + assert.equal(fakes.transport.sends.length, 1); + assert.equal(fakes.transport.edits.length, 1); + assert.equal(fakes.transport.edits[0].payload.content, 'second'); +}); + +// loops with other bridges + +test('a webhook message is never relayed, even with bots on, so another bridge on the same channels cannot make a loop', async () => { + const { dCh, fCh } = await bridged({ optionsD: { bots: true }, optionsF: { bots: true } }); + // Another bridge relays Alice's message into the Discord channel through its own webhook + await relay( + fromD(dCh, { + content: 'echo', + webhookId: snowflake(), + author: { bot: true, username: 'Alice [Fluxer]' }, + member: null, + }), + D + ); + await relay( + fromF(fCh, { + content: 'echo', + webhookId: snowflake(), + author: { bot: true, username: 'Bob [Discord]' }, + member: null, + }), + F + ); + // and a bot account that already wears a platform tag (a bridge's relay account) + await relay( + fromD(dCh, { content: 'echo', author: { bot: true, username: 'Alice [Fluxer]' }, member: null }), + D + ); + assert.equal(fakes.transport.sends.length, 0); + // an ordinary bot account still crosses + await relay(fromD(dCh, { content: 'real', author: { bot: true, username: 'Deploy' }, member: null }), D); + assert.equal(fakes.transport.sends.length, 1); +}); + +// sealed tokens + +test('webhook tokens are stored sealed; a token that cannot be opened leads to a new webhook, not a failure', async () => { + const { dCh, link } = await bridged(); + const stored = db.prepare('SELECT hook_a_token, hook_b_token FROM bridge_links WHERE id = ?').get(link.id); + for (const [column, token] of [ + [stored.hook_a_token, link.hookAToken], + [stored.hook_b_token, link.hookBToken], + ]) { + assert.ok(token.startsWith('tok-'), 'the code sees the real token'); + assert.ok(column.startsWith('v1.') && !column.includes(token), 'the table holds a sealed value'); + } + await relay(fromD(dCh, { content: 'works' }), D); + assert.equal(fakes.transport.sends.length, 1); + + // the session secret changed since: the stored value cannot be opened + db.prepare('UPDATE bridge_links SET hook_b_token = ? WHERE id = ?').run( + seal('another secret', 'bridge-webhook', 'tok-x'), + link.id + ); + dropBridgeCache(); + assert.equal((await store.getLink(link.id)).hookBToken, ''); + const before = fakes.transport.hooks.length; + await relay(fromD(dCh, { content: 'after the secret changed' }), D); + assert.equal(fakes.transport.hooks.length, before + 1, 'a new webhook was made'); + assert.equal(fakes.transport.sends.at(-1).payload.content, 'after the secret changed'); + assert.ok(fakes.transport.sends.at(-1).hook.token, 'sent with the new token'); + assert.equal((await store.getLink(link.id)).status, 'active'); + assert.ok(config.sessionSecret); +}); + +// download budget + +test('files that do not fit the process-wide download budget cross as links instead of waiting', async () => { + const gate = waitFor(); + fakes = useFakes({ files: {} }); + useBridgeParts({ + transport: fakes.transport, + queue: fakes.queue, + budget: { concurrent: 1, bytes: 1024 }, + download: async (url) => { + fakes.downloads.push({ url }); + await gate.open; + return { ok: true, status: 200, body: Buffer.from('FILE') }; + }, + }); + const one = await bridged(); + const other = p.extra('95'); + const otherF = `fluxer:${other.fGuild.id}`; + await setGuildModule(other.dGuild.id, 'bridge', { enabled: true }); + await setGuildModule(otherF, 'bridge', { enabled: true }); + const dCh2 = p.channel(other.dGuild, {}); + const fCh2 = p.channel(other.fGuild, {}); + const made = await requestLink({ guildKey: other.dGuild.id, channelId: dCh2.id, actor: 'A' }); + await acceptLink({ guildKey: otherF, channelId: fCh2.id, code: made.code, actor: 'B' }); + + const file = (name, size = 4) => ({ + attachments: new Map([['a', { name, url: `https://cdn/${name}`, size }]]), + }); + await handleMessage(fromD(one.dCh, file('first.png')), {}, D); // takes the only slot and waits + await new Promise((r) => setImmediate(r)); + await handleMessage(fromD(dCh2, file('second.png')), {}, other.dGuild.id); // another server: no room + await new Promise((r) => setImmediate(r)); + assert.deepEqual( + fakes.downloads.map((d) => d.url), + ['https://cdn/first.png'], + 'the second never started a download' + ); + const second = fakes.transport.sends.find((s) => s.payload.content?.includes('second.png')); + assert.equal(second.payload.content, '📎 second.png: https://cdn/second.png', 'it went as a link at once'); + assert.equal(second.payload.files, undefined); + gate.release(); + await bridgeIdle(); + const first = fakes.transport.sends.find((s) => s.payload.files?.length); + assert.equal(first.payload.files[0].name, 'first.png'); + // and a file larger than the whole byte budget is a link too + gate.release(); + await relay(fromD(one.dCh, file('huge.png', 2048)), D); + assert.ok(fakes.transport.sends.at(-1).payload.content.includes('huge.png: https://cdn/huge.png')); +}); + +// minor findings + +test('names that Sylo puts in its own text cannot carry markdown links', async () => { + p.fGuild.name = 'x](https://evil.example)[y'; + const { dCh } = await bridged(); + const notice = say(dCh)[0]; + assert.ok(!notice.includes('](https://evil.example)'), notice); + assert.ok(notice.includes('x\\]\\(https://evil.example\\)\\[y')); + await handleJoin({ displayName: 'a](https://evil.example)[b', user: { bot: false } }, {}, F); + const { link } = { link: (await store.linksOfGuild(F))[0] }; + await setSideOptions({ guildKey: F, id: link.id, options: { joinMessages: true }, actor: 'B' }); + await handleJoin({ displayName: 'a](https://evil.example)[b', user: { bot: false } }, {}, F); + await bridgeIdle(); + const join = fakes.transport.sends.at(-1).payload.content; + assert.ok(!join.includes('](https://evil.example)'), join); +}); + +test('sticker, poll and file names get the same mention rules as the text', async () => { + const { dCh } = await bridged(); + await relay( + fromD(dCh, { + content: 'x', + stickers: new Map([['s', { name: '@everyone', url: 'https://cdn/s.png' }]]), + poll: { + question: { text: '@here vote <@123456789012345678>' }, + answers: new Map([[1, { text: '@everyone' }]]), + }, + attachments: new Map([ + ['a', { name: '@everyone.zip', url: 'https://cdn/a.zip', size: 99 * 1024 * 1024 }], + ]), + }), + D + ); + const text = fakes.transport.sends[0].payload.content; + assert.ok(!/@everyone|@here|<@\d/.test(text), text); +}); + +test('a link a server paused is not turned into a stopped one by a late failure', async () => { + const { link } = await bridged(); + await pauseLink({ guildKey: D, id: link.id, actor: 'A' }); + await failLink(link.id, 'unreachable'); + const row = await store.getLink(link.id); + assert.deepEqual( + [row.status, row.pausedSide], + ['paused', 'a'], + 'still paused by Discord, so only Discord can resume it' + ); + await refuses(resumeLink({ guildKey: F, id: link.id, actor: 'B' }), /only they can resume/, 403); +}); + +test('the dashboard names only your own side, not the other server staff', async () => { + const { link } = await bridged(); + assert.equal(linkView(await store.getLink(link.id), D).by, 'Alice (dashboard)'); + assert.equal(linkView(await store.getLink(link.id), F).by, 'Bob (dashboard)'); + const text = JSON.stringify(linkView(await store.getLink(link.id), D)); + assert.ok(!text.includes('Bob (dashboard)')); +}); + +test('a platform that refuses a message gets a plain name, then the text without files and embeds', async () => { + const { dCh } = await bridged({ optionsD: { bots: true } }); + fakes.transport.sendFailures.push(apiError(400)); + await relay( + fromD(dCh, { content: 'name refused', member: null, author: { bot: true, username: 'Weird' } }), + D + ); + assert.equal(fakes.transport.sends[0].payload.username, 'Bridge [Discord]'); + + fakes.transport.sendFailures.push(apiError(400), apiError(400)); + await relay( + fromD(dCh, { + content: 'text survives', + member: null, + author: { bot: true, username: 'Deploy' }, + embeds: [{ type: 'rich', title: 'big embed' }], + }), + D + ); + const last = fakes.transport.sends.at(-1).payload; + assert.equal(last.content, 'text survives'); + assert.equal(last.embeds, undefined); + assert.equal(last.files, undefined); +}); + +// --- round 3: server owners only, and the smaller notes --------------------------------------------------- + +test('the owner of each server is recorded with the link', async () => { + const { link } = await bridged(); + const row = await store.getLink(link.id); + assert.deepEqual([row.ownerA, row.ownerB], [p.dGuild.ownerId, p.fGuild.ownerId]); +}); + +test('a change of owner pauses the link on the next message, until the new owner confirms it', async () => { + const { dCh, fCh, link } = await bridged(); + await relay(fromD(dCh, { content: 'before' }), D); + assert.equal(fakes.transport.sends.length, 1); + + p.fGuild.ownerId = 'new-fluxer-owner'; // the Fluxer server is transferred + await relay(fromD(dCh, { content: 'after the transfer' }), D); + await relay(fromF(fCh, { content: 'and from the new owner side' }), F); + assert.equal(fakes.transport.sends.length, 1, 'nothing crossed after the transfer'); + const row = await store.getLink(link.id); + assert.deepEqual([row.status, row.pausedSide, row.failReason], ['paused', 'b', 'owner-changed']); + assert.match(say(dCh).at(-1), /Bridge paused:\*\* the owner of Fluxer Home changed/); + assert.match(say(fCh).at(-1), /Bridge paused:\*\* the owner of Fluxer Home changed/); + assert.ok((await listAudit(D)).some((e) => e.action === 'bridge:pause' && /owner/.test(e.detail))); + assert.ok((await listAudit(F)).some((e) => e.action === 'bridge:pause' && /owner/.test(e.detail))); + const view = linkView(row, D); + assert.deepEqual([view.status, view.pausedByThem], ['paused', true]); + assert.match(view.pauseNote, /owner of one of the servers changed/); + assert.deepEqual( + (await bridgeProblems(F)).map((x) => x.code), + ['owner-changed'], + 'a Health item' + ); + + await refuses(resumeLink({ guildKey: D, id: link.id, actor: 'A' }), /only they can resume/, 403); + const resumed = await resumeLink({ guildKey: F, id: link.id, actor: 'B' }); + assert.equal(resumed.active, true); + const after = await store.getLink(link.id); + assert.deepEqual([after.status, after.ownerB], ['active', 'new-fluxer-owner'], 'the new owner is recorded'); + await relay(fromD(dCh, { content: 'confirmed' }), D); + assert.equal(fakes.transport.sends.length, 2); +}); + +test('when both servers changed owner, both owners have to confirm', async () => { + const { dCh, link } = await bridged(); + p.dGuild.ownerId = 'new-discord-owner'; + p.fGuild.ownerId = 'new-fluxer-owner'; + await relay(fromD(dCh, { content: 'x' }), D); + assert.equal((await store.getLink(link.id)).pausedSide, 'a'); + const first = await resumeLink({ guildKey: D, id: link.id, actor: 'A' }); + assert.equal(first.active, false); + const waiting = await store.getLink(link.id); + assert.deepEqual( + [waiting.status, waiting.pausedSide], + ['paused', 'b'], + 'now it waits for the Fluxer owner' + ); + await refuses(resumeLink({ guildKey: D, id: link.id, actor: 'A' }), /only they can resume/, 403); + assert.equal((await resumeLink({ guildKey: F, id: link.id, actor: 'B' })).active, true); + await relay(fromD(dCh, { content: 'both confirmed' }), D); + assert.equal(fakes.transport.sends.length, 1); +}); + +test('a quiet link is checked too (the sweep), and so is a join notice; an unknown owner is not a change', async () => { + const { link } = await bridged({ optionsD: { joinMessages: true } }); + p.fGuild.ownerId = undefined; // the platform did not tell: not a change + assert.equal(await sweepOwners(), 0); + p.fGuild.ownerId = 'someone-new'; + assert.equal(await sweepOwners(), 1, 'paused without any message'); + assert.equal((await store.getLink(link.id)).failReason, 'owner-changed'); + assert.equal(await sweepOwners(), 0, 'and only once'); + + await resumeLink({ guildKey: F, id: link.id, actor: 'B' }); + p.dGuild.ownerId = 'discord-owner-2'; + await handleJoin({ displayName: 'Zed', user: { bot: false } }, {}, D); + await bridgeIdle(); + assert.equal(fakes.transport.sends.length, 0, 'no join notice for a link that just lost its owner'); + assert.equal((await store.getLink(link.id)).status, 'paused'); +}); + +test('a code made by a previous owner is refused', async () => { + const dCh = p.channel(p.dGuild, {}); + const fCh = p.channel(p.fGuild, {}); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + p.dGuild.ownerId = 'the-new-owner'; + await refuses( + acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }), + /has a new owner\. Ask for a new code/, + 409 + ); + assert.equal(fakes.transport.hooks.length, 0); + assert.equal((await store.pendingCodes(D)).length, 1, 'nothing was used up'); +}); + +test('a resume by a stopped link also records the owner who confirms', async () => { + const { link } = await bridged(); + await store.setLinkStatus(link.id, { status: 'failed', reason: 'permissions' }); + p.dGuild.ownerId = 'discord-owner-2'; + const out = await resumeLink({ guildKey: D, id: link.id, actor: 'A' }); + assert.equal(out.active, true); + assert.equal((await store.getLink(link.id)).ownerA, 'discord-owner-2'); +}); + +// temporary errors on a delete + +test('a delete that hits a temporary error is tried again, and the map entry stays until the copy is gone', async () => { + const { dCh, link } = await bridged(); + const m = fromD(dCh, { content: 'delete me' }); + await relay(m, D); + fakes.transport.removeFailures.push(apiError(500), apiError(502)); + await handleDelete({ id: m.id, channelId: dCh.id }, {}, D); + await bridgeIdle(); + assert.equal(fakes.transport.removes.length, 1, 'removed on the third try'); + assert.deepEqual(fakes.sleeps, [1000, 2000], 'with growing pauses'); + assert.equal(await store.mappingOf(link.id, 'a', m.id), null, 'the entry went with the copy'); + + // it keeps failing: the entry stays, so a later delete can still find the copy + const m2 = fromD(dCh, { content: 'stubborn' }); + await relay(m2, D); + for (let i = 0; i < 4; i += 1) fakes.transport.removeFailures.push(apiError(503)); + await handleDelete({ id: m2.id, channelId: dCh.id }, {}, D); + await bridgeIdle(); + assert.ok(await store.mappingOf(link.id, 'a', m2.id), 'the copy is still known'); + assert.equal((await store.getLink(link.id)).status, 'active', 'and the link is not stopped for it'); +}); + +test('a delete that cannot be done for good (no webhook, no Manage Messages) keeps the map entry', async () => { + const { dCh, link } = await bridged(); + const m = fromD(dCh, { content: 'cannot be removed' }); + await relay(m, D); + fakes.transport.removeFailures.push(apiError(404, { code: 10015 })); + fakes.transport.botDeleteFailures.push(apiError(403)); + await handleDelete({ id: m.id, channelId: dCh.id }, {}, D); + await bridgeIdle(); + assert.ok(await store.mappingOf(link.id, 'a', m.id)); +}); + +// the download budget is held until the send is over + +function budgetParts(files, gate) { + const sent = []; + const transportSend = fakes.transport.send; + fakes.transport.send = async (key, hook, payload) => { + if (payload.files?.length && gate) await gate.open; + sent.push(payload); + return transportSend(key, hook, payload); + }; + useBridgeParts({ + transport: fakes.transport, + queue: fakes.queue, + budget: { concurrent: 1, bytes: 1024 }, + download: async (url) => { + fakes.downloads.push({ url }); + return { ok: true, status: 200, body: Buffer.from(files[url] ?? 'x') }; + }, + }); + return sent; +} +const pairOfLinks = async (suffix) => { + const one = await bridged(); + const other = p.extra(suffix); + const otherF = `fluxer:${other.fGuild.id}`; + await setGuildModule(other.dGuild.id, 'bridge', { enabled: true }); + await setGuildModule(otherF, 'bridge', { enabled: true }); + const dCh2 = p.channel(other.dGuild, {}); + const fCh2 = p.channel(other.fGuild, {}); + const made = await requestLink({ guildKey: other.dGuild.id, channelId: dCh2.id, actor: 'A' }); + await acceptLink({ guildKey: otherF, channelId: fCh2.id, code: made.code, actor: 'B' }); + return { one, dCh2, otherKey: other.dGuild.id }; +}; +const file = (name) => ({ attachments: new Map([['a', { name, url: `https://cdn/${name}`, size: 4 }]]) }); + +test('the download budget stays held while the copy is being sent, not only while the file downloads', async () => { + const gate = waitFor(); + budgetParts({}, gate); + const { one, dCh2, otherKey } = await pairOfLinks('96'); + await handleMessage(fromD(one.dCh, file('first.png')), {}, D); // downloaded, then the send waits at the gate + await new Promise((r) => setImmediate(r)); + assert.deepEqual( + fakes.downloads.map((d) => d.url), + ['https://cdn/first.png'] + ); + await handleMessage(fromD(dCh2, file('second.png')), {}, otherKey); + await new Promise((r) => setImmediate(r)); + assert.deepEqual( + fakes.downloads.map((d) => d.url), + ['https://cdn/first.png'], + 'the first file is still held: no room' + ); + assert.equal(fakes.transport.sends.at(-1).payload.content, '📎 second.png: https://cdn/second.png'); + gate.release(); + await bridgeIdle(); + // the copy is out: the budget is free again + await relay(fromD(dCh2, file('third.png')), otherKey); + assert.equal(fakes.downloads.at(-1).url, 'https://cdn/third.png'); + assert.equal(fakes.transport.sends.at(-1).payload.files[0].name, 'third.png'); +}); + +test('the download budget is given back when a send fails for good', async () => { + budgetParts({}, null); + const { one, dCh2, otherKey } = await pairOfLinks('97'); + for (let i = 0; i < 4; i += 1) fakes.transport.sendFailures.push(apiError(500)); + await relay(fromD(one.dCh, file('lost.png')), D); // four tries, then given up + assert.equal(fakes.transport.sends.length, 0); + await relay(fromD(dCh2, file('next.png')), otherKey); + assert.equal(fakes.transport.sends.at(-1).payload.files[0].name, 'next.png', 'the budget was free again'); +}); + +// an edit with a token that cannot be read + +test('an edit with an unreadable webhook token makes a new webhook and does not count as a failed delivery', async () => { + const { dCh, link } = await bridged(); + const m = fromD(dCh, { content: 'first' }); + await relay(m, D); + db.prepare('UPDATE bridge_links SET hook_b_token = ? WHERE id = ?').run( + seal('another secret', 'bridge-webhook', 'x'), + link.id + ); + dropBridgeCache(); + const hooks = fakes.transport.hooks.length; + for (let i = 0; i < FAIL_THRESHOLD + 2; i += 1) { + await handleUpdate({ old: m, new: { ...m, content: `edit ${i}` } }, {}, D); + await bridgeIdle(); + } + assert.equal(fakes.transport.hooks.length, hooks + 1, 'one new webhook, once'); + assert.equal(fakes.transport.edits.length, 0, 'the old copy cannot be edited by the new webhook'); + assert.equal((await store.getLink(link.id)).status, 'active', 'never stopped for it'); + await relay(fromD(dCh, { content: 'and new messages cross' }), D); + assert.equal(fakes.transport.sends.at(-1).hook.id, (await store.getLink(link.id)).hookBId); +}); + +// a deleted channel on a link that is not running + +test('a bridged channel deleted while the link is paused is shown as a problem', async () => { + const { dCh, link } = await bridged(); + await pauseLink({ guildKey: D, id: link.id, actor: 'A' }); + await handleChannelDelete(dCh, {}, D); + const row = await store.getLink(link.id); + assert.deepEqual([row.status, row.pausedSide, row.failReason], ['paused', 'a', 'channel-deleted']); + assert.deepEqual( + (await bridgeProblems(D)).map((x) => [x.linkId, x.code]), + [[link.id, 'channel-deleted']] + ); + assert.deepEqual( + (await bridgeProblems(F)).map((x) => x.code), + ['channel-deleted'], + 'the other server sees it too' + ); + assert.match(linkView(row, D).pauseNote, /channel was deleted/); + p.dGuild.channels.cache.delete(dCh.id); // it is gone on the platform + await refuses(resumeLink({ guildKey: D, id: link.id, actor: 'A' }), /normal text channel/); +}); + +test('a stopped link whose channel is deleted later says so', async () => { + const { dCh, link } = await bridged(); + await failLink(link.id, 'unreachable'); + await handleChannelDelete(dCh, {}, D); + const row = await store.getLink(link.id); + assert.deepEqual([row.status, row.failReason], ['failed', 'channel-deleted']); +}); diff --git a/test/bridge.postgres.test.js b/test/bridge.postgres.test.js new file mode 100644 index 00000000..36c894bd --- /dev/null +++ b/test/bridge.postgres.test.js @@ -0,0 +1,125 @@ +// Proves the Bridge storage against a real Postgres connection: the tables come from the bootstrap, ids and +// BIGINT times come back as numbers, a code is used up once, a channel is in one link, the message map is +// found from both sides and swept, and the guild purge removes the links of either side. +import './helpers/isolateSqlite.js'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { closePostgres } from '../src/db/driver.js'; + +const url = process.env.DATABASE_URL; + +test( + 'bridge against a real Postgres connection', + { skip: !url && 'DATABASE_URL not set (sqlite-only run)' }, + async (t) => { + const store = await import('../src/db/bridge.js'); + const purge = await import('../src/db/purge.js'); + const retention = await import('../src/db/retention.js'); + t.after(async () => { + await closePostgres(); + }); + + const run = Date.now(); + const G1 = `pgtest-bridge-${run}-a`; + const G2 = `fluxer:pgtest-bridge-${run}-b`; + const link = (over = {}) => ({ + guildA: G1, + channelA: `ca-${run}`, + guildB: G2, + channelB: `cb-${run}`, + hookA: { id: `ha-${run}`, token: 'ta' }, + hookB: { id: `hb-${run}`, token: 'tb' }, + requestedBy: 'Alice', + acceptedBy: 'Bob', + ...over, + }); + + await t.test( + 'a link is created, found from the channel and the webhook, and a channel is in one link only', + async () => { + const id = await store.createLink(link()); + assert.equal(typeof id, 'number'); + const found = await store.linkForChannel(G2, `cb-${run}`); + assert.equal(found.side, 'b'); + assert.equal(typeof found.link.createdAt, 'number'); + assert.equal((await store.linkForHook(`ha-${run}`)).id, id); + assert.equal( + await store.createLink(link({ channelB: `other-${run}` })), + null, + 'the same a-channel again' + ); + assert.equal( + await store.createLink(link({ channelA: `other-${run}` })), + null, + 'the same b-channel again' + ); + assert.equal(await store.linkCount(G1), 1); + await store.setLinkStatus(id, { status: 'paused', pausedSide: 'a' }); + assert.deepEqual( + [(await store.getLink(id)).status, (await store.getLink(id)).pausedSide], + ['paused', 'a'] + ); + await store.setLinkOptions(id, 'a', { bots: true }); + assert.deepEqual((await store.getLink(id)).optionsA, { bots: true }); + await store.setLinkHook(id, 'b', { id: `hb2-${run}`, token: 'tb2' }); + assert.equal((await store.getLink(id)).hookBId, `hb2-${run}`); + await store.deleteLink(id); + assert.equal(await store.getLink(id), null); + } + ); + + await t.test('a code is stored as a hash, expires, and is used up once', async () => { + const made = await store.createCode({ + guildId: G1, + channelId: 'c1', + options: { bots: true }, + requestedBy: 'Alice', + }); + assert.equal(await store.pendingCodeCount(G1), 1); + assert.equal(typeof (await store.pendingCodes(G1))[0].expiresAt, 'number'); + assert.equal((await store.peekCode(made.code)).guildId, G1); + const [a, b] = await Promise.all([store.claimCode(made.code), store.claimCode(made.code)]); + assert.equal([a, b].filter(Boolean).length, 1, 'exactly one claim wins'); + const old = await store.createCode({ guildId: G1, channelId: 'c2', ttlMs: -1000 }); + assert.equal(await store.claimCode(old.code), null); + assert.ok((await store.sweepCodes()) >= 1); + assert.equal(await store.cancelCode(G2, 1), false); + }); + + await t.test('the message map is found from both sides, replaced, and swept after 30 days', async () => { + const id = await store.createLink(link()); + await store.saveMapping(id, 'a', { + sourceId: 'm1', + copyId: 'c1', + hookId: 'h', + hash: 'x', + extra: { head: '> q' }, + }); + const fromCopy = await store.mappingOf(id, 'b', 'c1'); + assert.deepEqual([fromCopy.counterpartId, fromCopy.isOriginal], ['m1', false]); + assert.deepEqual((await store.mappingOf(id, 'a', 'm1')).extra, { head: '> q' }); + await store.saveMapping(id, 'a', { sourceId: 'm1', copyId: 'c2' }); + assert.equal((await store.mappingOf(id, 'a', 'm1')).counterpartId, 'c2'); + await store.saveMapping(id, 'b', { sourceId: 'old', copyId: 'old-copy' }); + const { prepare } = await import('../src/db/driver.js'); + await prepare('UPDATE bridge_messages SET created_at = ? WHERE a_message_id = ?').run( + Date.now() - 40 * 86_400_000, + 'old-copy' + ); + await retention.sweepRetention(); + assert.equal(await store.mappingOf(id, 'a', 'old-copy'), null); + assert.ok(await store.mappingOf(id, 'a', 'm1')); + assert.equal(await store.mappingCount(id), 1); + }); + + await t.test( + 'purging a server removes the links it is in, on either side, and its requests', + async () => { + await store.createCode({ guildId: G2, channelId: 'c9' }); + await purge.purgeGuild(G2); + assert.equal((await store.linksOfGuild(G1)).length, 0); + assert.equal(await store.pendingCodeCount(G2), 0); + } + ); + } +); diff --git a/test/bridgeHealth.test.js b/test/bridgeHealth.test.js new file mode 100644 index 00000000..6f9aa214 --- /dev/null +++ b/test/bridgeHealth.test.js @@ -0,0 +1,102 @@ +// The Health page for the bridge: the permissions it needs in a bridged channel, and a stopped link shown +// as a problem with its reason. +import './helpers/tmpDb.js'; +import test, { afterEach, beforeEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { setGuildModule } from '../src/db/modules.js'; +import { purgeGuild } from '../src/db/purge.js'; +import { buildHealthReport } from '../src/web/lib/permissionHealth.js'; +import { keyedGuild } from '../src/platform/platformOf.js'; +import { + acceptLink, + dropBridgeCache, + failLink, + handleChannelDelete, + pauseLink, + requestLink, + resumeLink, + sweepOwners, + useBridgeParts, +} from '../src/modules/bridge.js'; +import { D, F, installPlatforms, removePlatforms, useFakes } from './helpers/fakeBridge.js'; + +let p; +beforeEach(async () => { + p = installPlatforms(); + useFakes(); + dropBridgeCache(); + for (const key of [D, F]) await purgeGuild(key); + useFakes(); + await setGuildModule(D, 'bridge', { enabled: true }); + await setGuildModule(F, 'bridge', { enabled: true }); +}); +afterEach(() => { + useBridgeParts(); + removePlatforms(); +}); + +const forBridge = (guild) => + buildHealthReport(guild, { modules: [{ id: 'bridge', enabled: true, config: {} }] }); + +test('the bridge needs Manage Webhooks in a bridged channel, and Health says so', async () => { + const dCh = p.channel(p.dGuild, { name: 'discord-chat' }); + const fCh = p.channel(p.fGuild, { name: 'fluxer-chat' }); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + await acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }); + assert.deepEqual((await forBridge(p.dGuild)).checks, [], 'nothing to report while Sylo has what it needs'); + + dCh.permissionsFor = () => ({ has: (perm) => perm !== 'ManageWebhooks' }); + const report = await forBridge(p.dGuild); + assert.equal(report.checks.length, 1); + const [check] = report.checks; + assert.equal(check.module, 'bridge'); + assert.equal(check.severity, 'error'); + assert.deepEqual(check.permissions, ['ManageWebhooks']); + assert.match(check.message, /A bridged channel \(#discord-chat\): Sylo is missing Manage Webhooks/); +}); + +test('a link Sylo stopped is a problem with the reason and what to do, on both servers', async () => { + const dCh = p.channel(p.dGuild, { name: 'discord-chat' }); + const fCh = p.channel(p.fGuild, { name: 'fluxer-chat' }); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + const link = await acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }); + await failLink(link.id, 'permissions'); + // The dashboard hands the health check the guild with its key as `.id` (keyedGuild). + for (const guild of [keyedGuild(p.dGuild), keyedGuild(p.fGuild)]) { + const { checks } = await forBridge(guild); + const stopped = checks.find((c) => c.code === 'bridge-stopped'); + assert.ok(stopped, 'a stopped bridge is listed'); + assert.equal(stopped.severity, 'error'); + assert.match( + stopped.message, + /The bridge of #(discord|fluxer)-chat stopped: Sylo is missing a permission/ + ); + assert.match(stopped.fix, /press Resume/); + } +}); + +test('a server with the module off is not checked', async () => { + const report = await buildHealthReport(p.dGuild, { + modules: [{ id: 'bridge', enabled: false, config: {} }], + }); + assert.deepEqual(report.checks, []); +}); + +test('a paused link whose owner changed, or whose channel was deleted, is a problem too', async () => { + const dCh = p.channel(p.dGuild, { name: 'discord-chat' }); + const fCh = p.channel(p.fGuild, { name: 'fluxer-chat' }); + const made = await requestLink({ guildKey: D, channelId: dCh.id, actor: 'A' }); + const link = await acceptLink({ guildKey: F, channelId: fCh.id, code: made.code, actor: 'B' }); + p.fGuild.ownerId = 'new owner'; + await sweepOwners(); + const { checks } = await forBridge(keyedGuild(p.dGuild)); + const owner = checks.find((c) => c.code === 'bridge-stopped'); + assert.ok(owner, 'listed although it is paused, not stopped'); + assert.match(owner.message, /owner of one of the servers changed/); + + await resumeLink({ guildKey: F, id: link.id, actor: 'B' }); + await pauseLink({ guildKey: D, id: link.id, actor: 'A' }); + await handleChannelDelete(dCh, {}, D); + const after = await forBridge(keyedGuild(p.dGuild)); + assert.match(after.checks.find((c) => c.code === 'bridge-stopped').message, /channel was deleted/); +}); diff --git a/test/bridgeQueue.test.js b/test/bridgeQueue.test.js new file mode 100644 index 00000000..c6b9fc53 --- /dev/null +++ b/test/bridgeQueue.test.js @@ -0,0 +1,234 @@ +// The bridge's send queue: one lane per webhook, in order, spaced, with back-off on a rate limit, a cap on +// what waits and a limit on how long a job may wait; and how a failed webhook call is classified. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { BridgeQueueError, classifyError, createRelayQueue } from '../src/modules/lib/bridgeQueue.js'; + +const err = (props) => Object.assign(new Error('x'), props); + +test('a failed webhook call is classified by what it means for the link', () => { + assert.deepEqual(classifyError(err({ status: 429, retryAfter: 2 })), { kind: 'rate', retryAfterMs: 2000 }); + assert.deepEqual(classifyError(err({ statusCode: 429, retry_after: 0.5 })), { + kind: 'rate', + retryAfterMs: 500, + }); + assert.deepEqual( + classifyError(err({ status: 429, retryAfter: 3500 })), + { kind: 'rate', retryAfterMs: 3500 }, + 'milliseconds' + ); + assert.deepEqual( + classifyError(err({ status: 429 })), + { kind: 'rate', retryAfterMs: 2000 }, + 'no hint: two seconds' + ); + assert.equal(classifyError(err({ status: 429, retryAfter: 9999 })).retryAfterMs, 9999); + assert.equal( + classifyError(err({ status: 429, retry_after: 100 })).retryAfterMs, + 60000, + 'the API field is always seconds' + ); + assert.equal(classifyError(err({ status: 429, rawError: { retry_after: 0.1 } })).retryAfterMs, 250); + assert.equal( + classifyError(err({ status: 429, retryAfter: 600000 })).retryAfterMs, + 60000, + 'never longer than a minute' + ); + assert.equal( + classifyError(err({ status: 429, retryAfter: 0.001 })).retryAfterMs, + 250, + 'never shorter than 250 ms' + ); + assert.deepEqual( + classifyError(err({ cause: err({ statusCode: 429, retry_after: 1 }) })), + { kind: 'rate', retryAfterMs: 1000 }, + 'the SDKs wrap the HTTP error in a cause' + ); + + assert.equal(classifyError(err({ code: 10015, status: 404 })).kind, 'hook-gone', 'Unknown Webhook'); + assert.equal(classifyError(err({ code: 'UNKNOWN_WEBHOOK' })).kind, 'hook-gone'); + assert.equal(classifyError(err({ status: 401 })).kind, 'hook-gone', 'a bad webhook token'); + assert.equal(classifyError(err({ status: 404 }), 'send').kind, 'hook-gone'); + assert.equal(classifyError(err({ status: 404 }), 'edit').kind, 'message-gone'); + assert.equal(classifyError(err({ status: 404 }), 'delete').kind, 'message-gone'); + assert.equal(classifyError(err({ code: 10008, status: 404 }), 'edit').kind, 'message-gone'); + assert.equal(classifyError(err({ status: 403 })).kind, 'forbidden'); + assert.equal(classifyError(err({ code: 50013 })).kind, 'forbidden'); + assert.equal(classifyError(err({ status: 400 })).kind, 'bad-request'); + assert.equal(classifyError(err({ status: 413 })).kind, 'bad-request'); + assert.equal(classifyError(err({ status: 500 })).kind, 'transient'); + assert.equal(classifyError(new Error('ECONNRESET')).kind, 'transient'); + assert.equal(classifyError('a string').kind, 'transient'); + assert.equal(classifyError(null).kind, 'transient'); +}); + +function harness(opts = {}) { + const sleeps = []; + let clock = 1_000_000; + const queue = createRelayQueue({ + minGapMs: 100, + sleep: async (ms) => { + sleeps.push(ms); + clock += ms; + }, + now: () => clock, + ...opts, + }); + return { queue, sleeps, tick: (ms) => (clock += ms) }; +} + +test('jobs of one webhook run one at a time, in the order they came, spaced apart', async () => { + const { queue, sleeps } = harness(); + const order = []; + let running = 0; + let peak = 0; + const job = (name) => async () => { + running += 1; + peak = Math.max(peak, running); + await new Promise((r) => setImmediate(r)); + order.push(name); + running -= 1; + return name; + }; + const results = await Promise.all(['a', 'b', 'c'].map((n) => queue.enqueue('hook1', job(n)))); + assert.deepEqual(results, ['a', 'b', 'c']); + assert.deepEqual(order, ['a', 'b', 'c']); + assert.equal(peak, 1); + assert.deepEqual(sleeps, [100, 100], 'a gap after each job that has another behind it'); + assert.equal(queue.lanes(), 0, 'an idle lane is forgotten'); +}); + +test('webhooks do not wait for each other', async () => { + const { queue } = harness(); + const gate = {}; + gate.open = new Promise((r) => (gate.release = r)); + const slow = queue.enqueue('slow', () => gate.open.then(() => 'slow done')); + const fast = await queue.enqueue('fast', async () => 'fast done'); + assert.equal(fast, 'fast done', 'the second webhook finished while the first was stuck'); + gate.release(); + assert.equal(await slow, 'slow done'); +}); + +test('a rate limit is waited out as long as the platform said, then the same job runs again', async () => { + const { queue, sleeps } = harness(); + let calls = 0; + const out = await queue.enqueue('h', async () => { + calls += 1; + if (calls === 1) throw err({ status: 429, retryAfter: 1.25 }); + return 'ok'; + }); + assert.equal(out, 'ok'); + assert.equal(calls, 2); + assert.deepEqual(sleeps, [1250]); +}); + +test('a transient failure backs off 1 s, 2 s, 4 s and gives up after four tries', async () => { + const { queue, sleeps } = harness(); + let calls = 0; + await assert.rejects( + queue.enqueue('h', async () => { + calls += 1; + throw err({ status: 503 }); + }), + (e) => e.failure.kind === 'transient' && e.status === 503 + ); + assert.equal(calls, 4); + assert.deepEqual(sleeps, [1000, 2000, 4000]); + const again = harness(); + let n = 0; + assert.equal( + await again.queue.enqueue('h', async () => (++n < 3 ? Promise.reject(err({ status: 500 })) : 'fine')), + 'fine' + ); + assert.deepEqual(again.sleeps, [1000, 2000]); +}); + +test('a refusal is not retried: forbidden, webhook gone and bad request fail at once, with their kind', async () => { + for (const [props, kind] of [ + [{ status: 403 }, 'forbidden'], + [{ status: 404, code: 10015 }, 'hook-gone'], + [{ status: 400 }, 'bad-request'], + ]) { + const { queue, sleeps } = harness(); + let calls = 0; + await assert.rejects( + queue.enqueue('h', async () => { + calls += 1; + throw err(props); + }), + (e) => e.failure.kind === kind + ); + assert.equal(calls, 1, kind); + assert.deepEqual(sleeps, []); + } +}); + +test('a failed job does not stop the ones behind it', async () => { + const { queue } = harness(); + const bad = queue.enqueue('h', async () => { + throw err({ status: 403 }); + }); + const good = queue.enqueue('h', async () => 'next'); + await assert.rejects(bad); + assert.equal(await good, 'next'); +}); + +test('a full lane drops the newest job and counts it; the jobs already in keep their place', async () => { + const { queue } = harness({ maxQueued: 2 }); + const gate = {}; + gate.open = new Promise((r) => (gate.release = r)); + const first = queue.enqueue('h', () => gate.open.then(() => 1)); + const second = queue.enqueue('h', async () => 2); + const third = queue.enqueue('h', async () => 3); + await assert.rejects(third, (e) => e instanceof BridgeQueueError && e.reason === 'overflow'); + assert.deepEqual(queue.stats('h'), { queued: 2, dropped: 1 }); + assert.deepEqual(queue.stats('other'), { queued: 0, dropped: 0 }, 'another webhook has its own lane'); + gate.release(); + assert.deepEqual(await Promise.all([first, second]), [1, 2]); +}); + +test('a job that waited too long is dropped instead of arriving late', async () => { + const { queue, tick } = harness({ maxAgeMs: 60_000 }); + const gate = {}; + gate.open = new Promise((r) => (gate.release = r)); + const first = queue.enqueue('h', () => gate.open.then(() => 'first')); + const stale = queue.enqueue('h', async () => 'stale'); + const expired = assert.rejects(stale, (e) => e instanceof BridgeQueueError && e.reason === 'expired'); + tick(61_000); + gate.release(); + assert.equal(await first, 'first'); + await expired; +}); + +test('a delete is never dropped: not for a full lane, not for waiting too long, not by clear()', async () => { + const { queue, tick } = harness({ maxQueued: 1, maxAgeMs: 1000 }); + const gate = {}; + gate.open = new Promise((r) => (gate.release = r)); + const running = queue.enqueue('h', () => gate.open.then(() => 'ran')); + const del1 = queue.enqueue('h', async () => 'del1', { op: 'delete' }); + const del2 = queue.enqueue('h', async () => 'del2', { op: 'delete' }); + const edit = queue.enqueue('h', async () => 'edit', { op: 'edit' }); + await assert.rejects( + edit, + (e) => e.reason === 'overflow', + 'an ordinary job is turned away from a full lane' + ); + assert.equal(queue.stats('h').queued, 3, 'the deletes were accepted'); + assert.equal(queue.clear('h'), 0, 'clear() leaves the deletes'); + tick(60_000); + gate.release(); + assert.deepEqual(await Promise.all([running, del1, del2]), ['ran', 'del1', 'del2']); +}); + +test('clearing a lane drops what waits (a paused or unlinked bridge) but lets the running job finish', async () => { + const { queue } = harness(); + const gate = {}; + gate.open = new Promise((r) => (gate.release = r)); + const running = queue.enqueue('h', () => gate.open.then(() => 'ran')); + const waiting = queue.enqueue('h', async () => 'never'); + assert.equal(queue.clear('h'), 1); + await assert.rejects(waiting, (e) => e.reason === 'cleared'); + gate.release(); + assert.equal(await running, 'ran'); + assert.equal(queue.clear('nothing'), 0); +}); diff --git a/test/bridgeText.test.js b/test/bridgeText.test.js new file mode 100644 index 00000000..949ae3c0 --- /dev/null +++ b/test/bridgeText.test.js @@ -0,0 +1,232 @@ +// The pure rules of the bridge: how text, names, replies, files, embeds and options are made safe and +// fit for the other platform. No database and no platform needed. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + EMBED_LIMIT, + FILE_BYTES_LIMIT, + FILE_COUNT_LIMIT, + MESSAGE_LIMIT, + TOTAL_BYTES_LIMIT, + ZWSP, + clip, + escapeMarkdown, + fileLinkLines, + hashContent, + inertAt, + joinParts, + mapEmoji, + neutraliseMentions, + normaliseOptions, + planAttachments, + plainEmbed, + plainEmbeds, + plainTimestamps, + pollText, + replyHead, + senderName, + stickerLines, +} from '../src/modules/lib/bridgeText.js'; + +const plain = (s) => s.split(ZWSP).join(''); + +test('every kind of mention is replaced by a name, and @everyone and @here are broken', () => { + const names = { user: () => 'Bob', role: () => 'mods', channel: () => 'rules' }; + assert.equal( + plain(neutraliseMentions('<@111111> <@!111111> <@&222222> <#333333> @everyone @here @EVERYONE', names)), + '@Bob @Bob @mods #rules @everyone @here @EVERYONE' + ); + assert.equal(plain(neutraliseMentions('<@111111> <@&222222> <#333333>')), '@user @role #channel'); + assert.equal(neutraliseMentions(' and '), '/ban and /mod warn'); + assert.equal( + neutraliseMentions('plain text, an email a@b.c and 5 > 3'), + 'plain text, an email a@b.c and 5 > 3' + ); +}); + +test('a name cannot smuggle a mention: control characters go and an @ is broken', () => { + const out = neutraliseMentions('<@111111>', { user: () => '@everyone‮\u0000x' }); + assert.ok(!/@everyone/.test(out), out); + assert.equal(plain(inertAt('@a@b')), '@a@b'); +}); + +test('custom emoji map by name to the other server, else become :name:; Unicode is untouched', () => { + const find = (name) => + name === 'wave' + ? { id: '999', name: 'wave', animated: false } + : name === 'dance' + ? { id: '888', name: 'dance', animated: true } + : undefined; + assert.equal( + mapEmoji('a <:wave:12345> b c <:gone:11111> 🎉', find), + 'a <:wave:999> b c :gone: 🎉' + ); + assert.equal(mapEmoji('<:wave:12345>'), ':wave:', 'no lookup at all: names only'); +}); + +test('a Discord timestamp is written out in UTC', () => { + assert.equal( + plainTimestamps('at or '), + 'at 14 Nov 2023, 22:13 UTC or 1 Jan 1970, 00:00 UTC' + ); +}); + +test('the sender name carries the platform, fits 80 characters, is never empty, and cannot ping', () => { + assert.equal(senderName('Alice', 'discord'), 'Alice [Discord]'); + assert.equal(senderName('Bob', 'fluxer'), 'Bob [Fluxer]'); + assert.equal(plain(senderName('@everyone', 'fluxer')), '@everyone [Fluxer]'); + assert.equal(senderName('', 'fluxer'), 'Someone [Fluxer]'); + assert.equal(senderName('‮\u0000', 'fluxer'), 'Someone [Fluxer]'); + const long = senderName('x'.repeat(200), 'discord'); + assert.equal(long.length, 80); + assert.ok(long.endsWith('… [Discord]')); + // Discord refuses a webhook name with "discord" or "clyde" in it + const forDiscord = senderName('ClydeBot and Discordian', 'fluxer', { forDiscord: true }); + assert.ok(!/clyde|discord/i.test(forDiscord), forDiscord); + assert.ok(/Clyde/.test(forDiscord.split(ZWSP).join('')), 'the name is still readable'); + assert.ok(senderName('Zoë 🧑‍🚀', 'discord').includes('🧑‍🚀'), 'a joiner in an emoji sequence stays'); +}); + +test('a reply head is a short one-line quote with a link when the copy is known', () => { + assert.equal( + replyHead({ name: 'Bob', snippet: 'hello\nthere world', url: 'https://x/y' }), + '> ↪ [**Bob**](): hello there world' + ); + assert.equal(replyHead({ name: 'Bob', snippet: '' }), '> ↪ **Bob**'); + assert.equal(replyHead({ name: '', snippet: '> nested quote' }), '> ↪ **someone**: nested quote'); + const head = replyHead({ name: 'x'.repeat(100), snippet: 'y'.repeat(500) }); + assert.ok(head.length < 190); + assert.ok(head.endsWith('…')); +}); + +test('stickers become a picture link, and a poll becomes plain text', () => { + assert.deepEqual(stickerLines([{ name: 'wave', url: 'https://cdn/1.png' }, { name: 'nolink' }]), [ + '*sticker: wave* https://cdn/1.png', + '*sticker: nolink*', + ]); + assert.deepEqual(stickerLines(null), []); + assert.equal( + pollText({ question: 'Pizza?', answers: ['Yes', { text: 'No', emoji: '👎' }, { text: '' }] }), + '📊 **Poll:** Pizza?\n• Yes\n• 👎 No\n*Vote on the original server: polls do not cross the bridge.*' + ); + assert.equal(pollText(null), ''); +}); + +test('files are uploaded only within the per-file, total and count limits, the rest are links', () => { + const MB = 1024 * 1024; + const att = (name, size) => ({ name, url: `https://cdn/${name}`, size }); + const plan = planAttachments([ + att('a', 1 * MB), + att('b', 9 * MB), + att('c', 7 * MB), + att('d', 2 * MB), + att('e', 0), + ]); + assert.deepEqual( + plan.upload.map((a) => a.name), + ['a', 'c'], + 'a (1) + c (7) = 8 MB; d would pass the total' + ); + assert.deepEqual( + plan.links.map((a) => a.name), + ['b', 'd', 'e'] + ); + assert.deepEqual( + planAttachments([att('a', 5)], { upload: false }).links.map((a) => a.name), + ['a'] + ); + const many = planAttachments(Array.from({ length: 14 }, (_, i) => att(`f${i}`, 10))); + assert.equal(many.upload.length, FILE_COUNT_LIMIT); + assert.equal(many.links.length, 4); + assert.equal(FILE_BYTES_LIMIT, 8 * MB); + assert.equal(TOTAL_BYTES_LIMIT, 8 * MB); + assert.deepEqual(fileLinkLines([att('a b.png', 3)]), ['📎 a b.png: https://cdn/a b.png']); +}); + +test('only rich embeds cross, as plain API objects, with their text made harmless and their limits kept', () => { + assert.equal(plainEmbed({ type: 'link', url: 'https://x', title: 'preview' }), null); + assert.equal(plainEmbed({ type: 'image', url: 'https://x/a.png' }), null); + assert.equal(plainEmbed({ type: 'rich' }), null, 'an empty embed is nothing'); + const up = (t) => t.toUpperCase(); + const embed = plainEmbed( + { + type: 'rich', + title: 'Hi', + description: 'there', + url: 'javascript:alert(1)', + color: 255, + timestamp: '2030-01-01T00:00:00.000Z', + author: { name: 'me', iconURL: 'https://x/a.png' }, + footer: { text: 'f', iconUrl: 'https://x/f.png' }, + image: { url: 'https://x/i.png' }, + thumbnail: { url: 'ftp://nope' }, + fields: [ + { name: 'n', value: 'v' }, + { name: '', value: 'x' }, + ], + }, + up + ); + assert.deepEqual(embed, { + title: 'HI', + description: 'THERE', + color: 255, + timestamp: '2030-01-01T00:00:00.000Z', + footer: { text: 'F', icon_url: 'https://x/f.png' }, + image: { url: 'https://x/i.png' }, + author: { name: 'ME', icon_url: 'https://x/a.png' }, + fields: [{ name: 'N', value: 'V', inline: false }], + }); + assert.equal( + plainEmbed({ toJSON: () => ({ type: 'rich', title: 't' }) }).title, + 't', + 'a discord.js embed class' + ); + assert.equal( + plainEmbeds(Array.from({ length: 14 }, () => ({ type: 'rich', title: 't' }))).length, + EMBED_LIMIT + ); + assert.deepEqual(plainEmbeds(null), []); +}); + +test('text is cut to the limit without half an emoji, and parts join without blank lines', () => { + assert.equal(clip('abcdef', 4), 'abc…'); + assert.equal(clip('ab😀cd', 4), 'ab…', 'a surrogate pair is not cut in two'); + assert.equal(clip('short', 10), 'short'); + assert.equal(joinParts(['a', '', null, ' b ', undefined, 'c']), 'a\nb\nc'); + assert.equal(joinParts(['x'.repeat(5000)]).length, MESSAGE_LIMIT); +}); + +test('options are always booleans, bots and joins are off by default, files are copied by default', () => { + assert.deepEqual(normaliseOptions(undefined), { bots: false, joinMessages: false, uploadFiles: true }); + assert.deepEqual(normaliseOptions({}), { bots: false, joinMessages: false, uploadFiles: true }); + assert.deepEqual(normaliseOptions({ bots: 'yes', joinMessages: 1, uploadFiles: 'no' }), { + bots: false, + joinMessages: false, + uploadFiles: false, + }); + assert.deepEqual(normaliseOptions({ bots: true, joinMessages: true, uploadFiles: true, extra: 1 }), { + bots: true, + joinMessages: true, + uploadFiles: true, + }); + assert.deepEqual(normaliseOptions('junk'), { bots: false, joinMessages: false, uploadFiles: true }); +}); + +test('the content hash tells a changed text from an unchanged one', () => { + assert.equal(hashContent('a'), hashContent('a')); + assert.notEqual(hashContent('a'), hashContent('b')); + assert.equal(hashContent(null), hashContent('')); +}); + +test('a name cannot turn into a markdown link in Sylo’s own text', () => { + const evil = 'x](https://evil.example)[y'; + assert.ok(!escapeMarkdown(evil).includes('](https')); + assert.equal( + escapeMarkdown('a_b *c* ~d~ `e` |f| >g #h '), + String.raw`a\_b \*c\* \~d\~ \`e\` \|f\| \>g \#h \` + ); + assert.equal(escapeMarkdown('plain name 123'), 'plain name 123'); + assert.ok(!replyHead({ name: evil, snippet: 'hi', url: 'https://x/y' }).includes('](https://evil')); + assert.equal(plain(replyHead({ name: '@everyone', snippet: '' })), '> ↪ **@everyone**'); +}); diff --git a/test/bridgeTransport.test.js b/test/bridgeTransport.test.js new file mode 100644 index 00000000..0b173bd8 --- /dev/null +++ b/test/bridgeTransport.test.js @@ -0,0 +1,88 @@ +// The webhook transport (src/platform/webhooks.js): what both platforms are sent. The one rule that matters +// most is here: every body carries allowedMentions none, whatever the text says. +import './helpers/tmpDb.js'; +import test, { afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { runtime } from '../src/runtime.js'; +import { NO_MENTIONS, createWebhookTransport, webhookBody } from '../src/platform/webhooks.js'; + +afterEach(() => { + runtime.clients.discord = null; + runtime.clients.fluxer = null; +}); + +const FILE = { name: 'a.png', data: Buffer.from('x') }; + +test('a body never allows a mention, and has the platform’s own names for the avatar and files', () => { + const payload = { + content: '@everyone <@&123> hi', + username: 'Alice [Fluxer]', + avatarUrl: 'https://cdn/a.png', + embeds: [{ title: 't' }], + files: [FILE], + }; + const d = webhookBody('discord', payload); + assert.deepEqual(d.allowedMentions, { parse: [] }); + assert.deepEqual(NO_MENTIONS, { parse: [] }); + assert.equal(d.avatarURL, 'https://cdn/a.png'); + assert.equal(d.avatarUrl, undefined); + assert.deepEqual(d.files, [{ attachment: FILE.data, name: 'a.png' }]); + assert.equal(d.username, 'Alice [Fluxer]'); + assert.equal(d.content, '@everyone <@&123> hi'); + + const f = webhookBody('fluxer', payload); + assert.deepEqual(f.allowedMentions, { parse: [] }); + assert.equal(f.avatarUrl, 'https://cdn/a.png'); + assert.equal(f.avatarURL, undefined); + assert.deepEqual(f.files, [{ name: 'a.png', data: FILE.data }]); +}); + +test('an edit body has the text and embeds only, still with no mentions', () => { + for (const platform of ['discord', 'fluxer']) { + const body = webhookBody( + platform, + { content: 'new', username: 'x', avatarUrl: 'https://a/b', files: [FILE], embeds: [] }, + { forEdit: true } + ); + assert.deepEqual(body, { allowedMentions: { parse: [] }, content: 'new', embeds: [] }); + } + assert.deepEqual(webhookBody('discord', {}), { allowedMentions: { parse: [] } }); +}); + +test('a webhook is made through the channel of the right platform, and its token is returned', async () => { + const made = []; + const channel = { createWebhook: async (opts) => (made.push(opts), { id: '77', token: 'tok' }) }; + const guild = (client) => ({ + id: '710000000000000001', + client, + channels: { cache: new Map([['720000000000000001', channel]]) }, + }); + runtime.clients.discord = { guilds: { cache: new Map() } }; + runtime.clients.discord.guilds.cache.set('710000000000000001', guild(runtime.clients.discord)); + const transport = createWebhookTransport(); + assert.deepEqual(await transport.createHook('710000000000000001', '720000000000000001', 'Sylo bridge'), { + id: '77', + token: 'tok', + }); + assert.deepEqual(made, [{ name: 'Sylo bridge', reason: 'Sylo bridge' }]); +}); + +test('a server Sylo is not in, a channel without webhooks and a webhook without a token are refused', async () => { + const transport = createWebhookTransport(); + await assert.rejects( + transport.createHook('710000000000000009', '720000000000000001', 'x'), + (e) => e.code === 'NO_GUILD' + ); + runtime.clients.discord = { guilds: { cache: new Map() } }; + const noHooks = { id: '710000000000000001', channels: { cache: new Map([['720000000000000001', {}]]) } }; + runtime.clients.discord.guilds.cache.set(noHooks.id, noHooks); + await assert.rejects( + transport.createHook(noHooks.id, '720000000000000001', 'x'), + (e) => e.code === 'NO_CHANNEL' + ); + noHooks.channels.cache.set('720000000000000002', { createWebhook: async () => ({ id: '1' }) }); + await assert.rejects( + transport.createHook(noHooks.id, '720000000000000002', 'x'), + /did not return a webhook token/ + ); +}); diff --git a/test/helpers/fakeBridge.js b/test/helpers/fakeBridge.js new file mode 100644 index 00000000..860f36b1 --- /dev/null +++ b/test/helpers/fakeBridge.js @@ -0,0 +1,193 @@ +// A Discord and a Fluxer side for the bridge tests: two fake clients with one server each, text channels that +// record what Sylo says in them, and a fake webhook transport that records every send, edit and delete (and +// can be told to fail). The real transport (src/platform/webhooks.js) is covered separately. +import { ChannelType } from 'discord.js'; +import { runtime } from '../../src/runtime.js'; +import { createRelayQueue } from '../../src/modules/lib/bridgeQueue.js'; +import { useBridgeParts } from '../../src/modules/bridge.js'; + +export const D_RAW = '710000000000000001'; +export const F_RAW = '710000000000000002'; +export const D = D_RAW; +export const F = `fluxer:${F_RAW}`; +export const ME_D = '999000000000000001'; +export const ME_F = '999000000000000002'; + +let nextId = 80_000_000_000_000_000n; +export const snowflake = () => String((nextId += 1n)); + +/** + * `nsfwOverride` and `parentId` are what Fluxer's age restriction reads (own override, else the category's, + * else the server's); the plain `nsfw` is the channel's own mark. + */ +function makeChannel( + guild, + { id, name, nsfw = false, nsfwOverride = null, parentId = null, type = ChannelType.GuildText, deny = [] } +) { + const channel = { + id, + name, + type, + nsfw, + nsfwOverride, + parentId, + rawPosition: 0, + guild, + said: [], + permissionsFor: () => ({ has: (perm) => !deny.includes(perm) }), + send: async (payload) => { + channel.said.push(payload); + return { id: snowflake() }; + }, + }; + guild.channels.cache.set(id, channel); + return channel; +} + +function makeGuild(client, { id, name }) { + const guild = { + id, + name, + client, + ownerId: `owner-of-${id}`, // each server has its own owner (a platform user id) + nsfw: false, // the server's own age restriction (Fluxer) + channels: { cache: new Map() }, + emojis: { cache: new Map() }, + roles: { cache: new Map() }, + members: { me: { id: client.user.id }, cache: new Map() }, + }; + client.guilds.cache.set(id, guild); + return guild; +} + +/** Install the two fake clients; returns both servers and a way to add channels. */ +export function installPlatforms() { + const discord = { user: { id: ME_D }, guilds: { cache: new Map() }, isReady: () => true }; + const fluxer = { user: { id: ME_F }, guilds: { cache: new Map() }, isReady: () => true }; + runtime.clients.discord = discord; + runtime.clients.fluxer = fluxer; + const dGuild = makeGuild(discord, { id: D_RAW, name: 'Discord Home' }); + const fGuild = makeGuild(fluxer, { id: F_RAW, name: 'Fluxer Home' }); + return { + discord, + fluxer, + dGuild, + fGuild, + channel: (guild, opts) => makeChannel(guild, { id: snowflake(), name: 'chat', ...opts }), + /** Another pair of servers (for isolation tests). */ + extra: (suffix) => ({ + dGuild: makeGuild(discord, { id: `71000000000000${suffix}1`, name: `Discord ${suffix}` }), + fGuild: makeGuild(fluxer, { id: `71000000000000${suffix}2`, name: `Fluxer ${suffix}` }), + }), + }; +} + +export function removePlatforms() { + runtime.clients.discord = null; + runtime.clients.fluxer = null; +} + +/** An HTTP-ish error as the SDKs throw it. */ +export const apiError = (status, { code, retryAfter } = {}) => + Object.assign(new Error(`HTTP ${status}`), { + status, + ...(code && { code }), + ...(retryAfter && { retryAfter }), + }); + +/** A webhook transport that records what the bridge does, and fails on request. */ +export function fakeTransport() { + const t = { + hooks: [], + sends: [], + edits: [], + removes: [], + deleted: [], + botDeletes: [], + botDeleteFailures: [], + removeFailures: [], // errors thrown by the next webhook deletes + sendFailures: [], // errors thrown by the next sends, in order + hookFailures: [], // errors thrown by the next createHook calls + attempts: 0, + async createHook(guildKey, channelId, name) { + const next = t.hookFailures.shift(); + if (next) throw next; + const hook = { id: snowflake(), token: `tok-${t.hooks.length + 1}` }; + t.hooks.push({ guildKey, channelId, name, ...hook }); + return hook; + }, + async send(guildKey, hook, payload) { + t.attempts += 1; + const next = t.sendFailures.shift(); + if (next) throw next; + const id = snowflake(); + t.sends.push({ guildKey, hook, payload, id }); + return id; + }, + async edit(guildKey, hook, messageId, payload) { + t.edits.push({ guildKey, hook, messageId, payload }); + }, + async remove(guildKey, hook, messageId) { + const next = t.removeFailures.shift(); + if (next) throw next; + t.removes.push({ guildKey, hook, messageId }); + }, + async botDelete(guildKey, channelId, messageId) { + const next = t.botDeleteFailures.shift(); + if (next) throw next; + t.botDeletes.push({ guildKey, channelId, messageId }); + }, + async deleteHook(guildKey, hook) { + t.deleted.push({ guildKey, hook }); + }, + }; + return t; +} + +/** Use a fake transport, an instant queue and a downloader that serves `files` (url -> bytes). */ +export function useFakes({ files = {}, queue } = {}) { + const transport = fakeTransport(); + const sleeps = []; + const q = + queue ?? + createRelayQueue({ + minGapMs: 0, + sleep: async (ms) => { + if (ms > 0) sleeps.push(ms); // the spacing between sends is 0 here + }, + }); + const downloads = []; + useBridgeParts({ + transport, + queue: q, + download: async (url, maxBytes) => { + downloads.push({ url, maxBytes }); + const body = files[url]; + if (!body) return { ok: false, status: 404, body: Buffer.alloc(0) }; + return { ok: true, status: 200, body: Buffer.isBuffer(body) ? body : Buffer.from(body) }; + }, + }); + return { transport, sleeps, downloads, queue: q }; +} + +/** A chat message as the SDKs deliver it, with what the bridge reads. */ +export function message({ channelId, content = '', author = {}, ...rest }) { + const platformClient = rest.client; + return { + id: snowflake(), + channelId, + content, + type: 0, + system: false, + webhookId: null, + author: { id: snowflake(), username: 'alice', bot: false, ...author }, + member: { displayName: 'Alice' }, + attachments: new Map(), + embeds: [], + stickers: new Map(), + mentions: { users: new Map(), members: new Map(), roles: new Map() }, + reference: null, + client: platformClient, + ...rest, + }; +} diff --git a/test/helpers/mirrorFakes.js b/test/helpers/mirrorFakes.js new file mode 100644 index 00000000..3d392bea --- /dev/null +++ b/test/helpers/mirrorFakes.js @@ -0,0 +1,348 @@ +// Doubles for the server mirror tests: a Discord server in the shape of discord.js (read by snapshot.js), and a +// Fluxer server with the adapter the engine writes through (target.js), which records every call it gets. +import { createEngine } from '../../src/mirror/engine.js'; +import { rolesAbove } from '../../src/mirror/target.js'; + +export const SRC = '900000000000000301'; +export const DST_RAW = '800000000000000001'; +export const DST = `fluxer:${DST_RAW}`; +/** Sylo's own role on a fake Fluxer server: older than anything the mirror makes. */ +export const SYLO_ROLE = '810000000000000000'; + +let seq = 700000000000000000n; +export const nextId = () => String(++seq); + +const flag = (n) => 1n << BigInt(n); +export const BIT = { + ViewChannel: flag(10), + SendMessages: flag(11), + ManageMessages: flag(13), + ManageEvents: flag(33), + ManageRoles: flag(28), + Administrator: flag(3), + CreatePublicThreads: flag(35), +}; + +/** A Discord channel as discord.js shapes it. */ +export function dChannel(over) { + const id = over.id ?? nextId(); + const overwrites = new Map( + (over.overwrites ?? []).map((o) => [ + o.id, + { ...o, allow: { bitfield: o.allow ?? 0n }, deny: { bitfield: o.deny ?? 0n } }, + ]) + ); + const rest = { ...over }; + delete rest.overwrites; + return { + id, + name: 'channel', + type: 0, + parentId: null, + rawPosition: 0, + topic: null, + nsfw: false, + rateLimitPerUser: 0, + isThread: () => false, + permissionOverwrites: { cache: overwrites }, + ...rest, + }; +} + +export const dRole = (over) => ({ + id: over.id ?? nextId(), + name: 'role', + color: 0, + rawPosition: 1, + hoist: false, + mentionable: false, + managed: false, + permissions: { bitfield: 0n }, + ...over, + ...(over.permissions !== undefined ? { permissions: { bitfield: over.permissions } } : {}), +}); + +/** A Discord server. `channels`, `roles`, `emojis` and `stickers` are plain arrays of the shapes above. */ +export function fakeDiscord({ + ownerId = 'discord-owner', + id = SRC, + name = 'Source', + channels = [], + roles = [], + emojis = [], + stickers = [], + // What @everyone can do server-wide: by default what a new Discord server gives it. + everyonePermissions = flag(10) | flag(11) | flag(16), +} = {}) { + const everyone = dRole({ id, name: '@everyone', rawPosition: 0, permissions: everyonePermissions }); + const guild = { + id, + name, + ownerId, + icon: null, + banner: null, + iconURL: () => guild._icon ?? null, + bannerURL: () => guild._banner ?? null, + roles: { cache: new Map([everyone, ...roles].map((r) => [r.id, r])), everyone }, + channels: { cache: new Map(channels.map((c) => [c.id, c])) }, + emojis: { cache: new Map(emojis.map((e) => [e.id, e])) }, + stickers: { cache: new Map(stickers.map((s) => [s.id, s])) }, + }; + return guild; +} + +/** + * A Fluxer server and the adapter for it. The adapter keeps the state in the guild's maps and records each + * call in `calls` as { op, ... }. `failOn(op, errorOrFn)` makes the next call of an operation fail. + */ +export function fakeFluxer({ + id = DST_RAW, + name = 'Target', + existingChannels = [], + missing = [], + // The roles that already exist on the server, as Fluxer numbers them: { id, name, position }. A new server has only + // Sylo's own role, at position 1 (every role starts at position 1 on Fluxer; positions change only when reordered). + extraRoles = [], + syloPosition = 1, + emojiLimit = null, + ownerId = 'fluxer-owner', + clock = null, +} = {}) { + const calls = []; + const afterFailures = new Map(); + // Fluxer ids carry their creation time, like the real ones (the engine adopts only what was made recently). + let minted = 0n; + const mint = () => (clock ? String(((BigInt(clock.now) - 1420070400000n) << 22n) + ++minted) : nextId()); + const failures = new Map(); + const guild = { + id, + name, + ownerId, + channels: new Map(existingChannels.map((c) => [c.id, c])), + roles: new Map([ + [id, { id, name: '@everyone', position: 0, _permissions: String(flag(53) | flag(10)), managed: false }], + [SYLO_ROLE, { id: SYLO_ROLE, name: 'Sylo', position: syloPosition, _permissions: '8', managed: true }], + ...extraRoles.map((r) => [r.id, { _permissions: '0', managed: false, ...r }]), + ]), + emojis: new Map(), + stickers: new Map(), + }; + const fail = (op) => { + const f = failures.get(op) ?? failures.get('*'); + if (!f) return; + const err = typeof f.error === 'function' ? f.error(calls) : f.error; + if (f.times != null) { + f.times -= 1; + if (f.times <= 0) failures.delete(failures.has(op) ? op : '*'); + } + if (err) throw err; + }; + const attempted = []; + // A call counts when it succeeds: `begin` is where a failure injected with failOn() is thrown. + const begin = (op) => { + attempted.push(op); + fail(op); + }; + const done = (op, data = {}) => calls.push({ op, ...data }); + const gone = (what) => Object.assign(new Error(`Unknown ${what}`), { statusCode: 404 }); + const target = { + id, + name, + limits: () => ({ + emoji: emojiLimit ?? 50, + stickers: 50, + emojiBytes: 512 * 1024, + stickerBytes: 512 * 1024, + channelName: 100, + roleName: 100, + topic: 1024, + bitrateMin: 8000, + bitrateMax: 96000, + }), + missingPermissions: () => missing, + // The roles that rank above Sylo's own (Fluxer's order: position, then the older id), like the real adapter. + rolesAboveSylo: (ignore = []) => + rolesAbove(guild.roles.get(SYLO_ROLE), [...guild.roles.values()], new Set([id, ...ignore])), + listChannels: () => + [...guild.channels.values()].map((c) => ({ + id: c.id, + name: c.name, + type: c.type, + parentId: c.parentId ?? null, + })), + listRoles: () => + [...guild.roles.values()].map((r) => ({ + id: r.id, + name: r.name, + managed: Boolean(r.managed), + position: r.position, + })), + listEmojis: () => [...guild.emojis.values()], + listStickers: () => [...guild.stickers.values()], + async editGuild(body) { + begin('editGuild'); + if (body.name) guild.name = body.name; + done('editGuild', { body }); + }, + async createRole(spec) { + begin('createRole'); + const rid = mint(); + guild.roles.set(rid, { id: rid, name: spec.name, position: 1, _permissions: spec.permissions }); + done('createRole', { spec, id: rid }); + const after = afterFailures.get('createRole'); + if (after) { + afterFailures.delete('createRole'); + throw Object.assign(after, { createdId: rid }); + } + return rid; + }, + async editRole(rid, spec) { + begin('editRole'); + const r = guild.roles.get(rid); + if (!r) throw gone('role'); + Object.assign(r, { _permissions: spec.permissions, ...(spec.name ? { name: spec.name } : {}) }); + done('editRole', { id: rid, spec }); + }, + async deleteRole(rid) { + begin('deleteRole'); + if (!guild.roles.delete(rid)) throw gone('role'); + done('deleteRole', { id: rid }); + }, + // Like Fluxer: only the roles below Sylo's own are rearranged, into the slots they already hold, by the positions + // sent (ties keep their order); then every role is renumbered. Sylo's role never moves. + async setRolePositions(updates) { + begin('setRolePositions'); + const older = (a, b) => (BigInt(a.id) < BigInt(b.id) ? -1 : 1); + const current = [...guild.roles.values()] + .filter((r) => r.id !== id) + .sort((a, b) => b.position - a.position || older(a, b)); + const sylo = current.findIndex((r) => r.id === SYLO_ROLE); + const explicit = new Map(updates.map((u) => [u.id, u.position])); + const slots = current.map((_, i) => i).filter((i) => i > sylo); + const movable = slots + .map((i) => ({ role: current[i], i })) + .sort( + (a, b) => + (explicit.get(b.role.id) ?? b.role.position) - (explicit.get(a.role.id) ?? a.role.position) || + a.i - b.i + ); + const order = [...current]; + slots.forEach((slot, k) => { + order[slot] = movable[k].role; + }); + order.forEach((r, i) => { + r.position = order.length - i; + }); + done('setRolePositions', { updates }); + }, + async createChannel(spec) { + begin('createChannel'); + const cid = mint(); + guild.channels.set(cid, { + id: cid, + name: spec.name, + type: spec.type, + parentId: spec.parentId ?? null, + position: 0, + }); + done('createChannel', { spec, id: cid }); + return cid; + }, + async editChannel(cid, spec) { + begin('editChannel'); + const c = guild.channels.get(cid); + if (!c) throw gone('channel'); + Object.assign(c, { name: spec.name, parentId: spec.parentId ?? null }); + done('editChannel', { id: cid, spec }); + }, + async deleteChannel(cid) { + begin('deleteChannel'); + if (!guild.channels.delete(cid)) throw gone('channel'); + done('deleteChannel', { id: cid }); + }, + async setChannelPositions(updates) { + begin('setChannelPositions'); + done('setChannelPositions', { updates }); + }, + async createEmoji({ name, image }) { + begin('createEmoji'); + const eid = mint(); + guild.emojis.set(eid, { id: eid, name }); + done('createEmoji', { name, bytes: image.bytes.length, id: eid }); + return eid; + }, + async editEmoji(eid, { name }) { + begin('editEmoji'); + guild.emojis.get(eid).name = name; + done('editEmoji', { id: eid, name }); + }, + async deleteEmoji(eid) { + begin('deleteEmoji'); + guild.emojis.delete(eid); + done('deleteEmoji', { id: eid }); + }, + async createSticker(spec) { + begin('createSticker'); + const sid = mint(); + guild.stickers.set(sid, { id: sid, name: spec.name }); + done('createSticker', { name: spec.name, id: sid }); + return sid; + }, + async editSticker(sid, spec) { + begin('editSticker'); + done('editSticker', { id: sid, name: spec.name }); + }, + async deleteSticker(sid) { + begin('deleteSticker'); + guild.stickers.delete(sid); + done('deleteSticker', { id: sid }); + }, + }; + return { + guild, + target, + calls, + attempted, + failOn: (op, error, times = 1) => failures.set(op, { error, times }), + clearFailures: () => failures.clear(), + // The next create succeeds on Fluxer and then fails (a second call after the create), carrying the new id. + failAfterCreate: (op, error) => afterFailures.set(op, error), + ops: (op) => calls.filter((c) => c.op === op), + }; +} + +/** A fake clock and a sleep that moves it, so spacing and backoff are checked without waiting. */ +export function fakeClock(start = 1_800_000_000_000) { + const clock = { now: start, slept: [] }; + clock.fn = () => clock.now; + clock.sleep = async (ms) => { + clock.slept.push(ms); + clock.now += ms; + }; + return clock; +} + +/** An engine wired to the fakes. */ +export function engineFor({ discord, fluxer, clock = fakeClock(), images = true, ...rest }) { + const fetched = []; + const engine = createEngine({ + now: clock.fn, + sleep: clock.sleep, + minIntervalMs: 10, + backoffBaseMs: 100, + sourceGuild: (key) => (key === discord.id ? discord : undefined), + targetGuild: (key) => (key === `fluxer:${fluxer.guild.id}` ? fluxer.guild : undefined), + makeTarget: () => fluxer.target, + clientsReady: () => true, + enforceOwnership: () => false, + fetchImage: async (url, { maxBytes }) => { + fetched.push(url); + if (!images) throw new Error('no network in tests'); + const size = /big/.test(url) ? maxBytes + 1 : 64; + if (size > maxBytes) throw new Error(`Response too large (over ${maxBytes} bytes)`); + return { bytes: Buffer.alloc(size, 1), mime: 'image/png' }; + }, + ...rest, + }); + return { engine, fetched, clock }; +} diff --git a/test/helpers/mirrorScenario.js b/test/helpers/mirrorScenario.js new file mode 100644 index 00000000..fc6e2dea --- /dev/null +++ b/test/helpers/mirrorScenario.js @@ -0,0 +1,139 @@ +// A server worth copying, a Fluxer server, an engine and an agreed pair: the setup of the engine tests. +import * as db from '../../src/db/mirror.js'; +import { requestMirror } from '../../src/mirror/consent.js'; +import { BIT, dChannel, dRole, engineFor, fakeClock, fakeDiscord, fakeFluxer } from './mirrorFakes.js'; + +let n = 0; +export const ids = () => { + n += 1; + const tag = String(n).padStart(3, '0'); + return { + src: `93000000000000${tag}`, + dst: `83000000000000${tag}`, + cat: `73100000000000${tag}`, + rules: `73200000000000${tag}`, + chat: `73300000000000${tag}`, + cat2: `73400000000000${tag}`, + voice: `73500000000000${tag}`, + member: `73600000000000${tag}`, + mod: `73700000000000${tag}`, + bot: `73800000000000${tag}`, + emoji1: `73900000000000${tag}`, + emoji2: `74000000000000${tag}`, + emoji3: `74100000000000${tag}`, + sticker1: `74200000000000${tag}`, + sticker2: `74300000000000${tag}`, + }; +}; + +/** A Discord server worth copying, a Fluxer server, an engine, and a pair that both sides agreed to. */ +export async function scenario({ + fluxerOpts = {}, + discordOpts = {}, + options, + engineOpts = {}, + withExpressions = true, +} = {}) { + const k = ids(); + const discord = fakeDiscord({ + id: k.src, + name: 'Source', + roles: [ + dRole({ + id: k.member, + name: 'Member', + rawPosition: 1, + permissions: BIT.ViewChannel | BIT.SendMessages | BIT.ManageEvents, + color: 255, + hoist: true, + }), + dRole({ id: k.mod, name: 'Mod', rawPosition: 2, permissions: BIT.ManageMessages, mentionable: true }), + dRole({ id: k.bot, name: 'A bot', rawPosition: 3, managed: true }), + ], + channels: [ + dChannel({ id: k.cat, name: 'Info', type: 4, rawPosition: 0 }), + dChannel({ + id: k.rules, + name: 'rules', + type: 0, + parentId: k.cat, + rawPosition: 1, + topic: 'Read these', + overwrites: [ + { id: k.src, type: 0, deny: BIT.SendMessages }, + { id: k.member, type: 0, allow: BIT.ViewChannel }, + ], + }), + dChannel({ id: k.chat, name: 'chat', type: 0, rawPosition: 2, rateLimitPerUser: 5 }), + dChannel({ id: k.cat2, name: 'Voice', type: 4, rawPosition: 3 }), + dChannel({ id: k.voice, name: 'Lounge', type: 2, parentId: k.cat2, rawPosition: 4, userLimit: 4 }), + ], + emojis: withExpressions + ? [ + { + id: k.emoji1, + name: 'party', + animated: false, + managed: false, + imageURL: () => 'https://cdn.test/e1.png', + }, + { + id: k.emoji2, + name: 'wave', + animated: true, + managed: false, + imageURL: () => 'https://cdn.test/e2.gif', + }, + ] + : [], + stickers: withExpressions + ? [ + { + id: k.sticker1, + name: 'hello', + description: 'Hi', + tags: 'hi,wave', + format: 1, + url: 'https://cdn.test/s1.png', + }, + { + id: k.sticker2, + name: 'anim', + description: '', + tags: '', + format: 3, + url: 'https://cdn.test/s2.json', + }, + ] + : [], + ...discordOpts, + }); + discord._icon = 'https://cdn.test/icon.png'; + const clock = fakeClock(); + const fluxer = fakeFluxer({ id: k.dst, ...fluxerOpts, clock }); + const { engine, fetched } = engineFor({ discord, fluxer, clock, ...engineOpts }); + const { pair } = await requestMirror({ + sourceGuild: k.src, + targetGuild: `fluxer:${k.dst}`, + actor: 'Anna', + sourceUser: 'discord-owner', + targetUser: 'fluxer-owner', + targetOwner: true, + options, + now: clock.now, + }); + return { k, discord, fluxer, clock, engine, fetched, pair }; +} + +/** Work the queue until the copy is over (moving the clock past any wait). */ +export async function drain(s, { rounds = 60 } = {}) { + for (let i = 0; i < rounds; i += 1) { + await s.engine.runPair(s.pair.id); + const now = await db.getPair(s.pair.id); + if (!['copying'].includes(now.status)) return now; + s.clock.now += 5000; + } + throw new Error('the copy did not finish'); +} + +export const names = (calls) => calls.map((c) => c.spec?.name ?? c.name); diff --git a/test/mirror.db.test.js b/test/mirror.db.test.js new file mode 100644 index 00000000..3e3e2edf --- /dev/null +++ b/test/mirror.db.test.js @@ -0,0 +1,275 @@ +// The mirror's tables (src/db/mirror.js): pairs that are unique per server, the id map, and the queue that +// makes a copy resumable. The same file runs against Postgres in mirror.postgres.test.js. +import './helpers/tmpDb.js'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import * as db from '../src/db/mirror.js'; +import { MIGRATIONS, db as sqlite } from '../src/db/index.js'; +import { purgeGuild } from '../src/db/purge.js'; + +const A = '910000000000000001'; +const B = 'fluxer:810000000000000001'; +const C = '910000000000000002'; +const D = 'fluxer:810000000000000002'; + +test('a pair round-trips, and a server can be in only one pair', async () => { + const pair = await db.createPair({ + sourceGuild: A, + targetGuild: B, + options: { roles: false }, + requestedBy: 'x', + now: 1000, + }); + assert.equal(pair.status, 'pending'); + assert.deepEqual(pair.options, { roles: false }); + assert.equal(typeof pair.createdAt, 'number'); + await assert.rejects(db.createPair({ sourceGuild: A, targetGuild: D }), 'the source is taken'); + await assert.rejects(db.createPair({ sourceGuild: C, targetGuild: B }), 'the target is taken'); + assert.equal((await db.getPairBySource(A)).id, pair.id); + assert.equal((await db.getPairByTarget(B)).id, pair.id); + assert.equal((await db.pairsOfGuild(B)).length, 1); + assert.equal(await db.getPairBySource(C), null); + + const changed = await db.updatePair(pair.id, { status: 'ready', copyRun: 2, options: { name: false } }); + assert.equal(changed.status, 'ready'); + assert.equal(changed.copyRun, 2); + assert.deepEqual(changed.options, { name: false }); + assert.deepEqual( + (await db.listPairsIn('ready')).map((p) => p.id), + [pair.id] + ); +}); + +test('the id map keeps what Sylo made and whether it owns it', async () => { + const pair = await db.getPairBySource(A); + await db.putMapping({ + pairId: pair.id, + kind: 'channel', + sourceId: '1', + targetId: 't1', + owned: true, + specHash: 'h1', + }); + await db.putMapping({ pairId: pair.id, kind: 'role', sourceId: '2', targetId: 't2', owned: false }); + let m = await db.getMapping(pair.id, 'channel', '1'); + assert.deepEqual([m.targetId, m.owned, m.specHash], ['t1', true, 'h1']); + assert.equal((await db.getMappingByTarget(pair.id, 'channel', 't1')).sourceId, '1'); + assert.equal((await db.getMapping(pair.id, 'role', '2')).owned, false); + await db.putMapping({ + pairId: pair.id, + kind: 'channel', + sourceId: '1', + targetId: 't1', + owned: true, + specHash: 'h2', + }); + m = await db.getMapping(pair.id, 'channel', '1'); + assert.equal(m.specHash, 'h2', 'a second put replaces the first'); + assert.equal((await db.listMappings(pair.id, 'channel')).length, 1); + await db.deleteMapping(pair.id, 'channel', '1'); + assert.equal(await db.getMapping(pair.id, 'channel', '1'), null); +}); + +test('jobs: coalesce while waiting, claim once, retry, recover and count', async () => { + const pair = await db.getPairBySource(A); + const add = (extra = {}) => + db.enqueueJob({ + pairId: pair.id, + phase: 'sync', + op: 'sync', + kind: 'channel', + sourceId: '9', + stepKey: 'sync:channel:9', + now: 5000, + ...extra, + }); + assert.equal(await add(), true); + assert.equal(await add(), false, 'the same item is already waiting'); + const first = await db.nextDueJob(pair.id, 5000); + assert.equal(first.stepKey, 'sync:channel:9'); + assert.equal(await db.claimJob(first.id, 5000), true); + assert.equal(await db.claimJob(first.id, 5000), false, 'taken once'); + assert.equal(await add(), true, 'a change while the job runs is a new job'); + assert.equal(await db.openSyncCount(pair.id), 2); + + // A restart: the running job waits again, but the newer one for the same step wins, so it is dropped. + await db.recoverRunningJobs(6000); + const dropped = await db.getJob(first.id); + assert.equal(dropped.status, 'skipped'); + assert.equal(await db.openSyncCount(pair.id), 1); + const second = await db.nextDueJob(pair.id, 6000); + assert.equal(await db.claimJob(second.id, 6000), true); + await db.recoverRunningJobs(6500); + const recovered = await db.getJob(second.id); + assert.equal(recovered.status, 'pending'); + assert.equal(recovered.recovering, true); + + await db.retryJob(second.id, 2, 9000, 'later'); + assert.equal(await db.nextDueJob(pair.id, 8000), null, 'a retried job waits until it is due'); + assert.equal(await db.nextDueAt(pair.id), 9000); + assert.equal((await db.nextDueJob(pair.id, 9000)).id, second.id); + + await db.finishJob(second.id, 'failed', 'boom', [{ code: 'x' }], 7000); + const done = await db.getJob(second.id); + assert.deepEqual( + [done.status, done.detail, done.notes.length, done.recovering], + ['failed', 'boom', 1, false] + ); + assert.equal((await db.listFailedJobs(pair.id)).length, 1); + + await db.enqueueJob({ + pairId: pair.id, + run: 1, + phase: 'copy', + op: 'sync', + kind: 'role', + sourceId: '3', + stepKey: 'copy:1:role:3', + now: 1, + }); + await db.enqueueJob({ + pairId: pair.id, + run: 1, + phase: 'copy', + op: 'sync', + kind: 'role', + sourceId: '4', + stepKey: 'copy:1:role:4', + now: 1, + }); + const j = await db.nextDueJob(pair.id, 10_000); + await db.finishJob(j.id, 'done'); + const counts = await db.jobCounts(pair.id, 1); + assert.equal(counts.pending + counts.done, 2); + assert.equal(typeof counts.done, 'number'); + assert.equal(await db.openCopyCount(pair.id, 1), 1); + + await db.dropOpenJobs(pair.id, 'stopped'); + assert.equal(await db.openCopyCount(pair.id, 1), 0); + assert.equal(await db.openSyncCount(pair.id), 0); +}); + +test('purging a server removes every pair it is in, with the map and the queue', async () => { + const other = await db.createPair({ sourceGuild: C, targetGuild: D }); + await db.putMapping({ pairId: other.id, kind: 'role', sourceId: '1', targetId: 't' }); + await db.enqueueJob({ + pairId: other.id, + phase: 'sync', + op: 'sync', + kind: 'role', + sourceId: '1', + stepKey: 'k', + }); + const mine = await db.getPairBySource(A); + await purgeGuild(B); // the Fluxer side of the first pair + assert.equal(await db.getPair(mine.id), null); + assert.equal((await db.listMappings(mine.id, 'channel')).length, 0); + assert.ok(await db.getPair(other.id), 'the other pair is untouched'); + assert.equal((await db.listMappings(other.id, 'role')).length, 1); + await purgeGuild(C); + assert.equal(await db.getPair(other.id), null); + assert.equal(await db.openSyncCount(other.id), 0); +}); + +test('the id map notes what was removed on Discord, and what a pair has made', async () => { + const pair = await db.createPair({ + sourceGuild: '910000000000000010', + targetGuild: 'fluxer:810000000000000010', + }); + await db.putMapping({ pairId: pair.id, kind: 'channel', sourceId: '1', targetId: 't1', label: 'general' }); + await db.putMapping({ pairId: pair.id, kind: 'role', sourceId: '2', targetId: 't2', label: 'Mod' }); + assert.equal((await db.getMapping(pair.id, 'channel', '1')).label, 'general'); + assert.equal(await db.countMappings(pair.id), 2); + assert.deepEqual(await db.listRemoved(pair.id), []); + await db.markRemoved(pair.id, 'channel', '1', 5000); + const removed = await db.listRemoved(pair.id); + assert.deepEqual( + removed.map((m) => [m.kind, m.label, m.removedAt]), + [['channel', 'general', 5000]] + ); + // Syncing the same item again clears the mark. + await db.putMapping({ pairId: pair.id, kind: 'channel', sourceId: '1', targetId: 't1', label: 'general' }); + assert.deepEqual(await db.listRemoved(pair.id), []); + await db.deletePair(pair.id); +}); + +test('the burst brake counts the delete jobs a pair queued in a window', async () => { + const pair = await db.createPair({ + sourceGuild: '910000000000000011', + targetGuild: 'fluxer:810000000000000011', + }); + const queue = (op, id, now) => + db.enqueueJob({ + pairId: pair.id, + phase: 'sync', + op, + kind: 'channel', + sourceId: id, + stepKey: `${op}:${id}`, + now, + }); + await queue('delete', 'a', 1000); + await queue('delete', 'b', 2000); + await queue('sync', 'c', 2000); + await queue('delete', 'a', 3000); // the same item again while waiting: still one job + assert.equal(await db.countRecentDeletes(pair.id, 0), 2); + assert.equal(await db.countRecentDeletes(pair.id, 1500), 1); + await db.deletePair(pair.id); +}); + +test('a job remembers when it was first taken, and a retry after a timeout is marked as maybe done', async () => { + const pair = await db.createPair({ + sourceGuild: '910000000000000012', + targetGuild: 'fluxer:810000000000000012', + }); + await db.enqueueJob({ + pairId: pair.id, + phase: 'sync', + op: 'sync', + kind: 'role', + sourceId: '1', + stepKey: 'k', + now: 100, + }); + const job = await db.nextDueJob(pair.id, 100); + assert.equal(job.firstClaimAt, 0); + await db.claimJob(job.id, 500); + await db.retryJob(job.id, 1, 600, 'timed out', 550, { maybeDone: true }); + await db.claimJob(job.id, 700); + const again = await db.getJob(job.id); + assert.deepEqual([again.firstClaimAt, again.recovering], [500, true], 'the first claim time is kept'); + await db.deletePair(pair.id); +}); + +test('the SQLite migration can run twice without failing (IF NOT EXISTS)', () => { + const step = MIGRATIONS[MIGRATIONS.length - 1]; + assert.doesNotThrow(() => step(sqlite)); + assert.doesNotThrow(() => step(sqlite)); + const tables = sqlite + .prepare("SELECT name FROM sqlite_master WHERE type = 'table' AND name LIKE 'mirror_%' ORDER BY name") + .all() + .map((r) => r.name); + assert.deepEqual(tables, ['mirror_ids', 'mirror_jobs', 'mirror_links', 'mirror_pairs']); +}); + +test('the bridge links of a pair: one per Discord channel, found again by link, gone with the pair', async () => { + const pair = await db.createPair({ sourceGuild: C, targetGuild: D, now: 1000 }); + await db.putMirrorLink(pair.id, 'c1', 't1', 11, 2000); + await db.putMirrorLink(pair.id, 'c2', 't2', 12, 2100); + await db.putMirrorLink(pair.id, 'c1', 'other', 99, 2200); // the first record of a channel stays + const links = await db.listMirrorLinks(pair.id); + assert.deepEqual( + links.map((l) => [l.sourceChannel, l.targetChannel, l.linkId, l.createdAt]), + [ + ['c1', 't1', 11, 2000], + ['c2', 't2', 12, 2100], + ] + ); + assert.equal((await db.getMirrorLink(pair.id, 'c2')).linkId, 12); + assert.equal(await db.getMirrorLink(pair.id, 'nope'), null); + assert.equal(await db.mirrorPairOfLink(12), pair.id); + assert.equal(await db.mirrorPairOfLink(404), null, 'a link made by hand belongs to no mirror'); + await db.deletePair(pair.id); + assert.deepEqual(await db.listMirrorLinks(pair.id), []); + assert.equal(await db.mirrorPairOfLink(12), null); +}); diff --git a/test/mirror.postgres.test.js b/test/mirror.postgres.test.js new file mode 100644 index 00000000..aeaf3a78 --- /dev/null +++ b/test/mirror.postgres.test.js @@ -0,0 +1,176 @@ +// The mirror's tables against a real Postgres connection: serial ids, BIGINT times that come back as numbers, +// the partial unique index that coalesces waiting jobs, ON CONFLICT upserts and the purge. +import './helpers/isolateSqlite.js'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { closePostgres } from '../src/db/driver.js'; + +const url = process.env.DATABASE_URL; + +test( + 'server mirror tables against a real Postgres connection', + { skip: !url && 'DATABASE_URL not set (sqlite-only run)' }, + async (t) => { + const db = await import('../src/db/mirror.js'); + const { purgeGuild } = await import('../src/db/purge.js'); + t.after(async () => { + await closePostgres(); + }); + const n = Date.now(); + const A = `9${String(n).padStart(17, '0')}`.slice(0, 18); + const B = `fluxer:8${String(n).padStart(17, '0')}`.slice(0, 26); + const C = `9${String(n + 1).padStart(17, '0')}`.slice(0, 18); + + await t.test('pairs are unique per server and times are numbers', async () => { + const pair = await db.createPair({ + sourceGuild: A, + targetGuild: B, + options: { roles: false }, + now: 1_800_000_000_000, + }); + assert.equal(typeof pair.id, 'number'); + assert.equal(pair.createdAt, 1_800_000_000_000); + assert.equal(typeof pair.createdAt, 'number'); + await assert.rejects(db.createPair({ sourceGuild: A, targetGuild: `${B}9` })); + const next = await db.updatePair(pair.id, { status: 'ready', copyRun: 3, options: { name: false } }); + assert.deepEqual([next.status, next.copyRun, next.options.name], ['ready', 3, false]); + assert.equal((await db.pairsOfGuild(B)).length, 1); + }); + + await t.test('the id map upserts and deletes', async () => { + const pair = await db.getPairBySource(A); + await db.putMapping({ + pairId: pair.id, + kind: 'channel', + sourceId: '1', + targetId: 't1', + owned: true, + specHash: 'a', + }); + await db.putMapping({ + pairId: pair.id, + kind: 'channel', + sourceId: '1', + targetId: 't1', + owned: false, + specHash: 'b', + }); + const m = await db.getMapping(pair.id, 'channel', '1'); + assert.deepEqual([m.owned, m.specHash], [false, 'b']); + assert.equal((await db.getMappingByTarget(pair.id, 'channel', 't1')).sourceId, '1'); + await db.deleteMapping(pair.id, 'channel', '1'); + assert.equal(await db.getMapping(pair.id, 'channel', '1'), null); + }); + + await t.test('waiting jobs coalesce, claim once, requeue and count', async () => { + const pair = await db.getPairBySource(A); + const add = () => + db.enqueueJob({ + pairId: pair.id, + phase: 'sync', + op: 'sync', + kind: 'role', + sourceId: '5', + stepKey: 'sync:role:5', + now: 5000, + }); + assert.equal(await add(), true); + assert.equal(await add(), false); + const job = await db.nextDueJob(pair.id, 5000); + assert.equal(typeof job.nextAt, 'number'); + assert.equal(await db.claimJob(job.id), true); + assert.equal(await db.claimJob(job.id), false); + assert.equal(await add(), true, 'a new change while it runs'); + await db.recoverRunningJobs(); + assert.equal( + (await db.getJob(job.id)).status, + 'skipped', + 'the newer waiting job replaces the recovered one' + ); + const waiting = await db.nextDueJob(pair.id, 5000); + await db.retryJob(waiting.id, 1, 9000, 'later'); + assert.equal(await db.nextDueAt(pair.id), 9000); + assert.equal(typeof (await db.nextDueAt(pair.id)), 'number'); + assert.equal(await db.nextDueJob(pair.id, 8000), null); + await db.finishJob(waiting.id, 'failed', 'boom', [{ code: 'x' }]); + assert.equal((await db.listFailedJobs(pair.id))[0].notes.length, 1); + assert.equal(typeof (await db.jobCounts(pair.id, 0, 'sync')).failed, 'number'); + assert.equal(await db.openSyncCount(pair.id), 0); + }); + + await t.test('removed marks, mapping counts, the burst count and the first claim time', async () => { + const pair = await db.getPairBySource(A); + await db.putMapping({ + pairId: pair.id, + kind: 'channel', + sourceId: '7', + targetId: 't7', + label: 'general', + }); + await db.markRemoved(pair.id, 'channel', '7', 5000); + const removed = await db.listRemoved(pair.id); + assert.deepEqual( + removed.map((m) => [m.label, m.removedAt]), + [['general', 5000]] + ); + assert.equal(typeof removed[0].removedAt, 'number'); + assert.ok((await db.countMappings(pair.id)) >= 1); + await db.enqueueJob({ + pairId: pair.id, + phase: 'sync', + op: 'delete', + kind: 'channel', + sourceId: 'z', + stepKey: 'delete:z', + now: 9000, + }); + assert.equal(await db.countRecentDeletes(pair.id, 8000), 1); + assert.equal(typeof (await db.countRecentDeletes(pair.id, 0)), 'number'); + const job = await db.nextDueJob(pair.id, 9000); + await db.claimJob(job.id, 9500); + await db.retryJob(job.id, 1, 9600, 'x', 9550, { maybeDone: true }); + await db.claimJob(job.id, 9700); + const again = await db.getJob(job.id); + assert.deepEqual([again.firstClaimAt, again.recovering], [9500, true]); + assert.equal(typeof again.firstClaimAt, 'number'); + await db.finishJob(job.id, 'done'); + const next = await db.updatePair(pair.id, { + pauseKind: 'deletes', + pausedBy: 'system', + sourceConsentUser: 'u1', + }); + assert.deepEqual([next.pauseKind, next.pausedBy, next.sourceConsentUser], ['deletes', 'system', 'u1']); + }); + + await t.test('the bridge links of a pair are kept once per channel and go with the pair', async () => { + const pair = await db.createPair({ sourceGuild: C, targetGuild: `${B}5`, now: 1_800_000_000_000 }); + await db.putMirrorLink(pair.id, 'c1', 't1', 11, 1_800_000_000_100); + await db.putMirrorLink(pair.id, 'c1', 'x', 12); + await db.putMirrorLink(pair.id, 'c2', 't2', 13); + const links = await db.listMirrorLinks(pair.id); + assert.deepEqual( + links.map((l) => [l.sourceChannel, l.targetChannel, l.linkId]), + [ + ['c1', 't1', 11], + ['c2', 't2', 13], + ] + ); + assert.equal(typeof links[0].createdAt, 'number'); + assert.equal(await db.mirrorPairOfLink(13), pair.id); + await assert.rejects(db.putMirrorLink(pair.id, 'c3', 't3', 13), 'a link belongs to one record'); + await db.deletePair(pair.id); + assert.deepEqual(await db.listMirrorLinks(pair.id), []); + }); + + await t.test('purging a server removes its pair, map and queue', async () => { + const pair = await db.getPairBySource(A); + await db.putMapping({ pairId: pair.id, kind: 'role', sourceId: '2', targetId: 't2' }); + const other = await db.createPair({ sourceGuild: C, targetGuild: `${B}7` }); + await purgeGuild(A); + assert.equal(await db.getPair(pair.id), null); + assert.equal((await db.listMappings(pair.id, 'role')).length, 0); + assert.ok(await db.getPair(other.id)); + await purgeGuild(C); + }); + } +); diff --git a/test/mirrorBridgeEngine.test.js b/test/mirrorBridgeEngine.test.js new file mode 100644 index 00000000..ed3a2760 --- /dev/null +++ b/test/mirrorBridgeEngine.test.js @@ -0,0 +1,320 @@ +// How the mirror engine uses the bridge step and keeps its links in step, and the two small mirror fixes that came +// with the wiring: a role made again during "copy again" is ordered before it gets its powerful permissions, and a +// change of Fluxer owner turns "Mirror deletions" off until the new owner switches it on. +import './helpers/tmpDb.js'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import * as db from '../src/db/mirror.js'; +import { requestMirror } from '../src/mirror/consent.js'; +import { BIT, SYLO_ROLE, dChannel } from './helpers/mirrorFakes.js'; +import { drain, scenario } from './helpers/mirrorScenario.js'; + +const ADMIN = 1n << 3n; +const roleBits = (s, id) => BigInt(s.fluxer.guild.roles.get(id)._permissions); + +// --- the bridge step --------------------------------------------------------------------------------------- + +test('the bridge step is the last step of a copy, and its result shows in the report', async () => { + const calls = []; + const s = await scenario({ + withExpressions: false, + engineOpts: { + linkMirroredChannels: async (id, opts) => { + calls.push([id, [...opts.skip]]); + return { + linked: 3, + already: 0, + remaining: 0, + tried: [], + skipped: [ + { sourceId: '11', name: 'adult', code: 'age-restriction', message: 'the ages do not match' }, + ], + }; + }, + }, + }); + await s.engine.startCopy(s.pair.id); + assert.equal((await drain(s)).status, 'live'); + assert.deepEqual(calls, [[s.pair.id, []]]); + const report = await s.engine.report(await db.getPair(s.pair.id)); + assert.equal(report.bridge.status, 'done'); + assert.equal(report.bridge.detail, '3 channels linked, 1 not linked (see the notes)'); + const note = report.notes.find((n) => n.code === 'bridge-age-restriction'); + assert.deepEqual(note.ref, { kind: 'channel', id: '11', name: 'adult' }); + assert.match(note.message, /adult: not linked through the bridge, the ages do not match/); +}); + +test('when more channels wait than one run links, another run follows and does not try the same ones again', async () => { + const calls = []; + const answers = [ + { + linked: 2, + already: 1, + remaining: 2, + tried: ['a', 'b', 'c'], + skipped: [{ sourceId: 'c', name: 'c', code: 'permissions', message: 'Sylo lacks a permission' }], + }, + { linked: 2, already: 3, remaining: 0, tried: ['d', 'e'], skipped: [] }, + ]; + const s = await scenario({ + withExpressions: false, + engineOpts: { + linkMirroredChannels: async (id, opts) => { + calls.push([...opts.skip]); + return answers[calls.length - 1]; + }, + }, + }); + await s.engine.startCopy(s.pair.id); + assert.equal((await drain(s)).status, 'live', 'the copy waits for the whole bridge step'); + assert.deepEqual(calls, [[], ['a', 'b', 'c']]); + const report = await s.engine.report(await db.getPair(s.pair.id)); + assert.equal(report.bridge.status, 'done'); + assert.equal( + report.bridge.detail, + '2 channels linked, 1 already linked, 1 not linked (see the notes), 2 more follow; 2 channels linked, 3 already linked' + ); +}); + +test('a step that links nothing says why', async () => { + const s = await scenario({ + withExpressions: false, + engineOpts: { linkMirroredChannels: async () => ({ linked: 0, reason: 'nothing-to-link' }) }, + }); + await s.engine.startCopy(s.pair.id); + await drain(s); + assert.equal( + (await s.engine.report(await db.getPair(s.pair.id))).bridge.detail, + 'not linked: nothing-to-link' + ); +}); + +test('a channel made on Discord later is linked too: the bridge step is asked for again', async () => { + const calls = []; + const s = await scenario({ + withExpressions: false, + engineOpts: { + linkMirroredChannels: async (id) => { + calls.push(id); + return { linked: 1, already: 0, remaining: 0, tried: [], skipped: [] }; + }, + }, + }); + await s.engine.startCopy(s.pair.id); + await drain(s); + assert.equal(calls.length, 1); + const newId = '750000000000000077'; + s.discord.channels.cache.set(newId, dChannel({ id: newId, name: 'fresh', type: 0, rawPosition: 9 })); + await s.engine.noteSourceChange(s.k.src, { kind: 'channel', op: 'sync', id: newId }); + await s.engine.runPair(s.pair.id); + assert.equal(calls.length, 2, 'linked after the channel was made'); + // an edit of an existing channel does not ask again + s.discord.channels.cache.get(s.k.chat).name = 'chatter'; + await s.engine.noteSourceChange(s.k.src, { kind: 'channel', op: 'sync', id: s.k.chat }); + await s.engine.runPair(s.pair.id); + assert.equal(calls.length, 2); +}); + +// --- links follow the mirror --------------------------------------------------------------------------------- + +test('the links of a mirror follow its state: paused, resumed, stopped and released when removed', async () => { + const states = []; + const released = []; + const s = await scenario({ + withExpressions: false, + engineOpts: { + linkMirroredChannels: async () => ({ linked: 0, reason: 'nothing-to-link' }), + syncLinks: async (pair) => { + states.push(pair.status); + }, + releaseLinks: async (id, side) => { + released.push([id, side]); + }, + }, + }); + await s.engine.startCopy(s.pair.id); + await drain(s); + states.length = 0; + await s.engine.pause(s.pair.id, 'source'); + await s.engine.resume(s.pair.id, { side: 'source' }); + await s.engine.stopPair(s.pair.id); + assert.deepEqual(states, ['paused', 'live', 'stopped']); + assert.deepEqual(released, []); + await s.engine.removePair(s.pair.id, 'target'); + assert.deepEqual( + released, + [[s.pair.id, 'target']], + 'released, with the remover side, before the pair is gone' + ); + assert.equal(await db.getPair(s.pair.id), null); +}); + +test('a pause the mirror makes itself (a failure) pauses the links too', async () => { + const states = []; + const s = await scenario({ + withExpressions: false, + engineOpts: { + linkMirroredChannels: async () => ({ linked: 0, reason: 'nothing-to-link' }), + syncLinks: async (pair) => { + states.push(pair.status); + }, + enforceOwnership: () => true, + }, + }); + await s.engine.startCopy(s.pair.id); + s.fluxer.guild.ownerId = 'new-fluxer-owner'; + await s.engine.runPair(s.pair.id); + assert.equal((await db.getPair(s.pair.id)).pauseKind, 'ownership'); + assert.ok(states.includes('paused')); +}); + +test('a link that cannot be put right does not break the mirror; the timer tries again', async () => { + let fail = true; + let sweeps = 0; + const s = await scenario({ + withExpressions: false, + engineOpts: { + linkMirroredChannels: async () => ({ linked: 0, reason: 'nothing-to-link' }), + syncLinks: async () => { + sweeps += 1; + if (fail) throw new Error('database is busy'); + }, + }, + }); + await s.engine.startCopy(s.pair.id); + await drain(s); + const paused = await s.engine.pause(s.pair.id, 'source'); + assert.equal(paused.status, 'paused', 'the pause itself went through'); + const before = sweeps; + fail = false; + s.clock.now += 120_000; + await s.engine.tick(); + assert.ok(sweeps > before, 'the sweep of the timer ran again'); +}); + +// --- minor: a role made again in "copy again" ------------------------------------------------------------------ + +test('a role deleted on Fluxer and made again by "copy again" is ordered, then gets its powerful permissions', async () => { + const s = await scenario({ withExpressions: false }); + s.discord.roles.cache.get(s.k.mod).permissions = { bitfield: BIT.Administrator | BIT.ViewChannel }; + await s.engine.startCopy(s.pair.id); + await drain(s); + const first = (await db.getMapping(s.pair.id, 'role', s.k.mod)).targetId; + assert.equal(roleBits(s, first) & ADMIN, ADMIN, 'the first copy granted it'); + + // Somebody deletes the role on Fluxer, and the role changes on Discord (so the copy sends it again), but the order + // of the roles does not change, so the plan has no order step. + s.fluxer.guild.roles.delete(first); + s.discord.roles.cache.get(s.k.mod).name = 'Moderators'; + const preview = await s.engine.previewPair(await db.getPair(s.pair.id)); + assert.equal(preview.steps.find((x) => x.key === 'order-roles:roles').op, 'unchanged'); + s.fluxer.calls.length = 0; + await s.engine.startCopy(s.pair.id); + assert.equal((await drain(s)).status, 'live'); + + const again = (await db.getMapping(s.pair.id, 'role', s.k.mod)).targetId; + assert.notEqual(again, first, 'a new role was made'); + const calls = s.fluxer.calls; + const made = calls.findIndex((c) => c.op === 'createRole'); + const ordered = calls.findIndex((c) => c.op === 'setRolePositions'); + const granted = calls.findIndex((c) => c.op === 'editRole' && c.id === again); + assert.ok(made >= 0 && made < ordered && ordered < granted, 'made, then ordered, then granted'); + assert.equal(BigInt(calls[made].spec.permissions) & ADMIN, 0n, 'made without Administrator'); + assert.equal(roleBits(s, again) & ADMIN, ADMIN, 'and granted at the end'); + assert.ok((await db.getMapping(s.pair.id, 'role', s.k.mod)).specHash, 'and the fingerprint is final'); + assert.ok(s.fluxer.guild.roles.get(SYLO_ROLE).position > s.fluxer.guild.roles.get(again).position); +}); + +test('a first copy still sets the role order once for all its roles', async () => { + const s = await scenario({ withExpressions: false }); + s.discord.roles.cache.get(s.k.mod).permissions = { bitfield: BIT.Administrator }; + await s.engine.startCopy(s.pair.id); + await drain(s); + assert.equal(s.fluxer.ops('setRolePositions').length, 1); + assert.equal(roleBits(s, (await db.getMapping(s.pair.id, 'role', s.k.mod)).targetId) & ADMIN, ADMIN); +}); + +// --- minor: a new Fluxer owner starts with "Mirror deletions" off ------------------------------------------------ + +test('a change of Fluxer owner turns "Mirror deletions" off until the new owner switches it on', async () => { + const s = await scenario({ + withExpressions: false, + options: { mirrorDeletes: true }, + engineOpts: { enforceOwnership: () => true }, + }); + assert.deepEqual( + [s.pair.options.mirrorDeletes, s.pair.options.mirrorDeletesRequested], + [true, true], + 'the owner of both servers had it on' + ); + await s.engine.startCopy(s.pair.id); + await s.engine.runPair(s.pair.id, { maxJobs: 2 }); + s.fluxer.guild.ownerId = 'new-fluxer-owner'; + await s.engine.runPair(s.pair.id); + assert.equal((await db.getPair(s.pair.id)).pauseKind, 'ownership'); + assert.equal((await db.getPair(s.pair.id)).options.mirrorDeletes, true, 'still on until somebody confirms'); + + // The Discord owner confirming does not change who the Fluxer owner is. + await s.engine.resume(s.pair.id, { side: 'source' }); + assert.equal((await db.getPair(s.pair.id)).options.mirrorDeletes, true); + + const pair = await s.engine.resume(s.pair.id, { side: 'target' }); + assert.equal(pair.status, 'copying'); + assert.equal(pair.targetConsentUser, 'new-fluxer-owner'); + assert.deepEqual( + [pair.options.mirrorDeletes, pair.options.mirrorDeletesRequested], + [false, true], + 'off for the new owner, with the request still there for them to answer' + ); + await drain(s); + // A delete on Discord now only locks the Fluxer copy. + s.discord.channels.cache.delete(s.k.chat); + await s.engine.noteSourceChange(s.k.src, { kind: 'channel', op: 'delete', id: s.k.chat }); + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.ops('deleteChannel').length, 0, 'nothing is deleted on Fluxer'); + assert.ok((await db.getMapping(s.pair.id, 'channel', s.k.chat)).removedAt > 0); +}); + +test('a new request keeps real deletion only for the same Fluxer owner who owns both servers', async () => { + const D1 = '921000000000000001'; + const F1 = 'fluxer:821000000000000001'; + const ask = (over) => + requestMirror({ + sourceGuild: D1, + targetGuild: F1, + actor: 'Anna', + sourceUser: 'dc-owner', + targetUser: 'fx-owner', + targetOwner: true, + ...over, + }); + const first = await ask({ options: { mirrorDeletes: true } }); + assert.equal(first.pair.options.mirrorDeletes, true); + + const same = await ask({}); + assert.deepEqual( + [same.pair.options.mirrorDeletes, same.pair.options.mirrorDeletesRequested], + [true, true], + 'the same owner of both servers asking again keeps it' + ); + const newOwner = await ask({ targetUser: 'fx-new-owner' }); + assert.deepEqual( + [newOwner.pair.options.mirrorDeletes, newOwner.pair.options.mirrorDeletesRequested], + [false, true], + 'a new Fluxer owner starts with it off' + ); + await ask({ targetUser: 'fx-owner', options: { mirrorDeletes: true } }); + const stranger = await ask({ targetOwner: false }); + assert.deepEqual( + [stranger.pair.options.mirrorDeletes, stranger.pair.options.mirrorDeletesRequested], + [false, true], + 'a request from someone who is not the Fluxer owner never carries it' + ); + // The request brings its own options: the existing rules for them apply (asked, not on). + const explicit = await ask({ targetOwner: false, options: { mirrorDeletes: true } }); + assert.deepEqual( + [explicit.pair.options.mirrorDeletes, explicit.pair.options.mirrorDeletesRequested], + [false, true] + ); + await db.deletePair(explicit.pair.id); +}); diff --git a/test/mirrorBridgeLink.test.js b/test/mirrorBridgeLink.test.js new file mode 100644 index 00000000..caa45c1b --- /dev/null +++ b/test/mirrorBridgeLink.test.js @@ -0,0 +1,677 @@ +// A mirror links its channels through the bridge (src/mirror/bridgeLink.js, createMirrorLink and the held-link helpers in +// src/modules/bridge.js): which channels, what is recorded, every safety rule of the bridge, running it again, and +// links that follow the state of their mirror. Platforms and webhooks are the bridge tests' fakes. +import './helpers/tmpDb.js'; +import test, { afterEach, beforeEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { ChannelType } from 'discord.js'; +import * as mdb from '../src/db/mirror.js'; +import * as bridgeDb from '../src/db/bridge.js'; +import { listAudit } from '../src/db/audit.js'; +import { getGuildModule, setGuildModule } from '../src/db/modules.js'; +import { purgeGuild } from '../src/db/purge.js'; +import { requestMirror } from '../src/mirror/consent.js'; +import { + LINKS_PER_RUN, + linkMirroredChannels, + releaseMirrorLinks, + syncMirrorLinks, +} from '../src/mirror/bridgeLink.js'; +import { + BridgeError, + MAX_MIRROR_LINKS, + bridgeIdle, + bridgeProblems, + dropBridgeCache, + handleMessage, + linkView, + mirrorHold, + resumeLink, + unlinkLink, + useBridgeParts, +} from '../src/modules/bridge.js'; +import { D, F, installPlatforms, message, removePlatforms, useFakes } from './helpers/fakeBridge.js'; + +let p; +let fakes; +const noSleep = async () => {}; + +beforeEach(async () => { + p = installPlatforms(); + useFakes(); + dropBridgeCache(); + for (const key of [D, F]) await purgeGuild(key); + fakes = useFakes(); + await setGuildModule(D, 'bridge', { enabled: false }); + await setGuildModule(F, 'bridge', { enabled: false }); +}); +afterEach(() => { + useBridgeParts(); + removePlatforms(); +}); + +/** A live mirror of the Discord server onto the Fluxer server, agreed by both owners. */ +async function livePair(over = {}) { + const { pair } = await requestMirror({ + sourceGuild: D, + targetGuild: F, + actor: 'Anna', + sourceUser: p.dGuild.ownerId, + targetUser: p.fGuild.ownerId, + targetOwner: true, + ...over, + }); + return mdb.updatePair(pair.id, { status: 'live', targetConsentBy: 'Bo' }); +} +/** A Discord channel and its Fluxer copy, in the pair's channel map. */ +async function mirrored(pair, name, { type = ChannelType.GuildText, dOpts = {}, fOpts = {} } = {}) { + const d = p.channel(p.dGuild, { name, type, ...dOpts }); + const f = p.channel(p.fGuild, { name, type, ...fOpts }); + await mdb.putMapping({ + pairId: pair.id, + kind: 'channel', + sourceId: d.id, + targetId: f.id, + owned: true, + specHash: 'h', + label: name, + }); + return { d, f }; +} +const run = (pair, opts = {}) => linkMirroredChannels(pair.id, { sleep: noSleep, ...opts }); +const send = async (channel, key, over = {}) => { + await handleMessage( + message({ channelId: channel.id, client: key === D ? p.discord : p.fluxer, ...over }), + {}, + key + ); + await bridgeIdle(); +}; + +test('every mirrored text channel pair is linked, voice channels and categories are left alone', async () => { + const pair = await livePair(); + const chat = await mirrored(pair, 'chat'); + const news = await mirrored(pair, 'news', { type: ChannelType.GuildAnnouncement }); + await mirrored(pair, 'lounge', { type: ChannelType.GuildVoice }); + await mirrored(pair, 'Info', { type: ChannelType.GuildCategory }); + + const res = await run(pair); + assert.deepEqual([res.linked, res.already, res.remaining, res.skipped.length], [2, 0, 0, 0]); + for (const { d, f } of [chat, news]) { + const found = await bridgeDb.linkForChannel(D, d.id); + assert.ok(found, 'the Discord channel is linked'); + assert.deepEqual( + [found.link.guildA, found.link.channelA, found.link.guildB, found.link.channelB, found.link.status], + [D, d.id, F, f.id, 'active'] + ); + assert.equal((await mdb.getMirrorLink(pair.id, d.id)).linkId, found.link.id); + } + assert.equal(await bridgeDb.linkCount(D), 2); + assert.equal(fakes.transport.hooks.length, 4, 'a webhook in each of the four channels, none for voice'); +}); + +test('no link code is used: the link records the two owners who agreed to the mirror and their ids', async () => { + const pair = await livePair(); + const { d, f } = await mirrored(pair, 'chat'); + await run(pair); + const link = (await bridgeDb.linkForChannel(D, d.id)).link; + assert.equal(link.requestedBy, 'Anna (server mirror)'); + assert.equal(link.acceptedBy, 'Bo (server mirror)'); + assert.deepEqual([link.ownerA, link.ownerB], [p.dGuild.ownerId, p.fGuild.ownerId]); + assert.equal((await bridgeDb.pendingCodes(D)).length, 0, 'no request or code was made'); + assert.deepEqual([link.optionsA.bots, link.optionsB.bots], [false, false], 'bots are off on both sides'); + // the owners' own side of it, as the dashboard of each server shows it + assert.equal(linkView(link, D).by, 'Anna (server mirror)'); + assert.equal(linkView(link, F).by, 'Bo (server mirror)'); + // visible notice in both channels, naming the mirror + for (const channel of [d, f]) { + assert.equal(channel.said.length, 1); + assert.match(channel.said[0].content, /Bridge on/); + assert.match(channel.said[0].content, /server mirror/); + assert.deepEqual(channel.said[0].allowedMentions, { parse: [] }); + } + for (const key of [D, F]) { + const entry = (await listAudit(key)).find((e) => e.action === 'bridge:link'); + assert.ok(entry, 'both servers record it'); + assert.match(entry.detail, /through the server mirror/); + } +}); + +test('the Bridge module is switched on in both servers, and both audit logs say so', async () => { + const pair = await livePair(); + await mirrored(pair, 'chat'); + assert.equal((await getGuildModule(D, 'bridge')).enabled, false); + await run(pair); + for (const key of [D, F]) { + assert.equal((await getGuildModule(key, 'bridge')).enabled, true); + assert.ok((await listAudit(key)).some((e) => e.action === 'bridge:module')); + } +}); + +test('a linked pair relays messages both ways, with nobody pinged', async () => { + const pair = await livePair(); + const { d, f } = await mirrored(pair, 'chat'); + await run(pair); + await send(d, D, { content: 'hello from Discord @everyone' }); + await send(f, F, { content: 'hello from Fluxer' }); + assert.equal(fakes.transport.sends.length, 2); + assert.equal(fakes.transport.sends[0].guildKey, F); + assert.deepEqual(fakes.transport.sends[0].payload.allowedMentions ?? { parse: [] }, { parse: [] }); + assert.equal(fakes.transport.sends[1].guildKey, D); + // a bot's message stays on its side (bots are off) + await send(d, D, { content: 'beep', author: { bot: true } }); + assert.equal(fakes.transport.sends.length, 2); +}); + +test('running it again links only what is missing', async () => { + const pair = await livePair(); + await mirrored(pair, 'chat'); + await mirrored(pair, 'rules'); + assert.equal((await run(pair)).linked, 2); + const again = await run(pair); + assert.deepEqual([again.linked, again.already, again.reason], [0, 2, undefined]); + assert.equal(fakes.transport.hooks.length, 4, 'nothing was made twice'); + + await mirrored(pair, 'offers'); + const third = await run(pair); + assert.deepEqual([third.linked, third.already], [1, 2], 'only the new pair is linked'); + assert.equal(fakes.transport.hooks.length, 6); + assert.equal((await mdb.listMirrorLinks(pair.id)).length, 3); +}); + +test('a pair that has nothing to link says so', async () => { + const pair = await livePair(); + await mirrored(pair, 'lounge', { type: ChannelType.GuildVoice }); + assert.equal((await run(pair)).reason, 'nothing-to-link'); + assert.equal((await getGuildModule(D, 'bridge')).enabled, false, 'and the module is left alone'); +}); + +test('a channel somebody unlinked on purpose is not linked again', async () => { + const pair = await livePair(); + const { d } = await mirrored(pair, 'chat'); + await mirrored(pair, 'rules'); + await run(pair); + const link = (await bridgeDb.linkForChannel(D, d.id)).link; + await unlinkLink({ guildKey: D, id: link.id, actor: 'Anna' }); + assert.equal(await bridgeDb.linkForChannel(D, d.id), null); + const again = await run(pair); + assert.deepEqual([again.linked, again.already], [0, 2]); + assert.equal(await bridgeDb.linkForChannel(D, d.id), null, 'it stays unlinked'); +}); + +test('the age rule: an age-restricted channel is linked only to an age-restricted one', async () => { + const pair = await livePair(); + const plain = await mirrored(pair, 'chat'); + const mismatch = await mirrored(pair, 'adult', { dOpts: { nsfw: true } }); + const both = await mirrored(pair, 'adult-too', { dOpts: { nsfw: true }, fOpts: { nsfw: true } }); + const res = await run(pair); + assert.equal(res.linked, 2); + assert.deepEqual( + res.skipped.map((s) => [s.sourceId, s.code]), + [[mismatch.d.id, 'age-restriction']] + ); + assert.ok(await bridgeDb.linkForChannel(D, plain.d.id)); + assert.ok(await bridgeDb.linkForChannel(D, both.d.id)); + assert.equal(await bridgeDb.linkForChannel(D, mismatch.d.id), null); + assert.equal(mismatch.d.said.length + mismatch.f.said.length, 0, 'no notice, no webhook'); +}); + +test('a private channel is not linked to an open one, and two private channels are linked', async () => { + const pair = await livePair(); + const closed = (guild) => (target) => ({ + has: (perm) => !(perm === 'ViewChannel' && target === String(guild.id)), + }); + const leak = await mirrored(pair, 'staff'); + leak.d.permissionsFor = closed(p.dGuild); // private on Discord, open on Fluxer + const pulled = await mirrored(pair, 'staff-too'); + pulled.f.permissionsFor = closed(p.fGuild); // open on Discord, private on Fluxer + const same = await mirrored(pair, 'mods'); + same.d.permissionsFor = closed(p.dGuild); + same.f.permissionsFor = closed(p.fGuild); + const res = await run(pair); + assert.equal(res.linked, 1); + assert.deepEqual( + res.skipped.map((s) => s.code), + ['visibility', 'visibility'] + ); + assert.ok(await bridgeDb.linkForChannel(D, same.d.id)); + assert.equal(await bridgeDb.linkForChannel(D, leak.d.id), null); + assert.equal(await bridgeDb.linkForChannel(D, pulled.d.id), null); +}); + +test('with sign-in on, each owner must be able to see, read and send in the channel themselves', async () => { + const pair = await livePair(); + const ok = await mirrored(pair, 'chat'); + const hidden = await mirrored(pair, 'hidden'); + const owner = { id: p.dGuild.ownerId }; + p.dGuild.members.cache.set(owner.id, owner); + const fluxerOwner = { id: p.fGuild.ownerId }; + p.fGuild.members.cache.set(fluxerOwner.id, fluxerOwner); + // the Fluxer owner cannot read the history of one channel + const real = hidden.f.permissionsFor; + hidden.f.permissionsFor = (t) => + t?.id === fluxerOwner.id ? { has: (perm) => perm !== 'ReadMessageHistory' } : real(t); + const res = await run(pair, { open: false }); + assert.equal(res.linked, 1); + assert.deepEqual( + res.skipped.map((s) => [s.sourceId, s.code]), + [[hidden.d.id, 'visibility']] + ); + assert.ok(await bridgeDb.linkForChannel(D, ok.d.id)); +}); + +test('with sign-in on, an owner who cannot be found fails closed: nothing is linked', async () => { + const pair = await livePair(); + await mirrored(pair, 'chat'); + const res = await run(pair, { open: false }); + assert.equal(res.linked, 0); + assert.deepEqual( + res.skipped.map((s) => s.code), + ['visibility'] + ); +}); + +test('Sylo missing a permission in one channel skips only that pair', async () => { + const pair = await livePair(); + const fine = await mirrored(pair, 'chat'); + const broken = await mirrored(pair, 'locked'); + broken.f.permissionsFor = () => ({ has: (perm) => perm !== 'ManageWebhooks' }); + const res = await run(pair); + assert.equal(res.linked, 1); + assert.deepEqual( + res.skipped.map((s) => [s.sourceId, s.code]), + [[broken.d.id, 'permissions']] + ); + assert.ok(await bridgeDb.linkForChannel(D, fine.d.id)); +}); + +test('a channel already bridged by hand is left as it is, and a lost record of a mirror link is taken over', async () => { + const pair = await livePair(); + const other = await mirrored(pair, 'other'); + const lost = await mirrored(pair, 'lost'); + const handmade = await mirrored(pair, 'handmade'); + // somebody linked "other" to an unrelated Fluxer channel through the bridge page + const stray = p.channel(p.fGuild, { name: 'stray' }); + await bridgeDb.createLink({ + guildA: D, + channelA: other.d.id, + guildB: F, + channelB: stray.id, + hookA: { id: '1', token: 'a' }, + hookB: { id: '2', token: 'b' }, + }); + // a link of exactly the pair "lost" that the mirror made exists, but its record was never written + const made = await bridgeDb.createLink({ + guildA: D, + channelA: lost.d.id, + guildB: F, + channelB: lost.f.id, + hookA: { id: '3', token: 'c' }, + hookB: { id: '4', token: 'd' }, + requestedBy: 'Anna (server mirror)', + acceptedBy: 'Bo (server mirror)', + }); + // and the owners linked exactly the pair "handmade" themselves, with a code + const byHand = await bridgeDb.createLink({ + guildA: D, + channelA: handmade.d.id, + guildB: F, + channelB: handmade.f.id, + hookA: { id: '5', token: 'e' }, + hookB: { id: '6', token: 'f' }, + requestedBy: 'Anna (dashboard)', + acceptedBy: 'Bo (dashboard)', + }); + const res = await run(pair); + assert.deepEqual([res.linked, res.already], [0, 1]); + assert.deepEqual( + res.skipped.map((s) => [s.sourceId, s.code]), + [ + [other.d.id, 'exists'], + [handmade.d.id, 'exists'], + ] + ); + assert.match(res.skipped[1].message, /already bridged by hand/); + assert.equal( + (await mdb.getMirrorLink(pair.id, lost.d.id)).linkId, + made, + 'a lost mirror record is taken over' + ); + assert.equal(await mdb.getMirrorLink(pair.id, handmade.d.id), null, 'a hand-made link is never taken over'); + // so pausing the mirror leaves it alone + await mdb.updatePair(pair.id, { status: 'paused' }); + await syncMirrorLinks(pair.id); + assert.equal((await bridgeDb.getLink(byHand)).status, 'active'); + assert.equal((await bridgeDb.getLink(made)).status, 'paused'); + assert.equal(fakes.transport.hooks.length, 0, 'no new webhook'); +}); + +test('a batch links at most the limit, and the next run carries on without trying the same pairs again', async () => { + assert.ok(LINKS_PER_RUN >= 10); + const pair = await livePair(); + for (let i = 0; i < 5; i += 1) await mirrored(pair, `c${i}`); + const bad = await mirrored(pair, 'adult', { dOpts: { nsfw: true } }); // the sixth is refused + const first = await run(pair, { limit: 3 }); + assert.deepEqual([first.linked, first.remaining, first.tried.length], [3, 3, 3]); + const second = await run(pair, { limit: 3, skip: new Set(first.tried) }); + assert.deepEqual([second.linked, second.remaining], [2, 0]); + assert.deepEqual( + second.skipped.map((s) => s.sourceId), + [bad.d.id] + ); + // a third run that skips what was tried has nothing left + const third = await run(pair, { limit: 3, skip: new Set([...first.tried, ...second.tried]) }); + assert.deepEqual([third.linked, third.remaining, third.skipped.length], [0, 0, 0]); +}); + +test('the ceiling of links per server is respected and reported', async () => { + const pair = await livePair(); + const one = await mirrored(pair, 'chat'); + for (let i = 0; i < MAX_MIRROR_LINKS; i += 1) { + await bridgeDb.createLink({ + guildA: D, + channelA: `filler-${i}`, + guildB: F, + channelB: `filler-f-${i}`, + hookA: { id: `a${i}`, token: 'a' }, + hookB: { id: `b${i}`, token: 'b' }, + }); + } + const res = await run(pair); + assert.equal(res.linked, 0); + assert.deepEqual( + res.skipped.map((s) => [s.sourceId, s.code]), + [[one.d.id, 'limit']] + ); +}); + +test('a webhook that cannot be made skips the pair and leaves nothing half made', async () => { + const pair = await livePair(); + const a = await mirrored(pair, 'chat'); + fakes.transport.hookFailures.push(new Error('boom')); + const res = await run(pair); + assert.equal(res.linked, 0); + assert.deepEqual( + res.skipped.map((s) => s.code), + ['webhook'] + ); + assert.equal(await bridgeDb.linkForChannel(D, a.d.id), null); + assert.equal(await mdb.getMirrorLink(pair.id, a.d.id), null, 'so a later run tries it again'); + const later = await run(pair); + assert.equal(later.linked, 1); +}); + +test('when the clients are not both connected, nothing is linked and the reason says so', async () => { + const pair = await livePair(); + await mirrored(pair, 'chat'); + removePlatforms(); + assert.equal((await run(pair)).reason, 'bridge-not-available'); + assert.equal((await linkMirroredChannels(999_999, { sleep: noSleep })).reason, 'no-such-mirror'); +}); + +test('a mirror that is not running links nothing', async () => { + const pair = await livePair(); + await mirrored(pair, 'chat'); + await mdb.updatePair(pair.id, { status: 'paused' }); + assert.equal((await run(pair)).reason, 'mirror-not-running'); +}); + +// --- links follow the state of their mirror --------------------------------------------------------------- + +test('a paused mirror pauses its links, nothing crosses, and they run again with the mirror', async () => { + const pair = await livePair(); + const { d, f } = await mirrored(pair, 'chat'); + await run(pair); + await send(d, D, { content: 'one' }); + assert.equal(fakes.transport.sends.length, 1); + + await mdb.updatePair(pair.id, { status: 'paused' }); + assert.deepEqual(await syncMirrorLinks(pair.id), { held: 1, resumed: 0, waiting: [] }); + const held = (await bridgeDb.linkForChannel(D, d.id)).link; + assert.deepEqual([held.status, held.failReason], ['paused', 'mirror']); + await send(d, D, { content: 'two' }); + await send(f, F, { content: 'three' }); + assert.equal(fakes.transport.sends.length, 1, 'nothing crosses while the mirror is paused'); + assert.deepEqual(await bridgeProblems(D), [], 'the mirror has its own Health entry'); + assert.equal(await mirrorHold(held), 'held'); + assert.match(linkView(held, D, 'held').pauseNote, /server mirror/); + assert.equal(linkView(held, D, 'held').resumable, false, 'it comes back with the mirror, not by a button'); + + // the bridge page cannot resume it on its own + await assert.rejects( + resumeLink({ guildKey: D, id: held.id, actor: 'Anna' }), + (err) => err instanceof BridgeError && err.status === 409 && /mirror/.test(err.message) + ); + + await mdb.updatePair(pair.id, { status: 'live' }); + assert.deepEqual(await syncMirrorLinks(pair.id), { held: 0, resumed: 1, waiting: [] }); + assert.equal((await bridgeDb.getLink(held.id)).status, 'active'); + await send(d, D, { content: 'four' }); + assert.equal(fakes.transport.sends.length, 2, 'and it carries messages again'); +}); + +test('a stopped mirror pauses its links too, and only the ones the mirror paused come back', async () => { + const pair = await livePair(); + const a = await mirrored(pair, 'chat'); + const b = await mirrored(pair, 'rules'); + await run(pair); + const own = (await bridgeDb.linkForChannel(D, b.d.id)).link; + await bridgeDb.setLinkStatus(own.id, { status: 'paused', pausedSide: 'b' }); // the Fluxer server paused this one + await mdb.updatePair(pair.id, { status: 'stopped' }); + assert.equal((await syncMirrorLinks(pair.id)).held, 1); + await mdb.updatePair(pair.id, { status: 'live' }); + assert.equal((await syncMirrorLinks(pair.id)).resumed, 1); + assert.equal((await bridgeDb.linkForChannel(D, a.d.id)).link.status, 'active'); + const still = (await bridgeDb.linkForChannel(D, b.d.id)).link; + assert.deepEqual([still.status, still.pausedSide], ['paused', 'b'], "a server's own pause is left alone"); +}); + +test('a server that changed hands while the mirror was paused keeps the link paused until its new owner confirms', async () => { + const pair = await livePair(); + const { d } = await mirrored(pair, 'chat'); + await run(pair); + await mdb.updatePair(pair.id, { status: 'paused' }); + await syncMirrorLinks(pair.id); + p.fGuild.ownerId = 'new-owner'; + await mdb.updatePair(pair.id, { status: 'live' }); + const res = await syncMirrorLinks(pair.id); + assert.deepEqual([res.resumed, res.waiting], [0, ['owner-changed']]); + const link = (await bridgeDb.linkForChannel(D, d.id)).link; + assert.deepEqual([link.status, link.pausedSide, link.failReason], ['paused', 'b', 'owner-changed']); +}); + +test('a link whose channel can no longer be used stays paused with the mirror and is reported', async () => { + const pair = await livePair(); + const { d, f } = await mirrored(pair, 'chat'); + await run(pair); + await mdb.updatePair(pair.id, { status: 'paused' }); + await syncMirrorLinks(pair.id); + f.permissionsFor = () => ({ has: (perm) => perm !== 'ManageWebhooks' }); + await mdb.updatePair(pair.id, { status: 'live' }); + const res = await syncMirrorLinks(pair.id); + assert.equal(res.resumed, 0); + assert.match(res.waiting[0], /ManageWebhooks/); + assert.equal((await bridgeDb.linkForChannel(D, d.id)).link.failReason, 'mirror'); +}); + +test('removing a mirror releases its links, paused by the server that removed it: only that owner can resume, the other can unlink', async () => { + const pair = await livePair(); + const { d } = await mirrored(pair, 'chat'); + const other = await mirrored(pair, 'rules'); + await run(pair); + assert.equal(await releaseMirrorLinks(pair.id, 'target'), 2, 'removed from the Fluxer server'); + const link = (await bridgeDb.linkForChannel(D, d.id)).link; + assert.deepEqual([link.status, link.pausedSide, link.failReason], ['paused', 'b', 'mirror-ended']); + await mdb.deletePair(pair.id); + assert.equal((await mdb.listMirrorLinks(pair.id)).length, 0, 'the record goes with the pair'); + const view = async (key) => linkView(link, key, await mirrorHold(link)); + assert.equal(await mirrorHold(link), 'none', 'it is an ordinary paused link now'); + assert.match((await view(F)).pauseNote, /mirror that made this bridge was removed/); + assert.deepEqual([(await view(D)).resumable, (await view(F)).resumable], [false, true]); + assert.equal((await view(D)).pausedByThem, true); + // the Discord owner cannot start sharing again on their own... + await assert.rejects( + resumeLink({ guildKey: D, id: link.id, actor: 'Anna' }), + (err) => err instanceof BridgeError && err.status === 403 + ); + assert.equal((await bridgeDb.getLink(link.id)).status, 'paused'); + // ...but can unlink; the Fluxer owner who removed the mirror can resume + await unlinkLink({ + guildKey: D, + id: (await bridgeDb.linkForChannel(D, other.d.id)).link.id, + actor: 'Anna', + }); + await resumeLink({ guildKey: F, id: link.id, actor: 'Bo' }); + assert.equal((await bridgeDb.getLink(link.id)).status, 'active'); + // a second release does nothing to links that are no longer the mirror's to hold + assert.equal(await releaseMirrorLinks(pair.id, 'target'), 0); +}); + +test('a removal nobody owns (no side) leaves the links to either owner', async () => { + const pair = await livePair(); + const { d } = await mirrored(pair, 'chat'); + await run(pair); + assert.equal(await releaseMirrorLinks(pair.id), 1); + const link = (await bridgeDb.linkForChannel(D, d.id)).link; + assert.deepEqual([link.status, link.pausedSide], ['paused', '']); + await resumeLink({ guildKey: D, id: link.id, actor: 'Anna' }); + assert.equal((await bridgeDb.getLink(link.id)).status, 'active'); +}); + +test('a link whose mirror pair was replaced is not stranded: it counts as ended and can be resumed', async () => { + const pair = await livePair(); + const { d } = await mirrored(pair, 'chat'); + await run(pair); + await mdb.updatePair(pair.id, { status: 'stopped' }); + await syncMirrorLinks(pair.id); + const held = (await bridgeDb.linkForChannel(D, d.id)).link; + assert.deepEqual([held.failReason, await mirrorHold(held)], ['mirror', 'held']); + await mdb.deletePair(pair.id); // a new request for another Fluxer server replaced the stopped pair + assert.equal(await mirrorHold(held), 'ended'); + assert.equal(linkView(held, D, 'ended').resumable, true); + assert.match(linkView(held, D, 'ended').pauseNote, /was removed/); + await resumeLink({ guildKey: D, id: held.id, actor: 'Anna' }); + assert.equal((await bridgeDb.getLink(held.id)).status, 'active'); +}); + +test('while the mirror is not running, even a stopped or hand-paused link of it cannot be resumed on the Bridge page', async () => { + const pair = await livePair(); + const a = await mirrored(pair, 'chat'); + const b = await mirrored(pair, 'rules'); + await run(pair); + const failed = (await bridgeDb.linkForChannel(D, a.d.id)).link; + const byHand = (await bridgeDb.linkForChannel(D, b.d.id)).link; + await bridgeDb.setLinkStatus(failed.id, { status: 'failed', reason: 'permissions' }); + await bridgeDb.setLinkStatus(byHand.id, { status: 'paused', pausedSide: 'a' }); + await mdb.updatePair(pair.id, { status: 'stopped' }); + for (const [guildKey, link] of [ + [D, failed], + [D, byHand], + ]) { + await assert.rejects( + resumeLink({ guildKey, id: link.id, actor: 'Anna' }), + (err) => err instanceof BridgeError && err.status === 409 && /mirror/.test(err.message) + ); + } + const failedNow = await bridgeDb.getLink(failed.id); + const byHandNow = await bridgeDb.getLink(byHand.id); + assert.equal(linkView(failedNow, D, await mirrorHold(failedNow)).resumable, false); + assert.equal(linkView(byHandNow, D, await mirrorHold(byHandNow)).resumable, false); + // once the mirror runs again they are the owners' to resume + await mdb.updatePair(pair.id, { status: 'live' }); + assert.equal(linkView(byHandNow, D, await mirrorHold(byHandNow)).resumable, true); + await resumeLink({ guildKey: D, id: byHand.id, actor: 'Anna' }); + assert.equal((await bridgeDb.getLink(byHand.id)).status, 'active'); +}); + +// --- the module switch, privacy over time, recorded owners --------------------------------------------------- + +test('only the first linking switches the Bridge module on: a server that turned it off keeps it off', async () => { + const pair = await livePair(); + await mirrored(pair, 'chat'); + assert.equal((await run(pair)).linked, 1); + assert.equal((await getGuildModule(F, 'bridge')).enabled, true); + + // the Fluxer server's owner switches the module off; a channel is made on Discord and the mirror copies it + await setGuildModule(F, 'bridge', { enabled: false }); + const later = await mirrored(pair, 'offers'); + const res = await run(pair); + assert.deepEqual([res.linked, res.reason], [0, 'bridge-off']); + assert.deepEqual( + res.skipped.map((s) => [s.sourceId, s.code]), + [[later.d.id, 'bridge-off']] + ); + assert.equal((await getGuildModule(F, 'bridge')).enabled, false, 'still off'); + assert.equal(await bridgeDb.linkForChannel(D, later.d.id), null); + assert.equal( + (await listAudit(F)).filter((e) => e.action === 'bridge:module').length, + 1, + 'switched on once' + ); + // when the owner switches it on again, the missing pair is linked + await setGuildModule(F, 'bridge', { enabled: true }); + assert.equal((await run(pair)).linked, 1); +}); + +test('a first linking where every pair is refused still counts: the module is not switched on again later', async () => { + const pair = await livePair(); + await mirrored(pair, 'adult', { dOpts: { nsfw: true } }); + assert.equal((await run(pair)).linked, 0); + assert.equal((await getGuildModule(D, 'bridge')).enabled, true); + await setGuildModule(D, 'bridge', { enabled: false }); + assert.equal((await run(pair)).reason, 'bridge-off'); + assert.equal((await getGuildModule(D, 'bridge')).enabled, false); +}); + +test('a channel that turns private later pauses the mirror link instead of leaking: on resume and on the next message', async () => { + const pair = await livePair(); + const a = await mirrored(pair, 'chat'); + const b = await mirrored(pair, 'rules'); + await run(pair); + const closed = (guild) => (target) => ({ + has: (perm) => !(perm === 'ViewChannel' && target === String(guild.id)), + }); + // on the next message: the Discord channel became private while the Fluxer copy is still open + a.d.permissionsFor = closed(p.dGuild); + await send(a.d, D, { content: 'secret' }); + assert.equal(fakes.transport.sends.length, 0, 'nothing crosses'); + const stopped = (await bridgeDb.linkForChannel(D, a.d.id)).link; + assert.deepEqual([stopped.status, stopped.failReason], ['failed', 'visibility']); + assert.match((await bridgeProblems(D))[0].text, /no longer open to the same people/); + + // on resume of a link held with the mirror + await mdb.updatePair(pair.id, { status: 'paused' }); + await syncMirrorLinks(pair.id); + b.f.permissionsFor = closed(p.fGuild); + await mdb.updatePair(pair.id, { status: 'live' }); + const res = await syncMirrorLinks(pair.id); + assert.equal(res.resumed, 0); + assert.match(res.waiting[0], /no longer open to the same people/); + assert.equal((await bridgeDb.linkForChannel(D, b.d.id)).link.status, 'paused'); +}); + +test('the links record the people who agreed to the mirror as the owners the bridge checks', async () => { + const pair = await livePair(); + const { d } = await mirrored(pair, 'chat'); + await run(pair); + const link = (await bridgeDb.linkForChannel(D, d.id)).link; + assert.deepEqual([link.ownerA, link.ownerB], [pair.sourceConsentUser, pair.targetConsentUser]); + // when the Fluxer server has changed hands since, the bridge pauses the link by itself + p.fGuild.ownerId = 'someone-new'; + await handleMessage(message({ channelId: d.id, client: p.discord, content: 'hi' }), {}, D); + await bridgeIdle(); + assert.equal(fakes.transport.sends.length, 0); + assert.equal((await bridgeDb.linkForChannel(D, d.id)).link.failReason, 'owner-changed'); +}); + +test('a link the mirror made can be unlinked by either owner, and leaving the server purges it with the pair', async () => { + const pair = await livePair(); + const { d } = await mirrored(pair, 'chat'); + await run(pair); + const link = (await bridgeDb.linkForChannel(D, d.id)).link; + await unlinkLink({ guildKey: F, id: link.id, actor: 'Bo' }); + assert.equal(await bridgeDb.getLink(link.id), null); + await purgeGuild(F); + assert.equal(await mdb.getPair(pair.id), null); + assert.equal((await mdb.listMirrorLinks(pair.id)).length, 0); +}); diff --git a/test/mirrorConsent.test.js b/test/mirrorConsent.test.js new file mode 100644 index 00000000..05452646 --- /dev/null +++ b/test/mirrorConsent.test.js @@ -0,0 +1,185 @@ +// Consent for a mirror (src/mirror/consent.js): nothing may go ahead until someone with Edit on each side +// has agreed; the code that carries the Fluxer side's agreement is one-time, expires, is stored hashed, and +// cannot be used for another server. +import './helpers/tmpDb.js'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import * as db from '../src/db/mirror.js'; +import { + CODE_TTL_MS, + ConsentError, + MAX_CODE_ATTEMPTS, + acceptMirror, + newCode, + requestMirror, + requirePairConsent, + setConsentProvider, +} from '../src/mirror/consent.js'; + +const D1 = '920000000000000001'; +const D2 = '920000000000000002'; +const F1 = 'fluxer:820000000000000001'; +const F2 = 'fluxer:820000000000000002'; +const T0 = 1_800_000_000_000; + +const ask = (over = {}) => + requestMirror({ + sourceGuild: D1, + targetGuild: F1, + actor: 'Anna', + sourceUser: 'dc-owner', + targetOwner: false, + now: T0, + ...over, + }); +const rejects = (promise, code) => + assert.rejects(promise, (err) => err instanceof ConsentError && err.code === code, `expected ${code}`); + +test('a code is eight characters without lookalikes, and a new one each time', () => { + const a = newCode(); + assert.match(a, /^[A-HJKMNP-Z2-9]{4}-[A-HJKMNP-Z2-9]{4}$/); + assert.notEqual(a, newCode()); +}); + +test('asking creates a pending pair: the Discord side has agreed, the Fluxer side has not', async () => { + const { pair, code } = await ask(); + assert.ok(code); + assert.equal(pair.status, 'pending'); + assert.equal(pair.sourceConsentBy, 'Anna'); + assert.equal(pair.targetConsentBy, ''); + assert.equal(pair.codeExpiresAt, T0 + CODE_TTL_MS); + assert.ok(!JSON.stringify(pair).includes(code), 'the code is stored only as a hash'); + await assert.rejects(requirePairConsent(pair.id), /Fluxer server/); +}); + +test('someone with Edit on the other server agrees with the code; the code works once', async () => { + const { code } = await ask(); + const pair = await acceptMirror({ + targetGuild: F1, + code: code.toLowerCase().replace('-', ' '), + actor: 'Bo', + userId: 'fx-owner', + now: T0 + 1000, + }); + assert.equal(pair.status, 'ready'); + assert.equal(pair.targetConsentBy, 'Bo'); + assert.deepEqual( + [pair.sourceConsentUser, pair.targetConsentUser], + ['dc-owner', 'fx-owner'], + 'the owners are recorded' + ); + assert.equal(pair.codeHash, ''); + assert.equal((await requirePairConsent(pair.id)).id, pair.id); + await rejects(acceptMirror({ targetGuild: F1, code, actor: 'Cy', now: T0 + 2000 }), 'no-request'); +}); + +test('the same person with Edit on both servers agrees on both sides at once, with no code', async () => { + await db.deletePair((await db.getPairBySource(D1)).id); + const { pair, code } = await ask({ targetOwner: true, targetUser: 'fx-owner' }); + assert.deepEqual([pair.sourceConsentUser, pair.targetConsentUser], ['dc-owner', 'fx-owner']); + assert.equal(code, null); + assert.equal(pair.status, 'ready'); + assert.deepEqual([pair.sourceConsentBy, pair.targetConsentBy], ['Anna', 'Anna']); +}); + +test('a code is for one server: another Fluxer server cannot use it', async () => { + await db.deletePair((await db.getPairBySource(D1)).id); + const { code } = await ask(); + await rejects(acceptMirror({ targetGuild: F2, code, actor: 'Mallory', now: T0 + 1 }), 'no-request'); + assert.equal((await db.getPairBySource(D1)).status, 'pending', 'the request is untouched'); + assert.equal((await db.getPairBySource(D1)).codeAttempts, 0); +}); + +test('a wrong code is refused; five wrong tries cancel the request', async () => { + await db.deletePair((await db.getPairBySource(D1)).id); + const { code } = await ask(); + for (let i = 1; i < MAX_CODE_ATTEMPTS; i += 1) { + await rejects(acceptMirror({ targetGuild: F1, code: 'AAAA-AAAA', actor: 'x', now: T0 }), 'wrong-code'); + assert.equal((await db.getPairBySource(D1)).codeAttempts, i); + } + await rejects(acceptMirror({ targetGuild: F1, code: 'AAAA-AAAA', actor: 'x', now: T0 }), 'cancelled'); + assert.equal(await db.getPairBySource(D1), null); + await rejects(acceptMirror({ targetGuild: F1, code, actor: 'x', now: T0 }), 'no-request'); +}); + +test('an expired code is refused and the request goes away', async () => { + const { code } = await ask(); + await rejects(acceptMirror({ targetGuild: F1, code, actor: 'Bo', now: T0 + CODE_TTL_MS + 1 }), 'expired'); + assert.equal(await db.getPairBySource(D1), null); +}); + +test('a server cannot be mirrored onto itself, and a target belongs to one source', async () => { + await rejects(ask({ targetGuild: D1 }), 'same-server'); + await ask(); + await rejects(ask({ sourceGuild: D2 }), 'target-busy'); + // A stale request (its code has expired) does not keep the target. + const later = await ask({ sourceGuild: D2, now: T0 + CODE_TTL_MS + 5 }); + assert.equal(later.pair.sourceGuild, D2); + assert.equal(await db.getPairBySource(D1), null); + await db.deletePair(later.pair.id); +}); + +test('a source that already copies cannot ask again; a pair that has not copied can be replaced', async () => { + const first = await ask({ targetOwner: true }); + const replaced = await ask({ targetGuild: F2, targetOwner: true }); + assert.equal(replaced.pair.targetGuild, F2, 'a request that has not copied is replaced'); + assert.equal(await db.getPair(first.pair.id), null); + await db.updatePair(replaced.pair.id, { status: 'live' }); + await rejects(ask({ targetGuild: F1 }), 'already-mirrored'); + await rejects(ask({ targetGuild: F2 }), 'already-mirrored'); + await db.updatePair(replaced.pair.id, { status: 'stopped' }); + const again = await ask({ targetGuild: F2, targetOwner: true }); + assert.equal( + again.pair.id, + replaced.pair.id, + 'asking again after a stop reuses the pair, and so its id map' + ); + assert.equal(again.pair.status, 'ready'); +}); + +test('the code works for 15 minutes', () => { + assert.equal(CODE_TTL_MS, 15 * 60_000); +}); + +test('a stopped mirror that made something keeps its target: another server cannot take it over', async () => { + const first = await ask({ targetOwner: true }); + await db.putMapping({ pairId: first.pair.id, kind: 'channel', sourceId: '1', targetId: 't1', owned: true }); + await db.updatePair(first.pair.id, { status: 'stopped' }); + await rejects(ask({ sourceGuild: D2, targetOwner: true }), 'target-busy'); + assert.ok(await db.getPair(first.pair.id), 'the pair and its id map are kept'); + await db.deleteMapping(first.pair.id, 'channel', '1'); + const later = await ask({ sourceGuild: D2, targetOwner: true }); + assert.equal(later.pair.sourceGuild, D2, 'a stopped mirror that made nothing can be replaced'); + await db.deletePair(later.pair.id); +}); + +test('the consent check can be replaced (the bridge can plug its link codes in)', async () => { + await ask({ targetOwner: true }); + const pair = await db.getPairBySource(D1); + assert.equal((await requirePairConsent(pair.id)).id, pair.id); + const previous = setConsentProvider({ + name: 'strict', + check: () => ({ ok: false, reason: 'the bridge says no' }), + }); + await assert.rejects(requirePairConsent(pair.id), /the bridge says no/); + setConsentProvider(previous); + assert.equal((await requirePairConsent(pair.id)).id, pair.id); + await rejects(requirePairConsent(999_999), 'no-consent'); +}); + +test('real deletion asked for in a request takes effect only when the same person owns both servers', async () => { + await db.deletePair((await db.getPairBySource(D1))?.id ?? 0).catch(() => {}); + const other = await ask({ targetOwner: false, options: { mirrorDeletes: true } }); + assert.deepEqual( + [other.pair.options.mirrorDeletes, other.pair.options.mirrorDeletesRequested], + [false, true], + 'a request only, until the Fluxer owner turns it on' + ); + const own = await ask({ targetOwner: true, options: { mirrorDeletes: true } }); + assert.deepEqual([own.pair.options.mirrorDeletes, own.pair.options.mirrorDeletesRequested], [true, true]); + const plain = await ask({ targetOwner: false, options: { roles: false } }); + assert.deepEqual( + [plain.pair.options.mirrorDeletes, plain.pair.options.mirrorDeletesRequested], + [false, false] + ); +}); diff --git a/test/mirrorEngine.test.js b/test/mirrorEngine.test.js new file mode 100644 index 00000000..44f18b30 --- /dev/null +++ b/test/mirrorEngine.test.js @@ -0,0 +1,726 @@ +// The mirror engine (src/mirror/engine.js) against a fake Discord server and a fake Fluxer adapter: the dry run +// equals the copy, a copy survives a restart and a crash, changes on Discord follow, Sylo never deletes what it did +// not create, and trouble with Fluxer is spaced out, backed off and, when it keeps up, pauses the pair. +import './helpers/tmpDb.js'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import * as db from '../src/db/mirror.js'; +import { getGuildModule, setGuildModule } from '../src/db/modules.js'; +import { createEngine } from '../src/mirror/engine.js'; +import { linkMirroredChannels } from '../src/mirror/bridgeLink.js'; +import { dChannel, engineFor } from './helpers/mirrorFakes.js'; +import { drain, names, scenario } from './helpers/mirrorScenario.js'; + +test('the dry run creates nothing and lists everything with what cannot be carried over', async () => { + const s = await scenario(); + const preview = await s.engine.previewPair(s.pair); + assert.equal(preview.ok, true); + assert.equal(s.fluxer.calls.length, 0, 'a preview never writes to Fluxer'); + assert.equal(s.fetched.length, 0, 'and downloads nothing'); + assert.deepEqual( + preview.summary.role, + { create: 2, update: 1, unchanged: 0, skip: 1 }, + 'Member, Mod; @everyone is updated; the bot role is skipped' + ); + assert.deepEqual(preview.summary.channel, { create: 5, update: 0, unchanged: 0, skip: 0 }); + assert.deepEqual(preview.summary.emoji, { create: 2, update: 0, unchanged: 0, skip: 0 }); + assert.deepEqual(preview.summary.sticker, { create: 1, update: 0, unchanged: 0, skip: 1 }); + const codes = new Set(preview.notes.map((x) => x.code)); + for (const code of ['permissions-dropped', 'managed-role', 'sticker-lottie']) + assert.ok(codes.has(code), code); + assert.equal(preview.steps.at(-1).kind, 'bridge'); + assert.equal(preview.steps[0].kind, 'guild'); + // categories before the channels in them, roles before channels + const order = preview.steps.map((x) => x.key); + assert.ok(order.indexOf(`role:${s.k.member}`) < order.indexOf(`channel:${s.k.rules}`)); + assert.ok(order.indexOf(`channel:${s.k.cat}`) < order.indexOf(`channel:${s.k.rules}`)); +}); + +test('dry run equals real run: the same things, in the same order, with the same specs and notes', async () => { + const s = await scenario(); + const preview = await s.engine.previewPair(s.pair); + await s.engine.startCopy(s.pair.id); + const end = await drain(s); + assert.equal(end.status, 'live'); + const report = await s.engine.report(end); + + const planned = (kind, op) => preview.steps.filter((x) => x.kind === kind && x.op === op); + assert.equal(s.fluxer.ops('createRole').length, planned('role', 'create').length); + assert.equal(s.fluxer.ops('createChannel').length, planned('channel', 'create').length); + assert.equal(s.fluxer.ops('createEmoji').length, planned('emoji', 'create').length); + assert.equal(s.fluxer.ops('createSticker').length, planned('sticker', 'create').length); + assert.deepEqual( + names(s.fluxer.ops('createChannel')), + planned('channel', 'create').map((x) => x.name), + 'channels are made in the order the preview showed' + ); + assert.deepEqual( + names(s.fluxer.ops('createRole')), + planned('role', 'create').map((x) => x.name) + ); + + // The preview's spec, with its placeholders resolved through the id map, is what was sent. + const resolve = async (kind, id) => + id === 'everyone' ? s.fluxer.guild.id : (await db.getMapping(s.pair.id, kind, id)).targetId; + for (const step of planned('channel', 'create')) { + const madeId = (await db.getMapping(s.pair.id, 'channel', step.sourceId)).targetId; + const sent = s.fluxer.ops('createChannel').find((c) => c.id === madeId); + assert.equal(sent.spec.name, step.spec.name); + assert.equal(sent.spec.type, step.spec.type); + assert.equal(sent.spec.topic ?? null, step.spec.topic ?? null); + const wantParent = step.spec.parentId ? await resolve('channel', step.spec.parentId.split(':')[1]) : null; + assert.equal(sent.spec.parentId, wantParent); + const wantOverwrites = []; + for (const o of step.spec.permissionOverwrites) { + wantOverwrites.push({ + id: await resolve('role', o.id.split(':')[1]), + type: 0, + allow: BigInt(o.allow), + deny: BigInt(o.deny), + }); + } + assert.deepEqual( + sent.spec.permissionOverwrites.map((o) => ({ + id: o.id, + type: o.type, + allow: BigInt(o.allow), + deny: BigInt(o.deny), + })), + wantOverwrites + ); + } + + const key = (x) => `${x.code}|${x.ref.id}|${x.message}`; + assert.deepEqual( + report.notes.map(key).sort(), + preview.notes.map(key).sort(), + 'the report lists what the preview listed' + ); + + // Run again: everything is unchanged, so a second preview plans nothing to create. + const again = await s.engine.previewPair(end); + for (const kind of ['role', 'channel', 'emoji', 'sticker']) + assert.equal(again.summary[kind].create, 0, kind); + assert.equal(again.summary.channel.unchanged, 5); +}); + +test('a copy sends what the server has: ids remapped, @everyone adopted, settings carried over', async () => { + const s = await scenario(); + const welcome = { + joinEnabled: true, + joinChannel: s.k.chat, + joinMessage: 'Hi, read <#' + s.k.rules + '>', + autoRole: s.k.member, + panelMessageId: '999999999999999999', + userId: '123456789012345678', + }; + await setGuildModule(s.k.src, 'welcome', { enabled: true, config: welcome }); + await setGuildModule(s.k.src, 'logging', { enabled: true, config: { channel: s.k.bot } }); + await setGuildModule(`fluxer:${s.k.dst}`, 'logging', { enabled: false, config: { channel: '1' } }); + await s.engine.startCopy(s.pair.id); + const end = await drain(s); + assert.equal(end.status, 'live'); + const map = async (kind, id) => (await db.getMapping(s.pair.id, kind, id)).targetId; + + const category = s.fluxer.ops('createChannel').find((c) => c.spec.name === 'Info'); + const rules = s.fluxer.ops('createChannel').find((c) => c.spec.name === 'rules'); + assert.equal(rules.spec.parentId, category.id); + assert.deepEqual( + rules.spec.permissionOverwrites.map((o) => o.id).sort(), + [await map('role', s.k.member), s.fluxer.guild.id].sort() + ); + const everyone = await db.getMapping(s.pair.id, 'role', s.k.src); + assert.deepEqual([everyone.targetId, everyone.owned], [s.fluxer.guild.id, false]); + const edit = s.fluxer.ops('editRole').find((c) => c.id === s.fluxer.guild.id); + assert.ok( + BigInt(edit.spec.permissions) & (1n << 53n), + 'the Fluxer-only permission @everyone already had is kept' + ); + + const there = await getGuildModule(`fluxer:${s.k.dst}`, 'welcome'); + assert.equal(there.enabled, true); + assert.equal(there.config.joinChannel, await map('channel', s.k.chat)); + assert.equal(there.config.joinMessage, `Hi, read <#${await map('channel', s.k.rules)}>`); + assert.equal(there.config.autoRole, await map('role', s.k.member)); + assert.equal(there.config.panelMessageId, '', 'the id of a message that is not copied is cleared'); + assert.equal(there.config.userId, '', 'the id of a person means nothing on Fluxer'); + assert.deepEqual( + (await getGuildModule(`fluxer:${s.k.dst}`, 'logging')).config, + { channel: '1' }, + 'settings already on the Fluxer server are not overwritten' + ); + const report = await s.engine.report(end); + assert.ok( + report.items.some((i) => i.kind === 'module' && i.name === 'Server logging' && i.status === 'skipped') + ); + + assert.deepEqual( + s.fluxer.ops('editGuild').map((c) => c.body.name), + ['Source'], + 'the name is copied' + ); + assert.match(s.fluxer.ops('editGuild')[0].body.icon, /^data:image\/png;base64,/); + assert.equal(s.fluxer.ops('createEmoji').length, 2); + assert.equal(s.fluxer.ops('createSticker').length, 1, 'the Lottie sticker is left out'); + assert.equal(s.fluxer.ops('setRolePositions').length, 1); + assert.equal(s.fluxer.ops('setChannelPositions').length, 1); +}); + +test('the last step, linking through the bridge, says why it linked nothing when the platforms are not both up', async () => { + assert.equal((await linkMirroredChannels(999_999)).reason, 'no-such-mirror'); + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + const end = await drain(s); + const report = await s.engine.report(end); + assert.equal(report.bridge.status, 'done'); + assert.match(report.bridge.detail, /bridge-not-available/); + let calledWith = null; + const linked = await scenario({ + withExpressions: false, + engineOpts: { linkMirroredChannels: async (id) => ((calledWith = id), { linked: 3 }) }, + }); + await linked.engine.startCopy(linked.pair.id); + const done = await drain(linked); + assert.equal(calledWith, linked.pair.id); + assert.match((await linked.engine.report(done)).bridge.detail, /3 channels linked/); +}); + +test('a copy resumes after a restart without making anything twice', async () => { + const s = await scenario(); + await s.engine.startCopy(s.pair.id); + await s.engine.runPair(s.pair.id, { maxJobs: 6 }); + const before = await s.engine.progress(await db.getPair(s.pair.id)); + assert.ok(before.done > 0 && before.waiting > 0, 'stopped half way'); + assert.equal((await db.getPair(s.pair.id)).status, 'copying'); + + // A new process: a new engine, the same database and servers. + const { engine: second } = engineFor({ discord: s.discord, fluxer: s.fluxer, clock: s.clock }); + await second.resumeAfterRestart(); + s.engine = second; + const end = await drain(s); + assert.equal(end.status, 'live'); + const created = names(s.fluxer.ops('createChannel')); + assert.equal(new Set(created).size, created.length, 'no channel was made twice'); + assert.equal(created.length, 5); + assert.equal(s.fluxer.ops('createRole').length, 2); + const progress = await second.progress(end); + assert.deepEqual([progress.waiting, progress.percent], [0, 100]); +}); + +test('a job that was running when the process died adopts what it made, and never owns it', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + // Work until the next job is the "rules" channel. + for (;;) { + const job = await db.nextDueJob(s.pair.id, s.clock.now); + if (job.kind === 'channel' && job.sourceId === s.k.rules) break; + await s.engine.runPair(s.pair.id, { maxJobs: 1 }); + } + const job = await db.nextDueJob(s.pair.id, s.clock.now); + const parent = (await db.getMapping(s.pair.id, 'channel', s.k.cat)).targetId; + // The dead process claimed the job and created the channel on Fluxer, but never wrote the id map. + assert.equal(await db.claimJob(job.id, s.clock.now), true); + const madeId = await s.fluxer.target.createChannel({ + name: 'rules', + type: 0, + parentId: parent, + permissionOverwrites: [], + }); + s.fluxer.calls.length = 0; + + const { engine: second } = engineFor({ discord: s.discord, fluxer: s.fluxer, clock: s.clock }); + await second.resumeAfterRestart(); + s.engine = second; + const end = await drain(s); + assert.equal(end.status, 'live'); + assert.equal( + s.fluxer.ops('createChannel').filter((c) => c.spec.name === 'rules').length, + 0, + 'it was not made a second time' + ); + const m = await db.getMapping(s.pair.id, 'channel', s.k.rules); + assert.deepEqual([m.targetId, m.owned], [madeId, false]); + assert.equal([...s.fluxer.guild.channels.values()].filter((c) => c.name === 'rules').length, 1); +}); + +test('changes on Discord follow: rename, create, delete, reorder; a burst of events is one job', async () => { + const s = await scenario({ options: { mirrorDeletes: true } }); + await s.engine.startCopy(s.pair.id); + assert.equal((await drain(s)).status, 'live'); + s.fluxer.calls.length = 0; + const target = async (kind, id) => (await db.getMapping(s.pair.id, kind, id)).targetId; + const note = (change) => s.engine.noteSourceChange(s.k.src, change); + + s.discord.channels.cache.get(s.k.chat).name = 'chatter'; + assert.equal(await note({ kind: 'channel', op: 'sync', id: s.k.chat }), true); + for (let i = 0; i < 4; i += 1) + assert.equal(await note({ kind: 'channel', op: 'sync', id: s.k.chat }), false, 'already waiting'); + assert.equal(await db.openSyncCount(s.pair.id), 1); + + const newId = '750000000000000001'; + s.discord.channels.cache.set( + newId, + dChannel({ id: newId, name: 'new-one', type: 0, parentId: s.k.cat, rawPosition: 9 }) + ); + await note({ kind: 'channel', op: 'sync', id: newId, reorder: 'channels' }); + s.discord.roles.cache.get(s.k.mod).name = 'Moderators'; + await note({ kind: 'role', op: 'sync', id: s.k.mod }); + s.discord.emojis.cache.set('750000000000000002', { + id: '750000000000000002', + name: 'fresh', + animated: false, + managed: false, + imageURL: () => 'https://cdn.test/f.png', + }); + await note({ kind: 'emoji', op: 'sync', id: '750000000000000002' }); + const goneTarget = await target('emoji', s.k.emoji1); + s.discord.emojis.cache.delete(s.k.emoji1); + await note({ kind: 'emoji', op: 'delete', id: s.k.emoji1 }); + + await s.engine.runPair(s.pair.id); + assert.deepEqual( + s.fluxer.ops('editChannel').map((c) => c.spec.name), + ['chatter'] + ); + assert.deepEqual(names(s.fluxer.ops('createChannel')), ['new-one']); + assert.equal(s.fluxer.ops('createChannel')[0].spec.parentId, await target('channel', s.k.cat)); + assert.deepEqual( + s.fluxer.ops('editRole').map((c) => c.spec.name), + ['Moderators'] + ); + assert.deepEqual( + s.fluxer.ops('createEmoji').map((c) => c.name), + ['fresh'] + ); + assert.deepEqual( + s.fluxer.ops('deleteEmoji').map((c) => c.id), + [goneTarget] + ); + assert.equal(await db.getMapping(s.pair.id, 'emoji', s.k.emoji1), null); + assert.equal( + s.fluxer.ops('setChannelPositions').length, + 0, + 'the reorder waits a moment so a drag is one reorder' + ); + s.clock.now += 4000; + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.ops('setChannelPositions').length, 1); + + // A delete on Discord removes the Fluxer copy; a rename in between is dropped. + const chatTarget = await target('channel', s.k.chat); + s.discord.channels.cache.delete(s.k.chat); + await note({ kind: 'channel', op: 'sync', id: s.k.chat }); + await note({ kind: 'channel', op: 'delete', id: s.k.chat }); + s.fluxer.calls.length = 0; + await s.engine.runPair(s.pair.id); + assert.deepEqual( + s.fluxer.ops('deleteChannel').map((c) => c.id), + [chatTarget] + ); + assert.equal(s.fluxer.ops('editChannel').length, 0); + assert.equal(s.fluxer.guild.channels.has(chatTarget), false); +}); + +test('Sylo never deletes a Fluxer item it did not create', async () => { + const foreign = { id: '760000000000000001', name: 'general', type: 0, parentId: null }; + const mine = { id: '760000000000000002', name: 'rules', type: 0, parentId: null }; + const s = await scenario({ + fluxerOpts: { existingChannels: [foreign, mine] }, + withExpressions: false, + options: { mirrorDeletes: true }, + }); + await s.engine.startCopy(s.pair.id); + assert.equal((await drain(s)).status, 'live'); + assert.ok(s.fluxer.guild.channels.has(foreign.id), 'a channel that was already there stays'); + assert.ok(s.fluxer.guild.channels.has(mine.id), 'even one with the same name as a copied channel'); + assert.equal( + s.fluxer.ops('createChannel').filter((c) => c.spec.name === 'rules').length, + 1, + 'a copy is made next to it' + ); + + const note = (change) => s.engine.noteSourceChange(s.k.src, change); + // Deleting things on Discord that Sylo never copied does nothing on Fluxer. + assert.equal( + await note({ kind: 'channel', op: 'delete', id: '760000000000000099' }), + true, + 'queued; the job finds no copy' + ); + assert.equal( + await note({ kind: 'role', op: 'delete', id: s.k.bot }), + true, + 'a role that was skipped has no copy' + ); + // An item the id map holds as not owned is forgotten, never deleted. + await db.putMapping({ + pairId: s.pair.id, + kind: 'channel', + sourceId: '760000000000000050', + targetId: foreign.id, + owned: false, + }); + assert.equal(await note({ kind: 'channel', op: 'delete', id: '760000000000000050' }), true); + // Deleting the @everyone mapping (adopted) the same way. + await db.putMapping({ + pairId: s.pair.id, + kind: 'role', + sourceId: '760000000000000051', + targetId: s.fluxer.guild.id, + owned: false, + }); + assert.equal(await note({ kind: 'role', op: 'delete', id: '760000000000000051' }), true); + s.fluxer.calls.length = 0; + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.ops('deleteChannel').length, 0); + assert.equal(s.fluxer.ops('deleteRole').length, 0); + assert.ok(s.fluxer.guild.channels.has(foreign.id)); + assert.ok(s.fluxer.guild.roles.has(s.fluxer.guild.id)); + assert.equal( + await db.getMapping(s.pair.id, 'channel', '760000000000000050'), + null, + 'the mapping is dropped' + ); + // What Sylo did create is deleted when its original goes. + const mineOnFluxer = (await db.getMapping(s.pair.id, 'channel', s.k.rules)).targetId; + assert.notEqual(mineOnFluxer, mine.id); + s.discord.channels.cache.delete(s.k.rules); + await note({ kind: 'channel', op: 'delete', id: s.k.rules }); + await s.engine.runPair(s.pair.id); + assert.deepEqual( + s.fluxer.ops('deleteChannel').map((c) => c.id), + [mineOnFluxer] + ); + assert.ok(s.fluxer.guild.channels.has(mine.id)); +}); + +test('mirroring is one way: a change made on Fluxer is not read back, and only Discord is ever read', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + await drain(s); + const chat = (await db.getMapping(s.pair.id, 'channel', s.k.chat)).targetId; + s.fluxer.guild.channels.get(chat).name = 'renamed-on-fluxer'; + s.fluxer.calls.length = 0; + // Nothing on Fluxer is watched: no job exists until Discord changes. + await s.engine.tick(); + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.calls.length, 0); + // The next change on Discord overwrites it. + s.discord.channels.cache.get(s.k.chat).topic = 'new topic'; + await s.engine.noteSourceChange(s.k.src, { kind: 'channel', op: 'sync', id: s.k.chat }); + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.guild.channels.get(chat).name, 'chat'); +}); + +test('changes are not queued unless a mirror is running for that server, and continuous mirroring is on', async () => { + const s = await scenario({ withExpressions: false, options: { continuous: false } }); + const change = { kind: 'channel', op: 'sync', id: s.k.chat }; + assert.equal(await s.engine.noteSourceChange(s.k.src, change), false, 'agreed but not copied yet'); + await s.engine.startCopy(s.pair.id); + assert.equal( + (await drain(s)).status, + 'done', + 'with continuous mirroring off the pair is done after the copy' + ); + assert.equal(await s.engine.noteSourceChange(s.k.src, change), false); + assert.equal( + await s.engine.noteSourceChange('930000000000999999', change), + false, + 'another server has no pair' + ); + await db.updatePair(s.pair.id, { + status: 'live', + options: { ...s.pair.options, channels: false, continuous: true }, + }); + assert.equal(await s.engine.noteSourceChange(s.k.src, change), false, 'channels are turned off'); + await db.updatePair(s.pair.id, { options: { continuous: true } }); + assert.equal(await s.engine.noteSourceChange(s.k.src, change), true); +}); + +test('pausing keeps changes for later; stopping drops the work and the consent', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + await drain(s); + await s.engine.pause(s.pair.id); + assert.equal((await db.getPair(s.pair.id)).status, 'paused'); + s.discord.channels.cache.get(s.k.chat).name = 'while-paused'; + assert.equal(await s.engine.noteSourceChange(s.k.src, { kind: 'channel', op: 'sync', id: s.k.chat }), true); + s.fluxer.calls.length = 0; + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.calls.length, 0, 'a paused mirror does nothing'); + await s.engine.resume(s.pair.id); + await s.engine.runPair(s.pair.id); + assert.deepEqual( + s.fluxer.ops('editChannel').map((c) => c.spec.name), + ['while-paused'] + ); + + await s.engine.noteSourceChange(s.k.src, { kind: 'channel', op: 'sync', id: s.k.rules }); + const stopped = await s.engine.stopPair(s.pair.id); + assert.equal(stopped.status, 'stopped'); + assert.equal(stopped.sourceConsentBy, ''); + assert.equal(await db.openSyncCount(s.pair.id), 0); + assert.equal( + await s.engine.noteSourceChange(s.k.src, { kind: 'channel', op: 'sync', id: s.k.rules }), + false + ); + await assert.rejects(s.engine.startCopy(s.pair.id), /agree/); + assert.equal(await s.engine.removePair(s.pair.id), true); + assert.equal(await db.getPair(s.pair.id), null); +}); + +test('Fluxer asking to slow down: the job waits as long as asked, then goes on', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + s.fluxer.failOn('createRole', Object.assign(new Error('rate limited'), { statusCode: 429, retryAfter: 4 })); + await s.engine.runPair(s.pair.id); + const waiting = (await db.listRunJobs(s.pair.id, 1)).find( + (j) => j.status === 'pending' && j.attempts === 0 && j.detail + ); + assert.ok(waiting, 'the job is waiting'); + assert.equal(waiting.nextAt, s.clock.now + 4000, 'for what Fluxer asked'); + assert.equal(waiting.detail, 'Fluxer asked to slow down'); + assert.equal((await db.getPair(s.pair.id)).failures, 0, 'being asked to slow down is not a failure'); + const attemptsBefore = s.fluxer.attempted.length; + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.attempted.length, attemptsBefore, 'it does not try again before it is time'); + s.clock.now = waiting.nextAt + 1; + assert.equal((await drain(s)).status, 'live'); + assert.equal(s.fluxer.ops('createRole').length, 2); +}); + +test('Fluxer calls are spaced out: a wait after each job that called Fluxer', async () => { + const s = await scenario({ withExpressions: false, engineOpts: { minIntervalMs: 250 } }); + await s.engine.startCopy(s.pair.id); + await drain(s); + assert.ok(s.clock.slept.length >= 8, 'every role and channel, and the ordering'); + assert.ok(s.clock.slept.every((ms) => ms === 250)); +}); + +test('a network failure backs off with a doubling delay', async () => { + const s = await scenario({ withExpressions: false, engineOpts: { pauseAfterFailures: 99 } }); + await s.engine.startCopy(s.pair.id); + s.fluxer.failOn('*', Object.assign(new Error('fetch failed'), { code: 'ECONNRESET' }), 999); + const waits = []; + for (let i = 0; i < 4; i += 1) { + await s.engine.runPair(s.pair.id); + const job = (await db.listRunJobs(s.pair.id, 1)).find((j) => j.kind === 'guild'); + assert.equal(job.attempts, i + 1); + waits.push(job.nextAt - job.updatedAt); + s.clock.now = job.nextAt; + } + assert.deepEqual(waits, [100, 200, 400, 800]); +}); + +test('when Fluxer keeps failing the pair pauses with the reason, and resumes where it stopped', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + s.fluxer.failOn('*', Object.assign(new Error('fetch failed'), { code: 'ECONNRESET' }), 999); + await s.engine.runPair(s.pair.id); + const pair = await db.getPair(s.pair.id); + assert.equal(pair.status, 'paused'); + assert.equal(pair.resumeStatus, 'copying'); + assert.match(pair.lastError, /could not be reached/); + assert.equal(pair.failures, 3); + const tries = s.fluxer.attempted.length; + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.attempted.length, tries, 'paused: nothing is tried'); + s.fluxer.clearFailures(); + await s.engine.resume(s.pair.id); + assert.equal((await db.getPair(s.pair.id)).lastError, ''); + assert.equal((await drain(s)).status, 'live'); + assert.equal(s.fluxer.ops('createRole').length, 2); +}); + +test('a job that keeps failing is given up on after the attempts run out', async () => { + const s = await scenario({ engineOpts: { maxAttempts: 2, pauseAfterFailures: 99 } }); + await s.engine.startCopy(s.pair.id); + s.fluxer.failOn('createEmoji', Object.assign(new Error('bad gateway'), { statusCode: 502 }), 99); + const end = await drain(s); + assert.equal(end.status, 'live', 'the copy finishes without those items'); + const failed = await db.listFailedJobs(s.pair.id); + assert.deepEqual( + failed.map((j) => [j.kind, j.detail]), + [ + ['emoji', 'bad gateway'], + ['emoji', 'bad gateway'], + ] + ); + assert.equal(s.fluxer.attempted.filter((o) => o === 'createEmoji').length, 4, 'two attempts each'); +}); + +test('Fluxer refusing Sylo pauses the pair with a reason, and one refused item does not', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + s.fluxer.failOn('createChannel', Object.assign(new Error('Missing Permissions'), { statusCode: 403 }), 99); + await s.engine.runPair(s.pair.id); + const paused = await db.getPair(s.pair.id); + assert.equal(paused.status, 'paused'); + assert.match(paused.lastError, /Missing Permissions/); + assert.match(paused.lastError, /Check Sylo's permissions/); + + const t = await scenario({ withExpressions: false }); + await t.engine.startCopy(t.pair.id); + t.fluxer.failOn('createChannel', Object.assign(new Error('Invalid Form Body'), { statusCode: 400 }), 1); + const end = await drain(t); + assert.equal(end.status, 'live', 'a bad request for one channel is that channel only'); + assert.equal((await db.listFailedJobs(t.pair.id)).length, 1); + assert.equal((await t.engine.report(end)).failed.length, 1); +}); + +test('a pair that fails is isolated: the other pair carries on', async () => { + const a = await scenario({ withExpressions: false }); + const b = await scenario({ withExpressions: false }); + const engine = createEngine({ + now: a.clock.fn, + sleep: a.clock.sleep, + minIntervalMs: 1, + backoffBaseMs: 10, + clientsReady: () => true, + sourceGuild: (key) => [a, b].map((x) => x.discord).find((d) => d.id === key), + targetGuild: (key) => [a, b].map((x) => x.fluxer.guild).find((g) => `fluxer:${g.id}` === key), + makeTarget: (guild) => [a, b].map((x) => x.fluxer).find((f) => f.guild === guild).target, + fetchImage: async () => ({ bytes: Buffer.alloc(8), mime: 'image/png' }), + }); + a.engine = engine; + b.engine = engine; + await engine.startCopy(a.pair.id); + await engine.startCopy(b.pair.id); + a.fluxer.failOn('createRole', Object.assign(new Error('boom'), { statusCode: 500 }), 99); + await Promise.all([engine.runPair(a.pair.id), engine.runPair(b.pair.id)]); + assert.equal((await db.getPair(b.pair.id)).status, 'live', 'B finished'); + assert.equal(b.fluxer.ops('createRole').length, 2); + assert.notEqual((await db.getPair(a.pair.id)).status, 'live', 'A is still waiting for Fluxer'); + assert.equal(a.fluxer.ops('createRole').length, 0); + assert.equal( + b.fluxer.ops('createChannel').length, + 5, + 'and every channel of B went to the Fluxer server of B' + ); + assert.ok(a.fluxer.guild.channels.size < 5, 'none of them went to A'); +}); + +test('emoji: what does not fit is skipped and reported, in the preview and when Fluxer says it is full', async () => { + const s = await scenario({ fluxerOpts: { emojiLimit: 1 } }); + const preview = await s.engine.previewPair(s.pair); + assert.deepEqual(preview.summary.emoji, { create: 1, update: 0, unchanged: 0, skip: 1 }); + assert.ok(preview.notes.some((x) => x.code === 'emoji-limit')); + await s.engine.startCopy(s.pair.id); + await drain(s); + assert.equal(s.fluxer.ops('createEmoji').length, 1); + + // Fluxer's own limit is lower than ours: it refuses, and the rest of that kind is skipped. + const t = await scenario(); + t.fluxer.failOn( + 'createEmoji', + Object.assign(new Error('Maximum number of emoji reached (1)'), { statusCode: 400 }), + 99 + ); + await t.engine.startCopy(t.pair.id); + const end = await drain(t); + assert.equal(end.status, 'live'); + const report = await t.engine.report(end); + assert.equal(report.items.filter((i) => i.kind === 'emoji' && i.status === 'skipped').length, 2); + assert.ok(report.notes.some((x) => x.code === 'emoji-limit')); + assert.equal((await db.listFailedJobs(t.pair.id)).length, 0, 'being full is not a failure'); + assert.equal( + t.fluxer.attempted.filter((o) => o === 'createEmoji').length, + 1, + 'it asked once, then stopped asking' + ); +}); + +test('an emoji over the size limit is skipped with a note', async () => { + const s = await scenario({ discordOpts: {} }); + s.discord.emojis.cache.get(s.k.emoji1).imageURL = () => 'https://cdn.test/big.png'; + await s.engine.startCopy(s.pair.id); + const end = await drain(s); + const report = await s.engine.report(end); + const note = report.notes.find((x) => x.code === 'emoji-too-large'); + assert.ok(note); + assert.match(note.message, /512 KB/); + assert.equal(s.fluxer.ops('createEmoji').length, 1); +}); + +test('what cannot start is refused with the reason: no permission, a server missing, nobody agreed', async () => { + const noPerm = await scenario({ fluxerOpts: { missing: ['ManageRoles'] } }); + await assert.rejects(noPerm.engine.startCopy(noPerm.pair.id), /Sylo needs ManageRoles/); + assert.equal((await noPerm.engine.previewPair(noPerm.pair)).ok, false); + assert.equal(noPerm.fluxer.calls.length, 0); + + const s = await scenario({ withExpressions: false }); + const lonely = createEngine({ + clientsReady: () => true, + sourceGuild: () => undefined, + targetGuild: () => s.fluxer.guild, + }); + const preview = await lonely.previewPair(s.pair); + assert.equal(preview.ok, false); + assert.equal(preview.blockers[0].code, 'source-missing'); + + const pending = await db.updatePair(s.pair.id, { targetConsentBy: '', status: 'pending' }); + await assert.rejects(s.engine.startCopy(pending.id), /Fluxer server/); +}); + +test('a server that disappears pauses the pair once the platforms are up; while one connects it waits', async () => { + let up = false; + const s = await scenario({ withExpressions: false, engineOpts: { clientsReady: () => up } }); + await s.engine.startCopy(s.pair.id); + s.engine.deps.targetGuild = () => undefined; + await s.engine.runPair(s.pair.id); + assert.equal((await db.getPair(s.pair.id)).status, 'copying', 'waiting for the platform'); + up = true; + await s.engine.runPair(s.pair.id); + const pair = await db.getPair(s.pair.id); + assert.equal(pair.status, 'paused'); + assert.match(pair.lastError, /no longer in the Fluxer server/); +}); + +test('losing consent pauses the pair at the next job', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + await db.updatePair(s.pair.id, { targetConsentBy: '' }); + await s.engine.runPair(s.pair.id); + const pair = await db.getPair(s.pair.id); + assert.equal(pair.status, 'paused'); + assert.equal(s.fluxer.calls.length, 0); +}); + +test('only one worker runs per pair, and the timer starts workers for pairs that have work', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + const first = s.engine.runPair(s.pair.id); + const second = await s.engine.runPair(s.pair.id); + assert.equal(second.ran, 0, 'the second call finds a worker already running'); + await first; + const t = await scenario({ withExpressions: false }); + await t.engine.startCopy(t.pair.id); + await t.engine.tick(); + for (let i = 0; i < 100 && t.engine.isRunning(t.pair.id); i += 1) await new Promise((r) => setImmediate(r)); + for (let i = 0; i < 100 && (await db.getPair(t.pair.id)).status === 'copying'; i += 1) + await t.engine.tick(); + assert.equal((await db.getPair(t.pair.id)).status, 'live'); +}); + +test('a second copy after changes sends only what changed', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + await drain(s); + s.discord.channels.cache.get(s.k.chat).topic = 'a topic'; + s.discord.roles.cache.get(s.k.member).name = 'Members'; + const preview = await s.engine.previewPair(await db.getPair(s.pair.id)); + assert.deepEqual(preview.summary.channel, { create: 0, update: 1, unchanged: 4, skip: 0 }); + assert.deepEqual([preview.summary.role.create, preview.summary.role.update], [0, 1]); + s.fluxer.calls.length = 0; + await s.engine.startCopy(s.pair.id); + const end = await drain(s); + assert.equal(end.copyRun, 2); + assert.deepEqual( + s.fluxer.ops('editChannel').map((c) => c.spec.name), + ['chat'] + ); + assert.deepEqual( + s.fluxer.ops('editRole').map((c) => c.spec.name), + ['Members'] + ); + assert.equal(s.fluxer.ops('createChannel').length, 0); +}); diff --git a/test/mirrorGuards.test.js b/test/mirrorGuards.test.js new file mode 100644 index 00000000..7c0f5bc7 --- /dev/null +++ b/test/mirrorGuards.test.js @@ -0,0 +1,658 @@ +// What keeps a mirror from doing more than its owners agreed to (src/mirror/engine.js): only the servers' owners, +// deletions that lock instead of delete, a brake on bursts of deletes, the role hierarchy, privacy that never gets +// more open than on Discord, module settings that pass the dashboard's grant checks, and no second copy of anything. +import './helpers/tmpDb.js'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import * as db from '../src/db/mirror.js'; +import { getGuildModule, setGuildModule } from '../src/db/modules.js'; +import { createEngine } from '../src/mirror/engine.js'; +import { BIT, SYLO_ROLE, engineFor } from './helpers/mirrorFakes.js'; +import { drain, scenario } from './helpers/mirrorScenario.js'; + +const OWNERS = { enforceOwnership: () => true }; +const view = 1n << 10n; +const send = 1n << 11n; +const react = 1n << 6n; +const connect = 1n << 20n; +const mapped = async (s, kind, id) => (await db.getMapping(s.pair.id, kind, id)).targetId; + +test('the owners who agreed must still own both servers: a transfer pauses the mirror until the new owner confirms', async () => { + const s = await scenario({ withExpressions: false, engineOpts: OWNERS }); + await s.engine.startCopy(s.pair.id); + await s.engine.runPair(s.pair.id, { maxJobs: 3 }); + s.fluxer.guild.ownerId = 'new-fluxer-owner'; + s.fluxer.calls.length = 0; + await s.engine.runPair(s.pair.id); + let pair = await db.getPair(s.pair.id); + assert.deepEqual([pair.status, pair.pauseKind], ['paused', 'ownership']); + assert.match(pair.lastError, /changed hands/); + assert.equal(s.fluxer.calls.length, 0, 'nothing is done for a mirror whose server changed hands'); + assert.equal((await s.engine.previewPair(pair)).blockers[0].code, 'ownership'); + + // The Discord owner cannot confirm for the Fluxer server. + pair = await s.engine.resume(s.pair.id, { side: 'source' }); + assert.equal(pair.status, 'paused'); + assert.match(pair.lastError, /Waiting for the owner of the Fluxer server/); + // The new Fluxer owner confirms; the mirror goes on and records them as the person who agreed. + pair = await s.engine.resume(s.pair.id, { side: 'target' }); + assert.equal(pair.status, 'copying'); + assert.equal(pair.targetConsentUser, 'new-fluxer-owner'); + assert.equal((await drain(s)).status, 'live'); +}); + +test('when both servers change hands, both new owners have to confirm', async () => { + const s = await scenario({ withExpressions: false, engineOpts: OWNERS }); + await s.engine.startCopy(s.pair.id); + s.discord.ownerId = 'new-discord-owner'; + s.fluxer.guild.ownerId = 'new-fluxer-owner'; + await s.engine.runPair(s.pair.id); + assert.equal((await db.getPair(s.pair.id)).pauseKind, 'ownership'); + assert.equal((await s.engine.resume(s.pair.id, { side: 'target' })).status, 'paused'); + assert.equal((await s.engine.resume(s.pair.id, { side: 'source' })).status, 'copying'); +}); + +test('a copy cannot start when the people who agreed no longer own the servers', async () => { + const s = await scenario({ withExpressions: false, engineOpts: OWNERS }); + s.discord.ownerId = 'someone-else'; // an Administrator, not the owner, is not enough + await assert.rejects(s.engine.startCopy(s.pair.id), /changed hands/); + assert.equal(s.fluxer.calls.length, 0); + s.discord.ownerId = 'discord-owner'; + await s.engine.startCopy(s.pair.id); + assert.equal((await drain(s)).status, 'live'); +}); + +test('a pair with no recorded owners is not run when owners are enforced', async () => { + const s = await scenario({ withExpressions: false, engineOpts: OWNERS }); + await db.updatePair(s.pair.id, { sourceConsentUser: '' }); + await assert.rejects(s.engine.startCopy(s.pair.id), /changed hands/); +}); + +test('a delete on Discord locks the Fluxer copy by default; only "mirror deletions" deletes', async () => { + const s = await scenario(); + await s.engine.startCopy(s.pair.id); + await drain(s); + const chat = await mapped(s, 'channel', s.k.chat); + const mod = await mapped(s, 'role', s.k.mod); + s.discord.channels.cache.delete(s.k.chat); + s.discord.roles.cache.delete(s.k.mod); + s.discord.emojis.cache.delete(s.k.emoji1); + const note = (kind, id) => s.engine.noteSourceChange(s.k.src, { kind, op: 'delete', id }); + await note('channel', s.k.chat); + await note('role', s.k.mod); + await note('emoji', s.k.emoji1); + s.fluxer.calls.length = 0; + await s.engine.runPair(s.pair.id); + for (const op of ['deleteChannel', 'deleteRole', 'deleteEmoji']) + assert.equal(s.fluxer.ops(op).length, 0, op); + const lock = s.fluxer.ops('editChannel').find((c) => c.id === chat); + assert.deepEqual(lock.spec.permissionOverwrites, [ + { id: s.fluxer.guild.id, type: 0, allow: '0', deny: String(view | send | react | connect) }, + ]); + assert.equal(s.fluxer.ops('editRole').find((c) => c.id === mod).spec.permissions, '0'); + assert.ok(s.fluxer.guild.channels.has(chat), 'the channel is still on Fluxer, with its messages'); + assert.ok((await db.getMapping(s.pair.id, 'channel', s.k.chat)).removedAt > 0); + const progress = await s.engine.progress(await db.getPair(s.pair.id)); + assert.deepEqual(progress.removed.map((r) => [r.kind, r.label]).sort(), [ + ['channel', 'chat'], + ['emoji', 'party'], + ['role', 'Mod'], + ]); + + // With the option on, the next delete is real, and only of what Sylo made. + await db.updatePair(s.pair.id, { options: { ...s.pair.options, mirrorDeletes: true } }); + const rules = await mapped(s, 'channel', s.k.rules); + s.discord.channels.cache.delete(s.k.rules); + await note('channel', s.k.rules); + await s.engine.runPair(s.pair.id); + assert.deepEqual( + s.fluxer.ops('deleteChannel').map((c) => c.id), + [rules] + ); +}); + +test('the lock is not applied to something Sylo did not create', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + await drain(s); + await db.putMapping({ + pairId: s.pair.id, + kind: 'channel', + sourceId: '770000000000000001', + targetId: 'foreign', + owned: false, + }); + s.fluxer.calls.length = 0; + await s.engine.noteSourceChange(s.k.src, { kind: 'channel', op: 'delete', id: '770000000000000001' }); + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.calls.length, 0); + assert.equal(await db.getMapping(s.pair.id, 'channel', '770000000000000001'), null); +}); + +test('a burst of deletes pauses the mirror, and only the owner confirming applies them', async () => { + const s = await scenario({ options: { mirrorDeletes: true } }); + await s.engine.startCopy(s.pair.id); + await drain(s); + s.fluxer.calls.length = 0; + const targets = []; + const note = async (kind, id) => s.engine.noteSourceChange(s.k.src, { kind, op: 'delete', id }); + for (const id of [s.k.chat, s.k.rules, s.k.voice, s.k.cat, s.k.cat2]) { + targets.push(await mapped(s, 'channel', id)); + s.discord.channels.cache.delete(id); + await note('channel', id); + } + assert.equal((await db.getPair(s.pair.id)).status, 'live', 'five deletes are still fine'); + s.discord.emojis.cache.delete(s.k.emoji1); + await note('emoji', s.k.emoji1); + const pair = await db.getPair(s.pair.id); + assert.deepEqual([pair.status, pair.pauseKind, pair.pausedBy], ['paused', 'deletes', 'system']); + assert.match(pair.lastError, /6 things were deleted/); + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.ops('deleteChannel').length, 0, 'nothing is deleted while the mirror waits'); + assert.equal(await db.openSyncCount(s.pair.id), 6, 'the deletes wait in the queue'); + + // Real deletion is on, so the owner of the Discord server cannot confirm it: only the Fluxer owner can, and only after + // being shown what will be deleted. + await assert.rejects( + s.engine.resume(s.pair.id, { side: 'source', confirm: true }), + (e) => e.code === 'fluxer-owner-only' + ); + await assert.rejects(s.engine.resume(s.pair.id, { side: 'target' }), (e) => { + assert.equal(e.code, 'confirm-needed'); + assert.equal(e.deleteCount, 6); + assert.ok(e.deletes.every((d) => d.action === 'delete')); + assert.ok(e.deletes.some((d) => d.kind === 'channel' && d.name === 'chat')); + return true; + }); + assert.equal((await db.getPair(s.pair.id)).status, 'paused'); + const resumed = await s.engine.resume(s.pair.id, { side: 'target', confirm: true }); + assert.equal(resumed.status, 'live'); + await s.engine.runPair(s.pair.id); + assert.deepEqual( + s.fluxer + .ops('deleteChannel') + .map((c) => c.id) + .sort(), + targets.sort() + ); + assert.equal(s.fluxer.ops('deleteEmoji').length, 1); +}); + +const BOSS = { id: '810000000000000005', name: 'Admins', position: 5 }; +const roleBits = (s, id) => BigInt(s.fluxer.guild.roles.get(id)._permissions); +const ADMIN = 1n << 3n; +const MANAGE_ROLES = 1n << 28n; + +test('a fresh Fluxer server (only Sylo at position 1) is fine: Sylo outranks what it makes, by id', async () => { + const s = await scenario({ withExpressions: false }); + const preview = await s.engine.previewPair(s.pair); + assert.equal(preview.ok, true, 'position 1 is not a problem: it is the top role'); + assert.equal(preview.blockers.length, 0); + await s.engine.startCopy(s.pair.id); + assert.equal((await drain(s)).status, 'live'); + const sylo = s.fluxer.guild.roles.get(SYLO_ROLE); + for (const sourceId of [s.k.member, s.k.mod]) { + const role = s.fluxer.guild.roles.get(await mapped(s, 'role', sourceId)); + assert.ok(sylo.position > role.position, 'Sylo stays above every role it made'); + } +}); + +test("the copy is refused, with advice, while a role ranks above or level with Sylo's", async () => { + const above = await scenario({ withExpressions: false, fluxerOpts: { extraRoles: [BOSS] } }); + const preview = await above.engine.previewPair(above.pair); + assert.equal(preview.ok, false); + assert.equal(preview.blockers[0].code, 'role-hierarchy'); + assert.match(preview.blockers[0].message, /Admins/); + assert.match(preview.blockers[0].message, /Server Settings, then Roles/); + assert.match(preview.blockers[0].message, /Sylo cannot move its own role/); + await assert.rejects(above.engine.startCopy(above.pair.id), /Admins/); + assert.equal(above.fluxer.calls.length, 0); + + // Level with Sylo (same position) but older: it ranks above, so it blocks; a newer one does not. + const older = await scenario({ + withExpressions: false, + fluxerOpts: { extraRoles: [{ id: '800000000000000001', name: 'Older', position: 1 }] }, + }); + assert.equal((await older.engine.previewPair(older.pair)).blockers[0].code, 'role-hierarchy'); + const newer = await scenario({ + withExpressions: false, + fluxerOpts: { extraRoles: [{ id: '820000000000000001', name: 'Newer', position: 1 }] }, + }); + assert.equal((await newer.engine.previewPair(newer.pair)).ok, true); + + // Sylo's own role unknown: also refused, with the advice to check it. + const unknown = await scenario({ withExpressions: false }); + unknown.fluxer.target.rolesAboveSylo = () => null; + assert.match((await unknown.engine.previewPair(unknown.pair)).blockers[0].message, /could not be read/); + + // Roles the mirror made itself do not count, and with roles switched off there is nothing to order. + const off = await scenario({ + withExpressions: false, + fluxerOpts: { extraRoles: [BOSS] }, + options: { roles: false }, + }); + assert.equal((await off.engine.previewPair(off.pair)).ok, true); +}); + +test('role ranks on Fluxer follow Discord, even for a tie, and a Manage Roles role never outranks the Administrator role', async () => { + const s = await scenario({ withExpressions: false }); + s.discord.roles.cache.get(s.k.mod).rawPosition = 1; // a tie with Member + s.discord.roles.cache.get(s.k.member).permissions = { bitfield: MANAGE_ROLES }; + s.discord.roles.cache.get(s.k.mod).permissions = { bitfield: BIT.Administrator }; + await s.engine.startCopy(s.pair.id); + await drain(s); + const [sent] = s.fluxer.ops('setRolePositions'); + assert.equal(new Set(sent.updates.map((u) => u.position)).size, 2, 'no two roles share a position'); + const member = s.fluxer.guild.roles.get(await mapped(s, 'role', s.k.member)); + const mod = s.fluxer.guild.roles.get(await mapped(s, 'role', s.k.mod)); + assert.ok(mod.position > member.position, 'the Administrator role ranks above the Manage Roles role'); + assert.ok(s.fluxer.guild.roles.get(SYLO_ROLE).position > mod.position); +}); + +test('a role gets its powerful permissions only after its place in the order is set', async () => { + const s = await scenario({ withExpressions: false }); + s.discord.roles.cache.get(s.k.member).permissions = { bitfield: MANAGE_ROLES | BIT.ViewChannel }; + s.discord.roles.cache.get(s.k.mod).permissions = { bitfield: BIT.Administrator | BIT.ViewChannel }; + await s.engine.startCopy(s.pair.id); + const end = await drain(s); + assert.equal(end.status, 'live'); + const calls = s.fluxer.calls; + const member = await mapped(s, 'role', s.k.member); + const mod = await mapped(s, 'role', s.k.mod); + const made = (id) => calls.findIndex((c) => c.op === 'createRole' && c.id === id); + const ordered = calls.findIndex((c) => c.op === 'setRolePositions'); + const granted = (id) => calls.findIndex((c) => c.op === 'editRole' && c.id === id); + for (const id of [member, mod]) { + const created = calls[made(id)].spec; + assert.equal( + BigInt(created.permissions) & (ADMIN | MANAGE_ROLES), + 0n, + 'made without powerful permissions' + ); + assert.ok(made(id) < ordered && ordered < granted(id), 'created, then ordered, then granted'); + } + assert.equal(roleBits(s, member) & MANAGE_ROLES, MANAGE_ROLES); + assert.equal(roleBits(s, mod) & ADMIN, ADMIN); + assert.ok( + (await db.getMapping(s.pair.id, 'role', s.k.mod)).specHash, + 'and the role is fingerprinted when final' + ); +}); + +test('a single role is ordered too, so that it gets its permissions', async () => { + const s = await scenario({ withExpressions: false }); + for (const id of [s.k.member, s.k.bot]) s.discord.roles.cache.delete(id); + s.discord.roles.cache.get(s.k.mod).permissions = { bitfield: BIT.Administrator }; + await s.engine.startCopy(s.pair.id); + await drain(s); + assert.equal(s.fluxer.ops('setRolePositions').length, 1); + assert.equal(roleBits(s, await mapped(s, 'role', s.k.mod)) & ADMIN, ADMIN); +}); + +test('with a new powerful role and a role above Sylo, the mirror pauses and the role stays without its permissions', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + await drain(s); + s.fluxer.calls.length = 0; + s.fluxer.guild.roles.set(BOSS.id, { _permissions: '0', managed: false, ...BOSS }); // someone moved a role above Sylo + const id = '780000000000000001'; + s.discord.roles.cache.set(id, { + id, + name: 'New admin', + color: 0, + rawPosition: 5, + hoist: false, + mentionable: false, + managed: false, + permissions: { bitfield: BIT.Administrator }, + }); + await s.engine.noteSourceChange(s.k.src, { kind: 'role', op: 'sync', id, reorder: 'roles' }); + s.clock.now += 4000; + await s.engine.runPair(s.pair.id); + const pair = await db.getPair(s.pair.id); + assert.deepEqual([pair.status, pair.pauseKind], ['paused', 'hierarchy']); + assert.match(pair.lastError, /Admins/); + assert.equal(s.fluxer.ops('setRolePositions').length, 0, 'nothing was reordered'); + const made = s.fluxer.ops('createRole')[0]; + assert.equal(BigInt(made.spec.permissions) & ADMIN, 0n); + assert.equal(roleBits(s, made.id) & ADMIN, 0n, 'it stays without Administrator while the order is not set'); + + // The Fluxer owner moves Sylo's role to the top; the mirror resumes, orders, and only then grants. + s.fluxer.guild.roles.delete(BOSS.id); + await s.engine.resume(s.pair.id, { side: 'source' }); + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.ops('setRolePositions').length, 1); + assert.equal(roleBits(s, made.id) & ADMIN, ADMIN); +}); + +test('asking for real deletion needs the Fluxer owner; resuming with deletes waiting shows what they are', async () => { + const s = await scenario({ options: { mirrorDeletes: true } }); + await s.engine.startCopy(s.pair.id); + await drain(s); + await s.engine.pause(s.pair.id); // paused by hand, not by the brake + const note = (kind, id) => s.engine.noteSourceChange(s.k.src, { kind, op: 'delete', id }); + for (const id of [s.k.chat, s.k.rules, s.k.voice, s.k.cat, s.k.cat2]) { + s.discord.channels.cache.delete(id); + await note('channel', id); + } + s.discord.emojis.cache.delete(s.k.emoji1); + await note('emoji', s.k.emoji1); + assert.equal( + (await db.getPair(s.pair.id)).pauseKind, + 'owner', + 'deletes while paused do not change the pause' + ); + await assert.rejects(s.engine.resume(s.pair.id, { side: 'target' }), (e) => { + assert.equal(e.code, 'confirm-needed'); + assert.equal(e.deleteCount, 6); + return true; + }); + const progress = await s.engine.progress(await db.getPair(s.pair.id)); + assert.equal(progress.pendingDeletes.length, 6, 'the page can show what will be deleted'); + assert.equal((await s.engine.resume(s.pair.id, { side: 'target', confirm: true })).status, 'live'); +}); + +test('with roles off, a channel @everyone cannot see on Discord is still private on Fluxer', async () => { + const s = await scenario({ + withExpressions: false, + options: { roles: false }, + discordOpts: { everyonePermissions: 0n }, + }); + await s.engine.startCopy(s.pair.id); + await drain(s); + const made = s.fluxer.ops('createChannel'); + assert.equal(made.length, 5); + for (const c of made) { + const own = c.spec.permissionOverwrites.find((o) => o.id === s.fluxer.guild.id); + assert.ok(own && (BigInt(own.deny) & view) === view, `${c.spec.name} denies View Channel to @everyone`); + } +}); + +test('when the @everyone step fails, private channels are still private', async () => { + const s = await scenario({ withExpressions: false, discordOpts: { everyonePermissions: 0n } }); + await s.engine.startCopy(s.pair.id); + s.fluxer.failOn('editRole', Object.assign(new Error('Invalid Form Body'), { statusCode: 400 }), 1); + assert.equal((await drain(s)).status, 'live'); + const everyone = (await db.listFailedJobs(s.pair.id)).find((j) => j.kind === 'role'); + assert.ok(everyone, 'the @everyone step failed'); + for (const c of s.fluxer.ops('createChannel')) { + const own = c.spec.permissionOverwrites.find((o) => o.id === s.fluxer.guild.id); + assert.ok(own && (BigInt(own.deny) & view) === view, `${c.spec.name} stays hidden`); + } +}); + +test('copied settings do not hand out a powerful role or start pinging everyone', async () => { + const s = await scenario({ withExpressions: false }); + s.discord.roles.cache.get(s.k.mod).permissions = { bitfield: BIT.Administrator }; + s.discord.roles.cache.get(s.k.member).permissions = { bitfield: BIT.ViewChannel }; + await setGuildModule(s.k.src, 'roles', { enabled: true, config: { autoroles: [s.k.mod, s.k.member] } }); + await setGuildModule(s.k.src, 'giveaways', { + enabled: true, + config: { ping: 'everyone', channel: s.k.chat }, + }); + await setGuildModule(s.k.src, 'apple-connect', { enabled: true, config: { channelId: s.k.chat } }); + const preview = await s.engine.previewPair(s.pair); + assert.equal( + preview.steps.some((x) => x.key === 'module:apple-connect'), + false, + "the operator's own modules are never copied" + ); + const codes = preview.notes.map((n) => n.code); + assert.ok(codes.includes('module-grant-dropped') && codes.includes('module-ping-lowered')); + await s.engine.startCopy(s.pair.id); + const end = await drain(s); + const roles = await getGuildModule(`fluxer:${s.k.dst}`, 'roles'); + assert.deepEqual( + roles.config.autoroles, + [await mapped(s, 'role', s.k.member)], + 'the Administrator role is not an autorole' + ); + const giveaways = await getGuildModule(`fluxer:${s.k.dst}`, 'giveaways'); + assert.equal(giveaways.config.ping, 'none'); + assert.equal(giveaways.config.channel, await mapped(s, 'channel', s.k.chat)); + const report = await s.engine.report(end); + assert.deepEqual( + report.notes + .filter((n) => n.code.startsWith('module-')) + .map((n) => n.code) + .sort(), + ['module-grant-dropped', 'module-ids-dropped', 'module-ping-lowered'] + ); + assert.deepEqual((await getGuildModule(`fluxer:${s.k.dst}`, 'apple-connect')).config, {}); +}); + +test('a failed edit or look-up never makes a second copy', async () => { + const s = await scenario(); + await s.engine.startCopy(s.pair.id); + await drain(s); + s.fluxer.calls.length = 0; + const chat = await mapped(s, 'channel', s.k.chat); + s.discord.channels.cache.get(s.k.chat).name = 'chatter'; + s.fluxer.failOn('editChannel', Object.assign(new Error('unavailable'), { statusCode: 503 }), 1); + await s.engine.noteSourceChange(s.k.src, { kind: 'channel', op: 'sync', id: s.k.chat }); + await s.engine.runPair(s.pair.id); + assert.equal(await mapped(s, 'channel', s.k.chat), chat, 'the copy is still known'); + assert.equal(s.fluxer.ops('createChannel').length, 0, 'and no second one was made'); + s.clock.now += 5000; + await s.engine.runPair(s.pair.id); + assert.equal(s.fluxer.ops('createChannel').length, 0); + assert.equal(s.fluxer.guild.channels.get(chat).name, 'chatter', 'the retry updated the same channel'); +}); + +test('a role whose second call fails is tracked, and the retry updates it instead of making another', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + s.fluxer.failAfterCreate('createRole', Object.assign(new Error('timed out'), { statusCode: 503 })); + assert.equal((await drain(s)).status, 'live'); + assert.equal(s.fluxer.ops('createRole').length, 2, 'Member and Mod: one create each'); + assert.equal([...s.fluxer.guild.roles.values()].length, 4, 'and so four roles with @everyone and Sylo'); + const member = await db.getMapping(s.pair.id, 'role', s.k.member); + assert.equal(member.owned, true); + assert.ok(member.specHash, 'the retry updated it, so it is fingerprinted now'); + assert.ok(s.fluxer.ops('editRole').some((c) => c.id === member.targetId)); +}); + +test('a job that may have gone through before a timeout looks for what it made before it makes anything', async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + // The create reaches Fluxer but the answer is lost. + const real = s.fluxer.target.createChannel; + let lost = false; + s.fluxer.target.createChannel = async (spec) => { + const id = await real(spec); + if (!lost && spec.name === 'chat') { + lost = true; + throw Object.assign(new Error('fetch failed'), { code: 'ECONNRESET' }); + } + return id; + }; + assert.equal((await drain(s)).status, 'live'); + assert.equal( + [...s.fluxer.guild.channels.values()].filter((c) => c.name === 'chat').length, + 1, + 'one chat channel' + ); + assert.equal( + (await db.getMapping(s.pair.id, 'channel', s.k.chat)).owned, + false, + 'adopted, never deleted by Sylo' + ); +}); + +test('after a crash a job adopts only what was made since it started, never something older', async () => { + const foreign = { id: '760000000000000010', name: 'chat', type: 0, parentId: null }; + const s = await scenario({ withExpressions: false, fluxerOpts: { existingChannels: [foreign] } }); + await s.engine.startCopy(s.pair.id); + for (;;) { + const job = await db.nextDueJob(s.pair.id, s.clock.now); + if (job.kind === 'channel' && job.sourceId === s.k.chat) break; + await s.engine.runPair(s.pair.id, { maxJobs: 1 }); + } + const job = await db.nextDueJob(s.pair.id, s.clock.now); + assert.equal(await db.claimJob(job.id, s.clock.now), true); + const { engine } = engineFor({ discord: s.discord, fluxer: s.fluxer, clock: s.clock }); + await engine.resumeAfterRestart(); + s.engine = engine; + assert.equal((await drain(s)).status, 'live'); + assert.ok( + s.fluxer.ops('createChannel').some((c) => c.spec.name === 'chat'), + 'a new chat channel was made' + ); + assert.equal( + s.fluxer.ops('editChannel').some((c) => c.id === foreign.id), + false, + 'the older one was left alone' + ); +}); + +test("only one side's owner can resume what that side paused", async () => { + const s = await scenario({ withExpressions: false }); + await s.engine.startCopy(s.pair.id); + await drain(s); + await s.engine.pause(s.pair.id, 'source'); + await assert.rejects(s.engine.resume(s.pair.id, { side: 'target' }), /only that owner can resume/); + assert.equal((await s.engine.resume(s.pair.id, { side: 'source' })).status, 'live'); + await s.engine.pause(s.pair.id, 'target'); + assert.equal((await s.engine.resume(s.pair.id, { side: 'target' })).status, 'live'); +}); + +test('a server marked age-restricted gives every copied channel the age flag', async () => { + const s = await scenario({ withExpressions: false }); + s.discord.nsfwLevel = 3; + await s.engine.startCopy(s.pair.id); + await drain(s); + const text = s.fluxer.ops('createChannel').filter((c) => c.spec.type === 0); + assert.ok(text.length > 0 && text.every((c) => c.spec.nsfw === true)); +}); + +test('only a few mirrors are worked at once', async () => { + const a = await scenario({ withExpressions: false }); + const b = await scenario({ withExpressions: false }); + const engine = createEngine({ + now: a.clock.fn, + sleep: a.clock.sleep, + minIntervalMs: 1, + maxWorkers: 1, + clientsReady: () => true, + enforceOwnership: () => false, + sourceGuild: (key) => [a, b].map((x) => x.discord).find((d) => d.id === key), + targetGuild: (key) => [a, b].map((x) => x.fluxer.guild).find((g) => `fluxer:${g.id}` === key), + makeTarget: (guild) => [a, b].map((x) => x.fluxer).find((f) => f.guild === guild).target, + fetchImage: async () => ({ bytes: Buffer.alloc(8), mime: 'image/png' }), + }); + // Pairs of the other tests in this file must not take the worker slots. + for (const p of await db.listAllPairs()) { + if (![a.pair.id, b.pair.id].includes(p.id)) await db.updatePair(p.id, { status: 'stopped' }); + } + await engine.startCopy(a.pair.id); + await engine.startCopy(b.pair.id); + await engine.tick(); + assert.equal([a, b].filter((x) => engine.isRunning(x.pair.id)).length, 1, 'one worker at a time'); + for (let i = 0; i < 400; i += 1) { + await engine.tick(); + await new Promise((r) => setImmediate(r)); + const states = await Promise.all([a, b].map((x) => db.getPair(x.pair.id))); + if (states.every((p) => p.status === 'live')) break; + } + assert.equal((await db.getPair(a.pair.id)).status, 'live'); + assert.equal((await db.getPair(b.pair.id)).status, 'live'); +}); + +test('old copy runs are pruned when a new one starts', async () => { + const s = await scenario({ withExpressions: false }); + for (let i = 0; i < 3; i += 1) { + await s.engine.startCopy(s.pair.id); + await drain(s); + } + const pair = await db.getPair(s.pair.id); + assert.equal(pair.copyRun, 3); + assert.equal((await db.listRunJobs(pair.id, 1)).length, 0, 'run 1 is gone'); + assert.ok((await db.listRunJobs(pair.id, 2)).length > 0, 'the run before the latest is kept'); +}); + +test('a Discord emoji or sticker delete that arrives while its copy is still being made is not lost', async () => { + const s = await scenario(); + await s.engine.startCopy(s.pair.id); + s.discord.emojis.cache.delete(s.k.emoji1); + // The copy of the emoji has not been made yet, so there is no copy to find when the delete is noted. + assert.equal( + await s.engine.noteSourceChange(s.k.src, { kind: 'emoji', op: 'delete', id: s.k.emoji1 }), + true + ); + await drain(s); + assert.equal(s.fluxer.ops('createEmoji').length, 1, 'the deleted emoji was never made'); +}); + +test('a copied economy comes over switched off, with a note', async () => { + const s = await scenario({ withExpressions: false }); + await setGuildModule(s.k.src, 'economy', { enabled: true, config: { startingBalance: 100 } }); + const preview = await s.engine.previewPair(s.pair); + assert.ok(preview.notes.some((n) => n.code === 'module-economy-off')); + await s.engine.startCopy(s.pair.id); + await drain(s); + const there = await getGuildModule(`fluxer:${s.k.dst}`, 'economy'); + assert.equal(there.enabled, false); + assert.equal(there.config.startingBalance, 100, 'the settings are there for its owner to look at'); +}); + +test('a mirrored Administrator role deleted on Fluxer is made again without its permissions, ordered, then granted', async () => { + const s = await scenario({ withExpressions: false }); + s.discord.roles.cache.get(s.k.mod).permissions = { bitfield: BIT.Administrator | BIT.ViewChannel }; + await s.engine.startCopy(s.pair.id); + await drain(s); + const first = await mapped(s, 'role', s.k.mod); + assert.equal(roleBits(s, first) & ADMIN, ADMIN, 'the first copy granted it'); + + // Somebody deletes the role on Fluxer; it is edited on Discord, so the mirror syncs it. + s.fluxer.guild.roles.delete(first); + s.discord.roles.cache.get(s.k.mod).name = 'Moderators'; + s.fluxer.calls.length = 0; + await s.engine.noteSourceChange(s.k.src, { kind: 'role', op: 'sync', id: s.k.mod }); + await s.engine.runPair(s.pair.id); + + const again = await mapped(s, 'role', s.k.mod); + assert.notEqual(again, first, 'a new role was made'); + const calls = s.fluxer.calls; + const made = calls.findIndex((c) => c.op === 'createRole'); + const ordered = calls.findIndex((c) => c.op === 'setRolePositions'); + const granted = calls.findIndex((c) => c.op === 'editRole' && c.id === again); + assert.ok(made >= 0 && made < ordered && ordered < granted, 'made, then ordered, then granted'); + assert.equal(BigInt(calls[made].spec.permissions) & ADMIN, 0n, 'made without Administrator'); + assert.equal(roleBits(s, again) & ADMIN, ADMIN, 'and granted at the end'); + assert.ok((await db.getMapping(s.pair.id, 'role', s.k.mod)).specHash, 'final'); + assert.ok(s.fluxer.guild.roles.get(SYLO_ROLE).position > s.fluxer.guild.roles.get(again).position); +}); + +test('a role adopted after a crash waits for the order too, and gets its permissions after it', async () => { + const s = await scenario({ withExpressions: false }); + s.discord.roles.cache.get(s.k.mod).permissions = { bitfield: BIT.Administrator }; + await s.engine.startCopy(s.pair.id); + for (;;) { + const job = await db.nextDueJob(s.pair.id, s.clock.now); + if (job.kind === 'role' && job.sourceId === s.k.mod) break; + await s.engine.runPair(s.pair.id, { maxJobs: 1 }); + } + const job = await db.nextDueJob(s.pair.id, s.clock.now); + assert.equal(await db.claimJob(job.id, s.clock.now), true); + // The dead run made the role with everything it asked for, but never wrote it down. + const madeId = await s.fluxer.target.createRole({ + name: 'Mod', + color: 0, + hoist: false, + mentionable: true, + permissions: String(ADMIN), + }); + const { engine } = engineFor({ discord: s.discord, fluxer: s.fluxer, clock: s.clock }); + await engine.resumeAfterRestart(); + s.engine = engine; + s.fluxer.calls.length = 0; + assert.equal((await drain(s)).status, 'live'); + const adopted = await db.getMapping(s.pair.id, 'role', s.k.mod); + assert.deepEqual([adopted.targetId, adopted.owned], [madeId, false]); + const edits = s.fluxer.ops('editRole').filter((c) => c.id === madeId); + const ordered = s.fluxer.calls.findIndex((c) => c.op === 'setRolePositions'); + assert.equal(BigInt(edits[0].spec.permissions) & ADMIN, 0n, 'first stripped'); + assert.equal(roleBits(s, madeId) & ADMIN, ADMIN, 'and granted after the order'); + assert.ok(s.fluxer.calls.indexOf(edits.at(-1)) > ordered); + assert.ok(adopted.specHash || (await db.getMapping(s.pair.id, 'role', s.k.mod)).specHash); +}); diff --git a/test/mirrorPermissions.test.js b/test/mirrorPermissions.test.js new file mode 100644 index 00000000..e5b08f62 --- /dev/null +++ b/test/mirrorPermissions.test.js @@ -0,0 +1,117 @@ +// The Discord -> Fluxer permission table (src/mirror/permissions.js), checked bit by bit against the two +// libraries' own flag tables, so a renamed or renumbered permission cannot map silently wrong. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { PermissionFlagsBits } from 'discord.js'; +import { PermissionFlags } from '@fluxerjs/core'; +import { + ELEVATED_FLUXER_MASK, + FLUXER_ONLY_BITS, + PERMISSION_TABLE, + describeDropped, + fluxerBitOf, + isElevated, + keepFluxerOnly, + lostSomething, + mapPermissions, +} from '../src/mirror/permissions.js'; + +test('every row agrees with discord.js: the name has that bit', () => { + for (const row of PERMISSION_TABLE) { + assert.equal(PermissionFlagsBits[row.discord], 1n << BigInt(row.bit), `${row.discord} is bit ${row.bit}`); + } +}); + +test('every Discord permission discord.js knows is in the table (a new one must be decided)', () => { + // By bit: discord.js keeps a few old names (ManageEmojisAndStickers) for a bit that has a newer one. + const inTable = new Set(PERMISSION_TABLE.map((r) => r.bit)); + const missing = Object.entries(PermissionFlagsBits) + .filter(([, value]) => !inTable.has(Math.log2(Number(value)))) + .map(([name]) => name); + assert.deepEqual(missing, [], `new Discord permissions to map or drop: ${missing.join(', ')}`); +}); + +// Fluxer's thread permissions (packages/constants ThreadPermissionUtils): the SDK's flag table does not list them yet. +const THREAD_BITS = { + ManageThreads: 34n, + CreatePublicThreads: 35n, + CreatePrivateThreads: 36n, + SendMessagesInThreads: 38n, +}; + +test('every mapped row lands on the Fluxer permission of that name and bit', () => { + for (const row of PERMISSION_TABLE) { + if (row.fluxer === null) continue; + const sdk = + PermissionFlags[row.fluxer] ?? (row.fluxer in THREAD_BITS ? 1n << THREAD_BITS[row.fluxer] : undefined); + assert.ok(sdk !== undefined, `Fluxer has ${row.fluxer}`); + assert.equal(sdk, 1n << BigInt(fluxerBitOf(row.fluxer)), `${row.fluxer}'s bit`); + const { bits, dropped } = mapPermissions(1n << BigInt(row.bit)); + assert.equal(bits, sdk, `${row.discord} becomes ${row.fluxer}`); + assert.deepEqual(dropped, []); + } +}); + +test('the thread permissions are mapped, so a Discord deny of thread creation stays a deny', () => { + for (const [name, bit] of Object.entries(THREAD_BITS)) { + const row = PERMISSION_TABLE.find((r) => r.fluxer === name); + assert.ok(row, name); + assert.equal(BigInt(row.bit), bit); + assert.equal(mapPermissions(1n << bit).bits, 1n << bit); + } +}); + +test('powerful permissions are recognised, on Discord bits and on their Fluxer counterparts', () => { + assert.ok(isElevated(1n << 3n), 'Administrator'); + assert.ok(isElevated(1n << 28n), 'Manage Roles'); + assert.ok(isElevated(1n << 34n), 'Manage Threads'); + assert.ok(!isElevated((1n << 10n) | (1n << 11n)), 'View Channel and Send Messages are not'); + assert.ok(ELEVATED_FLUXER_MASK & PermissionFlags.Administrator); + assert.equal(ELEVATED_FLUXER_MASK & PermissionFlags.SendMessages, 0n); + assert.equal(ELEVATED_FLUXER_MASK & FLUXER_ONLY_BITS, 0n); +}); + +test('a Discord permission with no Fluxer counterpart is dropped and named', () => { + for (const row of PERMISSION_TABLE.filter((r) => r.fluxer === null)) { + const m = mapPermissions(1n << BigInt(row.bit)); + assert.equal(m.bits, 0n, `${row.discord} maps to nothing`); + assert.deepEqual(m.dropped, [row.label]); + } +}); + +test('the Fluxer permissions the table never produces are exactly the Fluxer-only ones', () => { + const produced = new Set(PERMISSION_TABLE.map((r) => r.fluxer).filter(Boolean)); + const unproduced = Object.keys(PermissionFlags).filter((n) => !produced.has(n)); + const only = unproduced.reduce((acc, n) => acc | PermissionFlags[n], 0n); + assert.equal(only, FLUXER_ONLY_BITS); +}); + +test('a mixed bitfield maps bit by bit, and unknown high bits are reported', () => { + const view = 1n << 10n; + const send = 1n << 11n; + const events = 1n << 33n; // Discord only + const future = 1n << 60n; // not in the table + const m = mapPermissions(view | send | events | future); + assert.equal(m.bits, PermissionFlags.ViewChannel | PermissionFlags.SendMessages); + assert.deepEqual(m.dropped, ['Manage Events']); + assert.deepEqual(m.unknown, [60]); + assert.ok(lostSomething(m)); + assert.equal(describeDropped(m), 'Manage Events, unknown permission (bit 60)'); + assert.equal(mapPermissions(0n).bits, 0n); + assert.equal(lostSomething(mapPermissions(view)), false); + assert.equal(mapPermissions('2048').bits, PermissionFlags.SendMessages, 'a string bitfield works too'); +}); + +test('Administrator maps to Administrator, and Fluxer-only bits are carried over, not invented', () => { + assert.equal(mapPermissions(1n << 3n).bits, PermissionFlags.Administrator); + assert.equal( + mapPermissions((1n << 53n) - 1n).bits & FLUXER_ONLY_BITS, + 0n, + 'no Discord bit turns into a Fluxer-only one' + ); + assert.equal( + keepFluxerOnly(PermissionFlags.UpdateRtcRegion | PermissionFlags.SendMessages), + PermissionFlags.UpdateRtcRegion + ); + assert.equal(keepFluxerOnly(null), 0n); +}); diff --git a/test/mirrorSpec.test.js b/test/mirrorSpec.test.js new file mode 100644 index 00000000..73ec04db --- /dev/null +++ b/test/mirrorSpec.test.js @@ -0,0 +1,345 @@ +// What the mirror sends for each thing (src/mirror/spec.js): channels with their permission overwrites, roles, the +// settings remap, and the placeholders that keep a spec free of Fluxer ids until the moment it is sent. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { PermissionFlags } from '@fluxerjs/core'; +import { snapshotDiscordGuild } from '../src/mirror/snapshot.js'; +import { + buildChannelSpec, + buildRoleSpec, + buildStickerSpec, + emojiName, + normaliseOptions, + placeholderConfig, + refsIn, + resolveSpec, + specHash, +} from '../src/mirror/spec.js'; +import { BIT, SRC, dChannel, dRole, fakeDiscord } from './helpers/mirrorFakes.js'; + +const MEMBER_ROLE = '710000000000000001'; +const BOT_ROLE = '710000000000000002'; +const CAT = '710000000000000010'; +const TEXT = '710000000000000011'; +const VOICE = '710000000000000012'; + +function server() { + return fakeDiscord({ + roles: [ + dRole({ + id: MEMBER_ROLE, + name: 'Member', + rawPosition: 1, + permissions: BIT.ViewChannel | BIT.SendMessages | BIT.ManageEvents, + }), + dRole({ id: BOT_ROLE, name: 'Some bot', rawPosition: 2, managed: true }), + ], + channels: [ + dChannel({ id: CAT, name: 'Info', type: 4, rawPosition: 0 }), + dChannel({ + id: TEXT, + name: 'rules', + type: 0, + parentId: CAT, + rawPosition: 1, + topic: 'Read these', + nsfw: true, + rateLimitPerUser: 30, + overwrites: [ + { id: SRC, type: 0, deny: BIT.SendMessages | BIT.ManageEvents }, + { id: MEMBER_ROLE, type: 0, allow: BIT.ViewChannel }, + { id: BOT_ROLE, type: 0, allow: BIT.ViewChannel }, + { id: '720000000000000001', type: 1, allow: BIT.ViewChannel }, + ], + }), + dChannel({ id: VOICE, name: 'Lounge', type: 2, bitrate: 1_000_000, userLimit: 5 }), + ], + }); +} + +const snap = () => snapshotDiscordGuild(server()); +const ctxOf = (source) => ({ source, copiedRoles: new Set([MEMBER_ROLE]) }); + +test('a text channel keeps every property and its overwrites, with ids as placeholders', () => { + const source = snap(); + const { spec, notes } = buildChannelSpec( + source.channels.find((c) => c.id === TEXT), + ctxOf(source) + ); + const { permissionOverwrites, ...rest } = spec; + assert.deepEqual(rest, { + name: 'rules', + type: 0, + parentId: `$channel:${CAT}`, + topic: 'Read these', + nsfw: true, + rateLimitPerUser: 30, + }); + assert.deepEqual(permissionOverwrites, [ + { id: `$role:${MEMBER_ROLE}`, type: 0, allow: String(PermissionFlags.ViewChannel), deny: '0' }, + { id: '$role:everyone', type: 0, allow: '0', deny: String(PermissionFlags.SendMessages) }, + ]); + const codes = notes.map((n) => n.code).sort(); + assert.deepEqual(codes, ['member-overwrite-dropped', 'overwrite-role-not-copied', 'permissions-dropped']); + assert.match(notes.find((n) => n.code === 'permissions-dropped').message, /Manage Events/); +}); + +test('voice and category channels carry what they have; a bitrate stays inside Fluxer limits', () => { + const source = snap(); + const voice = buildChannelSpec( + source.channels.find((c) => c.id === VOICE), + ctxOf(source) + ).spec; + assert.deepEqual([voice.type, voice.bitrate, voice.userLimit, voice.parentId], [2, 96000, 5, null]); + const cat = buildChannelSpec( + source.channels.find((c) => c.id === CAT), + ctxOf(source) + ).spec; + assert.deepEqual(cat, { name: 'Info', type: 4, parentId: null, permissionOverwrites: [] }); +}); + +test('a stage channel becomes voice with a note; a channel type Fluxer lacks is not copied', () => { + const source = snapshotDiscordGuild( + fakeDiscord({ + channels: [ + dChannel({ id: '710000000000000020', name: 'Stage', type: 13 }), + dChannel({ id: '710000000000000021', name: 'Dir', type: 14 }), + ], + }) + ); + const stage = buildChannelSpec(source.channels[0], { source, copiedRoles: new Set() }); + assert.equal(stage.spec.type, 2); + assert.deepEqual( + stage.notes.map((n) => n.code), + ['stage-as-voice'] + ); + const dir = buildChannelSpec(source.channels[1], { source, copiedRoles: new Set() }); + assert.equal(dir.spec, null); + assert.deepEqual( + dir.notes.map((n) => n.code), + ['channel-type-unsupported'] + ); +}); + +test('a channel @everyone cannot see on Discord is hidden from @everyone on Fluxer, whatever else is copied', () => { + const hidden = fakeDiscord({ + everyonePermissions: 0n, + roles: [dRole({ id: MEMBER_ROLE, name: 'Member', rawPosition: 1, permissions: BIT.ViewChannel })], + channels: [ + dChannel({ + id: TEXT, + name: 'staff', + overwrites: [{ id: MEMBER_ROLE, type: 0, allow: BIT.ViewChannel }], + }), + dChannel({ id: VOICE, name: 'staff-voice', type: 2 }), + dChannel({ + id: CAT, + name: 'open', + type: 0, + overwrites: [{ id: SRC, type: 0, allow: BIT.ViewChannel }], + }), + ], + }); + const source = snapshotDiscordGuild(hidden); + const byId = (id) => source.channels.find((c) => c.id === id); + assert.deepEqual([byId(TEXT).everyoneCanView, byId(CAT).everyoneCanView], [false, true]); + for (const copied of [new Set([MEMBER_ROLE]), new Set()]) { + // with roles copied, and with roles off: the private channel is private either way + const staff = buildChannelSpec(byId(TEXT), { source, copiedRoles: copied }).spec; + const own = staff.permissionOverwrites.find((o) => o.id === '$role:everyone'); + assert.equal(BigInt(own.deny) & (1n << 10n), 1n << 10n, 'View Channel is denied to @everyone'); + const voice = buildChannelSpec(byId(VOICE), { source, copiedRoles: copied }).spec; + const vown = voice.permissionOverwrites.find((o) => o.id === '$role:everyone'); + assert.equal(BigInt(vown.deny), (1n << 10n) | (1n << 20n), 'and Connect, for a voice channel'); + const open = buildChannelSpec(byId(CAT), { source, copiedRoles: copied }).spec; + const openOwn = open.permissionOverwrites.find((o) => o.id === '$role:everyone'); + assert.equal(BigInt(openOwn.deny) & (1n << 10n), 0n, 'an open channel stays open'); + } +}); + +test('an explicit @everyone overwrite keeps its other bits when the channel is hidden', () => { + const source = snapshotDiscordGuild( + fakeDiscord({ + everyonePermissions: 0n, + channels: [ + dChannel({ id: TEXT, name: 'x', overwrites: [{ id: SRC, type: 0, allow: BIT.SendMessages }] }), + ], + }) + ); + const { spec } = buildChannelSpec(source.channels[0], { source, copiedRoles: new Set() }); + const own = spec.permissionOverwrites.find((o) => o.id === '$role:everyone'); + assert.equal(BigInt(own.deny), 1n << 10n); + assert.equal(BigInt(own.allow), BIT.SendMessages); +}); + +test('Administrator on @everyone sees everything; a server marked age-restricted gives its channels the flag', () => { + const open = snapshotDiscordGuild( + fakeDiscord({ everyonePermissions: BIT.Administrator, channels: [dChannel({ id: TEXT, name: 'a' })] }) + ); + assert.equal(open.channels[0].everyoneCanView, true); + const g = fakeDiscord({ + channels: [dChannel({ id: TEXT, name: 'a' }), dChannel({ id: VOICE, name: 'v', type: 2 })], + }); + g.nsfwLevel = 3; + const source = snapshotDiscordGuild(g); + assert.equal(source.ageRestricted, true); + assert.equal(buildChannelSpec(source.channels[0], { source, copiedRoles: new Set() }).spec.nsfw, true); +}); + +test('@everyone never receives powerful permissions on Fluxer', () => { + const source = snapshotDiscordGuild( + fakeDiscord({ everyonePermissions: BIT.Administrator | BIT.ViewChannel }) + ); + const everyone = source.roles.find((r) => r.isEveryone); + const { spec, notes } = buildRoleSpec(everyone); + assert.equal(BigInt(spec.permissions), PermissionFlags.ViewChannel); + assert.ok(notes.some((n) => n.code === 'everyone-elevated-stripped')); +}); + +test('a thread permission deny is carried to Fluxer', () => { + const source = snapshotDiscordGuild( + fakeDiscord({ + channels: [ + dChannel({ id: TEXT, name: 'a', overwrites: [{ id: SRC, type: 0, deny: BIT.CreatePublicThreads }] }), + ], + }) + ); + const { spec, notes } = buildChannelSpec(source.channels[0], { source, copiedRoles: new Set() }); + assert.equal(BigInt(spec.permissionOverwrites[0].deny), 1n << 35n); + assert.equal( + notes.some((n) => n.code === 'permissions-dropped'), + false + ); +}); + +test('a sticker without a description sends none, as Fluxer wants 1 to 500 characters or nothing', () => { + const { spec } = buildStickerSpec({ id: '1', name: 'plain', format: 1, tags: [], description: '' }); + assert.equal(spec.description, null); +}); + +test('threads are not read from the server', () => { + const g = server(); + const id = '710000000000000030'; + g.channels.cache.set(id, dChannel({ id, name: 'a thread', type: 11, isThread: () => true })); + assert.equal(snapshotDiscordGuild(g).channels.length, 3); +}); + +test('a role maps its permissions and reports what Fluxer lacks; Administrator is flagged', () => { + const source = snap(); + const { spec, notes } = buildRoleSpec(source.roles.find((r) => r.id === MEMBER_ROLE)); + assert.equal(spec.permissions, String(PermissionFlags.ViewChannel | PermissionFlags.SendMessages)); + assert.deepEqual( + notes.map((n) => n.code), + ['permissions-dropped'] + ); + const admin = buildRoleSpec({ ...source.roles[1], id: '1', name: 'Admin', permissions: BIT.Administrator }); + assert.deepEqual( + admin.notes.map((n) => n.code), + ['administrator-role'] + ); +}); + +test('@everyone only sets permissions, and keeps the Fluxer-only bits it already has', () => { + const source = snap(); + const everyone = source.roles.find((r) => r.isEveryone); + const { spec } = buildRoleSpec( + { ...everyone, permissions: BIT.ViewChannel }, + { keep: PermissionFlags.UpdateRtcRegion } + ); + assert.deepEqual(Object.keys(spec), ['permissions']); + assert.equal(spec.permissions, String(PermissionFlags.ViewChannel | PermissionFlags.UpdateRtcRegion)); +}); + +test('options: only known switches, only booleans, the rest on', () => { + assert.deepEqual(normaliseOptions({ roles: false, emoji: 'no', bogus: true }), { + name: true, + icon: true, + banner: true, + roles: false, + channels: true, + emoji: true, + stickers: true, + modules: true, + continuous: true, + mirrorDeletes: false, + mirrorDeletesRequested: false, + }); + assert.equal(normaliseOptions(null).continuous, true); +}); + +test('emoji names are cleaned to what Fluxer takes; a Lottie sticker is not copied', () => { + assert.equal(emojiName('party-time!'), 'party_time_'); + assert.equal(emojiName('a'), 'a_'); + assert.equal(emojiName('ok'), 'ok'); + assert.equal(buildStickerSpec({ id: '1', name: 'x', format: 3, tags: [], description: '' }).spec, null); + assert.deepEqual( + buildStickerSpec({ id: '1', name: 'wave', format: 1, tags: ['hi'], description: 'hello' }).spec, + { + name: 'wave', + description: 'hello', + tags: ['hi'], + } + ); +}); + +test('the fingerprint ignores key order and changes with content', () => { + assert.equal(specHash({ a: 1, b: [1, { c: 2, d: 3 }] }), specHash({ b: [1, { d: 3, c: 2 }], a: 1 })); + assert.notEqual(specHash({ a: 1 }), specHash({ a: 2 })); +}); + +test('settings remap: copied ids become placeholders, dropped ones go, message ids are cleared', () => { + const ids = { + channels: new Set(['910000000000000001']), + roles: new Set(['910000000000000002']), + skippedChannels: new Set(['910000000000000003']), + skippedRoles: new Set(['910000000000000004']), + }; + const config = { + channel: '910000000000000001', + lost: '910000000000000003', + roles: ['910000000000000002', '910000000000000004'], + user: '123456789012345678', + panelMessageId: '999999999999999999', + text: 'see <#910000000000000001> and <@&910000000000000002> or <#910000000000000003>', + nested: { deep: [{ channelId: '910000000000000001' }] }, + count: 5, + }; + const out = placeholderConfig(config, ids); + assert.equal(out.config.channel, '$channel:910000000000000001'); + assert.equal(out.config.lost, ''); + assert.deepEqual(out.config.roles, ['$role:910000000000000002']); + assert.equal(out.config.user, '', 'an id of a person means nothing on Fluxer'); + assert.equal(out.config.panelMessageId, ''); + assert.equal(out.config.text, 'see <#$channel:910000000000000001> and <@&$role:910000000000000002> or '); + assert.equal(out.config.nested.deep[0].channelId, '$channel:910000000000000001'); + assert.equal(out.config.count, 5); + assert.deepEqual([out.remapped, out.dropped, out.cleared], [5, 3, 2]); +}); + +test('placeholders resolve to Fluxer ids; what cannot be resolved is dropped, and reported', () => { + const spec = { + parentId: '$channel:910000000000000001', + permissionOverwrites: [ + { id: '$role:910000000000000002', type: 0 }, + { id: '$role:910000000000000009', type: 0 }, + { id: '$role:everyone', type: 0 }, + ], + text: 'x <#$channel:910000000000000001> y', + roles: ['$role:910000000000000002', '$role:910000000000000009'], + }; + assert.deepEqual(refsIn(spec).sort(), [ + 'channel:910000000000000001', + 'role:910000000000000002', + 'role:910000000000000009', + ]); + const ids = { 'channel:910000000000000001': 'F1', 'role:910000000000000002': 'F2', 'role:everyone': 'F0' }; + const { value, unresolved } = resolveSpec(spec, (kind, id) => ids[`${kind}:${id}`] ?? null); + assert.equal(value.parentId, 'F1'); + assert.deepEqual( + value.permissionOverwrites.map((o) => o.id), + ['F2', null, 'F0'] + ); + assert.equal(value.text, 'x <#F1> y'); + assert.deepEqual(value.roles, ['F2']); + assert.deepEqual(unresolved, ['role:910000000000000009', 'role:910000000000000009']); +}); diff --git a/test/mirrorTarget.test.js b/test/mirrorTarget.test.js new file mode 100644 index 00000000..1476700f --- /dev/null +++ b/test/mirrorTarget.test.js @@ -0,0 +1,331 @@ +// The Fluxer adapter (src/mirror/target.js) against a double with the SDK's method names: what it passes to the +// SDK (bigint permissions, data URIs, the overwrites) and how a thing that is gone shows up (a 404). +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createFluxerTarget, rolesAbove, toDataUri } from '../src/mirror/target.js'; +import { classifyError, isNotFound } from '../src/mirror/errors.js'; +import { snapshotFluxerGuild } from '../src/mirror/snapshot.js'; + +function sdkGuild({ perms = [], admin = false, top = 7 } = {}) { + const log = []; + const entity = (kind, data) => ({ + ...data, + edit: async (body) => log.push([`${kind}.edit`, data.id, body]), + delete: async (arg) => log.push([`${kind}.delete`, data.id, arg]), + }); + const guild = { + id: '800000000000000050', + name: 'Target', + channels: new Map(), + roles: new Map(), + emojis: new Map(), + stickers: new Map(), + members: { + me: { + permissions: { has: (p) => admin || perms.includes(p) }, + roles: { highest: { id: '810000000000000000', position: top } }, + }, + }, + client: { channels: { fetch: async () => null } }, + createRole: async (body) => { + log.push(['createRole', body]); + const role = entity('role', { id: '810000000000000001', name: body.name, type: 'role' }); + guild.roles.set(role.id, role); + return role; + }, + createChannel: async (body) => { + log.push(['createChannel', body]); + const ch = entity('channel', { id: '820000000000000001', name: body.name, type: body.type }); + guild.channels.set(ch.id, ch); + return ch; + }, + createEmoji: async (body) => ( + log.push(['createEmoji', body]), + entity('emoji', { id: '830000000000000001' }) + ), + createSticker: async (body) => ( + log.push(['createSticker', body]), + entity('sticker', { id: '840000000000000001' }) + ), + setRolePositions: async (u) => log.push(['setRolePositions', u]), + setChannelPositions: async (u) => log.push(['setChannelPositions', u]), + edit: async (b) => log.push(['guild.edit', b]), + fetchRole: async () => { + throw Object.assign(new Error('Unknown Role'), { statusCode: 404 }); + }, + fetchEmoji: async () => { + throw Object.assign(new Error('Unknown Emoji'), { code: 'EMOJI_NOT_FOUND' }); + }, + }; + return { guild, log }; +} + +test("Sylo's rank is compared with the roles on the server: position first, then the older id", () => { + const { guild } = sdkGuild({ top: 1 }); + const add = (id, name, position) => guild.roles.set(id, { id, name, position }); + add('810000000000000000', 'Sylo', 1); + add('810000000000000007', 'Newer', 1); // the same position but a newer id: ranks below Sylo + add('800000000000000001', 'Older', 1); // the same position and older: ranks above + add('820000000000000002', 'High', 4); + const target = createFluxerTarget(guild); + assert.deepEqual( + target.rolesAboveSylo().map((r) => r.name), + ['High', 'Older'] + ); + assert.deepEqual( + target.rolesAboveSylo(['820000000000000002']).map((r) => r.name), + ['Older'], + "the mirror's own roles are ignored" + ); + assert.deepEqual(rolesAbove({ id: '5', position: 3 }, [{ id: '9', name: 'x', position: 2 }]), []); + delete guild.members.me.roles.highest; + assert.equal(target.rolesAboveSylo(), null, "unknown when Sylo's own role cannot be read"); + assert.equal(createFluxerTarget({ id: '1' }).rolesAboveSylo(), null); +}); + +test('what Sylo needs on the Fluxer server: Administrator covers it, otherwise the four permissions', () => { + assert.deepEqual(createFluxerTarget(sdkGuild({ admin: true }).guild).missingPermissions(), []); + assert.deepEqual( + createFluxerTarget(sdkGuild({ perms: ['ManageChannels', 'ManageGuild'] }).guild).missingPermissions(), + ['ManageRoles', 'ManageExpressions'] + ); + assert.deepEqual( + createFluxerTarget({ id: '1' }).missingPermissions(), + [], + 'unknown rights are not a blocker' + ); +}); + +test('a role is made with bigint permissions; hoist and mentionable are set in a second call', async () => { + const { guild, log } = sdkGuild(); + const target = createFluxerTarget(guild); + const id = await target.createRole({ + name: 'Mod', + color: 5, + hoist: true, + mentionable: false, + permissions: '2048', + }); + assert.equal(id, '810000000000000001'); + assert.deepEqual(log[0], ['createRole', { name: 'Mod', permissions: 2048n, color: 5 }]); + assert.deepEqual(log[1], ['role.edit', id, { hoist: true, mentionable: false }]); + await target.editRole(id, { + name: 'Moderators', + color: 6, + hoist: false, + mentionable: true, + permissions: '1024', + }); + assert.deepEqual(log.at(-1), [ + 'role.edit', + id, + { permissions: 1024n, name: 'Moderators', color: 6, hoist: false, mentionable: true }, + ]); + await target.editRole(id, { permissions: '8' }); + assert.deepEqual(log.at(-1), ['role.edit', id, { permissions: 8n }], '@everyone: permissions only'); + await target.deleteRole(id); + assert.equal(log.at(-1)[0], 'role.delete'); + await assert.rejects( + target.editRole('810000000000000099', { permissions: '1' }), + (e) => classifyError(e).kind === 'gone' + ); + await assert.rejects(target.deleteRole('810000000000000099'), (e) => classifyError(e).kind === 'gone'); +}); + +test('a channel keeps its overwrites as bigint allow and deny; only text and announcement change type', async () => { + const { guild, log } = sdkGuild(); + const target = createFluxerTarget(guild); + const id = await target.createChannel({ + name: 'rules', + type: 0, + parentId: '820000000000000009', + topic: 'Read these', + permissionOverwrites: [{ id: '810000000000000001', type: 0, allow: '1024', deny: '2048' }], + }); + assert.deepEqual(log[0][1].permissionOverwrites, [ + { id: '810000000000000001', type: 0, allow: 1024n, deny: 2048n }, + ]); + assert.equal(log[0][1].parentId, '820000000000000009'); + await target.editChannel(id, { name: 'news', type: 5, topic: null, permissionOverwrites: [] }); + assert.deepEqual(log.at(-1), [ + 'channel.edit', + id, + { type: 5, name: 'news', topic: null, permissionOverwrites: [] }, + ]); + guild.channels.get(id).type = 2; + await target.editChannel(id, { name: 'lounge', type: 2, userLimit: 3 }); + assert.deepEqual( + log.at(-1), + ['channel.edit', id, { name: 'lounge', userLimit: 3 }], + 'a voice channel never gets a type change' + ); + await target.deleteChannel(id); + assert.deepEqual(log.at(-1), ['channel.delete', id, { reason: 'Mirrored from the Discord server' }]); + await assert.rejects(target.deleteChannel('820000000000000099'), (e) => classifyError(e).kind === 'gone'); + await target.setChannelPositions([{ id, position: 1, parentId: null }]); + assert.equal(log.at(-1)[0], 'setChannelPositions'); +}); + +test('emoji, stickers, positions and the server itself go through the SDK with data URIs', async () => { + const { guild, log } = sdkGuild(); + const target = createFluxerTarget(guild); + const image = { mime: 'image/png', bytes: Buffer.from([1, 2, 3]) }; + assert.equal(toDataUri(image), 'data:image/png;base64,AQID'); + assert.equal(await target.createEmoji({ name: 'party', image }), '830000000000000001'); + assert.deepEqual(log.at(-1), ['createEmoji', { name: 'party', image: 'data:image/png;base64,AQID' }]); + assert.equal( + await target.createSticker({ name: 'hi', description: 'Hello', tags: ['a'], image }), + '840000000000000001' + ); + assert.deepEqual(log.at(-1), [ + 'createSticker', + { name: 'hi', description: 'Hello', tags: ['a'], image: 'data:image/png;base64,AQID' }, + ]); + await target.setRolePositions([{ id: '1', position: 2 }]); + assert.deepEqual(log.at(-1), ['setRolePositions', [{ id: '1', position: 2 }]]); + await target.editGuild({ name: 'New', icon: 'data:image/png;base64,AQID' }); + assert.deepEqual(log.at(-1), ['guild.edit', { name: 'New', icon: 'data:image/png;base64,AQID' }]); + await assert.rejects( + target.editEmoji('830000000000000099', { name: 'x' }), + (e) => classifyError(e).kind === 'gone' + ); +}); + +test('the Fluxer server is read as plain data: its own @everyone and what is already there', () => { + const { guild } = sdkGuild(); + guild.channels.set('1', { id: '1', name: 'general', type: 0, parentId: null, position: 0 }); + guild.roles.set(guild.id, { + id: guild.id, + name: '@everyone', + position: 0, + _permissions: String(1n << 53n), + }); + guild.roles.set('2', { id: '2', name: 'Mod', position: 1, _permissions: '8', managed: false }); + const snap = snapshotFluxerGuild(guild); + assert.deepEqual( + snap.channels.map((c) => c.name), + ['general'] + ); + assert.deepEqual( + snap.roles.map((r) => [r.name, r.isEveryone, r.permissions]), + [ + ['@everyone', true, 1n << 53n], + ['Mod', false, 8n], + ] + ); + const target = createFluxerTarget(guild); + assert.deepEqual( + target.listChannels().map((c) => c.name), + ['general'] + ); +}); + +test('errors are told apart by what to do about them', () => { + const kind = (err) => classifyError(err).kind; + assert.equal(kind(Object.assign(new Error('x'), { statusCode: 429, retryAfter: 2.5 })), 'rate-limit'); + assert.equal( + classifyError(Object.assign(new Error('x'), { statusCode: 429, retryAfter: 2.5 })).retryAfterMs, + 2500 + ); + assert.equal( + classifyError(Object.assign(new Error('x'), { statusCode: 429, retryAfter: 0 })).retryAfterMs, + 1000, + 'never less than a second' + ); + assert.equal(kind(Object.assign(new Error('x'), { statusCode: 403 })), 'permission'); + assert.equal(kind(Object.assign(new Error('x'), { statusCode: 401 })), 'permission'); + assert.equal(kind(Object.assign(new Error('x'), { statusCode: 404 })), 'gone'); + assert.equal(kind(Object.assign(new Error('x'), { statusCode: 503 })), 'transient'); + assert.equal( + kind(Object.assign(new Error('Maximum number of roles reached'), { statusCode: 400 })), + 'limit' + ); + assert.equal(kind(Object.assign(new Error('Invalid Form Body'), { statusCode: 400 })), 'item'); + assert.equal(kind(Object.assign(new Error('boom'), { code: 'ECONNRESET' })), 'transient'); + assert.equal(kind(Object.assign(new Error('x'), { cause: { code: 'UND_ERR_SOCKET' } })), 'transient'); + assert.equal(kind(new Error('fetch failed')), 'transient'); + assert.equal(kind(new Error('a bug')), 'item'); +}); + +test('only a real not-found answer is "gone": a failed look-up is thrown, so nothing is made twice', async () => { + const { guild } = sdkGuild(); + const target = createFluxerTarget(guild); + const down = Object.assign(new Error('unavailable'), { statusCode: 503 }); + guild.fetchRole = async () => { + throw down; + }; + guild.client.channels.fetch = async () => { + throw Object.assign(new Error('slow down'), { statusCode: 429, retryAfter: 1 }); + }; + guild.fetchEmoji = async () => { + throw Object.assign(new Error('fetch failed'), { code: 'ECONNRESET' }); + }; + await assert.rejects(target.editRole('810000000000000099', { permissions: '1' }), (e) => e === down); + await assert.rejects(target.deleteRole('810000000000000099'), (e) => e === down); + await assert.rejects( + target.editChannel('820000000000000099', { name: 'x', type: 0 }), + (e) => e.statusCode === 429 + ); + await assert.rejects( + target.deleteChannel('820000000000000099'), + (e) => classifyError(e).kind === 'rate-limit' + ); + await assert.rejects( + target.editEmoji('830000000000000099', { name: 'x' }), + (e) => classifyError(e).kind === 'transient' + ); +}); + +test('a look-up with nothing to look up with is an error, not "gone"', async () => { + const { guild } = sdkGuild(); + delete guild.fetchRole; + const target = createFluxerTarget(guild); + await assert.rejects( + target.editRole('810000000000000099', { permissions: '1' }), + (e) => classifyError(e).kind === 'item' + ); +}); + +test('a role whose second call fails is reported with its id, so it can be tracked', async () => { + const { guild } = sdkGuild(); + const real = guild.createRole; + guild.createRole = async (body) => { + const role = await real(body); + role.edit = async () => { + throw Object.assign(new Error('unavailable'), { statusCode: 503 }); + }; + return role; + }; + const target = createFluxerTarget(guild); + await assert.rejects( + target.createRole({ name: 'Mod', color: 0, hoist: true, mentionable: false, permissions: '0' }), + (e) => e.createdId === '810000000000000001' && classifyError(e).kind === 'transient' + ); +}); + +test("the SDK's own codes: not-found codes are gone, Fluxer's limit codes are limits", () => { + const kind = (err) => classifyError(err).kind; + for (const code of ['ROLE_NOT_FOUND', 'CHANNEL_NOT_FOUND', 'EMOJI_NOT_FOUND', 'STICKER_NOT_FOUND']) { + assert.equal(kind(Object.assign(new Error('x'), { code })), 'gone', code); + } + for (const code of ['MAX_EMOJIS', 'MAX_STICKERS', 'MAX_GUILD_ROLES', 'MAX_GUILD_CHANNELS']) { + assert.equal(kind(Object.assign(new Error('refused'), { code, statusCode: 400 })), 'limit', code); + } + assert.equal( + kind(Object.assign(new Error('x'), { statusCode: 429, code: 'ROLE_NOT_FOUND' })), + 'rate-limit' + ); + assert.equal(isNotFound(Object.assign(new Error('x'), { cause: { statusCode: 404 } })), true); + assert.equal(isNotFound(Object.assign(new Error('x'), { statusCode: 503 })), false); +}); + +test('a 5xx that only the SDK wrapper error carries in its cause is still retried, not failed', () => { + const wrapped = Object.assign(new Error('Failed to fetch role'), { cause: { statusCode: 503 } }); + assert.equal(classifyError(wrapped).kind, 'transient'); + const limited = Object.assign(new Error('x'), { cause: { statusCode: 429 }, retryAfter: 2 }); + assert.equal(classifyError(limited).kind, 'rate-limit'); + assert.equal( + classifyError(Object.assign(new Error('x'), { cause: { statusCode: 403 } })).kind, + 'permission' + ); +}); diff --git a/test/mobileInfo.test.js b/test/mobileInfo.test.js index efba5685..764864f4 100644 --- a/test/mobileInfo.test.js +++ b/test/mobileInfo.test.js @@ -51,6 +51,11 @@ test('GET /api/mobile/info describes a server with login on, without a cookie or assert.ok(info.features.includes(f), f); } assert.ok(info.features.includes('message-move'), 'a posted message moves when its channel changes'); + assert.ok( + info.features.includes('bridge'), + 'the Bridge module: GET /api/v2/guilds/:id/modules/bridge/config' + ); + assert.ok(info.features.includes('mirror'), 'a Discord server can be mirrored to Fluxer'); assert.ok(info.features.every((f) => typeof f === 'string')); }); diff --git a/test/v2.accessLevels.test.js b/test/v2.accessLevels.test.js index 8bfd002a..25d0f348 100644 --- a/test/v2.accessLevels.test.js +++ b/test/v2.accessLevels.test.js @@ -126,6 +126,7 @@ const READS = [ '/settings', '/access', '/modules/afk/config', + '/modules/bridge/config', '/audit', '/tickets', '/appeals', diff --git a/test/v2Bridge.test.js b/test/v2Bridge.test.js new file mode 100644 index 00000000..adc0ff66 --- /dev/null +++ b/test/v2Bridge.test.js @@ -0,0 +1,533 @@ +// The Bridge routes of the dashboard API, through the real app with both logins: who may see and change a +// server's bridge, the consent flow across two servers on two platforms, and that one server can never +// reach into another's links. The engine itself is in bridge.engine.test.js. +import './helpers/trustLoopbackProxy.js'; +import test, { beforeEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { + D_GUILD, + F_GUILD, + OTHER_D_GUILD, + browser, + profiles, + setProfiles, + signIn, + startOAuthApp, +} from './helpers/oauthApp.js'; + +await startOAuthApp(); + +const { runtime } = await import('../src/runtime.js'); +const { setGuildModule } = await import('../src/db/modules.js'); +const store = await import('../src/db/bridge.js'); +const { listAudit } = await import('../src/db/audit.js'); +const { purgeGuild } = await import('../src/db/purge.js'); +const { ROUTE_ACCESS } = await import('../src/web/lib/routeAccess.js'); +const { dropBridgeCache, sweepOwners } = await import('../src/modules/bridge.js'); +const { useFakes } = await import('./helpers/fakeBridge.js'); + +const F_KEY = `fluxer:${F_GUILD}`; +const CH = { + d1: '810000000000000001', + d2: '810000000000000002', + f1: '810000000000000003', + f2: '810000000000000004', + dNsfw: '810000000000000005', +}; +const dGuild = runtime.clients.discord.guilds.cache.get(D_GUILD); +const fGuild = runtime.clients.fluxer.guilds.cache.get(F_GUILD); +const addChannel = (guild, id, name, nsfw = false) => + guild.channels.cache.set(id, { + id, + name, + type: 0, + nsfw, + nsfwOverride: null, + rawPosition: Number(id.slice(-2)), + guild, + said: [], + // Sylo (no member) may do everything; a person may not use a channel that is hidden from them. + permissionsFor: (member) => ({ has: () => !member?.hidden?.has(id) }), + async send(payload) { + this.said.push(payload); + return { id: '1' }; + }, + }); + +let fakes; +beforeEach(async () => { + useFakes(); // the purge below talks to the webhook transport for links the previous test left + dropBridgeCache(); + for (const key of [D_GUILD, F_KEY, OTHER_D_GUILD]) await purgeGuild(key); + fakes = useFakes(); + dGuild.channels.cache.clear(); + fGuild.channels.cache.clear(); + fGuild.nsfw = false; + dGuild.members.cache.clear(); + fGuild.members.cache.clear(); + for (const guild of [dGuild, fGuild, runtime.clients.discord.guilds.cache.get(OTHER_D_GUILD)]) + guild.ownerId = 'x'; + addChannel(dGuild, CH.d1, 'discord-main'); + addChannel(dGuild, CH.d2, 'discord-two'); + addChannel(dGuild, CH.dNsfw, 'discord-nsfw', true); + addChannel(fGuild, CH.f1, 'fluxer-main'); + addChannel(fGuild, CH.f2, 'fluxer-two'); + await setGuildModule(D_GUILD, 'bridge', { enabled: true }); + await setGuildModule(F_KEY, 'bridge', { enabled: true }); +}); + +/** A browser that sends its requests from an address of its own, so the sign-in limit is not shared. */ +let clients = 0; +function ownBrowser() { + const b = browser(); + const ip = `10.8.${Math.floor(++clients / 250)}.${clients % 250}`; + return { + ...b, + get: (path, init = {}) => b.get(path, { ...init, headers: { ...init.headers, 'x-forwarded-for': ip } }), + }; +} + +/** Someone signed in on a platform, with JSON helpers that carry the CSRF token. */ +async function person(platform, { guilds, hidden = [], member = true } = {}) { + setProfiles(); + if (guilds) profiles[platform].guilds = guilds; + const b = ownBrowser(); + await signIn(b, platform); + // The platform reports the person as the owner of every server the profile lists as theirs. + const client = platform === 'discord' ? runtime.clients.discord : runtime.clients.fluxer; + for (const g of profiles[platform].guilds) { + if (g.owner === true) client.guilds.cache.get(g.id).ownerId = profiles[platform].id; + } + // The person's member in the server (the channel rights are read from it); `hidden` channels they cannot see. + if (member) { + const id = profiles[platform].id; + (platform === 'discord' ? dGuild : fGuild).members.cache.set(id, { id, hidden: new Set(hidden) }); + } + const token = (await (await b.get('/api/v2/csrf')).json()).token; + const headers = { accept: 'application/json', 'content-type': 'application/json', 'x-csrf-token': token }; + return { + get: (path) => b.get(`/api/v2${path}`, { headers: { accept: 'application/json' } }), + post: (path, body = {}) => + b.get(`/api/v2${path}`, { method: 'POST', headers, body: JSON.stringify(body) }), + }; +} +const dAdmin = () => person('discord'); +const fAdmin = () => person('fluxer'); +const base = (key) => `/guilds/${key}/modules/bridge`; +const json = (res) => res.json(); +const DEFAULTS = { bots: false, joinMessages: false, uploadFiles: true }; + +test('the routes are listed with the access they need: View reads, Edit changes', () => { + assert.equal(ROUTE_ACCESS['GET /guilds/:guildId/modules/bridge/config'], 'view'); + for (const [key, level] of Object.entries(ROUTE_ACCESS)) { + if (key.includes('/modules/bridge/') && key.startsWith('POST ')) assert.equal(level, 'edit', key); + } + assert.equal(Object.keys(ROUTE_ACCESS).filter((k) => k.includes('/modules/bridge/')).length, 8); +}); + +test('nobody signed out, and nobody without Manage Server, can read or change a bridge', async () => { + const b = ownBrowser(); + const read = await b.get(`/api/v2${base(D_GUILD)}/config`, { headers: { accept: 'application/json' } }); + assert.ok([401, 403].includes(read.status)); + const member = await person('discord', { guilds: [{ id: D_GUILD, owner: false, permissions: '0' }] }); + assert.equal((await member.get(`${base(D_GUILD)}/config`)).status, 403); + for (const [path, body] of [ + ['/request', { channelId: CH.d1 }], + ['/accept', { code: 'ABCD-EFGH-JKMN', channelId: CH.d1 }], + ['/links/1/pause', {}], + ['/links/1/resume', {}], + ['/links/1/unlink', {}], + ['/links/1/options', { options: { bots: true } }], + ['/requests/1/cancel', {}], + ]) { + assert.equal((await member.post(`${base(D_GUILD)}${path}`, body)).status, 403, path); + } + assert.equal((await store.pendingCodes(D_GUILD)).length, 0); + // an admin of one server has nothing in another + const stranger = await person('discord', { guilds: [{ id: OTHER_D_GUILD, owner: true }] }); + assert.equal((await stranger.get(`${base(D_GUILD)}/config`)).status, 403); + assert.equal((await stranger.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })).status, 403); +}); + +test('a state-changing call without the CSRF token is refused', async () => { + setProfiles(); + const b = ownBrowser(); + await signIn(b, 'discord'); + const res = await b.get(`/api/v2${base(D_GUILD)}/request`, { + method: 'POST', + headers: { 'content-type': 'application/json', accept: 'application/json' }, + body: JSON.stringify({ channelId: CH.d1 }), + }); + assert.equal(res.status, 403); + assert.equal((await store.pendingCodes(D_GUILD)).length, 0); +}); + +test('the overview lists channels, shows what is set up, and says when the module is off', async () => { + const d = await dAdmin(); + const res = await d.get(`${base(D_GUILD)}/config`); + assert.equal(res.status, 200); + const data = await json(res); + assert.equal(data.platform, 'discord'); + assert.equal(data.bothPlatforms, true); + assert.equal(data.enabled, true); + assert.deepEqual(data.limits, { links: 10, requests: 5 }); + assert.deepEqual(data.links, []); + assert.deepEqual( + data.channels.map((c) => [c.name, c.ageRestricted, c.taken]), + [ + ['discord-main', false, false], + ['discord-two', false, false], + ['discord-nsfw', true, false], + ] + ); + await setGuildModule(D_GUILD, 'bridge', { enabled: false }); + assert.equal((await json(await d.get(`${base(D_GUILD)}/config`))).enabled, false); + const refused = await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 }); + assert.equal(refused.status, 409); + assert.match((await json(refused)).error, /Turn the Bridge module on/); +}); + +test('asking, then accepting in the other server on the other platform, links the two channels', async () => { + const d = await dAdmin(); + const f = await fAdmin(); + + const asked = await d.post(`${base(D_GUILD)}/request`, { + channelId: CH.d1, + options: { joinMessages: true }, + }); + assert.equal(asked.status, 200); + const { code, expiresAt, overview } = await json(asked); + assert.match(code, /^[A-Z2-9]{4}-[A-Z2-9]{4}-[A-Z2-9]{4}$/); + assert.ok(expiresAt > Date.now()); + assert.equal(overview.requests.length, 1); + assert.ok(!JSON.stringify(overview).includes(code), 'the code is shown once and never listed'); + assert.equal(overview.channels.find((c) => c.id === CH.d1).taken, true); + + // asking again for the same channel is refused, and the Fluxer server sees none of it + assert.equal((await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })).status, 409); + assert.deepEqual((await json(await f.get(`${base(F_KEY)}/config`))).requests, []); + + const accepted = await f.post(`${base(F_KEY)}/accept`, { code: code.toLowerCase(), channelId: CH.f1 }); + assert.equal(accepted.status, 200); + const mine = (await json(accepted)).overview; + assert.equal(mine.platform, 'fluxer'); + assert.equal(mine.links.length, 1); + const view = mine.links[0]; + assert.equal(view.status, 'active'); + assert.deepEqual( + [view.mine.channel, view.theirs.channel, view.theirs.server, view.theirs.platform], + ['fluxer-main', 'discord-main', 'D-main', 'discord'] + ); + assert.deepEqual( + view.theirOptions, + { bots: false, joinMessages: true, uploadFiles: true }, + 'what the other side chose to share' + ); + assert.deepEqual(view.options, { bots: false, joinMessages: false, uploadFiles: true }); + + // Discord sees the same link from its side, and the request is gone + const theirs = await json(await d.get(`${base(D_GUILD)}/config`)); + assert.equal(theirs.requests.length, 0); + assert.equal(theirs.links.length, 1); + assert.deepEqual( + [theirs.links[0].mine.channel, theirs.links[0].theirs.channel], + ['discord-main', 'fluxer-main'] + ); + + // never a webhook or its token to the browser + const everything = JSON.stringify([mine, theirs]); + for (const hook of fakes.transport.hooks) assert.ok(!everything.includes(hook.token)); + assert.ok(!/token|webhook/i.test(everything)); + + assert.ok((await listAudit(D_GUILD)).some((e) => e.action === 'bridge:link')); + assert.ok((await listAudit(F_KEY)).some((e) => e.action === 'bridge:link')); + assert.ok(dGuild.channels.cache.get(CH.d1).said[0].content.includes('Bridge on')); + assert.ok(fGuild.channels.cache.get(CH.f1).said[0].content.includes('Bridge on')); +}); + +test('someone without Edit in the accepting server cannot use a code there, whatever they hold in the other', async () => { + const d = await dAdmin(); + const { code } = await json(await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })); + // the Discord admin has no right in the Fluxer server + const refused = await d.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 }); + assert.equal(refused.status, 403); + // and a Fluxer member without Manage Server cannot either + const member = await person('fluxer', { guilds: [{ id: F_GUILD, owner: false, permissions: '0' }] }); + assert.equal((await member.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 })).status, 403); + assert.equal((await store.pendingCodes(D_GUILD)).length, 1, 'the code is still unused'); + assert.equal(await store.linkForChannel(F_KEY, CH.f1), null); +}); + +test('a wrong code, a used code, your own code and the wrong platform are all refused', async () => { + const d = await dAdmin(); + const f = await fAdmin(); + const { code } = await json(await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })); + + const wrong = await f.post(`${base(F_KEY)}/accept`, { code: 'ABCD-EFGH-JKMN', channelId: CH.f1 }); + assert.equal(wrong.status, 404); + assert.match((await json(wrong)).error, /not valid, has expired or was already used/); + assert.equal((await f.post(`${base(F_KEY)}/accept`, { code: '', channelId: CH.f1 })).status, 404); + assert.equal( + (await f.post(`${base(F_KEY)}/accept`, { code, channelId: 'junk' })).status, + 400, + 'no channel' + ); + assert.equal( + (await d.post(`${base(D_GUILD)}/accept`, { code, channelId: CH.d2 })).status, + 409, + 'your own code' + ); + + assert.equal((await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 })).status, 200); + assert.equal( + (await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f2 })).status, + 404, + 'a used code' + ); +}); + +test('too many wrong codes lock the attempts for a while, for that person in that server', async () => { + const f = await fAdmin(); + for (let i = 0; i < 6; i += 1) { + assert.equal( + (await f.post(`${base(F_KEY)}/accept`, { code: 'ABCD-EFGH-JKM' + i, channelId: CH.f1 })).status, + 404 + ); + } + const locked = await f.post(`${base(F_KEY)}/accept`, { code: 'ABCD-EFGH-JKMN', channelId: CH.f1 }); + assert.equal(locked.status, 429); + assert.match((await json(locked)).error, /Wait a few minutes/); + // a valid code does not get around it + const d = await dAdmin(); + const { code } = await json(await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })); + assert.equal((await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 })).status, 429); + assert.equal((await store.pendingCodes(D_GUILD)).length, 1); +}); + +test('an age-restricted channel is refused for an ordinary one when asking and when accepting', async () => { + const d = await dAdmin(); + const f = await fAdmin(); + const { code } = await json(await d.post(`${base(D_GUILD)}/request`, { channelId: CH.dNsfw })); + const refused = await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 }); + assert.equal(refused.status, 400); + assert.match( + (await json(refused)).error, + /age-restricted channel can only be bridged with another age-restricted/ + ); + assert.equal(fakes.transport.hooks.length, 0); + addChannel(fGuild, '810000000000000006', 'fluxer-nsfw', true); + assert.equal( + (await f.post(`${base(F_KEY)}/accept`, { code, channelId: '810000000000000006' })).status, + 200 + ); +}); + +test('pause, resume and unlink from either side, with the rules about who may resume', async () => { + const d = await dAdmin(); + const f = await fAdmin(); + const { code } = await json(await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })); + await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 }); + const id = (await store.linksOfGuild(D_GUILD))[0].id; + + const paused = await d.post(`${base(D_GUILD)}/links/${id}/pause`); + assert.equal(paused.status, 200); + assert.equal((await json(paused)).overview.links[0].pausedByMe, true); + assert.equal((await json(await f.get(`${base(F_KEY)}/config`))).links[0].pausedByThem, true); + const blocked = await f.post(`${base(F_KEY)}/links/${id}/resume`); + assert.equal(blocked.status, 403); + assert.match((await json(blocked)).error, /only they can resume/); + assert.equal((await d.post(`${base(D_GUILD)}/links/${id}/resume`)).status, 200); + assert.equal((await store.getLink(id)).status, 'active'); + + const opts = await f.post(`${base(F_KEY)}/links/${id}/options`, { options: { bots: true, junk: 1 } }); + assert.deepEqual((await json(opts)).overview.links[0].options, { + bots: true, + joinMessages: false, + uploadFiles: true, + }); + assert.deepEqual( + (await store.getLink(id)).optionsA, + DEFAULTS, + "the other server's options are not touched" + ); + + const unlinked = await f.post(`${base(F_KEY)}/links/${id}/unlink`); + assert.equal(unlinked.status, 200); + assert.deepEqual((await json(unlinked)).overview.links, []); + assert.equal(await store.getLink(id), null); + assert.equal(fakes.transport.deleted.length, 2, 'both webhooks were deleted'); + assert.ok((await listAudit(D_GUILD)).some((e) => e.action === 'bridge:unlink')); +}); + +test("another server's link id is a 404 in every route, and nothing of it changes", async () => { + const d = await dAdmin(); + const f = await fAdmin(); + const { code } = await json(await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })); + await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 }); + const id = (await store.linksOfGuild(D_GUILD))[0].id; + + const stranger = await person('discord', { guilds: [{ id: OTHER_D_GUILD, owner: true }] }); + for (const path of ['pause', 'resume', 'unlink']) { + const res = await stranger.post(`${base(OTHER_D_GUILD)}/links/${id}/${path}`); + assert.equal(res.status, 404, path); + } + assert.equal( + (await stranger.post(`${base(OTHER_D_GUILD)}/links/${id}/options`, { options: { bots: true } })).status, + 404 + ); + assert.equal((await stranger.post(`${base(OTHER_D_GUILD)}/requests/1/cancel`)).status, 404); + assert.equal((await stranger.post(`${base(OTHER_D_GUILD)}/links/abc/pause`)).status, 404); + const link = await store.getLink(id); + assert.deepEqual([link.status, link.optionsA, link.optionsB], ['active', DEFAULTS, DEFAULTS]); +}); + +test('a request can be cancelled by the server that made it, which kills its code', async () => { + const d = await dAdmin(); + const f = await fAdmin(); + const { code, overview } = await json(await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })); + const requestId = overview.requests[0].id; + assert.equal( + (await f.post(`${base(F_KEY)}/requests/${requestId}/cancel`)).status, + 404, + 'not by the other server' + ); + const cancelled = await d.post(`${base(D_GUILD)}/requests/${requestId}/cancel`); + assert.equal(cancelled.status, 200); + assert.deepEqual((await json(cancelled)).overview.requests, []); + assert.equal((await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 })).status, 404); +}); + +test('a person must be able to view, read and send in the channel themselves: a hidden staff channel is refused', async () => { + const d = await person('discord', { hidden: [CH.d2] }); + const hiddenAsk = await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d2 }); + assert.equal(hiddenAsk.status, 403); + assert.match((await json(hiddenAsk)).error, /view, read and send in #discord-two yourself/); + assert.equal((await store.pendingCodes(D_GUILD)).length, 0, 'no request was made'); + + const ok = await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 }); + assert.equal(ok.status, 200); + const { code } = await json(ok); + + const f = await person('fluxer', { hidden: [CH.f2] }); + const hiddenAccept = await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f2 }); + assert.equal(hiddenAccept.status, 403); + assert.equal((await store.pendingCodes(D_GUILD)).length, 1, 'the code is still unused'); + assert.equal(fakes.transport.hooks.length, 0, 'and no webhook was made'); + assert.equal((await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 })).status, 200); +}); + +test('a person Sylo cannot look up in the server (no member) is refused: it fails closed', async () => { + const d = await person('discord', { member: false }); + const res = await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 }); + assert.equal(res.status, 403); + assert.match((await json(res)).error, /yourself/); +}); + +test('a Fluxer channel in an age-restricted category or server is refused for an ordinary Discord channel', async () => { + const d = await dAdmin(); + const f = await fAdmin(); + const { code } = await json(await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })); + // a category with its own override, the channel itself inheriting + fGuild.channels.cache.set('810000000000000007', { + id: '810000000000000007', + name: 'adult', + type: 4, + nsfwOverride: true, + nsfw: false, + }); + fGuild.channels.cache.get(CH.f1).parentId = '810000000000000007'; + const category = await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 }); + assert.equal(category.status, 400); + assert.match((await json(category)).error, /age-restricted/); + // the server itself is age-restricted + fGuild.channels.cache.get(CH.f1).parentId = null; + fGuild.nsfw = true; + assert.equal((await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 })).status, 400); + // what cannot be read counts as age-restricted + fGuild.nsfw = false; + fGuild.channels.cache.get(CH.f1).nsfwOverride = undefined; + assert.equal((await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 })).status, 400); + assert.equal(fakes.transport.hooks.length, 0); + fGuild.channels.cache.get(CH.f1).nsfwOverride = null; + assert.equal( + (await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 })).status, + 200, + 'clear: it links' + ); +}); + +const ALL_POSTS = (key) => + [ + ['/request', { channelId: CH.d1 }], + ['/accept', { code: 'ABCD-EFGH-JKMN', channelId: CH.d1 }], + ['/requests/1/cancel', {}], + ['/links/1/options', { options: { bots: true } }], + ['/links/1/pause', {}], + ['/links/1/resume', {}], + ['/links/1/unlink', {}], + ].map(([path, body]) => [`${base(key)}${path}`, body]); + +test('an Administrator who is not the owner cannot do anything to a bridge, but can see its status', async () => { + // a real link, made by the owners + const d = await dAdmin(); + const f = await fAdmin(); + const { code } = await json(await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })); + await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 }); + const id = (await store.linksOfGuild(D_GUILD))[0].id; + const before = await store.getLink(id); + + const admin = await person('discord', { guilds: [{ id: D_GUILD, owner: false, permissions: '8' }] }); + const read = await admin.get(`${base(D_GUILD)}/config`); + assert.equal(read.status, 200, 'Administrator reads the status (View)'); + const seen = await json(read); + assert.equal(seen.owner, false); + assert.equal(seen.links.length, 1); + for (const [path, body] of ALL_POSTS(D_GUILD)) { + const res = await admin.post(path.replace('/links/1/', `/links/${id}/`), body); + assert.equal(res.status, 403, path); + assert.match((await json(res)).error, /Only the server owner/, path); + } + assert.deepEqual(await store.getLink(id), before, 'nothing changed'); + assert.equal((await store.pendingCodes(D_GUILD)).length, 0); + + // the same on the Fluxer side + const fAdminOnly = await person('fluxer', { guilds: [{ id: F_GUILD, owner: false, permissions: '8' }] }); + for (const [path, body] of ALL_POSTS(F_KEY)) { + const res = await fAdminOnly.post(path.replace('/links/1/', `/links/${id}/`), body); + assert.equal(res.status, 403, path); + assert.match((await json(res)).error, /Only the server owner/, path); + } + assert.equal((await store.getLink(id)).status, 'active'); + + // the owner of the server reads `owner: true` and can act + assert.equal((await json(await d.get(`${base(D_GUILD)}/config`))).owner, true); + assert.equal((await d.post(`${base(D_GUILD)}/links/${id}/pause`)).status, 200); +}); + +test('a transfer of ownership pauses the link, and only the new owner can resume it', async () => { + const d = await dAdmin(); + const f = await fAdmin(); + const { code } = await json(await d.post(`${base(D_GUILD)}/request`, { channelId: CH.d1 })); + await f.post(`${base(F_KEY)}/accept`, { code, channelId: CH.f1 }); + const id = (await store.linksOfGuild(D_GUILD))[0].id; + + dGuild.ownerId = 'transfer-target'; // the Discord server is handed to someone else + assert.equal(await sweepOwners(), 1); + const paused = await store.getLink(id); + assert.deepEqual([paused.status, paused.pausedSide, paused.failReason], ['paused', 'a', 'owner-changed']); + + // the previous owner is no longer the owner: refused. The Fluxer owner cannot resume Discord's confirmation. + assert.equal((await d.post(`${base(D_GUILD)}/links/${id}/resume`)).status, 403); + const blocked = await f.post(`${base(F_KEY)}/links/${id}/resume`); + assert.equal(blocked.status, 403); + assert.match((await json(blocked)).error, /only they can resume/); + const view = (await json(await f.get(`${base(F_KEY)}/config`))).links[0]; + assert.deepEqual([view.status, view.pausedByThem], ['paused', true]); + assert.match(view.pauseNote, /owner of one of the servers changed/); + + const newOwner = await person('discord'); // becomes the owner the platform reports + const confirmed = await newOwner.post(`${base(D_GUILD)}/links/${id}/resume`); + assert.equal(confirmed.status, 200); + const row = await store.getLink(id); + assert.deepEqual([row.status, row.ownerA], ['active', dGuild.ownerId]); +}); diff --git a/test/v2Mirror.test.js b/test/v2Mirror.test.js new file mode 100644 index 00000000..32bc6bb4 --- /dev/null +++ b/test/v2Mirror.test.js @@ -0,0 +1,420 @@ +// The Mirror to Fluxer routes through the real app with both logins: asking, agreeing with the code, the +// preview, the copy and the controls; that only a server's owner can do any of it (not an Administrator, not an +// Edit holder); and that a code, a pair or a server id of another server reaches nothing. +import './helpers/trustLoopbackProxy.js'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + D_GUILD, + F_GUILD, + OTHER_D_GUILD, + browser, + profiles, + setProfiles, + signIn, + startOAuthApp, +} from './helpers/oauthApp.js'; + +await startOAuthApp(); + +const { runtime } = await import('../src/runtime.js'); +const db = await import('../src/db/mirror.js'); +const { listAudit } = await import('../src/db/audit.js'); + +const F_KEY = `fluxer:${F_GUILD}`; +const dGuild = runtime.clients.discord.guilds.cache.get(D_GUILD); +const fGuild = runtime.clients.fluxer.guilds.cache.get(F_GUILD); + +let people = 0; +/** A browser that sends its requests from an address of its own (the routes are rate limited per person). */ +function own() { + const b = browser(); + const ip = `10.7.${Math.floor(++people / 250)}.${people % 250}`; + return { + ...b, + get: (path, init = {}) => b.get(path, { ...init, headers: { ...init.headers, 'x-forwarded-for': ip } }), + }; +} + +/** + * Sign in with Discord, Fluxer or both; every request is JSON with the CSRF token. `admin` signs in with + * Administrator on the Discord server instead of owning it. The server's owner is set by the test (`owners`). + */ +async function person({ discord = false, fluxer = false, admin = false } = {}) { + setProfiles(); + if (admin) profiles.discord.guilds = [{ id: D_GUILD, owner: false, permissions: '8' }]; + const ids = { discord: profiles.discord.id, fluxer: profiles.fluxer.id }; + const b = own(); + if (discord) await signIn(b, 'discord'); + if (fluxer) await signIn(b, 'fluxer', { connect: discord }); + const token = (await (await b.get('/api/v2/csrf')).json()).token; + const headers = { accept: 'application/json', 'content-type': 'application/json', 'x-csrf-token': token }; + return { + ids, + get: (path) => b.get(`/api/v2${path}`, { headers: { accept: 'application/json' } }), + post: (path, body = {}) => + b.get(`/api/v2${path}`, { method: 'POST', headers, body: JSON.stringify(body) }), + }; +} + +/** Make these people the owners of the two fake servers. */ +const owners = ({ discord, fluxer }) => { + dGuild.ownerId = discord ?? 'nobody-discord'; + fGuild.ownerId = fluxer ?? 'nobody-fluxer'; +}; + +const reset = async () => { + for (const key of [D_GUILD, OTHER_D_GUILD, F_KEY]) + for (const p of await db.pairsOfGuild(key)) await db.deletePair(p.id); +}; + +test('the owner of the Discord server asks; the owner of the Fluxer server agrees with the one-time code; then the whole flow', async () => { + await reset(); + const anna = await person({ discord: true }); + const bo = await person({ fluxer: true }); + owners({ discord: anna.ids.discord, fluxer: bo.ids.fluxer }); + + const empty = await (await anna.get(`/guilds/${D_GUILD}/mirror`)).json(); + assert.deepEqual( + [empty.platform, empty.canMirror, empty.pair, empty.isOwner], + ['discord', true, null, true] + ); + assert.equal(empty.candidates.length, 0, 'this person owns no Fluxer server'); + + const res = await anna.post(`/guilds/${D_GUILD}/mirror/request`, { targetGuildId: F_GUILD }); + assert.equal(res.status, 200); + const body = await res.json(); + assert.match(body.code, /^[A-Z2-9]{4}-[A-Z2-9]{4}$/); + assert.equal(body.agreed, false); + assert.equal(body.pair.status, 'pending'); + assert.equal(body.pair.otherName, null, 'the Fluxer name is not shown before its owner agrees'); + assert.ok(!JSON.stringify(body).includes('codeHash')); + assert.ok( + !JSON.stringify(await (await anna.get(`/guilds/${D_GUILD}/mirror`)).json()).includes(body.code), + 'the code is shown once' + ); + const pending = await db.getPairBySource(D_GUILD); + assert.deepEqual([pending.sourceConsentUser, pending.targetConsentUser], [anna.ids.discord, '']); + + // Nothing can be previewed or started until both agree. + assert.equal((await anna.get(`/guilds/${D_GUILD}/mirror/preview`)).status, 409); + assert.equal((await anna.post(`/guilds/${D_GUILD}/mirror/start`, { confirm: true })).status, 409); + + const seen = await (await bo.get(`/guilds/${F_KEY}/mirror`)).json(); + assert.deepEqual([seen.platform, seen.pair.side, seen.pair.status], ['fluxer', 'target', 'pending']); + assert.equal(seen.pair.otherName, 'D-main', 'they see which server asks'); + assert.equal(seen.pair.requestedBy, '', "the other server's dashboard user is not named to them"); + assert.equal((await bo.post(`/guilds/${F_KEY}/mirror/accept`, { code: 'AAAA-AAAA' })).status, 403); + assert.equal( + (await anna.post(`/guilds/${D_GUILD}/mirror/accept`, { code: body.code })).status, + 400, + 'the code is entered on the Fluxer server' + ); + const ok = await bo.post(`/guilds/${F_KEY}/mirror/accept`, { code: body.code }); + assert.equal(ok.status, 200); + assert.equal((await ok.json()).pair.status, 'ready'); + assert.equal( + (await db.getPairBySource(D_GUILD)).targetConsentUser, + bo.ids.fluxer, + 'the Fluxer owner is recorded' + ); + assert.equal( + (await bo.post(`/guilds/${F_KEY}/mirror/accept`, { code: body.code })).status, + 404, + 'the code works once' + ); + const audit = (await listAudit(D_GUILD, 20)).map((e) => e.action); + assert.ok(audit.includes('mirror:request') && audit.includes('mirror:agree')); + + // The preview reads, the copy needs a confirmation, and a started copy shows progress and a report. + const preview = await (await anna.get(`/guilds/${D_GUILD}/mirror/preview`)).json(); + assert.equal(preview.ok, true); + assert.deepEqual( + preview.steps.map((s) => s.kind), + ['guild', 'bridge'] + ); + assert.equal(preview.target.name, 'F-main'); + assert.ok(!('spec' in preview.steps[0]), 'the preview names steps; it does not hand out whole specs'); + assert.equal((await anna.post(`/guilds/${D_GUILD}/mirror/start`, {})).status, 400); + assert.equal((await anna.post(`/guilds/${D_GUILD}/mirror/start`, { confirm: 'yes' })).status, 400); + const started = await anna.post(`/guilds/${D_GUILD}/mirror/start`, { confirm: true }); + assert.equal(started.status, 200); + const state = await started.json(); + assert.equal(state.pair.status, 'copying'); + assert.equal(state.progress.total, 2); + const report = await (await anna.get(`/guilds/${D_GUILD}/mirror/report`)).json(); + assert.deepEqual( + report.items.map((i) => i.kind), + ['guild', 'bridge'] + ); + // Real deletion: the Discord owner can only ask; the owner of the Fluxer server decides. + assert.equal( + (await bo.post(`/guilds/${F_KEY}/mirror/deletes`, { allow: true })).status, + 409, + 'nothing has been asked for yet' + ); + const opts = await anna.post(`/guilds/${D_GUILD}/mirror/options`, { + continuous: false, + mirrorDeletes: true, + bogus: 1, + }); + const optionsNow = (await opts.json()).pair.options; + assert.deepEqual( + [optionsNow.continuous, optionsNow.mirrorDeletes, optionsNow.mirrorDeletesRequested], + [false, false, true], + 'asking is not switching on' + ); + assert.equal(optionsNow.bogus, undefined); + assert.equal( + (await anna.post(`/guilds/${D_GUILD}/mirror/deletes`, { allow: true })).status, + 404, + 'not from the Discord side' + ); + const seenByBo = await (await bo.get(`/guilds/${F_KEY}/mirror`)).json(); + assert.equal(seenByBo.pair.options.mirrorDeletesRequested, true, 'the Fluxer owner sees the request'); + const allowed = await (await bo.post(`/guilds/${F_KEY}/mirror/deletes`, { allow: true })).json(); + assert.equal(allowed.pair.options.mirrorDeletes, true, 'the Fluxer owner switched it on'); + const again = await ( + await anna.post(`/guilds/${D_GUILD}/mirror/options`, { mirrorDeletes: true, continuous: false }) + ).json(); + assert.equal(again.pair.options.mirrorDeletes, true, 'asking again does not change it'); + const off = await (await anna.post(`/guilds/${D_GUILD}/mirror/options`, { mirrorDeletes: false })).json(); + assert.deepEqual( + [off.pair.options.mirrorDeletes, off.pair.options.mirrorDeletesRequested], + [false, false], + 'either side can switch it off' + ); + assert.ok((await listAudit(F_KEY, 20)).some((e) => e.action === 'mirror:deletes')); + + // Either owner can pause; only the owner that paused can resume it. + assert.equal((await bo.post(`/guilds/${F_KEY}/mirror/resume`)).status, 409, 'only a paused mirror resumes'); + const paused = await bo.post(`/guilds/${F_KEY}/mirror/pause`); + assert.equal((await paused.json()).pair.pausedBy, 'target'); + const refused = await anna.post(`/guilds/${D_GUILD}/mirror/resume`); + assert.equal(refused.status, 409); + assert.match((await refused.json()).error, /only that owner can resume/); + assert.equal((await bo.post(`/guilds/${F_KEY}/mirror/resume`)).status, 200); + assert.equal((await db.getPairBySource(D_GUILD)).status, 'copying'); + assert.equal( + (await bo.post(`/guilds/${F_KEY}/mirror/remove`)).status, + 409, + 'a running mirror is stopped before it is removed' + ); + const stopped = await bo.post(`/guilds/${F_KEY}/mirror/stop`); + assert.equal((await stopped.json()).pair.status, 'stopped'); + const pair = await db.getPairBySource(D_GUILD); + assert.deepEqual( + [pair.sourceConsentBy, pair.targetConsentBy, pair.sourceConsentUser, pair.targetConsentUser], + ['', '', '', ''], + 'stopping clears the agreement' + ); + assert.equal((await anna.post(`/guilds/${D_GUILD}/mirror/remove`)).status, 200); + assert.equal(await db.getPairBySource(D_GUILD), null); +}); + +test('the same person who owns both servers agrees on both sides at once', async () => { + await reset(); + const both = await person({ discord: true, fluxer: true }); + owners({ discord: both.ids.discord, fluxer: both.ids.fluxer }); + const state = await (await both.get(`/guilds/${D_GUILD}/mirror`)).json(); + assert.deepEqual( + state.candidates.map((c) => c.id), + [F_KEY], + 'the Fluxer servers this person owns are offered' + ); + const res = await both.post(`/guilds/${D_GUILD}/mirror/request`, { + targetGuildId: F_KEY, + options: { mirrorDeletes: true }, + }); + const body = await res.json(); + assert.equal( + body.pair.options.mirrorDeletes, + true, + 'the owner of both servers can switch deletion on at once' + ); + assert.equal(body.code, null); + assert.equal(body.agreed, true); + assert.equal(body.pair.status, 'ready'); + assert.equal(body.pair.otherName, 'F-main'); + const pair = await db.getPairBySource(D_GUILD); + assert.deepEqual([pair.sourceConsentUser, pair.targetConsentUser], [both.ids.discord, both.ids.fluxer]); +}); + +test('an owner of one server who is only an Administrator of the other is not enough', async () => { + await reset(); + const mixed = await person({ discord: true, fluxer: true }); + owners({ discord: mixed.ids.discord, fluxer: 'someone-else' }); // owns Discord, is not the Fluxer owner + const state = await (await mixed.get(`/guilds/${D_GUILD}/mirror`)).json(); + assert.deepEqual(state.candidates, [], 'a Fluxer server they do not own is not offered'); + const res = await mixed.post(`/guilds/${D_GUILD}/mirror/request`, { + targetGuildId: F_KEY, + options: { mirrorDeletes: true }, + }); + const body = await res.json(); + assert.equal(res.status, 200); + assert.equal(body.agreed, false, 'no automatic agreement for a server they do not own'); + assert.deepEqual( + [body.pair.options.mirrorDeletes, body.pair.options.mirrorDeletesRequested], + [false, true], + 'deletion is only asked for: the Fluxer owner has to turn it on' + ); + assert.ok(body.code, 'the Fluxer owner has to agree with the code'); + assert.equal((await mixed.post(`/guilds/${F_KEY}/mirror/accept`, { code: body.code })).status, 403); + assert.equal((await db.getPairBySource(D_GUILD)).status, 'pending'); +}); + +test('an Administrator who is not the owner can look, but cannot ask, agree, change, start, pause, resume, stop or remove', async () => { + await reset(); + const owner = await person({ discord: true, fluxer: true }); + owners({ discord: owner.ids.discord, fluxer: owner.ids.fluxer }); + await owner.post(`/guilds/${D_GUILD}/mirror/request`, { targetGuildId: F_KEY }); + await owner.post(`/guilds/${D_GUILD}/mirror/start`, { confirm: true }); + + const admin = await person({ discord: true, admin: true }); + const sees = await admin.get(`/guilds/${D_GUILD}/mirror`); + assert.equal(sees.status, 200, 'they have the dashboard access, so they can look'); + const seen = await sees.json(); + assert.equal(seen.isOwner, false); + assert.equal(seen.pair.status, 'copying', 'the status is readable'); + assert.equal((await admin.get(`/guilds/${D_GUILD}/mirror/preview`)).status, 200); + assert.equal((await admin.get(`/guilds/${D_GUILD}/mirror/report`)).status, 200); + for (const [path, body] of [ + ['request', { targetGuildId: F_KEY }], + ['options', { continuous: false }], + ['deletes', { allow: true }], + ['start', { confirm: true }], + ['pause', {}], + ['resume', {}], + ['stop', {}], + ['remove', {}], + ]) { + const res = await admin.post(`/guilds/${D_GUILD}/mirror/${path}`, body); + assert.equal(res.status, 403, path); + assert.match((await res.json()).error, /Only the server owner can/, path); + } + const pair = await db.getPairBySource(D_GUILD); + assert.equal(pair.status, 'copying', 'nothing changed'); + assert.equal(pair.options.continuous, true); + // The owner of the Discord server cannot act on the Fluxer side either, and a stranger gets nothing there. + assert.equal((await owner.post(`/guilds/${F_KEY}/mirror/pause`)).status, 200, 'the owner of both can'); + const other = await person({ fluxer: true }); + assert.equal((await other.post(`/guilds/${F_KEY}/mirror/stop`)).status, 403); + assert.equal((await db.getPairBySource(D_GUILD)).status, 'paused'); +}); + +test('a server that changes hands pauses the mirror until its new owner confirms', async () => { + await reset(); + const first = await person({ discord: true, fluxer: true }); + owners({ discord: first.ids.discord, fluxer: first.ids.fluxer }); + await first.post(`/guilds/${D_GUILD}/mirror/request`, { targetGuildId: F_KEY }); + await first.post(`/guilds/${D_GUILD}/mirror/start`, { confirm: true }); + const pair = await db.getPairBySource(D_GUILD); + + // The Fluxer server is handed to someone else: the engine pauses at the next piece of work. + const next = await person({ fluxer: true }); + fGuild.ownerId = next.ids.fluxer; + const { mirror } = await import('../src/mirror/index.js'); + await mirror.runPair(pair.id); + const paused = await db.getPair(pair.id); + assert.deepEqual([paused.status, paused.pauseKind], ['paused', 'ownership']); + + // The old owner cannot resume it; the new one confirms and the mirror goes on. + assert.equal((await first.post(`/guilds/${F_KEY}/mirror/resume`)).status, 403); + const resumed = await next.post(`/guilds/${F_KEY}/mirror/resume`); + assert.equal(resumed.status, 200); + const after = await db.getPair(pair.id); + assert.equal(after.status, 'copying'); + assert.equal(after.targetConsentUser, next.ids.fluxer); +}); + +test('another server cannot be reached: not by its id, not by a pair, not by a code', async () => { + await reset(); + const anna = await person({ discord: true }); + const bo = await person({ fluxer: true }); + owners({ discord: anna.ids.discord, fluxer: bo.ids.fluxer }); + // The owner of D_GUILD has no right in OTHER_D_GUILD or in the Fluxer server. + for (const [path, body] of [ + [`/guilds/${OTHER_D_GUILD}/mirror/request`, { targetGuildId: F_GUILD }], + [`/guilds/${OTHER_D_GUILD}/mirror/options`, {}], + [`/guilds/${OTHER_D_GUILD}/mirror/start`, { confirm: true }], + [`/guilds/${F_KEY}/mirror/accept`, { code: 'AAAA-AAAA' }], + [`/guilds/${F_KEY}/mirror/stop`, {}], + ]) { + assert.equal((await anna.post(path, body)).status, 403, path); + } + for (const path of [ + `/guilds/${OTHER_D_GUILD}/mirror`, + `/guilds/${F_KEY}/mirror`, + `/guilds/${F_KEY}/mirror/report`, + ]) { + assert.equal((await anna.get(path)).status, 403, path); + } + assert.equal(await db.getPairBySource(OTHER_D_GUILD), null); + + const { code } = await ( + await anna.post(`/guilds/${D_GUILD}/mirror/request`, { targetGuildId: F_GUILD }) + ).json(); + assert.equal( + (await bo.post(`/guilds/${D_GUILD}/mirror/accept`, { code })).status, + 403, + 'the Fluxer owner has no right on the Discord server' + ); + // A code for another server finds nothing here. + await db.deletePair((await db.getPairBySource(D_GUILD)).id); + const other = await db.createPair({ + sourceGuild: OTHER_D_GUILD, + targetGuild: 'fluxer:900000000000000299', + status: 'pending', + }); + assert.equal( + (await bo.post(`/guilds/${F_KEY}/mirror/accept`, { code })).status, + 404, + 'no request waits for this Fluxer server' + ); + await db.deletePair(other.id); +}); + +test('asking is checked: a Fluxer server cannot ask, the target must be a Fluxer server Sylo is in', async () => { + await reset(); + const both = await person({ discord: true, fluxer: true }); + owners({ discord: both.ids.discord, fluxer: both.ids.fluxer }); + assert.equal((await both.post(`/guilds/${F_KEY}/mirror/request`, { targetGuildId: D_GUILD })).status, 400); + assert.equal( + (await both.post(`/guilds/${D_GUILD}/mirror/request`, { targetGuildId: D_GUILD })).status, + 404, + 'a Discord id read as a Fluxer id: Sylo is not there' + ); + assert.equal( + (await both.post(`/guilds/${D_GUILD}/mirror/request`, { targetGuildId: 'nonsense' })).status, + 400 + ); + assert.equal( + (await both.post(`/guilds/${D_GUILD}/mirror/request`, { targetGuildId: '900000000000000777' })).status, + 404, + 'Sylo is not there' + ); + assert.equal(await db.getPairBySource(D_GUILD), null); + assert.equal( + (await both.get(`/guilds/${F_KEY}/mirror/preview`)).status, + 404, + 'a Fluxer server has no preview' + ); +}); + +test('signed out and without a CSRF token, nothing works', async () => { + const b = own(); + assert.ok( + [401, 403].includes( + (await b.get(`/api/v2/guilds/${D_GUILD}/mirror`, { headers: { accept: 'application/json' } })).status + ) + ); + const anna = await person({ discord: true }); + owners({ discord: anna.ids.discord }); + const noToken = await own().get(`/api/v2/guilds/${D_GUILD}/mirror/request`, { + method: 'POST', + headers: { accept: 'application/json', 'content-type': 'application/json' }, + body: '{}', + }); + assert.ok([401, 403].includes(noToken.status)); + assert.equal((await anna.get(`/guilds/${D_GUILD}/mirror`)).status, 200); +}); diff --git a/web-v2/src/App.jsx b/web-v2/src/App.jsx index e824fb23..03fb66df 100644 --- a/web-v2/src/App.jsx +++ b/web-v2/src/App.jsx @@ -33,6 +33,7 @@ const ModerationHub = lazy(() => import('./pages/ModerationHub.jsx')); const AuditLog = lazy(() => import('./pages/AuditLog.jsx')); const Changelog = lazy(() => import('./pages/Changelog.jsx')); const SetupWizard = lazy(() => import('./pages/SetupWizard.jsx')); +const Mirror = lazy(() => import('./pages/Mirror.jsx')); const MemberData = lazy(() => import('./pages/MemberData.jsx')); const Shop = lazy(() => import('./pages/Shop.jsx')); const Apply = lazy(() => import('./pages/Apply.jsx')); @@ -68,6 +69,7 @@ const router = createBrowserRouter( } /> } /> } /> + } /> } /> } /> } /> diff --git a/web-v2/src/__tests__/Bridge.test.jsx b/web-v2/src/__tests__/Bridge.test.jsx new file mode 100644 index 00000000..f9f1e907 --- /dev/null +++ b/web-v2/src/__tests__/Bridge.test.jsx @@ -0,0 +1,358 @@ +import { fireEvent, screen, waitFor } from '@testing-library/react'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import Bridge from '../moduleForms/Bridge.jsx'; +import { renderAtGuild } from './routerHarness.jsx'; + +vi.mock('../api.js', () => ({ + ApiError: class ApiError extends Error {}, + getBridge: vi.fn(), + requestBridgeLink: vi.fn(), + cancelBridgeRequest: vi.fn(), + acceptBridgeLink: vi.fn(), + setBridgeOptions: vi.fn(), + bridgeLinkAction: vi.fn(), +})); +vi.mock('../notify.js', () => ({ notify: vi.fn() })); +vi.mock('../components/ConfirmDialog.jsx', () => ({ confirmDialog: vi.fn().mockResolvedValue(true) })); +const api = await import('../api.js'); +const { confirmDialog } = await import('../components/ConfirmDialog.jsx'); +const { notify } = await import('../notify.js'); + +const GUILD = '123456789012345678'; +const MAIN = '666666666666666661'; +const TWO = '666666666666666662'; + +const link = (over = {}) => ({ + id: 7, + status: 'active', + pausedByMe: false, + pausedByThem: false, + problem: null, + mine: { channelId: MAIN, platform: 'discord', server: 'Home', channel: 'general', ageRestricted: false }, + theirs: { platform: 'fluxer', server: 'Fluxer Home', channel: 'chat', ageRestricted: false }, + options: { bots: false, joinMessages: false, uploadFiles: true }, + theirOptions: { bots: true, joinMessages: false, uploadFiles: true }, + createdAt: Date.UTC(2030, 0, 2), + by: 'Alice (dashboard)', + queued: 0, + dropped: 0, + ...over, +}); +const overview = (over = {}) => ({ + platform: 'discord', + bothPlatforms: true, + enabled: true, + owner: true, + limits: { links: 10, requests: 5 }, + links: [], + requests: [], + channels: [ + { id: MAIN, name: 'general', ageRestricted: false, taken: false }, + { id: TWO, name: 'off-topic', ageRestricted: true, taken: false }, + ], + ...over, +}); + +beforeEach(() => { + vi.resetAllMocks(); + confirmDialog.mockResolvedValue(true); + api.getBridge.mockResolvedValue(overview()); +}); + +describe('Bridge page', () => { + it('says there is nothing bridged yet and offers the two ways to start', async () => { + renderAtGuild(, GUILD); + expect(await screen.findByRole('heading', { name: 'Bridge' })).toBeTruthy(); + expect(screen.getByRole('heading', { name: 'Bridged channels (0 of 10)' })).toBeTruthy(); + expect(screen.getByText(/No channel is bridged yet/)).toBeTruthy(); + expect(screen.getByRole('heading', { name: 'Ask the other server' })).toBeTruthy(); + expect(screen.getByRole('heading', { name: 'Use a code' })).toBeTruthy(); + expect(api.getBridge).toHaveBeenCalledWith(GUILD); + }); + + it('names the other platform, and explains that nothing pings and bots are off', async () => { + renderAtGuild(, GUILD); + await screen.findByRole('heading', { name: 'Bridge' }); + expect(screen.getAllByText(/Fluxer/).length).toBeGreaterThan(0); + expect(screen.getAllByText(/pings/).length).toBeGreaterThan(0); + }); + + it('does not offer anything when this Sylo is not on both platforms, or the module is off', async () => { + api.getBridge.mockResolvedValue(overview({ bothPlatforms: false })); + const { unmount } = renderAtGuild(, GUILD); + expect(await screen.findByText(/not connected to both Discord and Fluxer/)).toBeTruthy(); + expect(screen.getByRole('button', { name: 'Get a code' }).disabled).toBe(true); + unmount(); + api.getBridge.mockResolvedValue(overview({ enabled: false })); + renderAtGuild(, GUILD); + expect(await screen.findByText(/Turn the module on above/)).toBeTruthy(); + expect(screen.getByRole('button', { name: 'Link the channels' }).disabled).toBe(true); + }); + + it('is read-only for everyone but the server owner, with a note that says so', async () => { + api.getBridge.mockResolvedValue( + overview({ + owner: false, + links: [link({ pauseNote: 'The owner of one of the servers changed.' })], + requests: [ + { + id: 3, + channelId: MAIN, + channel: 'general', + expiresAt: Date.now() + 900_000, + requestedBy: 'Alice', + }, + ], + }) + ); + renderAtGuild(, GUILD); + expect(await screen.findByText(/Only the server owner can ask for a link/)).toBeTruthy(); + expect(screen.getByText(/#general ↔ #chat/)).toBeTruthy(); + expect(screen.getByText('The owner of one of the servers changed.')).toBeTruthy(); + expect(screen.getByRole('button', { name: 'Pause' }).disabled).toBe(true); + expect(screen.queryByRole('button', { name: 'More actions for #general' })).toBeNull(); + expect(screen.queryByRole('button', { name: 'More actions for the request of #general' })).toBeNull(); + expect(screen.getByRole('button', { name: 'Get a code' }).disabled).toBe(true); + expect(screen.getByRole('button', { name: 'Link the channels' }).disabled).toBe(true); + expect( + screen.getByLabelText('Share messages from bots', { selector: '#opt-7-bots' }).matches(':disabled') + ).toBe(true); + expect(screen.getByLabelText('Code').closest('fieldset').disabled).toBe(true); + }); + + it('lists a link with the other side, its status, and what each side shares', async () => { + api.getBridge.mockResolvedValue(overview({ links: [link({ queued: 3 })] })); + renderAtGuild(, GUILD); + expect(await screen.findByText(/#general ↔ #chat/)).toBeTruthy(); + expect(screen.getByText('Running')).toBeTruthy(); + expect(screen.getByText(/Fluxer Home on Fluxer/)).toBeTruthy(); + expect(screen.getByText(/3 waiting to send/)).toBeTruthy(); + expect(screen.getByText(/Bot messages: shared. Who joins: not shared./)).toBeTruthy(); + expect( + screen + .getByLabelText('Share messages from bots', { selector: '#opt-7-bots' }) + .getAttribute('aria-checked') + ).toBe('false'); + }); + + it('shows why a stopped link stopped, and who paused a paused one', async () => { + api.getBridge.mockResolvedValue( + overview({ + links: [ + link({ + id: 1, + status: 'failed', + problem: { code: 'permissions', text: 'Sylo is missing a permission here.' }, + }), + link({ + id: 2, + status: 'paused', + pausedByThem: true, + mine: { + channelId: TWO, + platform: 'discord', + server: 'Home', + channel: 'off-topic', + ageRestricted: true, + }, + }), + ], + }) + ); + renderAtGuild(, GUILD); + expect(await screen.findByText('Stopped')).toBeTruthy(); + expect(screen.getByText('Sylo is missing a permission here.')).toBeTruthy(); + expect(screen.getByText('Paused by the other server')).toBeTruthy(); + expect(screen.getByText(/Fluxer Home paused this bridge. Only they can resume it./)).toBeTruthy(); + // a stopped link can be resumed; one the other server paused cannot + expect(screen.getAllByRole('button', { name: 'Resume' })).toHaveLength(1); + }); + + it('pauses a link and shows the fresh state the server answers with', async () => { + api.getBridge.mockResolvedValue(overview({ links: [link()] })); + api.bridgeLinkAction.mockResolvedValue({ + overview: overview({ links: [link({ status: 'paused', pausedByMe: true })] }), + }); + renderAtGuild(, GUILD); + fireEvent.click(await screen.findByRole('button', { name: 'Pause' })); + await waitFor(() => expect(api.bridgeLinkAction).toHaveBeenCalledWith(GUILD, 7, 'pause')); + expect(await screen.findByText('Paused by this server')).toBeTruthy(); + expect(screen.getByRole('button', { name: 'Resume' })).toBeTruthy(); + }); + + it('a bridge the server mirror holds has no Resume button; one whose mirror ended has', async () => { + api.getBridge.mockResolvedValue( + overview({ + links: [ + link({ + id: 7, + status: 'paused', + resumable: false, + pauseNote: 'This bridge was made by the server mirror and is paused with it.', + }), + link({ + id: 8, + status: 'paused', + resumable: true, + mine: { + channelId: TWO, + platform: 'discord', + server: 'Home', + channel: 'rules', + ageRestricted: false, + }, + pauseNote: 'The server mirror that made this bridge ended.', + }), + ], + }) + ); + renderAtGuild(, GUILD); + expect(await screen.findByText(/paused with it/)).toBeTruthy(); + expect(screen.getAllByRole('button', { name: 'Resume' })).toHaveLength(1); + }); + + it('changes a switch of this side through the server', async () => { + api.getBridge.mockResolvedValue(overview({ links: [link()] })); + api.setBridgeOptions.mockResolvedValue({ + overview: overview({ + links: [link({ options: { bots: true, joinMessages: false, uploadFiles: true } })], + }), + }); + renderAtGuild(, GUILD); + fireEvent.click(await screen.findByLabelText('Share messages from bots', { selector: '#opt-7-bots' })); + await waitFor(() => expect(api.setBridgeOptions).toHaveBeenCalledWith(GUILD, 7, { bots: true })); + await waitFor(() => + expect( + screen + .getByLabelText('Share messages from bots', { selector: '#opt-7-bots' }) + .getAttribute('aria-checked') + ).toBe('true') + ); + }); + + it('asks before unlinking, and only unlinks when confirmed', async () => { + api.getBridge.mockResolvedValue(overview({ links: [link()] })); + api.bridgeLinkAction.mockResolvedValue({ overview: overview() }); + confirmDialog.mockResolvedValueOnce(false); + renderAtGuild(, GUILD); + const open = async () => { + fireEvent.click(await screen.findByRole('button', { name: 'More actions for #general' })); + fireEvent.click(screen.getByRole('menuitem', { name: 'Unlink' })); + }; + await open(); + await waitFor(() => expect(confirmDialog).toHaveBeenCalledTimes(1)); + expect(confirmDialog.mock.calls[0][0].title).toBe('Unlink #general?'); + expect(api.bridgeLinkAction).not.toHaveBeenCalled(); + await open(); + await waitFor(() => expect(api.bridgeLinkAction).toHaveBeenCalledWith(GUILD, 7, 'unlink')); + expect(await screen.findByText(/No channel is bridged yet/)).toBeTruthy(); + }); + + it('asks for a code, shows it once with its expiry and a copy button', async () => { + api.requestBridgeLink.mockResolvedValue({ + code: 'ABCD-EFGH-JKMN', + expiresAt: Date.now() + 900_000, + overview: overview({ + requests: [ + { + id: 3, + channelId: MAIN, + channel: 'general', + expiresAt: Date.now() + 900_000, + requestedBy: 'Alice', + }, + ], + channels: [ + { id: MAIN, name: 'general', ageRestricted: false, taken: true }, + { id: TWO, name: 'off-topic', ageRestricted: true, taken: false }, + ], + }), + }); + renderAtGuild(, GUILD); + const button = await screen.findByRole('button', { name: 'Get a code' }); + expect(button.disabled).toBe(true); + fireEvent.change(screen.getByLabelText('Channel here', { selector: '#bridge-ask-channel' }), { + target: { value: MAIN }, + }); + fireEvent.click(screen.getByLabelText('Share who joins', { selector: '#ask-joinMessages' })); + fireEvent.click(screen.getByRole('button', { name: 'Get a code' })); + await waitFor(() => + expect(api.requestBridgeLink).toHaveBeenCalledWith(GUILD, { + channelId: MAIN, + options: { bots: false, joinMessages: true, uploadFiles: true }, + }) + ); + expect(await screen.findByText('ABCD-EFGH-JKMN')).toBeTruthy(); + expect(screen.getByText(/works once/)).toBeTruthy(); + expect(screen.getByRole('heading', { name: 'Waiting for the other server' })).toBeTruthy(); + expect(screen.getByText(/code valid until/)).toBeTruthy(); + // the channel that has a request waiting can no longer be picked + const options = [ + ...screen + .getByLabelText('Channel here', { selector: '#bridge-ask-channel' }) + .querySelectorAll('option'), + ].map((o) => o.textContent); + expect(options).toEqual(['Pick a channel', '#off-topic (age-restricted)']); + }); + + it('uses a code with a channel and says so', async () => { + api.acceptBridgeLink.mockResolvedValue({ overview: overview({ links: [link()] }) }); + renderAtGuild(, GUILD); + await screen.findByRole('button', { name: 'Link the channels' }); + fireEvent.change(screen.getByLabelText('Code'), { target: { value: ' abcd-efgh-jkmn ' } }); + fireEvent.change(screen.getByLabelText('Channel here', { selector: '#bridge-use-channel' }), { + target: { value: TWO }, + }); + fireEvent.click(screen.getByRole('button', { name: 'Link the channels' })); + await waitFor(() => + expect(api.acceptBridgeLink).toHaveBeenCalledWith(GUILD, { + code: 'abcd-efgh-jkmn', + channelId: TWO, + options: { bots: false, joinMessages: false, uploadFiles: true }, + }) + ); + expect(await screen.findByText(/#general ↔ #chat/)).toBeTruthy(); + expect(notify).toHaveBeenCalledWith('The bridge is on.'); + }); + + it('shows the refusal of the server as it is', async () => { + const { ApiError } = await import('../api.js'); + api.acceptBridgeLink.mockRejectedValue( + new ApiError('That code is not valid, has expired or was already used.') + ); + renderAtGuild(, GUILD); + await screen.findByRole('button', { name: 'Link the channels' }); + fireEvent.change(screen.getByLabelText('Code'), { target: { value: 'WRONG' } }); + fireEvent.change(screen.getByLabelText('Channel here', { selector: '#bridge-use-channel' }), { + target: { value: MAIN }, + }); + fireEvent.click(screen.getByRole('button', { name: 'Link the channels' })); + await waitFor(() => + expect(notify).toHaveBeenCalledWith('That code is not valid, has expired or was already used.') + ); + }); + + it('cancels a waiting request after asking', async () => { + api.getBridge.mockResolvedValue( + overview({ + requests: [ + { + id: 3, + channelId: MAIN, + channel: 'general', + expiresAt: Date.now() + 900_000, + requestedBy: 'Alice', + }, + ], + }) + ); + api.cancelBridgeRequest.mockResolvedValue({ overview: overview() }); + renderAtGuild(, GUILD); + fireEvent.click(await screen.findByRole('button', { name: 'More actions for the request of #general' })); + fireEvent.click(screen.getByRole('menuitem', { name: 'Cancel request' })); + await waitFor(() => expect(api.cancelBridgeRequest).toHaveBeenCalledWith(GUILD, 3)); + await waitFor(() => + expect(screen.queryByRole('heading', { name: 'Waiting for the other server' })).toBeNull() + ); + }); +}); diff --git a/web-v2/src/__tests__/Mirror.test.jsx b/web-v2/src/__tests__/Mirror.test.jsx new file mode 100644 index 00000000..b10d9236 --- /dev/null +++ b/web-v2/src/__tests__/Mirror.test.jsx @@ -0,0 +1,452 @@ +import { fireEvent, screen, waitFor, within } from '@testing-library/react'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import Mirror from '../pages/Mirror.jsx'; +import { renderAtGuild } from './routerHarness.jsx'; + +vi.mock('../api.js', () => ({ + ApiError: class ApiError extends Error {}, + getMirror: vi.fn(), + allowMirrorDeletes: vi.fn(), + getMirrorPreview: vi.fn(), + getMirrorReport: vi.fn(), + requestMirror: vi.fn(), + acceptMirror: vi.fn(), + setMirrorOptions: vi.fn(), + startMirror: vi.fn(), + mirrorAction: vi.fn(), +})); +vi.mock('../notify.js', () => ({ notify: vi.fn() })); +vi.mock('../components/ConfirmDialog.jsx', () => ({ confirmDialog: vi.fn().mockResolvedValue(true) })); +const api = await import('../api.js'); +const { confirmDialog } = await import('../components/ConfirmDialog.jsx'); + +const GUILD = '123456789012345678'; +const FLUXER = 'fluxer:223456789012345678'; +const OPTIONS = { + name: true, + icon: true, + banner: true, + roles: true, + channels: true, + emoji: true, + stickers: true, + modules: true, + continuous: true, + mirrorDeletes: false, + mirrorDeletesRequested: false, +}; + +const pair = (over = {}) => ({ + id: 1, + side: 'source', + status: 'ready', + resumeStatus: '', + options: OPTIONS, + otherId: FLUXER, + otherName: 'Fluxer copy', + otherPresent: true, + requestedBy: 'Anna (dashboard)', + consent: { source: { by: 'Anna', at: 1 }, target: { by: 'Bo', at: 2 } }, + codeExpiresAt: 0, + pauseKind: '', + pausedBy: '', + lastError: '', + copiedAt: 0, + copyRun: 0, + ...over, +}); +const state = (over = {}) => ({ + platform: 'discord', + canMirror: true, + isOwner: true, + fluxerConnected: true, + pair: null, + progress: null, + candidates: [{ id: FLUXER, name: 'Fluxer copy' }], + defaults: OPTIONS, + limits: { emoji: 50, stickers: 50 }, + ...over, +}); +const progress = (over = {}) => ({ + run: 1, + total: 8, + done: 0, + skipped: 0, + failed: 0, + waiting: 8, + percent: 0, + waitingChanges: 0, + ...over, +}); +const fluxerState = (p) => state({ platform: 'fluxer', canMirror: false, candidates: [], pair: p }); + +beforeEach(() => { + vi.resetAllMocks(); + confirmDialog.mockResolvedValue(true); +}); + +describe('Mirror: the Discord server', () => { + it('asks for a mirror with the id of the Fluxer server, and shows the one-time code that comes back', async () => { + api.getMirror.mockResolvedValue(state()); + api.requestMirror.mockResolvedValue({ + ...state({ pair: pair({ status: 'pending', otherName: null, codeExpiresAt: Date.now() + 3_000_000 }) }), + code: 'K7QW-H2NP', + }); + renderAtGuild(, GUILD); + const button = await screen.findByRole('button', { name: 'Ask for the mirror' }); + const input = screen.getByLabelText('Fluxer server'); + fireEvent.change(input, { target: { value: 'nonsense' } }); + expect(button.disabled).toBe(true); + fireEvent.change(input, { target: { value: FLUXER } }); + expect(button.disabled).toBe(false); + fireEvent.click(button); + await waitFor(() => expect(api.requestMirror).toHaveBeenCalledWith(GUILD, FLUXER)); + expect(await screen.findByText('K7QW-H2NP')).toBeTruthy(); + expect(screen.getByRole('button', { name: /Ask again with a new code/ })).toBeTruthy(); + }); + + it('shows the preview, what cannot be carried over, and only copies after a confirmation', async () => { + api.getMirror.mockResolvedValue(state({ pair: pair() })); + api.getMirrorPreview.mockResolvedValue({ + ok: true, + blockers: [], + warnings: [{ code: 'target-not-empty', message: 'The Fluxer server already has 2 channel(s).' }], + summary: { + role: { create: 2, update: 1, unchanged: 0, skip: 1 }, + channel: { create: 5, update: 0, unchanged: 0, skip: 0 }, + }, + notes: [ + { + code: 'managed-role', + scope: 'role', + ref: { id: '1' }, + message: 'A bot: managed by an integration.', + }, + ], + target: { id: '2', name: 'Fluxer copy' }, + steps: [{ key: 'role:1', kind: 'role', name: 'Member', op: 'create', reason: null, notes: 0 }], + }); + api.startMirror.mockResolvedValue( + state({ pair: pair({ status: 'copying', copyRun: 1 }), progress: progress() }) + ); + renderAtGuild(, GUILD); + fireEvent.click(await screen.findByRole('button', { name: 'Show the preview' })); + expect(await screen.findByText('Create 2 roles, update 1, skip 1')).toBeTruthy(); + expect(screen.getByText('Create 5 channels')).toBeTruthy(); + expect(screen.getByText(/already has 2 channel/)).toBeTruthy(); + expect(screen.getByText(/Roles \(1\)/)).toBeTruthy(); + expect(api.startMirror).not.toHaveBeenCalled(); + fireEvent.click(screen.getByRole('button', { name: 'Copy to Fluxer copy' })); + await waitFor(() => expect(confirmDialog).toHaveBeenCalled()); + await waitFor(() => expect(api.startMirror).toHaveBeenCalledWith(GUILD)); + expect(await screen.findByText('0 of 8 steps done')).toBeTruthy(); + }); + + it('does not offer to copy while something blocks it', async () => { + api.getMirror.mockResolvedValue(state({ pair: pair() })); + api.getMirrorPreview.mockResolvedValue({ + ok: false, + blockers: [{ code: 'target-permissions', message: 'Sylo needs ManageRoles on the Fluxer server.' }], + warnings: [], + summary: {}, + notes: [], + target: null, + steps: [], + }); + renderAtGuild(, GUILD); + fireEvent.click(await screen.findByRole('button', { name: 'Show the preview' })); + expect(await screen.findByText(/Sylo needs ManageRoles/)).toBeTruthy(); + expect(screen.getByRole('button', { name: /Copy to/ }).disabled).toBe(true); + }); + + it('shows why a mirror is paused, and resumes it', async () => { + api.getMirror.mockResolvedValue( + state({ + pair: pair({ + status: 'paused', + resumeStatus: 'copying', + copyRun: 1, + lastError: 'Fluxer refused Sylo: Missing Permissions.', + }), + progress: progress({ total: 10, done: 3, failed: 1, waiting: 6, percent: 40 }), + }) + ); + api.mirrorAction.mockResolvedValue( + state({ pair: pair({ status: 'copying', copyRun: 1 }), progress: progress() }) + ); + renderAtGuild(, GUILD); + expect(await screen.findByText(/Fluxer refused Sylo: Missing Permissions/)).toBeTruthy(); + expect(screen.getByText('4 of 10 steps done, 1 failed')).toBeTruthy(); + fireEvent.click(screen.getByRole('button', { name: 'Resume' })); + await waitFor(() => expect(api.mirrorAction).toHaveBeenCalledWith(GUILD, 'resume', {})); + }); + + it('stops the mirror only after a confirmation', async () => { + api.getMirror.mockResolvedValue( + state({ + pair: pair({ status: 'live', copyRun: 1, copiedAt: Date.now() - 60_000 }), + progress: progress({ total: 4, done: 4, waiting: 0, percent: 100 }), + }) + ); + api.mirrorAction.mockResolvedValue(state({ pair: pair({ status: 'stopped', copyRun: 1 }) })); + confirmDialog.mockResolvedValueOnce(false).mockResolvedValueOnce(true); + renderAtGuild(, GUILD); + const stop = await screen.findByRole('button', { name: 'Stop' }); + fireEvent.click(stop); + await waitFor(() => expect(confirmDialog).toHaveBeenCalledTimes(1)); + expect(api.mirrorAction).not.toHaveBeenCalled(); + fireEvent.click(stop); + await waitFor(() => expect(api.mirrorAction).toHaveBeenCalledWith(GUILD, 'stop')); + expect(await screen.findByText(/The mirror was stopped/)).toBeTruthy(); + }); + + it('lists what failed and what could not be carried over in the report', async () => { + api.getMirror.mockResolvedValue( + state({ + pair: pair({ status: 'live', copyRun: 1 }), + progress: progress({ total: 2, done: 1, failed: 1, waiting: 0, percent: 100 }), + }) + ); + api.getMirrorReport.mockResolvedValue({ + run: 1, + items: [], + notes: [ + { + code: 'permissions-dropped', + scope: 'channel', + ref: { id: '9' }, + message: 'rules: Fluxer has no permission for Manage Events.', + }, + ], + failed: [{ kind: 'channel', sourceId: '9', name: 'rules', detail: 'Invalid Form Body' }], + bridge: { status: 'done', detail: 'not linked: bridge-not-available' }, + summary: {}, + }); + renderAtGuild(, GUILD); + fireEvent.click(await screen.findByRole('button', { name: 'Show the report' })); + expect(await screen.findByText('rules: Invalid Form Body')).toBeTruthy(); + expect(screen.getByText(/bridge-not-available/)).toBeTruthy(); + }); +}); + +describe('Mirror: the Fluxer server', () => { + it('explains how to start when there is no mirror', async () => { + api.getMirror.mockResolvedValue(fluxerState(null)); + renderAtGuild(, FLUXER); + expect(await screen.findByText(/is not the mirror of a Discord server/)).toBeTruthy(); + }); + + it('shows who asks and agrees with the code', async () => { + const asked = pair({ side: 'target', status: 'pending', otherName: 'Home Discord', otherId: GUILD }); + api.getMirror.mockResolvedValue(fluxerState(asked)); + api.acceptMirror.mockResolvedValue(fluxerState({ ...asked, status: 'ready' })); + renderAtGuild(, FLUXER); + expect(await screen.findByText('Home Discord', { selector: 'strong' })).toBeTruthy(); + const agree = screen.getByRole('button', { name: 'Agree to the mirror' }); + expect(agree.disabled).toBe(true); + fireEvent.change(screen.getByLabelText('Code'), { target: { value: 'K7QW-H2NP' } }); + fireEvent.click(agree); + await waitFor(() => expect(api.acceptMirror).toHaveBeenCalledWith(FLUXER, 'K7QW-H2NP')); + expect(await screen.findByText('Agreed, not copied yet')).toBeTruthy(); + }); + + it('lets this side pause a running mirror', async () => { + const live = pair({ side: 'target', status: 'live', otherName: 'Home Discord', otherId: GUILD }); + api.getMirror.mockResolvedValue(fluxerState(live)); + api.mirrorAction.mockResolvedValue(fluxerState({ ...live, status: 'paused', resumeStatus: 'live' })); + renderAtGuild(, FLUXER); + fireEvent.click(await screen.findByRole('button', { name: 'Pause' })); + await waitFor(() => expect(api.mirrorAction).toHaveBeenCalledWith(FLUXER, 'pause')); + expect(await screen.findByRole('button', { name: 'Resume' })).toBeTruthy(); + const heading = screen.getByRole('heading', { level: 2 }); + expect(within(heading).getByText('Paused')).toBeTruthy(); + }); +}); + +describe('Mirror: only the owner, and the pauses', () => { + it('lets people without ownership look, but not change anything', async () => { + api.getMirror.mockResolvedValue(state({ isOwner: false, candidates: [] })); + renderAtGuild(, GUILD); + expect(await screen.findByText(/Only the server owner can do this/)).toBeTruthy(); + expect(screen.getByRole('button', { name: 'Ask for the mirror' }).matches(':disabled')).toBe(true); + expect(screen.getByLabelText('Fluxer server').matches(':disabled')).toBe(true); + }); + + it('shows the status and the stop button as locked for a non-owner on a running mirror', async () => { + api.getMirror.mockResolvedValue( + state({ + isOwner: false, + pair: pair({ status: 'live', copyRun: 1 }), + progress: progress({ total: 2, done: 2, waiting: 0, percent: 100 }), + }) + ); + renderAtGuild(, GUILD); + const stop = await screen.findByRole('button', { name: 'Stop' }); + expect(stop.matches(':disabled')).toBe(true); + expect(screen.getByText('Mirroring')).toBeTruthy(); + }); + + it('asks the owner to confirm a burst of deletions, with the list, before they are applied', async () => { + api.getMirror.mockResolvedValue( + state({ + pair: pair({ + status: 'paused', + pauseKind: 'deletes', + pausedBy: 'system', + copyRun: 1, + lastError: '6 things were deleted on Discord within 10 minutes.', + }), + progress: progress({ total: 2, done: 2, waiting: 0, percent: 100 }), + }) + ); + const needs = () => + Object.assign(new api.ApiError('Confirm the deletions first.'), { + code: 'confirm-needed', + deleteCount: 6, + deletes: [{ kind: 'channel', name: 'chat', action: 'lock' }], + }); + api.mirrorAction + .mockRejectedValueOnce(needs()) + .mockRejectedValueOnce(needs()) + .mockResolvedValueOnce(state({ pair: pair({ status: 'live', copyRun: 1 }) })); + confirmDialog.mockResolvedValueOnce(false).mockResolvedValueOnce(true); + renderAtGuild(, GUILD); + const review = await screen.findByRole('button', { name: 'Review the deletions' }); + fireEvent.click(review); + await waitFor(() => expect(confirmDialog).toHaveBeenCalledTimes(1)); + expect(confirmDialog.mock.calls[0][0].body).toMatch(/lock channel chat, and 5 more/); + expect(api.mirrorAction).toHaveBeenCalledTimes(1); + fireEvent.click(review); + await waitFor(() => + expect(api.mirrorAction).toHaveBeenLastCalledWith(GUILD, 'resume', { confirm: true }) + ); + }); + + it('asks a new owner to confirm the mirror after a server changed hands', async () => { + api.getMirror.mockResolvedValue( + state({ + pair: pair({ + status: 'paused', + pauseKind: 'ownership', + pausedBy: 'system', + copyRun: 1, + lastError: 'The owner of the Fluxer server is not the person who agreed to the mirror.', + }), + progress: progress({ total: 2, done: 2, waiting: 0, percent: 100 }), + }) + ); + renderAtGuild(, GUILD); + expect(await screen.findByRole('button', { name: 'Confirm the mirror as the owner' })).toBeTruthy(); + }); + + it('does not let the other side resume a mirror the first owner paused', async () => { + api.getMirror.mockResolvedValue( + fluxerState( + pair({ + side: 'target', + status: 'paused', + pauseKind: 'owner', + pausedBy: 'source', + otherName: 'Home Discord', + otherId: GUILD, + }) + ) + ); + renderAtGuild(, FLUXER); + const resume = await screen.findByRole('button', { name: 'Resume' }); + expect(resume.disabled).toBe(true); + expect(screen.getByText(/paused this mirror, so only that owner can resume it/)).toBeTruthy(); + }); + + it('lists what was removed on Discord and kept, locked, on Fluxer', async () => { + api.getMirror.mockResolvedValue( + state({ + pair: pair({ status: 'live', copyRun: 1 }), + progress: { + ...progress({ total: 2, done: 2, waiting: 0, percent: 100 }), + removed: [{ kind: 'channel', label: 'old-chat', at: 1 }], + }, + }) + ); + renderAtGuild(, GUILD); + expect(await screen.findByText('Removed on Discord (1)')).toBeTruthy(); + expect(screen.getByText('channel: old-chat')).toBeTruthy(); + }); + + it('warns about deleting before the option is turned on', async () => { + api.getMirror.mockResolvedValue(state({ pair: pair() })); + renderAtGuild(, GUILD); + expect(await screen.findByText(/it is deleted on Fluxer too, with all its messages/)).toBeTruthy(); + expect(screen.getByText('Delete on Fluxer what is deleted on Discord')).toBeTruthy(); + }); +}); + +describe('Mirror: real deletion needs the Fluxer owner', () => { + it('shows the Discord owner that the request waits for the Fluxer owner', async () => { + api.getMirror.mockResolvedValue( + state({ pair: pair({ options: { ...OPTIONS, mirrorDeletesRequested: true } }) }) + ); + renderAtGuild(, GUILD); + expect(await screen.findByText(/Waiting for the owner of the Fluxer server to allow it/)).toBeTruthy(); + }); + + it('lets the Fluxer owner allow deleting after a confirmation that says what it does', async () => { + const asked = pair({ + side: 'target', + status: 'live', + otherName: 'Home Discord', + otherId: GUILD, + options: { ...OPTIONS, mirrorDeletesRequested: true }, + }); + api.getMirror.mockResolvedValue(fluxerState(asked)); + api.allowMirrorDeletes.mockResolvedValue( + fluxerState({ ...asked, options: { ...asked.options, mirrorDeletes: true } }) + ); + confirmDialog.mockResolvedValueOnce(false).mockResolvedValueOnce(true); + renderAtGuild(, FLUXER); + const allow = await screen.findByRole('button', { name: 'Allow deleting' }); + fireEvent.click(allow); + await waitFor(() => expect(confirmDialog).toHaveBeenCalledTimes(1)); + expect(confirmDialog.mock.calls[0][0].body).toMatch(/loses all its messages/); + expect(api.allowMirrorDeletes).not.toHaveBeenCalled(); + fireEvent.click(allow); + await waitFor(() => expect(api.allowMirrorDeletes).toHaveBeenCalledWith(FLUXER, true)); + expect(await screen.findByRole('button', { name: 'Stop deleting here' })).toBeTruthy(); + }); + + it('shows what will be deleted before deletions that waited are applied', async () => { + api.getMirror.mockResolvedValue( + state({ + pair: pair({ + status: 'paused', + pauseKind: 'owner', + pausedBy: 'system', + copyRun: 1, + lastError: '', + }), + progress: progress({ total: 2, done: 2, waiting: 0, percent: 100 }), + }) + ); + const { ApiError } = api; + api.mirrorAction + .mockRejectedValueOnce( + Object.assign(new ApiError('Confirm the deletions first.'), { + code: 'confirm-needed', + deleteCount: 7, + deletes: [ + { kind: 'channel', name: 'chat', action: 'delete' }, + { kind: 'role', name: 'Mod', action: 'lock' }, + ], + }) + ) + .mockResolvedValueOnce(state({ pair: pair({ status: 'live', copyRun: 1 }) })); + confirmDialog.mockResolvedValueOnce(true); + renderAtGuild(, GUILD); + fireEvent.click(await screen.findByRole('button', { name: 'Resume' })); + await waitFor(() => expect(confirmDialog).toHaveBeenCalledTimes(1)); + expect(confirmDialog.mock.calls[0][0].body).toMatch(/delete channel chat, lock role Mod, and 5 more/); + await waitFor(() => + expect(api.mirrorAction).toHaveBeenLastCalledWith(GUILD, 'resume', { confirm: true }) + ); + }); +}); diff --git a/web-v2/src/__tests__/mirrorModel.test.js b/web-v2/src/__tests__/mirrorModel.test.js new file mode 100644 index 00000000..5a746de5 --- /dev/null +++ b/web-v2/src/__tests__/mirrorModel.test.js @@ -0,0 +1,108 @@ +import { describe, expect, it } from 'vitest'; +import { + canResume, + codeLeft, + deleteSummary, + groupNotes, + isBusy, + progressText, + statusOf, + stepOf, + summaryLines, + validTargetId, +} from '../mirrorModel.js'; + +describe('mirrorModel', () => { + it('puts a pair at the step of the wizard its state belongs to', () => { + expect(stepOf(null)).toBe('ask'); + expect(stepOf({ status: 'pending' })).toBe('agree'); + expect(stepOf({ status: 'ready' })).toBe('preview'); + for (const status of ['copying', 'live', 'done', 'paused', 'stopped']) { + expect(stepOf({ status })).toBe('copy'); + } + }); + + it('names every state the server uses, and an unknown one by its own word', () => { + for (const status of ['pending', 'ready', 'copying', 'live', 'done', 'paused', 'stopped']) { + expect(statusOf(status).label).not.toBe(status); + } + expect(statusOf('paused').tone).toBe('warning'); + expect(statusOf('weird')).toEqual({ label: 'weird', tone: 'neutral' }); + }); + + it('keeps asking for progress while a copy runs or changes wait', () => { + expect(isBusy({ status: 'copying' }, null)).toBe(true); + expect(isBusy({ status: 'live' }, { waitingChanges: 2 })).toBe(true); + expect(isBusy({ status: 'live' }, { waitingChanges: 0 })).toBe(false); + expect(isBusy({ status: 'paused' }, { waitingChanges: 2 })).toBe(false); + expect(isBusy(null, null)).toBe(false); + }); + + it('sums the preview up in one line per kind', () => { + const lines = summaryLines({ + role: { create: 2, update: 1, unchanged: 0, skip: 1 }, + channel: { create: 1, update: 0, unchanged: 4, skip: 0 }, + emoji: { create: 0, update: 0, unchanged: 0, skip: 0 }, + sticker: { create: 0, update: 2, unchanged: 0, skip: 0 }, + }); + expect(lines).toEqual([ + { kind: 'role', text: 'Create 2 roles, update 1, skip 1' }, + { kind: 'channel', text: 'Create 1 channel, 4 already match' }, + { kind: 'sticker', text: 'Update 2 stickers' }, + ]); + expect(summaryLines(undefined)).toEqual([]); + }); + + it('groups the notes by what they are about, roles first', () => { + const groups = groupNotes([ + { scope: 'channel', message: 'a' }, + { scope: 'role', message: 'b' }, + { scope: 'channel', message: 'c' }, + ]); + expect(groups.map((g) => [g.title, g.items.length])).toEqual([ + ['Roles', 1], + ['Channels and permissions', 2], + ]); + }); + + it('says how far a copy is', () => { + expect(progressText(null)).toBe('Nothing queued.'); + expect(progressText({ total: 0 })).toBe('Nothing queued.'); + expect(progressText({ total: 40, waiting: 28, failed: 1, skipped: 2 })).toBe( + '12 of 40 steps done, 1 failed, 2 skipped' + ); + }); + + it('counts down a code, and says nothing once it ran out', () => { + const now = 1_000_000; + expect(codeLeft(now + 41 * 60_000 - 5, now)).toBe('41 minutes'); + expect(codeLeft(now + 30_000, now)).toBe('1 minute'); + expect(codeLeft(now - 1, now)).toBeNull(); + expect(codeLeft(0, now)).toBeNull(); + }); + + it('accepts a Fluxer server id as a number or as fluxer:', () => { + expect(validTargetId('123456789012345678')).toBe(true); + expect(validTargetId(' fluxer:123456789012345678 ')).toBe(true); + expect(validTargetId('12345')).toBe(false); + expect(validTargetId('discord:123456789012345678')).toBe(false); + expect(validTargetId('')).toBe(false); + }); + + it('lets only the side that paused a mirror resume it, and either side after a pause Sylo made', () => { + expect(canResume({ side: 'source', pausedBy: '' })).toBe(true); + expect(canResume({ side: 'source', pausedBy: 'system' })).toBe(true); + expect(canResume({ side: 'source', pausedBy: 'source' })).toBe(true); + expect(canResume({ side: 'target', pausedBy: 'source' })).toBe(false); + }); + + it('says what a confirmation is about: the first deletions, then how many more', () => { + const items = [ + { kind: 'channel', name: 'chat', action: 'delete' }, + { kind: 'role', name: 'Mod', action: 'lock' }, + ]; + expect(deleteSummary(items)).toBe('delete channel chat, lock role Mod'); + expect(deleteSummary(items, 7)).toBe('delete channel chat, lock role Mod, and 5 more'); + expect(deleteSummary([], 0)).toBe(''); + }); +}); diff --git a/web-v2/src/__tests__/overviewModel.test.js b/web-v2/src/__tests__/overviewModel.test.js index 7eb92c6c..97ca8b9c 100644 --- a/web-v2/src/__tests__/overviewModel.test.js +++ b/web-v2/src/__tests__/overviewModel.test.js @@ -182,7 +182,7 @@ describe('navModel', () => { 'Leaderboard', 'Audit log', ]); - expect(nav.server.map((i) => i.label)).toEqual(['Server settings', 'Commands']); + expect(nav.server.map((i) => i.label)).toEqual(['Server settings', 'Commands', 'Mirror to Fluxer']); }); it('opens the Applications entry on the queue when applications wait, and on the page default when none do', () => { diff --git a/web-v2/src/api.js b/web-v2/src/api.js index e93b45e1..bf6a2736 100644 --- a/web-v2/src/api.js +++ b/web-v2/src/api.js @@ -15,6 +15,9 @@ export class ApiError extends Error { // A refusal the page can act on (the settings changed since an audit entry: confirm to undo anyway). this.code = body?.code; this.laterChanges = body?.laterChanges; + // What a mirror resume asks to confirm: the deletions that wait (src/web/routes/v2Mirror.js). + this.deletes = body?.deletes; + this.deleteCount = body?.deleteCount; this.token = body?.token; } } @@ -157,6 +160,19 @@ export const moveSuggestionBack = (guildId, number) => // { moved, remaining, failed, next, error? }; ask again with after = next while remaining > 0. export const moveAllSuggestionsBack = (guildId, after = 0) => postJson(`/api/v2/guilds/${gid(guildId)}/modules/suggestions/move-all-back`, { after }); +// Bridge: the links of a server and what the page needs (GET), then the calls that change them. Every +// answer to a change carries the fresh `overview`. +export const getBridge = (guildId) => apiFetch(`/api/v2/guilds/${gid(guildId)}/modules/bridge/config`); +export const requestBridgeLink = (guildId, body) => + postJson(`/api/v2/guilds/${gid(guildId)}/modules/bridge/request`, body); +export const cancelBridgeRequest = (guildId, id) => + postJson(`/api/v2/guilds/${gid(guildId)}/modules/bridge/requests/${Number(id)}/cancel`, {}); +export const acceptBridgeLink = (guildId, body) => + postJson(`/api/v2/guilds/${gid(guildId)}/modules/bridge/accept`, body); +export const setBridgeOptions = (guildId, id, options) => + postJson(`/api/v2/guilds/${gid(guildId)}/modules/bridge/links/${Number(id)}/options`, { options }); +export const bridgeLinkAction = (guildId, id, action) => + postJson(`/api/v2/guilds/${gid(guildId)}/modules/bridge/links/${Number(id)}/${action}`, {}); export const getQuotes = (guildId, { q = '' } = {}) => apiFetch(`/api/v2/guilds/${gid(guildId)}/modules/quote-board/list?q=${encodeURIComponent(q)}`); export const deleteQuote = (guildId, number) => @@ -478,6 +494,27 @@ export const duplicateShopItem = (guildId, id) => duplicate(guildId, `modules/ec // The setup wizard: what Sylo proposes for this server, and applying the ticked items. export const getSetupSuggestion = (guildId) => apiFetch(`/api/v2/guilds/${gid(guildId)}/setup/suggest`); export const applySetup = (guildId, body) => postJson(`/api/v2/guilds/${gid(guildId)}/setup/apply`, body); +// Mirror to Fluxer (src/mirror/): the state of this server's mirror, the dry run and the report, and the +// actions: ask, agree with the code, change options, start the copy, pause, resume, stop and remove. +export const getMirror = (guildId) => apiFetch(`/api/v2/guilds/${gid(guildId)}/mirror`); +export const getMirrorPreview = (guildId) => apiFetch(`/api/v2/guilds/${gid(guildId)}/mirror/preview`); +export const getMirrorReport = (guildId) => apiFetch(`/api/v2/guilds/${gid(guildId)}/mirror/report`); +export const requestMirror = (guildId, targetGuildId, options) => + postJson(`/api/v2/guilds/${gid(guildId)}/mirror/request`, { + targetGuildId, + ...(options ? { options } : {}), + }); +export const acceptMirror = (guildId, code) => + postJson(`/api/v2/guilds/${gid(guildId)}/mirror/accept`, { code }); +export const setMirrorOptions = (guildId, options) => + postJson(`/api/v2/guilds/${gid(guildId)}/mirror/options`, options); +// The owner of the Fluxer server allows, or stops, real deletion (the Discord owner can only ask for it). +export const allowMirrorDeletes = (guildId, allow) => + postJson(`/api/v2/guilds/${gid(guildId)}/mirror/deletes`, { allow }); +export const startMirror = (guildId) => + postJson(`/api/v2/guilds/${gid(guildId)}/mirror/start`, { confirm: true }); +export const mirrorAction = (guildId, action, body = {}) => + postJson(`/api/v2/guilds/${gid(guildId)}/mirror/${action}`, body); export const getMemberData = (guildId, user) => apiFetch(`/api/v2/guilds/${gid(guildId)}/member-data?user=${encodeURIComponent(user ?? '')}`); export const forgetMemberData = (guildId, userId, reason) => diff --git a/web-v2/src/components/icons.jsx b/web-v2/src/components/icons.jsx index 9db2147c..36e810cf 100644 --- a/web-v2/src/components/icons.jsx +++ b/web-v2/src/components/icons.jsx @@ -185,6 +185,7 @@ export const ICONS = { ), chevron: , + copy: , sparkles: ( ), diff --git a/web-v2/src/main.jsx b/web-v2/src/main.jsx index e97b7ce7..f8059e25 100644 --- a/web-v2/src/main.jsx +++ b/web-v2/src/main.jsx @@ -19,9 +19,11 @@ import './styles/wp4-inbox.css'; import './styles/wp5-lists.css'; import './styles/p6-import.css'; import './styles/setup-wizard.css'; +import './styles/mirror.css'; import './styles/access.css'; import './styles/history.css'; import './styles/health.css'; +import './styles/bridge.css'; createRoot(document.getElementById('root')).render( diff --git a/web-v2/src/mirrorModel.js b/web-v2/src/mirrorModel.js new file mode 100644 index 00000000..873f1998 --- /dev/null +++ b/web-v2/src/mirrorModel.js @@ -0,0 +1,139 @@ +// What the Mirror to Fluxer page says about a mirror (pages/Mirror.jsx), kept apart from the markup so the +// wording and the decisions can be tested: the state of a pair in words, which steps of the wizard are done, how +// far a copy is, and how the preview is summed up. + +/** The pair states the server uses (src/db/mirror.js), in plain words and with the tone of their pill. */ +export const STATUS = { + pending: { label: 'Waiting for the Fluxer server', tone: 'warning' }, + ready: { label: 'Agreed, not copied yet', tone: 'neutral' }, + copying: { label: 'Copying', tone: 'success' }, + live: { label: 'Mirroring', tone: 'success' }, + done: { label: 'Copied', tone: 'success' }, + paused: { label: 'Paused', tone: 'warning' }, + stopped: { label: 'Stopped', tone: 'neutral' }, +}; + +export const statusOf = (status) => STATUS[status] ?? { label: status ?? 'Unknown', tone: 'neutral' }; + +/** The wizard's steps on the Discord side, and which one a pair is at. */ +export const STEPS = [ + ['ask', 'Ask'], + ['agree', 'Agree'], + ['preview', 'Preview'], + ['copy', 'Copy'], +]; + +export function stepOf(pair) { + if (!pair) return 'ask'; + if (pair.status === 'pending') return 'agree'; + if (pair.status === 'ready') return 'preview'; + return 'copy'; +} + +/** Is a copy or a sync running, so the page should keep asking for progress? */ +export const isBusy = (pair, progress) => + pair?.status === 'copying' || (pair?.status === 'live' && (progress?.waitingChanges ?? 0) > 0); + +const KIND = { + guild: ['the server name, icon and banner', 'the server name, icon and banner'], + role: ['role', 'roles'], + channel: ['channel', 'channels'], + emoji: ['emoji', 'emoji'], + sticker: ['sticker', 'stickers'], + module: ['module setup', 'module setups'], +}; + +/** Singular or plural by count: plural(3, 'role', 'roles') is "3 roles". */ +const count = (n, one, many) => `${n} ${n === 1 ? one : many}`; + +/** The preview's summary as one line per kind: "Create 5 channels, update 1, skip 2". */ +export function summaryLines(summary) { + const lines = []; + for (const [kind, names] of Object.entries(KIND)) { + const s = summary?.[kind]; + if (!s) continue; + const parts = []; + if (s.create) parts.push(`create ${count(s.create, names[0], names[1])}`); + if (s.update) + parts.push(s.create ? `update ${s.update}` : `update ${count(s.update, names[0], names[1])}`); + if (s.skip) parts.push(`skip ${s.skip}`); + if (s.unchanged) parts.push(`${s.unchanged} already match`); + if (parts.length) lines.push({ kind, text: parts.join(', ').replace(/^./, (c) => c.toUpperCase()) }); + } + return lines; +} + +/** Notes grouped by what they are about, so a long list reads as a few sections. */ +export function groupNotes(notes) { + const groups = new Map(); + for (const n of notes ?? []) { + if (!groups.has(n.scope)) groups.set(n.scope, []); + groups.get(n.scope).push(n); + } + const order = ['role', 'channel', 'emoji', 'sticker', 'module', 'guild']; + return [...groups] + .sort(([a], [b]) => order.indexOf(a) - order.indexOf(b)) + .map(([scope, items]) => ({ scope, title: SCOPE_TITLE[scope] ?? scope, items })); +} + +const SCOPE_TITLE = { + role: 'Roles', + channel: 'Channels and permissions', + emoji: 'Emoji', + sticker: 'Stickers', + module: "Sylo's settings", + guild: 'The server itself', +}; + +/** The progress line under the bar: "12 of 40 steps done, 1 failed". */ +export function progressText(progress) { + if (!progress || progress.total === 0) return 'Nothing queued.'; + const finished = progress.total - progress.waiting; + const parts = [`${finished} of ${progress.total} steps done`]; + if (progress.failed) parts.push(`${progress.failed} failed`); + if (progress.skipped) parts.push(`${progress.skipped} skipped`); + return parts.join(', '); +} + +/** What the options are called on the page, in order. `copy` ones can still change after a copy. */ +export const OPTION_ROWS = [ + ['name', 'Server name'], + ['icon', 'Server icon'], + ['banner', 'Server banner'], + ['roles', 'Roles'], + ['channels', 'Channels and categories'], + ['emoji', 'Emoji'], + ['stickers', 'Stickers'], + ['modules', "Sylo's own settings (welcome, tickets, moderation and the rest)"], + ['continuous', 'Keep the Fluxer server in step afterwards'], + ['mirrorDeletes', 'Delete on Fluxer what is deleted on Discord'], +]; + +/** The deletions that wait, as a short text for a confirmation: "lock channel #chat, delete role Mod, and 3 more". */ +export function deleteSummary(items = [], count = items.length) { + const verb = { lock: 'lock', delete: 'delete' }; + const shown = items.slice(0, 10).map((i) => `${verb[i.action] ?? i.action} ${i.kind} ${i.name}`); + const more = count - shown.length; + return shown.join(', ') + (more > 0 ? `, and ${more} more` : ''); +} + +/** What sits under an option row, when it needs a word of warning. */ +export const OPTION_HINTS = { + mirrorDeletes: + 'Off (the default): a channel or role deleted on Discord stays on Fluxer, locked and marked as removed, and its messages are kept. On: it is deleted on Fluxer too, with all its messages. Only what Sylo made is ever deleted. Turning it on only asks: the owner of the Fluxer server has to allow it in their dashboard.', +}; + +/** May this side resume a paused mirror? Only the side whose owner paused it can; a pause Sylo made is open to both. */ +export const canResume = (pair) => + !pair.pausedBy || pair.pausedBy === 'system' || pair.pausedBy === pair.side; + +/** Time left of a code, as "42 minutes", or null once it has run out. */ +export function codeLeft(expiresAt, now = Date.now()) { + const ms = expiresAt - now; + if (!(ms > 0)) return null; + const minutes = Math.ceil(ms / 60_000); + return count(minutes, 'minute', 'minutes'); +} + +/** The id the person typed or picked as the Fluxer server: a raw id or "fluxer:" is accepted. */ +export const validTargetId = (text) => /^(fluxer:)?\d{17,20}$/.test(String(text ?? '').trim()); diff --git a/web-v2/src/moduleForms/Bridge.jsx b/web-v2/src/moduleForms/Bridge.jsx new file mode 100644 index 00000000..6e87bc89 --- /dev/null +++ b/web-v2/src/moduleForms/Bridge.jsx @@ -0,0 +1,426 @@ +import { useState } from 'react'; +import { useParams } from 'react-router-dom'; +import { + acceptBridgeLink, + bridgeLinkAction, + cancelBridgeRequest, + getBridge, + requestBridgeLink, + setBridgeOptions, +} from '../api.js'; +import { notify } from '../notify.js'; +import { useApiData } from '../useApiData.js'; +import LoadError from '../components/LoadError.jsx'; +import HowItWorks from '../components/HowItWorks.jsx'; +import CopyButton from '../components/CopyButton.jsx'; +import EmptyList from '../components/EmptyList.jsx'; +import RowActions from '../components/RowActions.jsx'; +import Switch from '../components/Switch.jsx'; +import Meta from '../components/Meta.jsx'; +import { confirmDialog } from '../components/ConfirmDialog.jsx'; +import { formatDateTime } from '../formatTime.js'; + +const PLATFORM = { discord: 'Discord', fluxer: 'Fluxer' }; +// The code is XXXX-XXXX-XXXX; a pasted one may carry spaces. +const CODE_FIELD_LENGTH = 24; + +const OPTION_ROWS = [ + { + key: 'bots', + label: 'Share messages from bots', + hint: 'Off by default. Messages from other bot accounts of this server cross. Webhook messages (what bridges post) and Sylo itself never cross.', + }, + { + key: 'joinMessages', + label: 'Share who joins', + hint: 'Off by default. A short "joined" line appears on the other side when someone joins this server.', + }, + { + key: 'uploadFiles', + label: 'Copy files across', + hint: 'Files up to 8 MB are re-uploaded to the other side; bigger ones, or all of them when this is off, cross as a link.', + }, +]; + +function statusOf(link) { + if (link.status === 'active') return { tone: 'success', text: 'Running' }; + if (link.status === 'paused') { + return { + tone: 'warning', + text: link.pausedByMe ? 'Paused by this server' : 'Paused by the other server', + }; + } + return { tone: 'danger', text: 'Stopped' }; +} + +/** One channel picker: the text channels of this server that are not in a link or a request yet. */ +function ChannelSelect({ id, channels, value, onChange }) { + const free = channels.filter((c) => !c.taken); + return ( + + ); +} + +/** The two switches of a side that are off until asked for, as a small group. */ +function OptionSwitches({ options, onChange, idPrefix, disabled }) { + return ( +
+ {OPTION_ROWS.map((o) => ( +
+ onChange({ [o.key]: on })} + label={o.label} + disabled={disabled} + /> + + + {o.hint} + +
+ ))} +
+ ); +} + +const DEFAULT_OPTIONS = { bots: false, joinMessages: false, uploadFiles: true }; + +export default function Bridge() { + const { guildId } = useParams(); + const { data, dataKey, loading, error, setData } = useApiData(() => getBridge(guildId), [guildId], guildId); + const [busy, setBusy] = useState(false); + // Ask the other server + const [askChannel, setAskChannel] = useState(''); + const [askOptions, setAskOptions] = useState(DEFAULT_OPTIONS); + const [issued, setIssued] = useState(null); // { code, expiresAt } + // Use a code + const [code, setCode] = useState(''); + const [useChannel, setUseChannel] = useState(''); + const [useOptions, setUseOptions] = useState(DEFAULT_OPTIONS); + + if (error) return ; + // The previous server's page stays in `data` while the next one loads: never show or change it as this one. + if (!data || dataKey !== guildId) return loading ?

Loading…

: null; + + const other = data.platform === 'fluxer' ? 'Discord' : 'Fluxer'; + const here = PLATFORM[data.platform]; + const ready = data.bothPlatforms && data.enabled; + // Only the owner of the server can change a bridge; everyone with View access reads the status. + const owner = data.owner === true; + + async function run(fn, done) { + setBusy(true); + try { + const out = await fn(); + if (out?.overview) setData(out.overview); + done?.(out); + } catch (err) { + notify(err.message); + } finally { + setBusy(false); + } + } + + const onAsk = (e) => { + e.preventDefault(); + if (!askChannel) return notify('Pick a channel.'); + return run( + () => requestBridgeLink(guildId, { channelId: askChannel, options: askOptions }), + (out) => { + setIssued({ code: out.code, expiresAt: out.expiresAt }); + setAskChannel(''); + } + ); + }; + + const onUse = (e) => { + e.preventDefault(); + if (!code.trim()) return notify('Paste the code from the other server.'); + if (!useChannel) return notify('Pick a channel.'); + return run( + () => acceptBridgeLink(guildId, { code: code.trim(), channelId: useChannel, options: useOptions }), + () => { + setCode(''); + setUseChannel(''); + notify('The bridge is on.'); + } + ); + }; + + const onOption = (link, patch) => run(() => setBridgeOptions(guildId, link.id, patch)); + + async function onUnlink(link) { + const ok = await confirmDialog({ + title: `Unlink #${link.mine.channel}?`, + body: `Messages stop crossing to #${link.theirs.channel} in ${link.theirs.server}. Sylo deletes both webhooks and tells both channels. Messages already posted stay where they are, and deleting an original later no longer removes its copy.`, + confirmLabel: 'Unlink', + }); + if (ok) await run(() => bridgeLinkAction(guildId, link.id, 'unlink')); + } + + async function onCancelRequest(request) { + const ok = await confirmDialog({ + title: 'Cancel this request?', + body: 'The code stops working. You can ask again.', + confirmLabel: 'Cancel request', + cancelLabel: 'Keep it', + }); + if (ok) await run(() => cancelBridgeRequest(guildId, request.id)); + } + + return ( + <> +

Bridge

+

+ One conversation between a channel here on {here} and a channel on {other}. Members on both sides + write as they always do, and Sylo shows each message on the other side with the author's name and + picture. +

+ +

+ Both servers agree. One asks for a link and gets a code that works for 15 minutes; the owner of the + other server uses it and picks a channel. Sylo has to be in both servers. Either side can unlink at + any time. +

+

+ What crosses: messages, replies (as a short quote), files up to 8 MB (otherwise a link), custom + emoji with the same name, stickers (as a picture link), embeds, polls (as text: the vote stays where + the poll is) and, if you turn it on, who joins. Edits and deletes follow for 30 days. +

+

+ What never crosses: pings. @everyone, @here, roles and people are shown as plain text on the other + side, and nothing the bridge posts notifies anyone. Sylo's own bridge messages are never + relayed again, so there are no loops. Other bots are off until you turn them on. An age-restricted + channel can only be bridged with another age-restricted channel. Both channels show a notice while a + bridge is on. +

+
+ + {!data.bothPlatforms ? ( +

+ This Sylo is not connected to both Discord and Fluxer, so there is nothing to bridge to. +

+ ) : null} + {data.bothPlatforms && !data.enabled ? ( +

Turn the module on above to ask for a link or use a code.

+ ) : null} + +
+

+ Bridged channels ({data.links.length} of {data.limits.links}) +

+ {data.links.length === 0 ? ( + + No channel is bridged yet. Ask for a link below, or use a code from the other server. + + ) : null} +
+ {data.links.map((link) => { + const status = statusOf(link); + const canResume = link.resumable ?? (link.status === 'failed' || link.pausedByMe); + return ( +
+
+

+ #{link.mine.channel} ↔ #{link.theirs.channel} + {status.text} +

+

+ 0 && `${link.queued} waiting to send`, + link.dropped > 0 && `${link.dropped} dropped while busy`, + ]} + /> +

+ {link.problem ?

{link.problem.text}

: null} + {link.pauseNote ?

{link.pauseNote}

: null} + {link.pausedByThem ? ( +

+ {link.theirs.server} paused this bridge. Only they can resume it. +

+ ) : null} +
+ What this server shares +
+ onOption(link, patch)} + /> +
+

+ The other server decides for itself what it shares. Bot messages:{' '} + {link.theirOptions.bots ? 'shared' : 'not shared'}. Who joins:{' '} + {link.theirOptions.joinMessages ? 'shared' : 'not shared'}. +

+
+
+ run(() => bridgeLinkAction(guildId, link.id, 'pause')), + } + : canResume + ? { + label: 'Resume', + disabled: busy || !owner, + onClick: () => run(() => bridgeLinkAction(guildId, link.id, 'resume')), + } + : null + } + items={ + owner + ? [{ label: 'Unlink', danger: true, disabled: busy, onClick: () => onUnlink(link) }] + : [] + } + /> +
+ ); + })} +
+
+ + {data.requests.length ? ( +
+

Waiting for the other server

+
+ {data.requests.map((r) => ( +
+
+

#{r.channel}

+

+ +

+
+ onCancelRequest(r), + }, + ] + : [] + } + /> +
+ ))} +
+
+ ) : null} + + {!owner ? ( +

+ Only the server owner can ask for a link, use a code, change what this server shares, or pause, + resume and unlink a bridge. You can see the status here. +

+ ) : null} +
+
+

Ask the other server

+

+ Pick the channel here that should be bridged. You get a one-time code to give to the owner of the{' '} + {other} server. +

+ +
+ + +
+ setAskOptions((o) => ({ ...o, ...patch }))} + /> + + + {issued ? ( +
+

+ Give this code to the {other} server. It works once, until {formatDateTime(issued.expiresAt)}, + and is not shown again. +

+

+ {issued.code}{' '} + +

+
+ ) : null} +
+ +
+

Use a code

+

+ Someone on {other} asked for a link and gave you a code. Pick the channel here that joins it. +

+
+
+ + setCode(e.target.value)} + /> +
+
+ + +
+ setUseOptions((o) => ({ ...o, ...patch }))} + /> + + +
+
+ + ); +} diff --git a/web-v2/src/moduleForms/index.js b/web-v2/src/moduleForms/index.js index 1001f944..216a81d0 100644 --- a/web-v2/src/moduleForms/index.js +++ b/web-v2/src/moduleForms/index.js @@ -19,6 +19,7 @@ const Suggestions = lazy(() => import('./Suggestions.jsx')); const TempRoles = lazy(() => import('./TempRoles.jsx')); const QuoteBoard = lazy(() => import('./QuoteBoard.jsx')); const Introductions = lazy(() => import('./Introductions.jsx')); +const Bridge = lazy(() => import('./Bridge.jsx')); const Applications = lazy(() => import('./Applications.jsx')); const Economy = lazy(() => import('./Economy.jsx')); const AutoThreads = lazy(() => import('./AutoThreads.jsx')); @@ -86,6 +87,7 @@ export const MODULE_FORMS = { 'temp-roles': TempRoles, 'quote-board': QuoteBoard, introductions: Introductions, + bridge: Bridge, applications: Applications, economy: Economy, 'auto-threads': AutoThreads, @@ -104,6 +106,8 @@ export const SPECIAL_V2_PATHS = { insights: (guildId) => `/guilds/${guildId}/insights`, // The Commands overview card — V2's page is the Moderation hub's Commands tab. commands: (guildId) => `/guilds/${guildId}/moderation/commands`, + // The server mirror is a page of its own, not a module (src/mirror/). + mirror: (guildId) => `/guilds/${guildId}/mirror`, }; export function v2Href(card, guildId) { diff --git a/web-v2/src/overviewModel.js b/web-v2/src/overviewModel.js index 4069a3fc..46bb5ab6 100644 --- a/web-v2/src/overviewModel.js +++ b/web-v2/src/overviewModel.js @@ -267,6 +267,8 @@ export function navModel({ guildId, data, operator }) { const server = [ { key: 'settings', label: 'Server settings', icon: 'gear', href: g('/settings') }, { key: 'commands', label: 'Commands', icon: 'terminal', href: g('/moderation/commands') }, + // Copy this Discord server to a Fluxer server and keep it in step (the page works on the Fluxer side too). + { key: 'mirror', label: 'Mirror to Fluxer', icon: 'copy', href: g('/mirror') }, ]; const more = [ { key: 'command-list', label: 'Command list', icon: 'list', href: '/commands' }, diff --git a/web-v2/src/pages/Mirror.jsx b/web-v2/src/pages/Mirror.jsx new file mode 100644 index 00000000..6048eaf9 --- /dev/null +++ b/web-v2/src/pages/Mirror.jsx @@ -0,0 +1,924 @@ +import { createContext, useCallback, useContext, useEffect, useState } from 'react'; +import { useParams } from 'react-router-dom'; +import { + acceptMirror, + allowMirrorDeletes, + getMirror, + getMirrorPreview, + getMirrorReport, + mirrorAction, + requestMirror, + setMirrorOptions, + startMirror, +} from '../api.js'; +import { useApiData } from '../useApiData.js'; +import { notify } from '../notify.js'; +import LoadError from '../components/LoadError.jsx'; +import Reveal from '../components/Reveal.jsx'; +import Callout from '../components/Callout.jsx'; +import Field from '../components/Field.jsx'; +import Switch from '../components/Switch.jsx'; +import Time from '../components/Time.jsx'; +import CopyButton from '../components/CopyButton.jsx'; +import { confirmDialog } from '../components/ConfirmDialog.jsx'; +import { + OPTION_HINTS, + OPTION_ROWS, + STEPS, + canResume, + codeLeft, + deleteSummary, + groupNotes, + isBusy, + progressText, + statusOf, + stepOf, + summaryLines, + validTargetId, +} from '../mirrorModel.js'; + +const POLL_MS = 3000; + +// Only the owner of a server can change anything about its mirror; everyone with access can look. +const OwnerContext = createContext(true); + +/** Disables every control inside unless the signed-in person owns this server, and says why. */ +function OwnerOnly({ note = false, children }) { + const isOwner = useContext(OwnerContext); + return ( + <> + {!isOwner && note ? ( +

+ Only the server owner can do this. You can look at everything here, but not change it. +

+ ) : null} +
+ {children} +
+ + ); +} + +// Copy a Discord server's structure to a Fluxer server, and keep it in step (src/mirror/). The same page serves +// both servers of a pair: the Discord server asks, previews and copies; the Fluxer server agrees with the code and +// may pause or stop the mirror. +export default function Mirror() { + const { guildId } = useParams(); + const { data, error, setData } = useApiData(() => getMirror(guildId), [guildId], guildId); + // The one-time code is shown only in the answer to the request; keep it for as long as the request waits. + const [code, setCode] = useState({ guildId: null, text: null }); + const onState = useCallback( + (next) => { + if (next?.code) setCode({ guildId, text: next.code }); + setData(next); + }, + [guildId, setData] + ); + + // While a copy runs, ask for its progress. + const busy = data ? isBusy(data.pair, data.progress) : false; + useEffect(() => { + if (!busy) return undefined; + let live = true; + const timer = setInterval(() => { + getMirror(guildId) + .then((next) => live && setData(next)) + .catch(() => {}); + }, POLL_MS); + return () => { + live = false; + clearInterval(timer); + }; + }, [busy, guildId, setData]); + + if (error) return ; + if (!data) return

Loading…

; + + return ( + +

Mirror to Fluxer

+ + {data.platform === 'discord' ? ( + + ) : ( + + )} + +
+ ); +} + +/** Run an action that answers with the new state of the page; a refusal is shown as a message. */ +function useAct(onState) { + const [working, setWorking] = useState(false); + const act = useCallback( + async (call, done) => { + setWorking(true); + try { + const next = await call(); + onState(next); + if (done) notify(done, 'info'); + return next; + } catch (err) { + notify(err.message || 'That did not work.'); + return null; + } finally { + setWorking(false); + } + }, + [onState] + ); + return { working, act }; +} + +function StatusPill({ status }) { + const s = statusOf(status); + return {s.label}; +} + +function Stepper({ step }) { + const at = STEPS.findIndex(([key]) => key === step); + return ( +
    + {STEPS.map(([key, label], i) => ( +
  1. + + {label} +
  2. + ))} +
+ ); +} + +const Limits = ({ limits }) => ( +
    +
  • + Copied: categories and channels with their topics, slowmode, age restriction and + permission overwrites; roles with their colour, order and permissions; up to {limits.emoji} emoji and{' '} + {limits.stickers} stickers; the server's name, icon and banner; and Sylo's own settings, with channels + and roles pointing at the new ones. +
  • +
  • + Not copied: members (they join Fluxer themselves), roles given to people, message + history, threads, overwrites for single members, and anything Fluxer has no counterpart for. The preview + lists every such case before anything is created. +
  • +
  • + One way: Discord is the source. Changes you make to mirrored channels on Fluxer are + overwritten the next time that item changes on Discord. Sylo never deletes anything on Fluxer that it + did not create itself, and by default does not delete there when you delete on Discord: the copy is + locked and marked as removed. Changing a mirrored category also changes the channels on Fluxer that are + synced with it. +
  • +
+); + +/** + * Why a mirror is paused, and what resuming it takes: a burst of deletes needs the owner's confirmation (the deletes + * that waited are then applied), a server that changed hands needs its new owner to confirm, and a mirror one side's + * owner paused can be resumed only from that side. + */ +function PauseBlock({ guildId, pair, onState }) { + const { working, act } = useAct(onState); + if (pair.status !== 'paused') return null; + const mine = canResume(pair); + const kind = pair.pauseKind; + const label = + kind === 'deletes' + ? 'Review the deletions' + : kind === 'ownership' + ? 'Confirm the mirror as the owner' + : 'Resume'; + async function go() { + // Ask without a confirmation first: the server says when deletions wait and what they are, and the person + // confirms that list (or the mirror simply resumes when none wait). + try { + onState(await mirrorAction(guildId, 'resume', {})); + notify('The mirror goes on.', 'info'); + } catch (err) { + if (err.code !== 'confirm-needed') { + notify(err.message || 'That did not work.'); + return; + } + const ok = await confirmDialog({ + title: 'Apply the deletions on Fluxer?', + body: `These were deleted on Discord and wait to be applied on Fluxer: ${deleteSummary(err.deletes, err.deleteCount)}. A locked channel or role stays on Fluxer; a deleted channel loses its messages. Stop the mirror instead if this was a mistake.`, + confirmLabel: 'Apply the deletions', + tone: 'danger', + }); + if (ok) await act(() => mirrorAction(guildId, 'resume', { confirm: true }), 'The mirror goes on.'); + } + } + return ( + <> + + {pair.lastError || 'It was paused.'} + + +
+ +
+
+ {!mine ? ( +

+ The owner of the {pair.pausedBy === 'source' ? 'Discord' : 'Fluxer'} server paused this mirror, so + only that owner can resume it. +

+ ) : null} + + ); +} + +/** Copies left on Fluxer whose Discord original was deleted: locked, not deleted, for the Fluxer owner to look at. */ +function RemovedList({ progress }) { + const removed = progress?.removed ?? []; + if (!removed.length) return null; + return ( +
+

Removed on Discord ({removed.length})

+

+ These were deleted on Discord. Their copies on Fluxer were locked, not deleted, so their messages are + kept. Delete them on Fluxer if you do not need them. +

+
    + {removed.map((r, i) => ( +
  • + {r.kind}: {r.label} +
  • + ))} +
+
+ ); +} + +// --------------------------------------------------------------------------------------------------- +// The Discord server + +function SourceView({ guildId, state, onState, code }) { + const { pair } = state; + const step = stepOf(pair); + return ( + <> +

+ Mirror this server to a Fluxer server: Sylo creates the same categories, channels, roles and emoji + there and keeps them in step when you change them here. Only the owner of this server, and the owner + of the Fluxer server, can set up and run a mirror. +

+ + {!pair ? : null} + {pair?.status === 'pending' ? ( + + ) : null} + {pair?.status === 'ready' ? : null} + {pair && !['pending', 'ready'].includes(pair.status) ? ( + + ) : null} + + ); +} + +function AskStep({ guildId, state, onState }) { + const [target, setTarget] = useState(state.candidates[0]?.id ?? ''); + const { working, act } = useAct(onState); + const valid = validTargetId(target); + + const ask = () => act(() => requestMirror(guildId, target.trim())); + return ( +
+

+ Ask for a mirror +

+ {!state.fluxerConnected ? ( + + This Sylo is not connected to Fluxer, so there is nothing to mirror to. + + ) : null} +
    +
  1. On Fluxer, create an empty server for the mirror.
  2. +
  3. Invite Sylo to it and give Sylo's role the Administrator permission.
  4. +
  5. Enter the id of that Fluxer server below.
  6. +
+ + + + setTarget(e.target.value)} + placeholder="Server id" + autoComplete="off" + spellCheck={false} + /> + + + {state.candidates.map((c) => ( + + ))} + +
+ +
+
+
+ ); +} + +function AgreeStep({ guildId, pair, onState, code }) { + const { working, act } = useAct(onState); + const left = codeLeft(pair.codeExpiresAt); + return ( +
+

+ Waiting for the Fluxer server +

+

+ Nothing is created until the owner of the Fluxer server has agreed. Ask them to open Sylo's dashboard + for that server, go to Mirror to Fluxer and enter the code. +

+ {code ? ( +

+ {code} +

+ ) : ( +

+ The code is shown once, when you ask. If you lost it, ask again for a new one. +

+ )} +

+ {left ? `The code works for another ${left}.` : 'The code has expired.'} Asking again gives a new one. + If you own the Fluxer server yourself, sign in with Fluxer too (Account) and ask again: both sides + agree at once. +

+ +
+ + +
+
+
+ ); +} + +function OptionsForm({ guildId, pair, onState, only }) { + const { working, act } = useAct(onState); + return ( +
+ {OPTION_ROWS.filter(([key]) => !only || only.includes(key)).map(([key, label]) => ( +
+ {label} + act(() => setMirrorOptions(guildId, { [key]: on }))} + /> + {key === 'mirrorDeletes' && pair.options.mirrorDeletesRequested && !pair.options.mirrorDeletes ? ( +

+ Waiting for the owner of the Fluxer server to allow it in their dashboard. Until then deleted + items are locked, not deleted. +

+ ) : null} + {OPTION_HINTS[key] ? ( +

{OPTION_HINTS[key]}

+ ) : null} +
+ ))} +
+ ); +} + +function PreviewStep({ guildId, state, onState }) { + const { pair } = state; + const [shown, setShown] = useState(null); + const [loading, setLoading] = useState(false); + const [problem, setProblem] = useState(''); + const { working, act } = useAct(onState); + + // The options changed: what the preview showed no longer holds. + const optionsKey = JSON.stringify(pair.options); + const preview = shown?.key === optionsKey ? shown.data : null; + + async function look() { + setLoading(true); + setProblem(''); + try { + setShown({ key: optionsKey, data: await getMirrorPreview(guildId) }); + } catch (err) { + setProblem(err.message || 'The preview could not be made.'); + } finally { + setLoading(false); + } + } + + async function start() { + const ok = await confirmDialog({ + title: `Copy this server to ${pair.otherName ?? 'the Fluxer server'}?`, + body: 'Sylo creates the channels, roles and emoji you saw in the preview on the Fluxer server. It does not delete anything there. You can pause or stop the mirror at any time.', + confirmLabel: 'Start the copy', + }); + if (ok) await act(() => startMirror(guildId)); + } + + return ( +
+

+ Preview{pair.otherName ? ` for ${pair.otherName}` : ''} +

+

+ Both servers agreed. The preview shows what the copy would do and what cannot be carried over. It + creates nothing. +

+ + + +
+ + + + +
+ {problem ?

{problem}

: null} + {loading ?

Reading both servers…

: null} + {preview ? : null} +
+ ); +} + +function PreviewResult({ preview, onStart, working }) { + const lines = summaryLines(preview.summary); + const groups = groupNotes(preview.notes); + return ( +
+ {preview.blockers.map((b) => ( + + {b.message} + + ))} + {preview.warnings.map((w) => ( + + {w.message} + + ))} +

What the copy does

+
    + {lines.map((l) => ( +
  • {l.text}
  • + ))} +
  • + Last, every copied text channel is linked through the bridge, if the channels allow it (the reasons + are in the report). +
  • +
+

What cannot be carried over{groups.length ? '' : ': nothing'}

+ {groups.map((g) => ( +
+ + {g.title} ({g.items.length}) + +
    + {g.items.map((n, i) => ( +
  • {n.message}
  • + ))} +
+
+ ))} +
+ Every step ({preview.steps.length}) +
    + {preview.steps.map((s) => ( +
  • + {s.op} {s.name} + {s.reason ? ({s.reason}) : null} +
  • + ))} +
+
+ +
+ +
+
+
+ ); +} + +function RunningStep({ guildId, state, onState }) { + const { pair, progress } = state; + const { working, act } = useAct(onState); + const [reported, setReported] = useState(null); + const [again, setAgain] = useState(null); + + // A finished or running copy changes the report: an older one is not shown. + const reportKey = `${pair.copyRun}:${pair.status}`; + const report = reported?.key === reportKey ? reported.data : null; + + async function showReport() { + try { + setReported({ key: reportKey, data: await getMirrorReport(guildId) }); + } catch (err) { + notify(err.message || 'The report could not be loaded.'); + } + } + const running = pair.status === 'copying' || pair.status === 'live'; + return ( +
+

+ {pair.otherName ?? 'Fluxer server'} +

+ + {pair.status === 'stopped' ? ( +

+ The mirror was stopped. What Sylo made on Fluxer is still there. To mirror again, ask for a new + mirror; the same Fluxer server picks up where this one left off. +

+ ) : null} + {progress && progress.total > 0 ? ( +
+ +

{progressText(progress)}

+
+ ) : null} + {pair.status === 'live' ? ( +

+ Changes to channels, roles, emoji and stickers on this server are applied on Fluxer + {progress?.waitingChanges ? ` (${progress.waitingChanges} waiting)` : ''}. + {pair.copiedAt ? ( + <> + {' '} + The first copy finished

+ ) : null} + +
+ {running ? ( + + ) : null} + {['live', 'done'].includes(pair.status) ? ( + + ) : null} + {pair.status !== 'stopped' ? ( + + ) : ( + + )} +
+
+ {again ? setAgain(null)} /> : null} + +

What is mirrored

+ + + +

+ Changes on Fluxer to mirrored channels and roles are overwritten the next time that item changes on + Discord. Sylo never deletes something on Fluxer that it did not create. +

+ {pair.copyRun > 0 ? ( + <> +
+ +
+ {report ? : null} + + ) : null} +
+ ); +} + +// "Copy again": the same preview as the first time, then a new copy of what changed. +function AgainPanel({ guildId, onState, onClose }) { + const [preview, setPreview] = useState(null); + const { working, act } = useAct(onState); + useEffect(() => { + let live = true; + getMirrorPreview(guildId) + .then((p) => live && setPreview(p)) + .catch((err) => notify(err.message || 'The preview could not be made.')); + return () => { + live = false; + }; + }, [guildId]); + return ( +
+

Copy again

+ {!preview ?

Reading both servers…

: null} + {preview ? ( + { + const next = await act(() => startMirror(guildId)); + if (next) onClose(); + }} + /> + ) : null} + +
+ ); +} + +function Report({ report }) { + const groups = groupNotes(report.notes); + return ( +
+ {report.failed.length ? ( + <> +

Could not be copied ({report.failed.length})

+
    + {report.failed.map((f) => ( +
  • + {f.name}: {f.detail} +
  • + ))} +
+ + ) : ( +

Every step that could be done was done.

+ )} + {groups.map((g) => ( +
+ + {g.title} ({g.items.length}) + +
    + {g.items.map((n, i) => ( +
  • {n.message}
  • + ))} +
+
+ ))} + {report.bridge ? ( +

+ Linking channels through the bridge:{' '} + {report.bridge.status === 'done' ? report.bridge.detail : 'waiting'}. +

+ ) : null} +
+ ); +} + +/** On the Fluxer server: the Discord side asks to delete here too, and only the owner here can allow it. */ +function DeletesPanel({ guildId, pair, onState }) { + const { working, act } = useAct(onState); + const asked = pair.options.mirrorDeletesRequested; + const allowed = pair.options.mirrorDeletes; + if (!asked && !allowed) return null; + async function allow() { + const ok = await confirmDialog({ + title: 'Allow deleting on this server?', + body: 'From now on, what is deleted on Discord is deleted here too: a channel loses all its messages, including ones that only exist here. Only what Sylo made is ever deleted. Without this, deleted items are only locked and listed for you to look at.', + confirmLabel: 'Allow deleting', + tone: 'danger', + }); + if (ok) await act(() => allowMirrorDeletes(guildId, true), 'Deleting is allowed.'); + } + return ( +
+

Deleting on this server

+ {allowed ? ( +

+ Deleting is allowed: what is deleted on Discord is deleted here, with its messages. Only what Sylo + made is deleted. +

+ ) : ( +

+ The Discord server asks to delete here too, when something is deleted on Discord. Until you allow + it, those items are locked and listed, and nothing is deleted. +

+ )} + +
+ {allowed ? ( + + ) : ( + + )} +
+
+
+ ); +} + +// --------------------------------------------------------------------------------------------------- +// The Fluxer server + +function TargetView({ guildId, state, onState }) { + const { pair } = state; + const { working, act } = useAct(onState); + const [code, setCode] = useState(''); + + if (!pair) { + return ( + <> +

+ This Fluxer server is not the mirror of a Discord server. To make it one, open Sylo's dashboard for + the Discord server, go to Mirror to Fluxer and ask for a mirror. You will be given + a code to enter here. +

+ + + ); + } + return ( +
+

+ Mirror of {pair.otherName ?? 'a Discord server'} +

+ {pair.status === 'pending' ? ( + <> +

+ {pair.requestedBy ? `${pair.requestedBy} asked` : 'Someone asked'} to copy the Discord server{' '} + {pair.otherName ?? 'above'} to this server. Sylo would create channels, roles and + emoji here, and keep them in step with Discord. Nothing is deleted. If you agree, enter the code + they gave you. +

+ + + setCode(e.target.value)} + placeholder="XXXX-XXXX" + autoComplete="off" + spellCheck={false} + /> + +
+ + +
+
+ + ) : ( + <> +

+ The Discord server is the source: what changes there is applied here. Changes made here to + mirrored channels and roles are overwritten the next time that item changes on Discord. Sylo never + deletes something here that it did not create, and by default does not delete here when something + is deleted on Discord: the copy is locked and marked as removed. Changing a mirrored category also + changes the channels here that are synced with it. Only the server owner can pause or stop the + mirror. +

+ + + + +
+ {['copying', 'live'].includes(pair.status) ? ( + + ) : null} + {pair.status !== 'stopped' ? ( + + ) : null} +
+
+ + )} +
+ ); +} diff --git a/web-v2/src/pages/ModulePage.jsx b/web-v2/src/pages/ModulePage.jsx index 5e1605b3..c02f5ccc 100644 --- a/web-v2/src/pages/ModulePage.jsx +++ b/web-v2/src/pages/ModulePage.jsx @@ -22,7 +22,7 @@ const INBOX_PATH = { tickets: '/tickets', appeals: '/appeals' }; // A module's settings need Edit access (access.jsx), so below it the whole form is disabled. Not for // pages with tabs or a search a reader still uses: there the save bar, the switches, the row actions // and the queue's own buttons (Applications, Suggestions: Inbox) check the level themselves. -const NOT_LOCKED_AS_A_WHOLE = new Set(['applications', 'suggestions', 'economy', 'quote-board']); +const NOT_LOCKED_AS_A_WHOLE = new Set(['applications', 'suggestions', 'economy', 'quote-board', 'bridge']); // The module's on/off state with a switch, so a module can be turned on from // its own settings page. Reads and patches the same shared overview state the diff --git a/web-v2/src/styles/bridge.css b/web-v2/src/styles/bridge.css new file mode 100644 index 00000000..ccd0aa2c --- /dev/null +++ b/web-v2/src/styles/bridge.css @@ -0,0 +1,45 @@ +/* The Bridge page (moduleForms/Bridge.jsx): the switches of one side, the one-time code and the forms. */ +.v2-bridge-options { + display: grid; + gap: 12px; + margin: 12px 0; +} +.v2-bridge-option { + display: flex; + align-items: flex-start; + gap: 12px; +} +.v2-bridge-option > span { + display: grid; + gap: 2px; + min-width: 0; +} +.v2-bridge-option label { + font-weight: 500; +} +.v2-bridge-details { + margin-top: 8px; +} +.v2-bridge-details > summary { + cursor: pointer; + color: var(--text-muted); + font-size: 0.875rem; +} +.v2-bridge-form { + display: grid; + gap: 12px; + max-width: 480px; +} +.v2-bridge-code { + margin-top: 16px; + padding: 12px 16px; + border: 1px solid var(--border-strong); + border-radius: var(--radius-md, 10px); + background: color-mix(in srgb, var(--success) 8%, transparent); +} +.v2-bridge-code-text { + font-size: 1.25rem; + letter-spacing: 0.08em; + font-weight: 600; + overflow-wrap: anywhere; +} diff --git a/web-v2/src/styles/mirror.css b/web-v2/src/styles/mirror.css new file mode 100644 index 00000000..dbcc2fb9 --- /dev/null +++ b/web-v2/src/styles/mirror.css @@ -0,0 +1,141 @@ +/* The Mirror to Fluxer page (pages/Mirror.jsx). The stepper reuses the setup wizard's (setup-wizard.css). */ + +.v2-mirror-facts { + margin: 12px 0 18px; + padding-left: 20px; + display: grid; + gap: 8px; + color: var(--text-muted); + font-size: var(--fs-sm); +} +.v2-mirror-facts strong { + color: var(--text); +} +.v2-mirror-todo { + margin: 8px 0 14px; + padding-left: 22px; + display: grid; + gap: 4px; +} + +.v2-mirror-code { + display: flex; + align-items: center; + flex-wrap: wrap; + gap: 10px; + margin: 12px 0; +} +.v2-mirror-code code { + padding: 8px 14px; + border: 1px solid var(--border-control); + border-radius: 8px; + background: var(--bg-elevated, var(--bg-main)); + font-size: 1.15rem; + letter-spacing: 0.12em; + font-weight: 600; +} + +.v2-mirror-options { + display: grid; + gap: 2px; + margin: 10px 0 16px; + max-width: 560px; +} +.v2-mirror-option { + display: flex; + align-items: center; + justify-content: space-between; + gap: 16px; + padding: 8px 0; + border-bottom: 1px solid var(--border); +} + +.v2-mirror-subtitle { + margin: 20px 0 8px; + font-size: var(--fs-md, 1rem); + font-weight: 600; +} +.v2-mirror-summary, +.v2-mirror-steps { + margin: 0 0 12px; + padding-left: 20px; + display: grid; + gap: 4px; +} +.v2-mirror-steps { + list-style: none; + padding-left: 0; + max-height: 320px; + overflow: auto; +} +.v2-mirror-op { + display: inline-block; + min-width: 78px; + padding: 1px 8px; + border-radius: 999px; + background: var(--bg-hover, var(--bg-main)); + color: var(--text-muted); + font-size: var(--fs-xs); + text-transform: uppercase; + letter-spacing: 0.04em; + text-align: center; +} +.v2-mirror-op.is-create { + color: var(--success); +} +.v2-mirror-op.is-skip { + color: var(--warning, var(--text-muted)); +} +.v2-mirror-notes { + margin: 6px 0; +} +.v2-mirror-notes summary { + cursor: pointer; + font-weight: 600; + padding: 4px 0; +} +.v2-mirror-notes ul { + margin: 4px 0 8px; + padding-left: 22px; + display: grid; + gap: 4px; + overflow-wrap: anywhere; +} + +.v2-mirror-progress { + margin: 12px 0; + max-width: 560px; +} +.v2-mirror-progress progress { + width: 100%; + height: 10px; + accent-color: var(--success); +} +.v2-mirror-again, +.v2-mirror-report { + margin-top: 14px; + padding: 14px 16px; + border: 1px solid var(--border); + border-radius: 10px; +} + +@media (max-width: 600px) { + .v2-mirror-option { + align-items: flex-start; + } +} + +.v2-mirror-option { + flex-wrap: wrap; +} +.v2-mirror-option-hint { + flex-basis: 100%; + margin: 0 0 6px; +} +.v2-mirror-removed ul { + margin: 4px 0 8px; + padding-left: 22px; + display: grid; + gap: 4px; + overflow-wrap: anywhere; +}