Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,11 @@ WEB_PORT=3000
# network (127.0.0.1, 10.x, 192.168.x, 169.254.x ...) are refused. Allow them on a trusted LAN:
# ALLOW_PRIVATE_URLS=true

# ---- Discord <-> Fluxer bridge (optional) ----
# The most bridged channels one server can have, links made by hand and by a server mirror together.
# A whole mirrored server is linked channel by channel, so the default is 100. 1 to 500.
# BRIDGE_MAX_LINKS=100

# ---- Push notifications for the Sylo app (optional) ----
# Postal Pigeon, the Hedgerow push relay: with this set, the Sylo app gets notifications with no
# text in them (Pigeon and Apple only see a random reference; the app asks this instance for the
Expand Down
7 changes: 5 additions & 2 deletions docs/mirror.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,11 @@ so a message written on one side shows up on the other. Voice channels, categori
is in the report ("Linking channels through the bridge" and the notes), and the rest are linked.
- **Safe to run again.** "Copy again" links only the pairs that have no link yet. A channel that an owner
unlinked on the Bridge page stays unlinked, and a pair the owners bridged by hand is never taken over or
changed (it is reported as skipped). A server can have at most 50 bridged channels this way (10 when
linking by hand), and a copy links 25 channels at a time. A channel made on Discord later (continuous mirroring)
changed (it is reported as skipped). A server can have at most 100 bridged channels, links made by hand
included (the operator can change this with `BRIDGE_MAX_LINKS`, 1 to 500). A mirror of a bigger server links
up to the limit and lists the rest in the report as "not linked: the server's bridge limit (100) is reached";
they are linked on a later "Copy again" once the limit is raised or links are removed. A copy links 25
channels at a time. A channel made on Discord later (continuous mirroring)
is linked too.
- **The links follow the mirror.** While the mirror is paused (by an owner, by Sylo after a failure, or because a
server changed owner) or stopped, its links are paused, with no notice in each channel, and they run again when
Expand Down
18 changes: 14 additions & 4 deletions docs/modules/bridge.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ A link is never made from one side alone.
shared with #chat in Fluxer Home on Fluxer ...".

A link joins a Discord channel to a Fluxer channel, never two on the same platform. A channel is in one link
at a time, and a server has at most 10 links and 5 waiting requests. The Bridge module must be on in both
at a time, and a server has at most 100 links (the operator can change this, see below) and 5 waiting requests. The Bridge module must be on in both
servers. Wrong codes are rate limited, and a few wrong ones in a row lock the attempts for a while.

**Either side can unlink at any time** (the owner). Both webhooks are deleted, the message map goes, and
Expand All @@ -47,8 +47,8 @@ is told too.
**Links made by the server mirror.** A [server mirror](../mirror.md) links its copied text channels through the
bridge without codes, because both owners already agreed to the mirror. Those links show "Name (server mirror)"
as who made them, follow the mirror (paused with it, running again with it; the Bridge page cannot resume one on
its own while the mirror is paused) and follow every rule on this page. A server can have 50 of them (10 when you
link by hand). Unlink works for them as for any other link, and an unlinked channel is not linked again. If the
its own while the mirror is paused) and follow every rule on this page. They count toward the same limit as links
made by hand. Unlink works for them as for any other link, and an unlinked channel is not linked again. If the
mirror is removed, its links stay, paused by the server that removed it: only that server's owner can resume
them, and either owner can unlink them. The mirror switches this module on only the first time it links, and a
server that switches it off keeps it off.
Expand Down Expand Up @@ -92,7 +92,17 @@ stopped itself (see below) can be resumed by either server once the cause is fix
- **Rate limits.** Each webhook has its own queue: sends are spaced out, a rate limit from the platform is
waited out, a short outage is retried with growing pauses, and a message that waited more than five
minutes is dropped. A queue holds at most 100 messages; a flooded channel drops the newest and counts it.
One server's queue never slows another's.
One server's queue never slows another's. The spacing follows what each platform allows for one webhook:
Fluxer allows 60 sends and 30 edits or deletes a minute (Sylo waits 1 second between sends and 2 between
edits and deletes), Discord does not publish its figures and answers with a retry time, which Sylo waits out
(it spaces sends about half a second apart).
- **How many links.** A server can have **100** bridged channels by default, made by hand or by a server
mirror together. The people who run the Sylo instance can change the number with `BRIDGE_MAX_LINKS` (1 to
500, see [self-hosting](../self-hosting.md)). The limit is about Sylo's own work, not the platforms'
webhook limits: each link uses one webhook in each channel (a channel can hold 15 webhooks on Discord and on
Fluxer, a Fluxer server 1000), and each webhook has its own queue. When a server is at the limit, new
requests are refused and a mirror reports the channels it could not link; the Bridge page says so and how to
raise it.
- **A failing link stops.** No permission, a webhook Sylo cannot make again, a deleted channel, or five
failed deliveries in a row stop the link. Sylo says so in both channels, the **Health** page shows it as a
problem with the reason, and it is not retried until someone presses **Resume**.
Expand Down
1 change: 1 addition & 0 deletions docs/self-hosting.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ At least one platform must be configured.
| `APPLE_PRIVATE_KEY` | — | The key's `.p8` file: PEM text, PEM on one line with literal `\n`, or base64 |
| `APPLE_VENDOR_NUMBER` | — | Optional vendor number for a planned daily downloads digest |
| `TRUST_PROXY` | one private hop with `DASHBOARD_URL` | Which proxies may set `X-Forwarded-For`: `false`, a hop count, or addresses/subnets. See [Behind a reverse proxy](#behind-a-reverse-proxy) |
| `BRIDGE_MAX_LINKS` | `100` | The most bridged channels one server can have, links by hand and links made by a [server mirror](mirror.md) together (1 to 500; Sylo does not start with another value). Each link uses a webhook in each of its two channels |
| `ALLOW_PRIVATE_URLS` | `false` | Let server admins use feed and image URLs on private, loopback or link-local addresses (your own network). Off by default: those URLs are fetched by the server |
| `PUSH_RELAY_URL` | `https://app.sylobot.com` | Where this instance sends [Sylo app notifications](mobile-app.md#notifications); empty turns them off |
| `APNS_KEY_ID` | — | Only for the instance that talks to Apple itself (the hosted Sylo): APNs auth key id |
Expand Down
9 changes: 9 additions & 0 deletions src/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { resolveSessionSecret } from './lib/sessionSecret.js';
import { normalisePrivateKey } from './lib/appStoreConnect.js';
import { checkPigeonUrl } from './lib/pigeonUrl.js';
import { commandPrefixError } from './lib/commandPrefix.js';
import { parseBridgeMaxLinks } from './lib/bridgeLimit.js';

/**
* Read a required string env var, or exit with a clear message.
Expand Down Expand Up @@ -282,6 +283,13 @@ if (commandPrefixProblem) {
process.exit(1);
}

// The most bridged channels one server can have (the Bridge module and the server mirror share it).
const bridgeMaxLinks = parseBridgeMaxLinks(process.env.BRIDGE_MAX_LINKS);
if (bridgeMaxLinks.error) {
console.error(`[config] ${bridgeMaxLinks.error}`);
process.exit(1);
}

export const config = Object.freeze({
// Discord (null when this install does not run on Discord)
discordToken,
Expand All @@ -294,6 +302,7 @@ export const config = Object.freeze({
fluxerWebUrl,
fluxerApiUrl,
commandPrefix,
bridgeMaxLinks: bridgeMaxLinks.value,
// Optional: register slash commands to these guild(s) for instant availability
// during development (one id, or a comma/space-separated list). When empty,
// commands are registered globally (can take up to ~1 hour to propagate).
Expand Down
28 changes: 28 additions & 0 deletions src/lib/bridgeLimit.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
// The most bridged channels a server can have, set by the operator with BRIDGE_MAX_LINKS (src/config.js).
//
// The real limits are the platforms' webhook limits and the relay queue, not this number: Sylo makes one
// webhook per bridged channel (Discord and Fluxer both allow 15 per channel, Fluxer 1000 per server), and the
// relay spaces the sends of each webhook inside what the platform allows (src/modules/lib/bridgeQueue.js).
// A server mirror links every copied text channel, so the default has to be well above a typical server.

export const BRIDGE_MAX_LINKS_DEFAULT = 100;
export const BRIDGE_MAX_LINKS_MIN = 1;
export const BRIDGE_MAX_LINKS_MAX = 500;

/**
* Read BRIDGE_MAX_LINKS. Unset or blank gives the default.
* @param {string | undefined | null} raw
* @returns {{ value: number, error: null } | { value: null, error: string }}
*/
export function parseBridgeMaxLinks(raw) {
const text = raw == null ? '' : String(raw).trim();
if (text === '') return { value: BRIDGE_MAX_LINKS_DEFAULT, error: null };
const n = /^\d+$/.test(text) ? Number(text) : NaN;
if (!Number.isInteger(n) || n < BRIDGE_MAX_LINKS_MIN || n > BRIDGE_MAX_LINKS_MAX) {
return {
value: null,
error: `BRIDGE_MAX_LINKS must be a whole number from ${BRIDGE_MAX_LINKS_MIN} to ${BRIDGE_MAX_LINKS_MAX} (default ${BRIDGE_MAX_LINKS_DEFAULT}).`,
};
}
return { value: n, error: null };
}
31 changes: 30 additions & 1 deletion src/mirror/bridgeLink.js
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ import { isModuleEnabled } from '../db/modules.js';
import {
BridgeError,
createMirrorLink,
getMaxLinks,
enableBridgeForMirror,
holdMirrorLink,
releaseMirrorLink,
Expand Down Expand Up @@ -223,8 +224,10 @@ export async function linkMirroredChannels(

let linked = 0;
let attempted = 0;
let full = false;
const tried = [];
for (const item of todo) {
if (full) break;
if (attempted >= limit) break;
attempted += 1;
tried.push(item.sourceId);
Expand All @@ -240,6 +243,18 @@ export async function linkMirroredChannels(
linked += 1;
} catch (err) {
if (!(err instanceof BridgeError)) throw err;
if (err.code === 'limit') {
// A server is at its bridge limit: this pair and every pair still waiting cannot be linked. They are
// reported (not "remaining": nothing more will be tried until the limit is raised or a link is removed).
full = true;
const rest = todo.filter((t) => !tried.includes(t.sourceId) || t === item);
for (const t of rest) skipped.push(limitSkip(t));
log.info(
'mirror',
`pair ${pair.id}: ${rest.length} channels not linked, the bridge limit is reached`
);
continue;
}
log.info(
'mirror',
`pair ${pair.id}: ${item.name} not linked (${err.code || err.status}): ${err.message}`
Expand All @@ -253,5 +268,19 @@ export async function linkMirroredChannels(
}
if (attempted < limit && attempted < todo.length) await sleep(PAUSE_BETWEEN_MS);
}
return { linked, already, remaining: Math.max(0, todo.length - attempted), tried, skipped };
return {
linked,
already,
remaining: full ? 0 : Math.max(0, todo.length - attempted),
tried,
skipped,
};
}

/** A pair that is not linked because a server is at its bridge limit (BRIDGE_MAX_LINKS). */
const limitSkip = (item) => ({
sourceId: item.sourceId,
name: item.name,
code: 'limit',
message: `the server's bridge limit (${getMaxLinks()}) is reached`,
});
Loading
Loading