Skip to content

feat(mirror): link mirrored channels through the bridge - #387

Open
Ferdinand99 wants to merge 18 commits into
v5.3from
feat/53-mirror-bridge-link
Open

Ferdinand99 wants to merge 18 commits into
v5.3from
feat/53-mirror-bridge-link

Conversation

@Ferdinand99

Copy link
Copy Markdown
Owner

What this does

Wires the server mirror (#386) to the Bridge (#385): the last step of a copy now links every mirrored text channel pair through the bridge. This PR is stacked on #386 and #385 (it merges feat/53-bridge into feat/53-mirror and adds the wiring on top), so until those two are merged its diff also shows their commits. Review the last commits: everything after "Merge origin/feat/53-bridge into feat/53-mirror".

Linking (src/mirror/bridgeLink.js, src/modules/bridge.js)

  • linkMirroredChannels(pairId) links every pair of text and announcement channels in the pair's channel map, with the bridge's own link model and relay. Voice channels, categories and forums are left alone.
  • No link code: both server owners already agreed to the mirror. The link records them (Name (server mirror) as the requested-by / accepted-by names, shown on the Bridge page and in both audit logs) and the owners of both servers for the ownership check.
  • Every bridge rule holds (createMirrorLink): the age rule; a channel everyone can see is never linked to a private one and the other way round; each server owner must be able to see, read and send in the channel (fails closed when sign-in is on and the member cannot be found); Sylo needs Manage Webhooks, View Channel and Send Messages in both; a notice in both channels (it says the link came from the server mirror); bots off by default; no pings across. A pair that fails a rule is skipped, listed in the report and the notes with a reason code, and the rest go on.
  • The Bridge module is switched on in both servers when it is off (it has to be on for a link to carry anything); both audit logs record it.
  • Idempotent: a re-run ("Copy again") links only pairs without a link. A channel unlinked on purpose on the Bridge page is not linked again. A link of exactly this pair whose record was lost is taken over, not duplicated. Batches of 25 with a continuation run; a channel made on Discord later (continuous mirroring) is linked too.
  • A mirrored server can have 50 bridged channels (10 when linking by hand).
  • The links follow the mirror: while it is paused (by a server owner, by Sylo after a failure, or by an ownership change) or stopped, its links are paused (reason mirror, no notice per channel) and run again when the mirror does. A server that changed hands keeps its link paused until the new server owner confirms on the Bridge page. Removing a mirror pauses its links and leaves them (mirror-ended); either server owner can resume or unlink them. A link held by its mirror cannot be resumed from the Bridge page, which now gets a resumable flag per link.
  • New table mirror_links (part of migration 84 and the Postgres bootstrap): which bridge link belongs to which Discord channel of a pair.

Two mirror fixes from the re-review

  • A role made again during "copy again" now goes through order-then-grant: the role-order request is a copy job with the key of the plan's order step, so it joins that step when it is waiting and runs on its own when "copy again" left the step out.
  • A change of the Fluxer server owner turns "Mirror deletions" off until the new server owner switches it on themselves (the request stays). A new request that brings no options keeps real deletion only for the same Fluxer server owner who owns both servers.

Integration order and migrations

Merge order: #368 -> #383 -> #385 -> #386 -> this PR. On SQLite the migrations end as 81 public look (#368), 82 tickets index (#383), 83 bridge (#385), 84 mirror (#386, with mirror_links added here). This branch keeps #385's two empty placeholder steps (81 and 82) so the bridge stays the 83rd; when #368 and #383 are merged first, drop the placeholders and keep the real steps (nothing else needs them). Postgres bootstraps its tables and is order independent. src/web/lib/guildOwner.js is identical on both branches.

Tests

New: test/mirrorBridgeLink.test.js (the linking, every rule, idempotence, batches, hold / resume / release) and test/mirrorBridgeEngine.test.js (the engine step, continuations, links following the mirror, the two fixes), mirror.db and its Postgres twin, a web test for the Resume button. Lint and prettier clean; SQLite 4285 tests, Postgres 16 4492 tests, test:v2 884, build:v2 ok.

Not tested live: linking against a real Discord and Fluxer server (webhook creation and the notices go through the same code as a link made by hand).

A new Bridge module shares one conversation between a Discord channel and a
Fluxer channel. Both servers agree: one asks for a link and gets a one-time
code (hashed, valid 15 minutes), someone with Edit access in the other server
uses it. Either side can pause or unlink at any time.

Messages cross through webhooks Sylo makes in each channel, with the author's
name and picture. Replies are quoted, files are re-uploaded within limits
(otherwise a link), custom emoji map by name, stickers become a picture link,
rich embeds cross, polls become text and joins are optional. Edits and deletes
follow through a message map that keeps a message for 30 days.

Safety: no ping crosses (allowed mentions none, mentions turned into names),
Sylo's own bridge messages are never relayed again, bots are off by default,
an age-restricted channel only links to an age-restricted channel (checked on
every message), a notice shows in both channels, each webhook has its own
rate-limited queue with back-off, and a failing link is stopped, shown in
Health and not retried.

Adds migration 83 (versions 81 and 82 stay empty placeholders until the
other 5.3 changes land), the Bridge page in the dashboard, the `bridge`
feature string in /api/mobile/info and a docs page.
A Discord server can be copied to a Fluxer server: categories, channels with
their properties and permission overwrites, roles with their permissions mapped
bit by bit, emoji and stickers within Fluxer's limits, the server's name, icon
and banner, and Sylo's own module settings with channel and role ids remapped.
Afterwards channel, role, emoji and sticker changes on Discord are applied on
Fluxer (one way, and Sylo never deletes what it did not create).

- migration 84: mirror pairs, the id map and a job queue, so a copy resumes
  after a restart and never makes anything twice
- dry run with a report of everything that cannot be carried over; the real
  copy builds the same specs from the same plan
- consent from both servers (Edit on each side, a one-time code or the same
  person on both) behind requirePairConsent()
- per-server rate-limited queue with backoff; a mirror that keeps failing
  pauses itself and shows up in Health
- "Mirror to Fluxer" page in the dashboard, classified in routeAccess.js
- `mirror` feature string in /api/mobile/info, docs/mirror.md
- the final bridge-linking step is a stub (linkMirroredChannels) until the
  bridge lands
… sealed tokens

- Age restriction uses the platform's real rule: on Fluxer the channel's own
  setting, its category's or the server's (fail closed when unreadable), and
  a Discord server marked age-restricted counts for its channels. Checked when
  linking, resuming and on every message.
- The person linking (request and accept) must be able to view, read and send
  in the channel themselves; Sylo's own rights that cannot be read count as
  missing.
- Deletes always reach the copy: looked up inside the queued job, run while the
  link is paused or stopped, not dropped for a full or slow lane, one job for a
  purge, the webhook that really posted a copy is remembered, the bot deletes
  the copy when the webhook cannot, and a delete that wins the race against the
  relay is honoured. Edits are looked up inside the job too.
- Webhook messages never cross, even with bots on, and bot accounts that wear a
  platform tag are skipped, so another bridge cannot make a loop.
- Webhook tokens are sealed at rest with the AES-GCM helper now shared in
  lib/seal.js (the Pigeon credential keeps its format); an unreadable token
  leads to a new webhook.
- One process-wide budget for file downloads (4 at a time, 32 MB); a file that
  does not fit crosses as a link.
- Names in Sylo's own text are escaped, poll, sticker and file names get the
  mention rules, a late failure no longer overrides a pause, the leaving
  server's own webhook is deleted, a refused message falls back to a plain name
  and then to text only, pause is rate limited, and the dashboard shows only
  the own side's name.
…ixes

- Only the owner of a server (Discord owner, Fluxer owner) can ask for a link,
  use a code, change options, pause, resume or unlink; Administrators, Edit
  users and bot masters are refused. Everyone with View access sees the status,
  read-only. requireGuildOwner(req, guildKey) in web/lib/guildOwner.js, checked
  on every action from the owner the platform reports.
- The owner of each server is recorded with the link. If either owner changes,
  the link pauses (on the next message or join, and by a five-minute sweep)
  until the new owner confirms with Resume; a code made by a previous owner is
  refused.
- A delete that hits a temporary error is retried and keeps its map entry
  until the copy is gone; the download budget is held until the copy is sent;
  an edit with an unreadable webhook token makes a new webhook instead of
  counting as a failed delivery; plain-text webhook tokens are sealed on first
  read; a paused or stopped link whose channel was deleted shows a Health item.
…ole order, privacy and grant checks

Review round 1 for the server mirror.

- Only the owner of the Discord server and the owner of the Fluxer server can
  ask for, agree to, change, start, pause, resume, stop or remove a mirror
  (guildOwner.js, the same helper the bridge uses). The owners' account ids are
  recorded with the agreement and checked again before every piece of work; a
  server that changes hands pauses the mirror until its new owner confirms.
- A delete on Discord locks the Fluxer copy (channel hidden and read-only, role
  stripped) and marks it removed; real deletion only with "Mirror deletions"
  (off by default). More than five deletes in ten minutes pause the mirror
  until the owner confirms them.
- The copy is blocked while Sylo's role on Fluxer is too low to give every
  mirrored role its own position; positions are always distinct and in Discord's
  order, and a new role with no room pauses the mirror.
- A channel @everyone cannot see on Discord is hidden on Fluxer whatever else is
  copied; @everyone never gets powerful permissions; an age-restricted server
  flags its channels.
- Copied module settings pass the dashboard's grant and ping checks, ids that
  mean nothing on Fluxer are cleared, and an explicit list keeps the operator's
  modules and the bridge out.
- Only a real 404 means "gone"; a failed look-up, a timeout or a 5xx never makes
  a second copy; a role created but not finished is tracked; a job that may have
  gone through looks for what it made (made since the job started) first.
- The SQLite migration is idempotent; thread permissions are mapped; Fluxer's
  real limits (500 emoji and stickers, 512 KB); a delete during the copy is not
  lost; a pause can be resumed only by the side that made it; old copy runs are
  pruned; a few mirrors run at a time.
…er ordering, Fluxer owner allows deletion

Review round 2 for the server mirror.

- The role-hierarchy check compares Sylo's rank on Fluxer (position, then the
  older id), not its position number: a fresh server with Sylo at position 1 is
  fine, a role above or level with Sylo's blocks the copy with advice naming the
  roles and how the Fluxer owner moves Sylo's role. The same test pauses a
  continuous reorder.
- A new role is made without its powerful permissions and gets them only after
  the role order is set (the order step also runs for a single role); if the
  order cannot be set the role stays without them.
- Real deletion ("Mirror deletions") is only asked for by the Discord owner and
  takes effect when the Fluxer owner allows it; resuming with deletes waiting
  lists them for confirmation, and with real deletion on only the Fluxer owner
  can confirm.
- A 5xx whose status is only in the SDK error's cause is retried, a copied
  economy comes over switched off, and the docs note which channels carry the
  age flag.
… asked for at request time is only a request

Review round 3 for the server mirror.

- A role made, adopted or made again (a mirrored role deleted on Fluxer, then
  edited on Discord) is created without its powerful permissions, the order
  step runs (continuous mode asks for it at once), and only then are the
  permissions granted: one path for the first copy and for continuous mode.
  Adopted roles are queued for the same grant step.
- "Mirror deletions" in a mirror request only takes effect when the same person
  owns both servers; otherwise it is stored as a request that the owner of the
  Fluxer server turns on with their own switch.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Code graph out of date: Health delta baseline: last indexed commit a9ec8af (diverged from this PR's base — delta is approximate). Impact and hotspot numbers below may be wrong.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.

Not checked: tests were not run; no formal proof of the changed code.


Graphify review — findings

Adds Mirror to Fluxer, a one-way copy of a Discord server's categories, channels, roles, emoji, stickers and Sylo settings onto a Fluxer server, kept in step as the Discord server changes. Only the owner of each server can ask, agree, start, pause, resume or stop a mirror, via a one-time code or automatically when one account owns both. Everyone with dashboard access can view the page, preview and report. Ownership is rechecked before each piece of work, and a mirror pauses if either server changes hands until the new owner confirms it. The preview also refuses to start unless Sylo's role is the top role on the Fluxer server. Documents the new Bridge module (a Discord channel shared with a Fluxer channel through webhooks), bumping the module count from 43 to 44. Adds docs/mirror.md and links it from the README, the dashboard v2 notes and the Fluxer guide.

Worth a look

  • Mirror authorization docs contradict the open-mode exception — docs/mobile-app.md · Escalate · medium
    • NOT verified (no proof, no reproducing execution) — needs human review

Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 2155 functions depend on the 1293 functions this change touches.

Health — this change adds coupling hotspots:

  • worse: setGuildModule() — 159 callers, 6 callees
  • worse: getGuildModule() — 197 callers, 3 callees
  • worse: purgeGuild() — 40 callers, 5 callees
  • worse: apiFetch() — 66 callers, 3 callees
  • new: requestMirror() — 8 callers, 12 callees
  • new: acceptLink() — 5 callers, 17 callees
  • worse: buildHealthReport() — 5 callers, 15 callees
  • new: handleMessage() — 4 callers, 17 callees
  • …and 58 more — each is listed as a finding

Verification — 2155 functions in the blast radius were not formally verified this run (proofs are advisory here).

Health delta baseline: last indexed commit a9ec8af (diverged from this PR's base — delta is approximate).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 2112 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

695 of 695 test file(s) selected (100%) via static blast radius.

Escalated to a full run for safety (noncode-change, uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.

  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Access/DashboardAccessTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Access/Round3Tests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Account/AccountConnectTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Account/AccountModelTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Account/ConnectCallbackSourceTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Account/NoDashboardLinkSourceTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Appeals/AppealsReviewTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Applications/ApplicationFormsTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Applications/ApplicationPassthroughTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Applications/ApplicationSubmissionsTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Applications/ApplicationWireTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Audit/AuditHistoryTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Audit/AuditLogTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/CallbackIssuerAddressTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/CallbackIssuerTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/KeychainTokenStoreTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/PKCETests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/SignInRequestTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/TokenStoreTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Changelog/ChangelogTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Commands/CommandOverridesTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/CollectiblePictureTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/EconomyAdminTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/EconomyMembersTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/InventoryTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/ShopExtrasTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/ShopItemsTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/ShopModelTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Embeds/EmbedDraftTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Embeds/ModuleLimitsTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Events/CommunityReviewFixesTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Events/EventEndpointsTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Giveaways/GiveawayActionsTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/GuildSettings/CommandPrefixTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/GuildSettings/GuildSettingsTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Health/HealthTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Import/P6Tests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Insights/InsightsTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Leaderboard/LeaderboardTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Members/MemberDataTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Members/MemberSearchTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Messages/ComposedMessagesModelTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Messages/ComposedMessagesTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Models/ModelDecodingTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Models/ServerListTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Models/Sylo52PolishTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Models/UnknownKeysTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Models/Utf16TextLimitTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/Moderation/ModerationTests.swift — full-run-safety
  • apps/sylo-ios/SyloKit/Tests/SyloKitTests/ModuleSettings/AfkSettingsTests.swift — full-run-safety
  • … and 645 more

non-code file(s) changed (README.md, docs/dashboard-v2.md, docs/fluxer.md, docs/mirror.md, docs/mobile-app.md …) → running the full suite for safety (a code graph can't see config/fixture/data deps)

changed code file(s) with no mapped test (README.md, docs/dashboard-v2.md, docs/fluxer.md, docs/mirror.md, docs/mobile-app.md …) — a coverage gap or a missing link — running the full suite rather than only the selected tests

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant