Repository navigation
feat(mirror): mirror a Discord server to Fluxer and keep it in step - #386
Ferdinand99 wants to merge 4 commits into
Conversation
A Discord server can be copied to a Fluxer server: categories, channels with their properties and permission overwrites, roles with their permissions mapped bit by bit, emoji and stickers within Fluxer's limits, the server's name, icon and banner, and Sylo's own module settings with channel and role ids remapped. Afterwards channel, role, emoji and sticker changes on Discord are applied on Fluxer (one way, and Sylo never deletes what it did not create). - migration 84: mirror pairs, the id map and a job queue, so a copy resumes after a restart and never makes anything twice - dry run with a report of everything that cannot be carried over; the real copy builds the same specs from the same plan - consent from both servers (Edit on each side, a one-time code or the same person on both) behind requirePairConsent() - per-server rate-limited queue with backoff; a mirror that keeps failing pauses itself and shows up in Health - "Mirror to Fluxer" page in the dashboard, classified in routeAccess.js - `mirror` feature string in /api/mobile/info, docs/mirror.md - the final bridge-linking step is a stub (linkMirroredChannels) until the bridge lands
There was a problem hiding this comment.
Graphify reviewed this change.
Code graph out of date: Health delta baseline: last indexed commit a9ec8af (diverged from this PR's base — delta is approximate). Impact and hotspot numbers below may be wrong.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.
Not checked: tests were not run; no formal proof of the changed code.
Graphify review — findings
Adds a "Mirror to Fluxer" feature that copies a Discord server's categories, channels, roles, emoji, stickers and Sylo module settings onto a Fluxer server, then keeps the copy in step one-way (Discord to Fluxer). Both servers must agree before anything is created, either automatically when the same account has Edit access on both or via a one-time code entered on the Fluxer side. The flow is ask, agree, preview (dry run), copy, report, with pause, resume and stop from either side. A failing mirror pauses itself and appears in Health, and it is advertised as the mirror feature in GET /api/mobile/info.
Worth a look
- requeue: twin check and requeue are not atomic, so two waiting jobs for one step can result —
src/db/mirror.js· Escalate · medium- NOT verified (no proof, no reproducing execution) — needs human review
- requeueJob can violate the partial unique index when a coalesced twin was inserted during the run —
src/db/mirror.js· Escalate · medium- NOT verified (no proof, no reproducing execution) — needs human review
- finishJob/requeueJob lack a status guard and can overwrite a job that dropOpen already skipped —
src/db/mirror.js· Escalate · medium- NOT verified (no proof, no reproducing execution) — needs human review
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 1231 functions depend on the 791 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
apiFetch()— 65 callers, 3 callees - worse:
purgeGuild()— 35 callers, 3 callees - new:
requestMirror()— 5 callers, 10 callees - worse:
safeFetch()— 9 callers, 5 callees - new:
buildChannelSpec()— 5 callers, 8 callees - new:
buildRoleSpec()— 4 callers, 6 callees - new:
buildPlan()— 1 callers, 15 callees - new:
RunningStep()— 1 callers, 14 callees - …and 12 more — each is listed as a finding
Verification — 1231 functions in the blast radius were not formally verified this run (proofs are advisory here).
Health delta baseline: last indexed commit a9ec8af (diverged from this PR's base — delta is approximate).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 1227 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
683 of 683 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety (
noncode-change,uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.
apps/sylo-ios/SyloKit/Tests/SyloKitTests/Access/DashboardAccessTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Access/Round3Tests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Account/AccountConnectTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Account/AccountModelTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Account/ConnectCallbackSourceTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Account/NoDashboardLinkSourceTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Appeals/AppealsReviewTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Applications/ApplicationFormsTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Applications/ApplicationPassthroughTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Applications/ApplicationSubmissionsTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Applications/ApplicationWireTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Audit/AuditHistoryTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Audit/AuditLogTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/CallbackIssuerAddressTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/CallbackIssuerTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/KeychainTokenStoreTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/PKCETests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/SignInRequestTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Auth/TokenStoreTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Changelog/ChangelogTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Commands/CommandOverridesTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/CollectiblePictureTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/EconomyAdminTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/EconomyMembersTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/InventoryTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/ShopExtrasTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/ShopItemsTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Economy/ShopModelTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Embeds/EmbedDraftTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Embeds/ModuleLimitsTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Events/CommunityReviewFixesTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Events/EventEndpointsTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Giveaways/GiveawayActionsTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/GuildSettings/CommandPrefixTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/GuildSettings/GuildSettingsTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Health/HealthTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Import/P6Tests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Insights/InsightsTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Leaderboard/LeaderboardTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Members/MemberDataTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Members/MemberSearchTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Messages/ComposedMessagesModelTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Messages/ComposedMessagesTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Models/ModelDecodingTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Models/ServerListTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Models/Sylo52PolishTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Models/UnknownKeysTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Models/Utf16TextLimitTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/Moderation/ModerationTests.swift— full-run-safetyapps/sylo-ios/SyloKit/Tests/SyloKitTests/ModuleSettings/AfkSettingsTests.swift— full-run-safety- … and 633 more
non-code file(s) changed (
README.md,docs/dashboard-v2.md,docs/fluxer.md,docs/mirror.md,docs/mobile-app.md…) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
README.md,docs/dashboard-v2.md,docs/fluxer.md,docs/mirror.md,docs/mobile-app.md…) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
…ole order, privacy and grant checks Review round 1 for the server mirror. - Only the owner of the Discord server and the owner of the Fluxer server can ask for, agree to, change, start, pause, resume, stop or remove a mirror (guildOwner.js, the same helper the bridge uses). The owners' account ids are recorded with the agreement and checked again before every piece of work; a server that changes hands pauses the mirror until its new owner confirms. - A delete on Discord locks the Fluxer copy (channel hidden and read-only, role stripped) and marks it removed; real deletion only with "Mirror deletions" (off by default). More than five deletes in ten minutes pause the mirror until the owner confirms them. - The copy is blocked while Sylo's role on Fluxer is too low to give every mirrored role its own position; positions are always distinct and in Discord's order, and a new role with no room pauses the mirror. - A channel @everyone cannot see on Discord is hidden on Fluxer whatever else is copied; @everyone never gets powerful permissions; an age-restricted server flags its channels. - Copied module settings pass the dashboard's grant and ping checks, ids that mean nothing on Fluxer are cleared, and an explicit list keeps the operator's modules and the bridge out. - Only a real 404 means "gone"; a failed look-up, a timeout or a 5xx never makes a second copy; a role created but not finished is tracked; a job that may have gone through looks for what it made (made since the job started) first. - The SQLite migration is idempotent; thread permissions are mapped; Fluxer's real limits (500 emoji and stickers, 512 KB); a delete during the copy is not lost; a pause can be resumed only by the side that made it; old copy runs are pruned; a few mirrors run at a time.
…er ordering, Fluxer owner allows deletion
Review round 2 for the server mirror.
- The role-hierarchy check compares Sylo's rank on Fluxer (position, then the
older id), not its position number: a fresh server with Sylo at position 1 is
fine, a role above or level with Sylo's blocks the copy with advice naming the
roles and how the Fluxer owner moves Sylo's role. The same test pauses a
continuous reorder.
- A new role is made without its powerful permissions and gets them only after
the role order is set (the order step also runs for a single role); if the
order cannot be set the role stays without them.
- Real deletion ("Mirror deletions") is only asked for by the Discord owner and
takes effect when the Fluxer owner allows it; resuming with deletes waiting
lists them for confirmation, and with real deletion on only the Fluxer owner
can confirm.
- A 5xx whose status is only in the SDK error's cause is retried, a copied
economy comes over switched off, and the docs note which channels carry the
age flag.
… asked for at request time is only a request Review round 3 for the server mirror. - A role made, adopted or made again (a mirrored role deleted on Fluxer, then edited on Discord) is created without its powerful permissions, the order step runs (continuous mode asks for it at once), and only then are the permissions granted: one path for the first copy and for continuous mode. Adopted roles are queued for the same grant step. - "Mirror deletions" in a mirror request only takes effect when the same person owns both servers; otherwise it is stored as a request that the owner of the Fluxer server turns on with their own switch.
What
A "Mirror to Fluxer" page and engine (Sylo 5.3, plan section 4b). The server owner creates an empty Fluxer server, invites Sylo with the Administrator permission, and Sylo copies the Discord server's structure and keeps it in step.
owned); consent from an Edit user on both servers; each pair has its own queue, pace and failure count; rate limits andretry_afterare honoured, transient failures back off; a mirror that keeps failing pauses itself with the reason and shows in Health; the queue is in the database and resumes after a restart without making anything twice (a job interrupted mid-create adopts what it may have made, as something Sylo may not delete).src/web/lib/guildOwner.js, identical to the bridge's). An Administrator, an Edit holder or a bot master cannot; View users can look. The owners' account ids are recorded and checked again before every piece of work; a server that changes hands pauses the mirror until the new owner confirms.requirePairConsent()(setConsentProvider()swaps the check), so the bridge's link codes can replace it later. The Fluxer side agrees with a one-time code (hashed, one hour, five tries, only for that Fluxer server), or at once when the same person holds Edit on both servers.routeAccess.js(View to see, Edit to change) and rate limited.linkMirroredChannels(pairId)(src/mirror/bridgeLink.js), which returns{ linked: 0, reason: 'bridge-not-available' }for now and is shown as such in the report.mirrorin thefeaturesof/api/mobile/info; docs indocs/mirror.md,docs/mobile-app.md,docs/fluxer.md,docs/dashboard-v2.md.Migration
The migration is entry 84 of the 5.3 line (now
CREATE ... IF NOT EXISTS, and run twice in a test). Merge order at integration: after the bridge's 83, so this entry has to be moved to follow it.The migration is entry 84 of the 5.3 line. This branch is cut from
v5.3before 81 to 83 exist, so in this branch it is the next entry in the list (version 81). When it is merged after the bridge's 83 it must sit after it. Postgres needs no version number: the tables are created in the bootstrap ofsrc/db/mirror.js. The three tables have noguild_idcolumn (a pair belongs to two servers);purgeGuildremoves every pair a server is part of.Tests
SQLite and Postgres 16 full suites pass (4037 tests / 4238 tests), lint, prettier on the changed files,
test:v2(857) andbuild:v2.New tests: the permission table checked against discord.js's and the Fluxer SDK's own flag tables, spec building and the settings remap, consent (code, expiry, five tries, another server, replacing a request), the engine against a fake Discord and Fluxer server (dry run equals real run, resume after a restart, adoption after a crash, continuous events, never deleting foreign items, rate limit and backoff, pausing, isolation between pairs, emoji limits), the Fluxer adapter against an SDK-shaped double, the routes through the real app with both logins (cross-server access, CSRF, levels), the database layer on both drivers, and the page.
Not covered
Nothing was run against a live Fluxer server. The Fluxer limits Sylo plans with (50 emoji, 50 stickers, 256 KB and 512 KB, bitrate 8 to 96 kbit/s) are assumptions; if Fluxer refuses an item as over its limit the copy stops that kind and reports the rest.