Skip to content

feat: add external API administrator UI - #809

Open
bballdavis wants to merge 102 commits into
DialmasterOrg:devfrom
bballdavis:feature/external-api-gui
Open

bballdavis wants to merge 102 commits into
DialmasterOrg:devfrom
bballdavis:feature/external-api-gui

Conversation

@bballdavis

@bballdavis bballdavis commented Sep 8, 2026 •

Copy link
Copy Markdown
Collaborator

Dependency and scope

Depends on #810, which must be reviewed and merged before this PR. #750 is already merged. This remains the existing cumulative branch targeting dev; the runtime head included here is 9badb936aaec4a84a7403a9d75f03dd76785c4b7 (verified runtime CI).

The original GUI head 353d05bb and its integration history are preserved. UI-only comparison against the runtime head. Once #810 lands, this PR can be synchronized with dev for its final phase-three review.

Result

Adds Settings controls for external key permissions, rating/media limits, quotas, and channel grants, plus the administrator Requests page for video, channel, and deletion requests.

  • Uses actual channel database IDs from the production DTO. Creation/editing have independent state, abort obsolete reads, wait for grant loading, and prevent duplicate submissions. Failed saves retain editable values; expanding access uses the application confirmation dialog.
  • Handles approved-request retries after queue saturation, preserving the original channel-grant decision. Stale detail/filter responses cannot overwrite a later selection; failed detail loads do not offer decisions based on stale data.
  • Keeps raw numeric input editable, clamps removed pagination pages, resets cancelled actions, and exposes review buttons directly on mobile cards.
  • Shows legacy key errors when external access is disabled, clears one-time credentials when their dialog closes, and handles clipboard failures. External controls and Requests navigation use the effective server feature flag and fail closed while it is unavailable.
  • Keeps policy editing, channel selection, key cards, dialogs, queue loading, and review state in focused feature-local components/hooks. Documentation and Storybook reflect the completed UI.

The follow-up cross-boundary review found unavailable channel grants were incorrectly returned as generic 500 errors. That fix and real-database HTTP/rollback regressions were committed to #810 first, verified in CI, and then merged here.

Persistence and dependency safety

Production backend code, migrations, and dependency locks match the verified runtime branch. The four merged #750 migrations remain authoritative; obsolete markers, duplicate migration chains, channel uniqueness/deduplication, and the obsolete GUI index migration are absent. Duplicate channel rows remain intact, and no global dependency override was restored.

Validation

  • Backend: 223 suites / 6,050 tests passed. Frontend: 284 suites / 5,211 tests passed; frontend line coverage is 89.46%. No skipped tests.
  • MariaDB 10.3.39 and MySQL 8.0.46: all 23 migration/runtime/HTTP integration checks passed on each.
  • Chromium with the production React build, production management/external routes, actual model queries, and disposable databases: nine flows passed on each engine. These cover creation/grant IDs, video approval, queue-full retry with a false grant choice, deletion approval/rejection, 390px mobile/keyboard behavior, stale-channel save rollback/retry, grant removal, regeneration/revocation, and feature-off gating.
  • Browser execution of the Storybook key-creation and mobile-request scenarios passed. ESLint, TypeScript, client/Storybook production builds, documentation checks/strict build/generated anchors, and MSW worker consistency passed.

Browser fixtures stub startup configuration and use synthetic download/provisioning/deletion executors. No real customer data or external-service operations were used. Neither PR has been merged or deployed.

Published head: 1f20ea4ad1780a5ec94c6f24478992c7dd5ec2a9. CI: all 11 checks passed on this exact head, including both database engines and their query-plan artifacts. The existing review request is retained; no new review request is being sent while #810 remains the prerequisite.

@bballdavis
bballdavis force-pushed the feature/external-api-gui branch from c5febdc to 2743176 Compare September 8, 2026 23:23
@bballdavis
bballdavis force-pushed the feature/external-api-gui branch from 2743176 to fb04fad Compare September 8, 2026 23:36
@bballdavis
bballdavis force-pushed the feature/external-api-gui branch from fb04fad to 4961ba0 Compare September 9, 2026 12:50
Comment thread client/src/components/Configuration/sections/ApiKeysSection.tsx Outdated
Comment thread client/src/components/Configuration/sections/ApiKeysSection.tsx Outdated
Comment thread client/src/components/Configuration/sections/ApiKeysSection.tsx Outdated
Comment thread client/src/components/Configuration/sections/ApiKeysSection.tsx Outdated
Comment thread client/src/components/Configuration/sections/ApiKeysSection.tsx
Comment thread client/src/components/ExternalRequests/RequestsPage.tsx Outdated
Comment thread client/src/components/ExternalRequests/RequestsPage.tsx Outdated
Comment thread client/src/components/ExternalRequests/RequestsPage.tsx
Comment thread client/src/config/configSchema.ts
Comment thread client/src/components/ui/select.tsx Outdated
@bballdavis
bballdavis force-pushed the feature/external-api-gui branch 2 times, most recently from ffca52a to dc54292 Compare September 19, 2026 05:05
@bballdavis
bballdavis marked this pull request as ready for review September 19, 2026 05:09
Restore the downloads-directory and cookie temp-dir log lines, the
x-request-id correlation header outside the external API namespace,
TV-Y7's text rating fallback, and unredacted yt-dlp stderr in server
logs. Drop the bare 'key' redaction path so config upgrade logs keep
naming the corrected setting. None of these were needed by the API.
Dev reworked the API keys page to lead with the "Send to Youtarr"
bookmarklet. Keep that: the bookmarklet steps, key list and rate limit
come first, and the key-created dialog shows the draggable bookmarklet
and phone shortcut details again. The external access keys section
follows only when the external API is enabled.

The page is split into BookmarkletKeysSection, ExternalKeysSection,
CreatedKeyDialog and ConfirmKeyActionDialog to keep ApiKeysSection
under the size target.
Restore the blank line the earlier edit removed so the only change in
select.tsx is the isTrusted check on the synthetic mousedown opener.
The 5/30/200 per-key ceilings were repeated in the policy defaults, the
fallbacks for older keys, the editor fields and the validation. Define
them once in WORKLOAD_LIMITS next to normalizePolicy, and move the
useApiKeys test into the sections __tests__ folder.
- Move RequestActions, RequestTarget, RequestDetailsDialog and
  RequestActionDialog into ExternalRequests/components/, and the
  misplaced hook test into hooks/__tests__/, matching the feature
  directory layout used elsewhere.
- Rewrite RequestTarget as readable JSX with lookup tables for type
  labels and status colors instead of nested ternaries on one line.
- A video with no known channel now reads "Unknown channel" instead of
  "Channel <database id>"; pending channel requests still read "New
  channel request".
- Cancelling an approve or reject started from the details dialog goes
  back to the details; one started from the list goes back to the list.
  Every new action resets the grant choice.
- Capitalize status names in the status filter, reuse one label helper
  for it and the status chip, and name the page size constant.
- Add tests for RequestTarget, RequestActions and the cancel paths.
USAGE_GUIDE: restore the single-video note for bookmarklet keys, drop
the legacy wording, and explain that the external keys section and the
Requests page only appear with EXTERNAL_API_ENABLED=true, what the nav
toggle does, and that the queue works without login when
AUTH_ENABLED=false.

CLAUDE.md: replace the thin ExternalRequests one-liner with what each
file owns, add an ApiKeysSection entry, and drop the stray line that
had landed above App.tsx in the frontend list.
…UTF-8 bodies

With AUTH_ENABLED=false, verifyToken never sets req.authType, so the
review routes' session check returned 403 for every call. Deny API keys
instead, matching the key-management routes. External API POST bodies
declaring a charset other than UTF-8 now get a 415 rather than a parse
error or 500.
Move the external API services, their tests, and the channel grant module
into a feature folder like channel/, mediaServers/, and notifications/.
Give TV-Y7 its PG-level policy rank inside externalEligibility instead of
changing ratingMapper, which also drives NFO output for every video.
Express routes /EXTERNAL-API/v1/... into the external router, but the
app-level body parser bypass, error envelopes, degraded-database 503, and
request id handling compared req.path case-sensitively. Share one
isExternalApiPath check that matches the mount the way Express does.
Creating or approving a request now only records the decision and
charges quota. A small in-process executor picks approved rows up
outside any lock, re-checks the key's authority, and does the slow work,
so no yt-dlp call, download queueing, or file unlink runs under a row
lock or inside an HTTP handler:

- Channel provisioning resolves the channel without enabling it and
  takes locks only for the final write, where a terminated channel now
  fails the request instead of being re-enabled.
- Deletions go through deleteVideos, so the channel .m3u, playlists,
  and storage pause are refreshed like an in-app delete.
- A storage pause or a busy video holds the request approved instead
  of failing it; the executor's sweep retries it.
- A jobModule.onJobEnded listener closes video requests when their job
  ends, so ?status=completed no longer depends on who reads the list.
  Reads and quota checks no longer reconcile or lock anything.
- A new approval_source column records whether a request was
  auto-approved, so the executor re-checks auto-approval only for
  those rows.
- The download job id is passed as an option to doSpecificDownloads
  rather than read from the job data, and addJob rejects a preferred
  job id that is not a UUID.
Approving now only queues the work; the server's executor moves the
request from approved to processing to a final status, and approving
an approved or processing request again does nothing.

- Only pending requests can be approved, so the "Retry approval" path
  and its locked grant checkbox are gone.
- The queue refreshes itself every 5 seconds while any request on the
  page is approved or processing (not while the tab is hidden), without
  the loading spinner, and an open details dialog follows the refresh.
- The details dialog says whether a request was approved automatically
  or by an administrator.
- The approve dialog says the work runs in the background.
Creating the request service in server.js eagerly required the channel,
deletion and storage modules, which broke the server startup suites.
- Video detail reads metadata only from the cached .info.json and never
  runs yt-dlp or writes backfills.
- Raw catalog SQL uses snake_case aliases and POSITION() instead of LOCATE().
- Video list ordering and keyset cursors (now v3) tie-break on channel video id.
- Single-channel lookups use Channel.findOne with the grant include.
Document approval, background execution and held requests, cached-only
video detail, auto-approved channel grants, delete and idempotency
responses, the work queue env vars and the approval_source column. Bump
the contract fixture to version 5 for the approved create response.
A background wake after approval could sweep before the test changed
state (for example revoking a grant), which made the database job fail
at random on either engine. The wake scheduler is now injectable and
the database tests turn it off.
@bballdavis
bballdavis force-pushed the feature/external-api-gui branch from 768d826 to 446058c Compare October 22, 2026 13:37

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants