diff --git a/docs-developers/architecture/cross-repo-contract.md b/docs-developers/architecture/cross-repo-contract.md index 64872ae8..6723ab2f 100644 --- a/docs-developers/architecture/cross-repo-contract.md +++ b/docs-developers/architecture/cross-repo-contract.md @@ -950,7 +950,8 @@ both repos, with a test on each side. [Read-along](../bridge/readalong.md). - **Setup and contributions (Phase 2b):** the heartbeat's `setup` block (the Bridge's onboarding checklist: `paired`, `libraries`, `asr`, `agent`, `budget`, - `github`, plus `can_contribute` and `github_login`) reaches the console only, as + `github`, plus `can_contribute` and `github_login`; `agent` and `github` are + optional) reaches the console only, as `setup` on `GET /admin/bridge` ([the checklist](../bridge/pairing-and-jobs.md#the-setup-checklist)). Two new kinds, both owned by the `contributions` module (`404 module_off` while it is @@ -1026,7 +1027,9 @@ both repos, with a test on each side. /bridge/progress?account=&since=&limit=` (`{rows, next, more}`, keyset on `progress.change_seq`, the account's own rows left out). Both answer `403 sync_off`, `403 user_unavailable`, `404 unknown_account`; the feed also `400 - invalid_request`. The account's `sync` is the console's now: the Bridge's PUT + invalid_request`. A `preorder` title takes no part in either direction (not in + the params, `not_synced` on the listening route, not in the feed), and its + release offers its book's place again (server `0071`). The account's `sync` is the console's now: the Bridge's PUT sends it and the server ignores it. The `audible:` device id is reserved (`/me/listening/sync` refuses it; a progress PUT saves it as none). Server migration `0066`, Bridge `0011`; the heartbeat gains no block. The Bridge mirrors diff --git a/docs-developers/bridge/audible.md b/docs-developers/bridge/audible.md index 802b713c..bef1d48c 100644 --- a/docs-developers/bridge/audible.md +++ b/docs-developers/bridge/audible.md @@ -218,10 +218,13 @@ the new one (a claimed poll runs as claimed). 1. checks the account is this Bridge's and was mapped on the paired server (else it fails, asking to map it again); -2. lists the Audible library (a refusal is the account's: the job just fails), - then adds the account's pending [preorders](#preorders) +2. lists the Audible library, then adds the account's pending [preorders](#preorders) (`audibleacct.Manager.Preorders`: from the orders, remembered, described by the - catalog; it never fails the poll); + catalog; it never fails the poll). A refused listing fails the poll with a + coded `audiblejob.Failure` (result and error, as an import's): + `audible_refused` with `ThrottledText` for a refused device, else + `audible_sign_in` for a refused sign-in. The server folds no poll's code; the + console's Bridge jobs list words both; 3. reads the account's library on the server, 500 books a page (`GET /bridge/libraries/{id}/books`, at most 2000 pages); 4. skips records without a usable ASIN, and states the rest from @@ -237,9 +240,13 @@ the new one (a claimed poll runs as claimed). [Preorders](#preorders)), then `importjob.KeepBest` settles books claimed by several titles. `how` is `asin` when the book's ASIN is the title's, else `fuzzy`. A book path goes out only as `.` (a library that is one book) or a - clean path below the root, within the server's 4096-byte limit. The Bridge - skips over-limit candidates before building a titles PUT batch, so one - invalid path does not reject the other matches; + clean path below the root, within the server's 4096-byte limit + (`MaxBookPath`). A title matched to a book whose path is over it can't name + that book, so it goes out unmatched with `reason: "contested"` and no + `contested_path` (counted in `overlong`, a warning logs the path's length). + Plain unmatched, Back up everything would download a second copy of a book the + library already has; contested, it is skipped. One over-limit path never + rejects the other matches of the batch; 6. **plans** each unmatched title (contested included, never a matched one, nor a preorder with no title yet): where a backup would go, `planned` (see [Planned destinations](#planned-destinations)), @@ -249,15 +256,18 @@ the new one (a claimed poll runs as claimed). call, so every title becomes `gone`), with each title's authors cut to 20; 8. **backfills ASINs** (`PUT /bridge/libraries/{id}/enrichment`) for each matched book without one, unless the book is [contested](#matching-rules) (derive - nothing durable from a doubtful match) or the title is a preorder (not out yet: - the book is another edition). `asin_set` and a moved book are skipped, other + nothing durable from a doubtful match), the title is a preorder (not out yet: + the book is another edition), or the match is by words to a book with an + identical twin in the library (`importjob.Twins`: the same folded title, + authors and series; the match is kept, but the ASIN isn't stamped on an + arbitrary copy). `asin_set` and a moved book are skipped, other failures counted. The PUTs are paced (see [Enrichment pacing](#enrichment-pacing)). The result is counts only, never a title or a credential: `account`, `poll_id`, `titles`, `available`, `preorder`, `unavailable`, `skipped`, `matched`, `matched_asin`, `matched_fuzzy`, `contested` (books more than one title claimed), -`enriched`, `enrich_failed`, `enrich_deferred` (omitted when 0), `books`. +`enriched`, `enrich_failed`, `enrich_deferred` and `overlong` (both omitted when 0), `books`. `matched` counts the titles that kept a match. A poll that finished records `last_poll_at`, the counts and clears a refusal on the Bridge's row; one stopped, cancelled or lost records nothing. @@ -298,7 +308,10 @@ book's keeps it even over a higher score (`MatchScored` itself is not gated by it). Then the single best score keeps the match. A tie for the best keeps none, unless every tied title is the same work (`WorkKey`: the folded title, the folded authors in any order and the series position); then all keep it -(one book owned twice, under two ASINs). A losing title whose position +(one book owned twice, under two ASINs). A title that folds to nothing has no +key, nor does a title with no series position that is just its series' name +(folded alike, or alike under `match.NormalizeSeries`): Audible lists some +volumes so, and a tie between them keeps none (contested) rather than all. A losing title whose position **disagrees** with the book's is another volume: it goes out plainly unmatched, no reason. The other losers are PUT with no `match`, `reason: "contested"` (`link.ReasonContested`, set only on a title with no other reason) and @@ -398,7 +411,8 @@ library's): `store/audiblepreorders.go`): one row per `(account, asin)` with `ordered_at` (the order's date, else when first seen), `last_seen_at` (last seen pending in the orders), `record` (the catalog's last description as JSON, `''` until a read - succeeds; a cache the next fill replaces) and `filled_at`. No credential, no + succeeds; a cache the next fill replaces) and `filled_at` (no longer written + or read; the column stays so a downgrade works). No credential, no payment field. Each poll reports every remembered row, so a preorder whose order has left the six-month window is still reported. The rows go with the account (`ON DELETE CASCADE`). @@ -413,7 +427,10 @@ library's): (cancelled). A delivered order (`Order.Delivered`: `state` `ORDER_CLOSED`, `detail_state` `ORDER_SUCCEEDED`) keeps it a preorder until the library lists it, which can be a little after the order closes, so it never drops out of a final - poll in between; + poll in between. That wait is capped: a delivered one the library still + doesn't list is dropped `DeliveredDropAfter` (14 days) after the later of its + `last_seen_at` (last seen pending) and its release date, both bounds of when + the order closed (a refund, or a release under another ASIN); - the listing's oldest order is older than its `ordered_at` and the listing doesn't hold it: it was in the window and is gone, so it was cancelled (an empty listing, or one with no dated order, drops nothing); @@ -550,10 +567,11 @@ a park's code, else the result's `code`, else an error starting with a known cod | `disk_full` | free space short in the Bridge's temp folder (2.2x the size, less what the partial holds) or at the library's nearest existing folder (1.1x, `transfer.DefaultHeadroom`), checked before a byte is written; or ENOSPC/EDQUOT during the download, decrypt (ffmpeg's words too) or copy. A free space that can't be read is no refusal | | `offline_url_expired` | the presigned download URL answered 401/403/410 again after the one new license | | `unknown_account` | not this Bridge's account, or mapped on another server | +| `drm_free` | Audible offers the title only as a DRM-free MP3 (`audible.ErrDRMFree`), which the Bridge doesn't back up yet | -A title offered only as a DRM-free MP3 (`audible.ErrDRMFree`) fails with no code -(not backed up in 4b), as do a download over `DownloadOptions.MaxBytes` -(`audible.DefaultMaxDownloadBytes`, 4 GiB) and other errors. +A download over `DownloadOptions.MaxBytes` (`audible.DefaultMaxDownloadBytes`, +4 GiB) and other errors fail with no code. The server's `bridge.AudibleImportCodes` +lists every code above, and the console words each one. **Not writable while running.** A mapping check not made yet for the mount is waited for (`mappingPoll`, 1 s), no park. An unwritable mapping refuses a **new** @@ -875,7 +893,13 @@ session, `listening_daily` or history row is ever written for Audible's listenin backups never read it. A title syncs when `syncTitleSQL` holds: matched, not `rejected`, not `ignored`, -not `no_sync`, not `gone`. An account syncs while the module is on, its `sync` is +not `no_sync`, not `gone`, not `preorder`. A poll may match a preorder to another +edition on disk, and Audible holds no place for an unreleased ASIN, so the server +leaves a preorder out of both directions: not in `audible.sync`'s params, +`rejected` `not_synced` on the listening route, and not in the progress feed. The +Bridge's own `skipped_preorder` stays as a second guard. Once the title is +released (`preorder` to `available`), `0071`'s trigger offers its book's place to +Audible again, as for a title back from `gone`. An account syncs while the module is on, its `sync` is on, it isn't paused, `refused_at` is NULL and its person may still be synced for (`bridgeUserSQL`: not disabled, not a kid, not a demo account). @@ -936,7 +960,22 @@ not negative. 6. records `heard_ms` for each place the server answered (not rejected): `store.AudibleHeard` clears a `last_ms` written before the job began, and an applied one drops a place the push was waiting to write from the row it had - then (it would move Audible back). + then (it would move Audible back); +7. **re-asserts** a place the Audible app overwrote. The app may upload the older + place it still held over one the push wrote, and the sync then reads it as + `behind`. When the Bridge wrote the title's last place (`last_ms`), the server + row it last read is still that place (within 5 s) and Audible's place is more + than 5 s behind it, `store.RearmAudiblePush` sets `want_ms` to the row's place + and `row_seen` to now, so the push writes it again after `StillFor`. It is + guarded in SQL: only while the title is listed, nothing is waiting, and no push + and no new server row came since the job read it. The push's read before the + write still drops it if Audible moved further on meanwhile. + **At most once per written place** (`reasserted_ms`, Bridge migration `0014`): + a sync can't tell the app's stale upload from a listener who rewound in the + Audible app and listens on there. So once a place was written again, Audible + behind it again is left alone (an Info log says so) until the push writes a + new place (`AudiblePushed` of another place, or another book for the title, + resets it). A `duplicate` answer re-arms nothing. `sync_off` from the server ends the job quietly (done, `sync_off`); `user_unavailable` fails it with that code. The **event id** is @@ -948,8 +987,8 @@ again to another book is a new event. The job's **result** is counts and flags only: `account`, `titles`, `heard`, `sent`, `finished`, the server's `applied`, `duplicate`, `behind`, `not_applied`, -`rejected`, and the skips (`skipped_echo`, `skipped_past_runtime`, `skipped_old`, -`skipped_start`, `skipped_preorder`), `mismatched` (events the server rejected +`rejected`, `reasserted` (step 7, omitted when 0), and the skips (`skipped_echo`, +`skipped_past_runtime`, `skipped_old`, `skipped_start`, `skipped_preorder`), `mismatched` (events the server rejected `duration_mismatch`: counted, sent once a job and never retried in it); `waiting` with `wait_until`, or `sync_off`. `titles` counts the titles kept, after `skipped_preorder`. A failure's result is @@ -961,7 +1000,11 @@ Audible's places (`bridge.AudibleSynced`: neither `waiting` nor `sync_off`) sets `audible_refused:` error prefix, `AudibleSyncCode`) sets `refused_at`, which stops the scheduler. Any account PUT of the Bridge's (a fresh sign-in, an edit) and a complete poll (`PutAudibleTitles` with `final`: Audible answered the account) clear -it. +it and stamp `refusal_ended_at` (`0071`). A refused sync claimed in an earlier +second than that stamp ran before the fresh sign-in, so it marks nothing +(`refusalEndedSince`; `claimed_at` is whole seconds, so one claimed in the same +second still marks). `audible_sign_in` (`bridge.SyncAudibleSignIn`) is a sync's +code only, never an import's, and an ordinary failure. ### The server's side of a batch: furthest only @@ -971,7 +1014,9 @@ TokenID: 0, FurthestOnly: true}`. No request body can set `FurthestOnly` or `TokenID`: `POST /me/listening/sync` refuses an unknown field. Each event's book is resolved under the **person's** scopes (`userPathResolver`, `catalog.UserScopes`), then must be a synced title's match (`AudibleSyncBooks`) whose Audible length -agrees with the book's (`audibleRuntimeAgrees`, listen and finish alike). +agrees with the book's (`audibleRuntimeAgrees`, listen and finish alike). The +book's length is its indexed duration, else the `duration` on the person's own +progress row for it: the progress feed's rule, so the two never disagree. `FurthestOnly` (`furthestListen`) then applies only a place more than `furthestSlack` ahead, over any device's row, keeping the row's `updated_at`, speed and offer, and compares `finish_changed_at` for a place from before a finish @@ -1067,7 +1112,9 @@ The Bridge mirrors them by hand in `internal/link/audiblesync.go` `last_ms`/`at` written to Audible, `heard_ms` sent from it (each clears the other: `AudiblePushed`, and `AudibleHeard` for a write before the job began), `want_ms` waiting, and the feed row last read: `row_ms` and `row_seen` (when this Bridge -first read that place, the stillness clock). On `audible_accounts`: `push_cursor`/`push_from`, the health columns +first read that place, the stillness clock). Migration `0014_audible_reasserted` +adds `reasserted_ms` (`-1` none): the written place a sync already wrote again +once (step 7 above). On `audible_accounts`: `push_cursor`/`push_from`, the health columns `sync_at`/`sync_error`/`sync_retry_at`, and `refusal_kind` (above). ## On the server (`0063`, `catalog/audible.go`) @@ -1089,8 +1136,11 @@ the accounts, `no_sync` to the titles and `change_seq` to `progress` (see library was deleted"), so no titles call can match into it in between. The scheduler skips a paused account and Check now is `409 account_paused` until the Bridge maps it again. -- A title's match **moves with its book**: a move (`moveBookState`, a scan's move - detection or a Bridge rescan's `from`) and a join (`joinBookState`) carry it. +- A title's match **moves with its book**: a move (a scan's move detection or a + Bridge rescan's `from`) and a join carry it, in the per-user transaction after + the book's places (`moveAudibleTitles`). So the re-match trigger offers the moved + book's place to Audible, never a stale row an earlier book left at the new path, + and a carry that fails leaves the titles where they were. - Unpairing keeps the rows; deleting a user removes their accounts (the FK). - The Bridge's writes are user-state writes (`store/userstate.go`): titles and the poll times don't move the content generation; a mapping change does. diff --git a/docs-developers/bridge/chapters.md b/docs-developers/bridge/chapters.md index 7034ebe0..16f009f6 100644 --- a/docs-developers/bridge/chapters.md +++ b/docs-developers/bridge/chapters.md @@ -224,10 +224,20 @@ resumed job's own book found again, moves followed), and when new: whole run, which it never clears. 2. **Writes from an earlier run** with no new transcription when an engine book at the source path already has a `chapters_detected.json` current for its manifest. -3. Else **creates** an engine book with goal `chapters` and no series, and follows +3. **Takes up a cancelled chapters run** (`cancelledChapters`, `resumeChapters`): + a goal-chapters run on a single file that was **cancelled** at `asr` or + `chapter_detecting` (`scheduler.IsCancelled`) continues, under the engine's + hold, through `store.ContinueReadalong` with goal `chapters` + (`resumesAt`, shared with the generate continue below: `splitting` for a + continuable transcription, `chapter_detecting` for a rerunnable detection). Its + transcription is kept, and the log says "taking up the book's stopped chapters + run, its transcription kept". A run that **failed** for a reason of its own, is + parked, or can't be trusted (the file replaced, the windows gone, swept) is + cleared, so asking again never fails the same way forever. +4. Else **creates** an engine book with goal `chapters` and no series, and follows it: its progress is generate's without `cost_usd` (`stage` `chapter_detecting`, the ETA, a park's code and hint). -4. At `done` it writes `chapters.json` and merges `bridge.json` (every other field +5. At `done` it writes `chapters.json` and merges `bridge.json` (every other field kept, the [`chapters` object](attachments.md#chaptersjson) added or replaced), then asks for the attachments rescan. @@ -276,6 +286,10 @@ job's identity, `contrib_mode` and `work_id`, its status, error, park and retry cleared, and the new job is mapped to it (a job cancelled meanwhile leaves the run as it was). +Give-way waits wake when a job ends: each job end (`release`) closes the +handler's end signal, and `awaitChaptersGivingWay` and `awaitOpenJobEnd` wake on it, +with the poll kept as the bound (both bounds are time, not a count of wake-ups). + The server keeps the two kinds apart (the table in [Kinds are allow-listed twice](pairing-and-jobs.md#kinds-are-allow-listed-twice)): a generate job is refused while a detect job is queued or working (until an admin diff --git a/docs-developers/bridge/development.md b/docs-developers/bridge/development.md index ba49c02c..f854f051 100644 --- a/docs-developers/bridge/development.md +++ b/docs-developers/bridge/development.md @@ -109,7 +109,10 @@ internal/ config/ config.yaml + AUDIOSILO_BRIDGE_* (bridge.go: the Bridge's own settings) store/ SQLite, append-only migrations (0001 the engine, 0002 server_jobs, 0003 generate_books, server_jobs.server_id, books.contrib_mode, - 0004 workers, 0005 books.goal, 0006 accounts) + 0004 workers, 0005 books.goal, 0006 accounts, 0007-0008 podcasts, + 0009-0012 and 0014 Audible, 0013 idx_books_unfinished: a partial + index whose WHERE is unfinishedBook's word for word, so the + periodic passes scan only the books the engine may act on) -- Phase 2b -- ghauth/ the GitHub App's device flow, refresh, revoke and /user; a leaf accounts/ the Accounts tab: the GitHub sign-in and the agent credentials (vault-sealed) @@ -174,7 +177,11 @@ Conventions that carry the design: ## The page and its API The Bridge serves its own page and a small JSON API under `/api` on its `listen` -address (`:8091`). The page has seven tabs: **Status** (led by a **Get set up** card, +address (`:8091`). It follows the system colour scheme: dark by default, and under +`prefers-color-scheme: light` `index.css` redefines the brand tokens and the state +shades it uses with the server console's light palette (`src/theme.test.ts` fails +a token or a used shade the light block misses). `public/favicon.svg` (also in the +placeholder page) is its icon. The page has seven tabs: **Status** (led by a **Get set up** card, the [setup checklist](pairing-and-jobs.md#the-setup-checklist) with an **Open** link to the tab of each step still to do, until every required step is done; pairing, the last heartbeat, the job kinds and lanes, where the secret key came from, and a **Characters and @@ -190,7 +197,9 @@ with cancel, a `bridge.verify` result as a file table, and for a `sidecars.generate`, `sidecars.describe` or `chapters.detect` job its stage, progress, cost, the park's code, hint and retry time, and **Retry** while it is parked; once done, the files it saved and whether a description came, for a describe job what its check measured against, and -for a chapters job how many chapters it found), **Settings** (its public address, +for a chapters job how many chapters it found; a known kind is named in words +only, its raw kind the label's tooltip, and a park leads with its hint for people, +the engine's own message under a collapsed **Details** line), **Settings** (its public address, the check interval, the ASR fallback, the monthly agent budget, the password), **Accounts** (Phase 2b: the GitHub sign-in and the agent's credentials; see [Contributions](contributions.md#the-pages-routes)), **Workers** (the [ASR workers](workers.md): add one, its token shown once in the commands to run, its state and figures, remove) and **Logs** diff --git a/docs-developers/bridge/pairing-and-jobs.md b/docs-developers/bridge/pairing-and-jobs.md index d660cf5f..145eec7b 100644 --- a/docs-developers/bridge/pairing-and-jobs.md +++ b/docs-developers/bridge/pairing-and-jobs.md @@ -165,7 +165,7 @@ Bridge's pure `setup.Checklist` from what the app already knows, in a fixed orde | `paired` | yes | the Bridge is paired | | | `libraries` | yes | the Bridge has checked libraries and every one is mapped and writable (a library counts once it has been checked) | "checking the libraries..." (`setup.DetailLibrariesChecking`, from v0.5.0) from pairing until a heartbeat has listed the server's libraries and each has been checked (`Input.LibrariesPending`: no `lastBeat` yet, or `libs.Unchecked()`), never done and no `library_ids` meanwhile; then "1 of 2 libraries writable" (or "the server has no libraries yet"); `library_ids` lists the ones still to fix | | `asr` | yes | the engine has started and something can transcribe now, not waiting for a worker | `local`, `worker`, "waiting for an ASR worker", "the engine is starting", or why it isn't available | -| `agent` | yes | the agent container is available (with its own or a handed-over credential) | the backend, or why it isn't ready in plain words (below) | +| `agent` | no | the agent container is available (with its own or a handed-over credential); only characters, recaps and descriptions need it | the backend, or why it isn't ready in plain words (below) | | `budget` | yes | the monthly budget was saved on the page or is set by the environment (`Config.BudgetReviewed`) | "$100 a month" or "no monthly cap" | | `github` | no | a GitHub token resolves and isn't refused | the login, "a token from the environment" or "the gh CLI's token" | | `audible` | no | Phase 4a: at least one Audible account signed in, none refused and none on an older device profile (an additive id: older servers and pages leave it out) | "1 account signed in", "1 account needs signing in again", or "... signing in again to keep backups working" when any is on an older profile; Phase 4c adds "; places don't sync until then" while one is refused, and "; places not syncing for 1 account: " and the reason when an account's last position sync failed (that alone never makes it not done) | @@ -176,7 +176,7 @@ Bridge's pure `setup.Checklist` from what the app already knows, in a fixed orde {"id": "paired", "done": true, "required": true}, {"id": "libraries", "done": false, "required": true, "detail": "1 of 2 libraries writable", "library_ids": [3]}, {"id": "asr", "done": true, "required": true, "detail": "local"}, - {"id": "agent", "done": false, "required": true, "detail": "set the agent's sign-in on the Accounts tab; the agent signs in to Claude or ChatGPT and is needed for characters, recaps and descriptions"}, + {"id": "agent", "done": false, "required": false, "detail": "set the agent's sign-in on the Accounts tab; the agent signs in to Claude or ChatGPT and is needed for characters, recaps and descriptions"}, {"id": "budget", "done": true, "required": true, "detail": "$100 a month"}, {"id": "github", "done": false, "required": false} ], @@ -200,9 +200,13 @@ recaps and descriptions"), but the backend mismatch, which names both CLIs: | `no_credential` | `DetailAgentNoCredential`: "set the agent's sign-in on the Accounts tab; ..." | | a refused credential (`Refused`) | the container's own detail, saying where to replace it (else `DetailAgentNoCredential`) | -The item stays required (FIRST-RUN F2). The console's Setup card shows the +The item is optional (`required: false`, an owner decision reversing FIRST-RUN +F2): podcasts, read-along, chapters and Audible need no agent, so setup reads +complete without an agent container. The console's Setup card shows the Bridge's detail when it sends one, and otherwise its own hint for the id -(`bridge.setup.hint.agent`, from server 2.5.1 also saying what the agent is for). +(`bridge.setup.hint.agent`, also saying what the agent is for). The console still +holds Generate back while the agent step isn't done, required or not +(`generateNeed`): a run without the agent would park `agent_unavailable`. The `asr` item's "waiting for an ASR worker" is a stable string (`setup.DetailASRWaiting`): from v0.4.0 the Bridge's Status tab, and the console's @@ -583,8 +587,11 @@ listener routes. (`template` the account's placement template, omitted when empty) with no library or path, deduped per account, and it promotes an open poll to a person's priority when Check now asks. It runs in the `light` lane and writes nothing into a -library (an ASIN it backfills goes through the server). The module's scheduler -queues it; queued polls are held while the module is off. See +library (an ASIN it backfills goes through the server). A refused poll fails +with a code in its result, `audible_refused` (the device) or `audible_sign_in` +(the sign-in, for now); the server folds neither, and the console's Bridge jobs +list words both. The module's scheduler queues it; queued polls are held while +the module is off. See [Audible](audible.md#the-poll-audiblepoll). `audible.import` (Phase 4b) backs up one title into the account's library. It diff --git a/docs-developers/server/api/index.md b/docs-developers/server/api/index.md index e954f606..e1f93809 100644 --- a/docs-developers/server/api/index.md +++ b/docs-developers/server/api/index.md @@ -260,6 +260,7 @@ Failures a person can fix also carry a machine-readable **`code`** next to |---|---|---| | `username_taken` | `409` | creating an account with a username that exists (`POST /admin/users`) | | `name_taken` | `409` | a library or share name already in use (create, rename) | +| `library_grant` | `409` | `POST`/`DELETE /admin/shares/{id}/paths` on a whole-library grant: its rules change only by granting or revoking the library | | `config_library` | `409` | a library the server's config declares: renaming it or moving its folder (`PATCH /admin/libraries/{id}`, from 2.4.0), or deleting it (`DELETE /admin/libraries/{id}`, from 2.4.1) | | `last_admin` | `409` | demoting or disabling the last enabled admin | | `admin_needs_password` | `400` | an admin without a password (creating one, promoting a password-less account, or clearing an admin's password) | @@ -287,7 +288,7 @@ Failures a person can fix also carry a machine-readable **`code`** next to | `invalid_setting` | `400` | `PATCH /admin/settings` with a value the server refuses (including a `demo.library` that names no library, and turning `metadata.enabled` on when no metadata service is configured); the body also carries `field`, the setting's id (`network.bind`) | | `unknown_setting` | `400` | `PATCH /admin/settings` naming something that isn't a setting (`field` names it) | | `setting_read_only` | `400` | `PATCH /admin/settings` changing a setting shown but not changeable there (`players.web_dir`, `backups.dir`; `field` names it) | -| `setting_locked` | `409` | `PATCH /admin/settings` changing a setting an `AUDIOSILO_*` variable or the launcher sets (`field` names it) | +| `setting_locked` | `409` | `PATCH /admin/settings` changing a setting an `AUDIOSILO_*` variable sets (`field` names it) | | `update_check_off` | `409` | `POST /admin/update/check` while the update check is turned off | | `not_mirror_mode` | `409` | `GET /admin/meta/mirror` or `POST /admin/meta/mirror/check` on a server that isn't keeping a local copy of the community metadata (remote mode, or `metadata.mode: mirror` saved but not restarted into) | | `backup_running` | `409` | `POST /admin/backups` while a backup is being made | @@ -452,18 +453,23 @@ round-trips reliably. The value is the slash-separated path relative to the library root, exactly as returned by `/fs` listings, book `rel_path` fields, and chapter `file_path` fields. -Two server-side guarantees apply to every `?path=`, plus a length cap on the -content routes: - -- **Length cap** - the content routes that address a book or file by - `?path=` (`item`, `chapters`, `cover`, `stream`, progress, bookmarks, notes, - history and the like, every route whose path goes through the server's - `pathInScope`) answer a path over 4096 bytes (as sent, - `catalog.MaxPathBytes`) with 400 `path is too long`, before it is cleaned, - looked up or stored. In a [listening sync](reference.md#post-apiv1melisteningsync) - batch such an event is `rejected` `invalid` and the rest of the batch applies. - Routes that take their path on their own (browse, removing from a list, the - podcast routes, the admin folder-override and enrichment routes) don't apply it. +Two server-side guarantees apply to every `?path=`, plus a length cap: + +- **Length cap** - a path over 4096 bytes (as sent, `relpath.MaxBytes`, which + `catalog.MaxPathBytes` names) is answered 400 `path is too long` by the content + routes (`item`, `chapters`, `cover`, `stream`, progress, bookmarks, notes, + history, `POST /cast/token` and the like), browse (`/fs`), removing from the + queue or a collection, the admin folder override (`PUT`/`DELETE`) and the + Bridge's book read. The check comes **first**: before the library is looked + up (an unknown or ungranted library answers 400 for such a path, not 404 or + 403), and before the path is cleaned or stored. In a + [listening sync](reference.md#post-apiv1melisteningsync) batch, the Bridge's + Audible listening batch included, such an event is `rejected` `invalid` + whatever its library, and the rest of the batch applies. The podcast routes + (unfollow, fetch, retention and policy) and the Bridge's ASIN enrichment refuse + such a path as their usual `400 invalid_request` (a show or episode path + `feed.ShowPath`/`EpisodePath` doesn't take, or no book path), and a podcast + show or episode folder name over the cap in a Bridge's feed is invalid. - **Traversal safety** - the path is resolved through `library.SafeJoin`, which rejects `..` escapes, absolute-path injection, and symlinks pointing outside the library root (400 `invalid path`). diff --git a/docs-developers/server/api/reference.md b/docs-developers/server/api/reference.md index 361ae637..0ac29cfb 100644 --- a/docs-developers/server/api/reference.md +++ b/docs-developers/server/api/reference.md @@ -563,7 +563,9 @@ the account had revokes every `session` token of the caller but the one making t request, with their open pairing codes (unlinked `pairing` tokens from [`POST /auth/pair`](#post-apiv1authpair), or an owner's [`POST /me/household/members/{id}/pair`](#post-apiv1mehouseholdmembersidpair) for a member) and their recovery code (`has_recovery` -turns `false`); API keys, the [calendar feed token](#the-calendar-feed) and Bridge +turns `false`), and their household members' sessions made on the caller's other +devices with every member's open pairing code (a member's own paired device stays); +API keys, the [calendar feed token](#the-calendar-feed) and Bridge tokens stay, and a first password ends nothing ([Password changes end sessions](../auth-and-security.md#password-changes-end-sessions)). An ended token is refused (`401`) on its very next request, media `?token=` included; @@ -878,7 +880,7 @@ The podcast branch caps a page at 2,000 entries and applies private scope. | Query param | Type | Default | Notes | |---|---|---|---| -| `path` | string | `""` (library root) | directory to list, relative to the root | +| `path` | string | `""` (library root) | directory to list, relative to the root; over 4096 bytes is `400 path is too long` | | `offset` | int | `0` | | | `limit` | int | `200` | values ≤ 0 or > 500 fall back to 200 | @@ -1015,8 +1017,8 @@ Keyset-paginated (see [conventions](index.md#pagination)). } ``` -Conditional book fields: `asin`/`isbn` appear only when known (attached via -[enrichment](#put-apiv1adminlibrariesidenrichment) or set by an +Conditional book fields: `asin`/`isbn` appear only when known (attached by a +Bridge's [ASIN backfill](#put-apiv1bridgelibrariesidenrichment) or set by an [admin edit](#patch-apiv1adminlibrariesidbook)); `codec` is omitted when never probed; `added_at` when unknown. Every book response (lists, search, recent, `/item`, `/next`) also carries, each omitted when empty: @@ -1534,7 +1536,8 @@ response's HTTP status. Codes include `invalid_feed_url`, `blocked_address`, `not_a_feed`, `not_a_podcast`, `not_podcast_library`, `library_not_writable`, `feed_secret_missing`, `request_expired`, `write_failed`, and server-only `bridge_lost`. A private request lost before claim on a server restart is -`request_expired`; follow again. Lost Bridge credentials are +`request_expired`; follow again. A private request whose requester's account was +deleted fails `request_expired` before any claim too, its URL dropped. Lost Bridge credentials are `feed_secret_missing`; follow the URL again to restore them. No podcast job sends a finished/failed notification. See [the Bridge implementation](../../bridge/podcasts.md). @@ -1742,9 +1745,11 @@ type: `application/xml`, `text/xml`, `text/x-opml` or `application/x-opml+xml`, with no parameter but `charset=utf-8` and no `Content-Encoding` but `identity`, else `415 unsupported_opml_type`. Then the caller's import budget (2/minute, burst 1: `429 opml_rate_limited` with `Retry-After`, or `503 opml_busy` with -`Retry-After: 1` while the server's per-user budget table is full), the body -(read whole, at most 1 MiB), and one of two server-wide import slots -(`503 opml_busy`, `Retry-After: 1`). +`Retry-After: 1` while the server's per-user budget table is full), whether one of +two server-wide import slots is free (`503 opml_busy`, `Retry-After: 1`, answered +before the body is read), the body (read whole, at most 1 MiB), then the slot. A +busy answer refunds the budget it took, so the `Retry-After` retry runs rather than +meeting `429`. The whole document is validated before any row acts, so a refused document changes nothing: @@ -2243,8 +2248,13 @@ Otherwise the local steps decide, in order, and the first one with an answer win omnibus of 1-3 is no step forward); a book inside the current omnibus in a series ranked above is passed over. The first series with a later book that isn't passed over answers. When none has a later book but some other book in one of them is - numbered, the answer is `{"source": "series"}`, the end of the series; when - nothing else is numbered the step falls through. + numbered, the series has ended: in the series' own folder (every other indexed + book there shares a series with this one, and there is at least one), a + folder no scan has reached after the book (the pending rule of step 3) answers + as a `folder` answer with `next` and no `book`, so a series still being + scanned is walked book by book. Otherwise, or with nothing pending, the answer + is `{"source": "series"}`, the end of the series. When nothing else is + numbered the step falls through. 3. **`folder`** - the book's parent folder, listed whole as the caller may open it (their share scope and the library's ignore rules): the first book or folder whose name sorts after the current one, preferring a book: an indexed one or, @@ -2962,7 +2972,7 @@ as stored, `audio/x-matroska`. See | `401` | missing/invalid token | | `403` | path outside the caller's scope | | `404` | file does not exist, or isn't audio; `code: "rendition_unavailable"` when the server has no ffmpeg or no data directory (`renditions` is `false`) | -| `422` | `code: "rendition_failed"`: ffmpeg refused this file, or its copy came out more than 1 s short of the source; remembered for an hour (or until the file changes), then tried again. `Retry-After` is the seconds the remembered refusal has left (1 to 3600) | +| `422` | `code: "rendition_failed"`: ffmpeg refused this file, its copy came out more than 1 s short of the source, or the encode ran past its 2-hour timeout; remembered for an hour (or until the file changes), then tried again. `Retry-After` is the seconds the remembered refusal has left (1 to 3600) | | `503` | `transcode=1` but ffmpeg is not configured (check the `transcode` capability), or every transcode slot is busy (`server busy transcoding; try again shortly`, with `Retry-After: 5`; also a cold `.mka` answered with the transcode); `the server is shutting down` for a remux or an encode cut short by shutdown; `code: "rendition_pending"` (`Retry-After: 15`) for a rendition still being made; `code: "rendition_error"` (`Retry-After: 60`) for an encode that failed for a reason that isn't the file's (low disk, a run cut short, the file changed while it was read, the encode queue full) | #### Watch renditions @@ -3023,7 +3033,7 @@ one file), each `path` as `/stream` takes it, with its `state`: |---|---| | `ready` | the rendition is cached; `bytes` is its size (sent only for `ready`) | | `pending` | queued or encoding (one encode runs at a time across the server) | -| `failed` | ffmpeg refused the file, or its copy came out short or with no audio track (the stream answers `422 rendition_failed`), tried again after an hour or once the file changes; `retry_after_seconds` is then the seconds until it is tried again (sent only for a remembered refusal, additive). Also no regular file at that path (gone, a folder: the stream answers `404`; no `retry_after_seconds`) | +| `failed` | ffmpeg refused the file, its copy came out short or with no audio track, or the encode ran past its 2-hour timeout (the stream answers `422 rendition_failed`), tried again after an hour or once the file changes; `retry_after_seconds` is then the seconds until it is tried again (sent only for a remembered refusal, additive). Also no regular file at that path (gone, a folder: the stream answers `404`; no `retry_after_seconds`) | | `missing` | not cached and not queued: always on a `GET` for a file never asked for; on a `POST` when the server's encode queue is full (at most 256 files wait or run across the server), so ask again later | | Status | Meaning | @@ -3237,7 +3247,7 @@ Response `200`: the map file's bytes as they are on disk. |---|---| | `Content-Type` | `application/json; charset=utf-8` | | `Content-Encoding` | `gzip` when the request's `Accept-Encoding` allows it (`Vary: Accept-Encoding`) | -| `ETag` | `"-"` of the file (the same for both encodings); a matching `If-None-Match` is `304` | +| `ETag` | `"-"` of the file, `"--gz"` for the gzipped answer (so a cache never pairs one encoding's validator with the other's bytes); a matching `If-None-Match` is `304`, checked before anything is gzipped | | `Cache-Control` | `private, no-cache` (the Bridge may rewrite the map at any moment) | **Header auth only**, as [`/ebook`](#get-apiv1librariesidebook): `?token=` is @@ -3646,7 +3656,7 @@ fixed-width UTC milliseconds. | `duplicate` | the id is already stored for this user, or an earlier copy in the same batch passed the checks: nothing was done again (a copy that was `rejected` doesn't count, so the next copy is checked on its own) | | `diverged` | the event was made without seeing the stored row (below); `kept` is `event` or `stored`, and `other_place` is the place **not** kept | | `not_applied` | stored, but the place didn't move; `reason` is `older` (an older write under last-write-wins, or a finish/unfinish not newer than the row's last finish-state change) or `resume_behind` (a `resume`/`continue` span behind the stored place, below; `other_place` is the event's place); `behind` only on the Bridge's [Audible listening route](#post-apiv1bridgeaudibleaccountsaccountlistening) (furthest only) | -| `rejected` | not stored and changed nothing; `reason` is `invalid` (a field check failed, the event couldn't be decoded, or its path cleans to nothing or is over 4096 bytes), `unknown_kind`, `too_old` (it ended more than 90 days ago, after the clock correction), `duration_mismatch` (the device's `duration` is more than 10 s from the indexed book's), `no_access` (the path is outside the caller's access, a library they hold no grant on, or a podcast library their client doesn't see: what `PUT` answers with a `403`) or `not_found` (an unknown library to an admin, or a hidden path); on the [Audible listening route](#post-apiv1bridgeaudibleaccountsaccountlistening) also `not_synced` (not a synced title's book), and `duration_mismatch` there means Audible's length disagrees | +| `rejected` | not stored and changed nothing; `reason` is `invalid` (a field check failed, the event couldn't be decoded, or its path cleans to nothing or is over 4096 bytes, checked before its library is looked up), `unknown_kind`, `too_old` (it ended more than 90 days ago, after the clock correction), `duration_mismatch` (the device's `duration` is more than 10 s from the indexed book's), `no_access` (the path is outside the caller's access, a library they hold no grant on, or a podcast library their client doesn't see: what `PUT` answers with a `403`) or `not_found` (an unknown library to an admin, or a hidden path); on the [Audible listening route](#post-apiv1bridgeaudibleaccountsaccountlistening) also `not_synced` (not a synced title's book), and `duration_mismatch` there means Audible's length disagrees | Every status but `rejected` is final for the sender: the event is stored and a resend is a `duplicate`. Every stored `listen` counts for sessions and history, @@ -3737,8 +3747,10 @@ opens one): saw the finish keeps playing without undoing it. - **Sessions.** A stored listen is folded into the token's session on the book at its corrected times (`recordSpan`, the rules in - [the data model](../data-model.md#listening-sessions-how-they-are-derived)); - `finish` and `unfinish` record no session. + [the data model](../data-model.md#listening-sessions-how-they-are-derived)). + A `finish` marks the token's session on the book finished when its last span + ended within 10 minutes before the finish; it opens none. `unfinish` records + nothing. - **History.** A listen with at least 1 s listened extends the newest history row on the book that ended up to 60 s before it began (2 s of overlap allowed) within 5 s of where it starts, else it adds a row, so a run of 15 s spans is one row and @@ -4186,7 +4198,7 @@ stays where it is (so adding twice is harmless). See ### `DELETE /api/v1/me/queue` *Session.* Removes one book (`?library_id=&path=`). `204 No Content`, -idempotent, with no access check. +idempotent, with no access check. A `path` over 4096 bytes is `400 path is too long`. ### `GET /api/v1/me/collections` @@ -4326,7 +4338,8 @@ the collection as the owner sees it, with the queue's ### `DELETE /api/v1/me/collections/{id}/items` *Session.* Removes one book (`?library_id=&path=`, owner only). `204 No -Content`, idempotent, with no access check on the path. `403` +Content`, idempotent, with no access check on the path (one over 4096 bytes is +`400 path is too long`). `403` `code: "not_owner"` for a viewer, `404` for anyone else. ### `PUT /api/v1/me/collections/{id}/shares` @@ -4981,7 +4994,7 @@ Adds a member. | Field | Type | Required | Notes | |---|---|---|---| -| `name` | string | yes | trimmed; 1-40 characters, no control characters | +| `name` | string | yes | trimmed; 1-40 characters with something visible, no control or invisible characters (bidi controls and marks, zero-width spaces, the BOM, fillers; right-to-left scripts, ZWJ emoji and ZWNJ are fine); else `400 invalid_name` | | `kind` | string | yes | `"kid"` or `"grownup"` | | `access` | string | yes | `"all"` or `"collections"` | | `sign_in_here` | bool | no | also sign the member in on the caller's device | @@ -5167,7 +5180,7 @@ person keeps, what ends it and how dead ones are pruned: | Status | Meaning | |---|---| | `201` | `{ token, expires_at }` | -| `400` | `invalid request` (a malformed body or an unknown field), `path is required` | +| `400` | `invalid request` (a malformed body or an unknown field), `path is required`, `path is too long` (over 4096 bytes, before the library is looked up) | | `401` | no session (a cast token can't mint another: it is never a bearer credential) | | `403` | the library or the path is outside the caller's access; `error.code: "not_a_book"` (`Only books can be cast.`) for a path in a podcast library, since episodes are not cast (from 2.4.0; before, the same 403 with no code); or `not available when authenticating with an API key` (a key never mints another credential) | | `404` | `book not found` (no book at the path), or `library not found` (an admin naming an unknown library; anyone else gets the `403`) | @@ -5427,7 +5440,8 @@ password. It also forgets the browsers they signed in from, so each one's next password sign-in is announced as a `new_device`. API keys stay, revoked one at a time with [`DELETE /admin/devices/{id}`](#delete-apiv1admindevicesid). No body. -Response `200`, the number of sessions it ended (`0` when there were none): +Response `200`, the number of the person's own sessions it ended (`0` when there +were none; household members' sessions it also ended aren't counted): ```json { "signed_out": 3 } @@ -5610,13 +5624,12 @@ Sending the existing `kind` is accepted; a different kind is `400 kind_immutable`. Root changes enforce the same `409 root_overlaps` rule as create. -A library the server's config declares (`config.yaml`, or an embedder using -the public `launcher.Options.Libraries` API; no environment variable declares one) keeps the name and +A library the server's config declares (`config.yaml`; no environment variable +declares one) keeps the name and folder declared there, since every start upserts it by name (`launcher.syncLibraries`) and the server never writes a library to `config.yaml`. A different `name` or `root` on such a library is -`409 config_library` (from 2.4.0; before, the rename stuck until the next -start brought the declared library back as a second one); the same values +`409 config_library`; the same values resent write nothing, and its other settings stay editable. The console reads the declared names from `GET /admin/setup`'s `config_libraries` and locks the fields. Renaming another library to a declared name at another folder is @@ -5676,28 +5689,15 @@ any other folder without audio it changes nothing. Clearing it splits the discs out again. Response `200`: `{ "status": "override set", "path": "…", "mode": "collection" }`. -`400` for any other mode; `404` library not found. +`400` for any other mode or a path over 4096 bytes (`path is too long`, before +the library is looked up); `404` library not found. ### `DELETE /api/v1/admin/libraries/{id}/folder-override` -Clears the override (back to auto-detection) and rescans. `?path=` required. +Clears the override (back to auto-detection) and rescans. `?path=` required +(over 4096 bytes is `400 path is too long`). Response `200`: `{ "status": "override cleared", "path": "…" }`. -### `PUT /api/v1/admin/libraries/{id}/enrichment` - -Attaches durable, path-keyed external identifiers to a book. Survives -rescans (every re-index of the book layers it back on); modifies no file on -disk. `?path=` required. An admin's own edit of `asin`/`isbn` -([`PATCH …/book`](#patch-apiv1adminlibrariesidbook)) wins over the enrichment -for that field; reverting the edit falls back to the enrichment. - -| Body field | Type | Required | -|---|---|---| -| `asin` | string | at least one of the two | -| `isbn` | string | at least one of the two | - -Response `200`: `{ "status": "enrichment set", "path": "…" }`. - ### `GET /api/v1/admin/libraries/{id}/export` Downloads the library's book list as a JSON **file**, in the envelope the @@ -5822,8 +5822,8 @@ Per-book notes: ``` - `runtime_min` is the book's duration in whole minutes (rounded); `chapters` is the number of indexed chapters. Both are omitted when unknown. -- `asin` / `isbn` come from the book's identifiers, including any attached via - [`PUT …/enrichment`](#put-apiv1adminlibrariesidenrichment). +- `asin` / `isbn` come from the book's identifiers, including any a Bridge + attached ([`PUT /bridge/libraries/{id}/enrichment`](#put-apiv1bridgelibrariesidenrichment)). - `subtitle`, `published` (`YYYY`, `YYYY-MM` or `YYYY-MM-DD`), `publisher`, `language`, `abridged`, `genres`, (with `include=description`) `description` and (with `include=tags`) `tags` are the effective values, as on the player's @@ -5898,8 +5898,8 @@ Progress of the library's (possibly running) scan: nothing was pruned. A scan an admin request queues - `POST …/scan`, creating a library, an edit -that changes the root or ignore rules, setting or clearing a folder override, -or a `POST /setup` that sends library fields - reads as `queued` (or `running`) before that request +that changes the root or ignore rules, or setting or clearing a folder +override - reads as `queued` (or `running`) before that request returns, so a status poll made right after it sees the scan. ### `GET /api/v1/admin/fs/dirs` @@ -6022,7 +6022,10 @@ Adds / removes one path rule. Body for both: selection of books to a share. Response: `204 No Content`. `400` when a `library_id` is missing/zero (nothing is -added), or with code `too_large` for more than 1000 rules. +added), or with code `too_large` for more than 1000 rules. `409` with code +`library_grant` on a whole-library grant (a share with `whole_library_id`): its +rules change only by granting or revoking the library +([`POST /admin/library-access`](#post-apiv1adminlibrary-access)). ### `POST /api/v1/admin/share-access` · `DELETE /api/v1/admin/share-access` @@ -6103,7 +6106,7 @@ folder and file names yield), `tag` (any other scanned value - it came from the file's embedded tags), `abs` (from an Audiobookshelf `metadata.json` in the book's folder), `edited` (typed by an admin), `community` (accepted from a community-metadata match, or an ASIN/ISBN attached through -[enrichment](#put-apiv1adminlibrariesidenrichment)), or `""` when the field has +[enrichment](#put-apiv1bridgelibrariesidenrichment)), or `""` when the field has no value. The scan does not store sources; `path` vs `tag` is worked out when the book is read, by the same rule for every book, so a tag that happens to match the folder name reads as `path`. @@ -6199,7 +6202,7 @@ is a `400` too (`too many format or codec values`). "edited_fields": ["narrator"], "scan_error": "probe_failed", "scan_error_file": "Andy Weir/The Martian/The Martian.m4b", - "scan_error_detail": "[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f8c] moov atom not found; Invalid data found when processing input", + "scan_error_detail": "[mov,mp4,m4a,3gp,3g2,mj2] moov atom not found; Invalid data found when processing input", "chapters_source": "files", "chapters_check": "refine" } @@ -6238,11 +6241,16 @@ is a `400` too (`too many format or codec values`). value), `[]` when none; `edited` is also true for a chapter-title edit alone. - `file_count` is `1` for a single-file book. - `scan_error` - the first problem reading the book's files on its last - indexing: `unreadable` (a file couldn't be opened), `empty_file` (0 bytes) - or `probe_failed` (ffprobe couldn't read it). `scan_error_file` is the - library-relative file, `scan_error_detail` the OS's or ffprobe's own message - (shown as is; may be empty for `empty_file`). An ffprobe message keeps every - distinct line ffprobe printed, joined with `"; "`, at most 300 bytes. A + indexing: `unreadable` (a file couldn't be opened, or its read was cut short + on a share that stopped answering: `reading it timed out`, or `too many + earlier reads are stuck (on this or another library), so it wasn't tried`), + `empty_file` (0 bytes) or `probe_failed` (ffprobe couldn't read it). + `scan_error_file` is the library-relative file, `scan_error_detail` the OS's, + the scan's or ffprobe's own message (may be empty for `empty_file`). An ffprobe + message keeps every distinct line ffprobe printed, joined with `"; "`, at most + 300 bytes, with FFmpeg's context address (`@ 0x...`) dropped. When a read of + an already-indexed book is cut short, the book keeps its stored length, + chapters and names, and only the problem is recorded. A recorded problem is re-checked on every scan, so it clears once the file reads (fixed permissions, a share back online). See [Read problems](../scanner.md#read-problems). @@ -8824,7 +8832,7 @@ effect. Lists are never `null`. | `backups.keep` | how many scheduled backups are kept, 1-365 (manual and before-restore backups stay until deleted; the newest three before-upgrade backups are kept apart) | | `backups.dir` | read-only: changed only in `config.yaml` or `AUDIOSILO_BACKUP_DIR`; `""` = `/backups` (`GET /admin/backups`' `status.dir` has the folder in use) | | `modules.` | each [managed module's](../modules.md) toggle, by module id (`false` by default; config key `modules..enabled`, variable `AUDIOSILO_MODULES__ENABLED`). Applies at once, with no restart. [`GET /admin/modules`](#get-apiv1adminmodules) says more about each | -| `locked` | setting id → why the console can't change it: the `AUDIOSILO_*` variable that set it, or `"launcher"` (a launcher override for `bind`, `tls.mode` or `public_url`) | +| `locked` | setting id → why the console can't change it: the `AUDIOSILO_*` variable that set it (before 2.7.0 also `"launcher"`, a launcher override, which only the archived desktop manager set) | | `restart_settings` | the setting ids read only at start (fixed) | | `restart_pending` | restart settings whose saved value differs from the one the server started with | @@ -9356,8 +9364,8 @@ records the outcome on the destination, and answers: ``` (`target` is the whole destination as above.) `error` is `""` when it arrived, -and `"failed"` from a server built without the notification service (an -embedder's; never one the launcher starts). +and `"failed"` from a server built without the notification service (never +one the launcher starts). A failed delivery is not an HTTP error here: the answer is `200` either way, `404` when there is no such destination. @@ -10532,7 +10540,7 @@ this never indexes a path on demand. | Status | Meaning | |---|---| | `200` | `{ library_id, path, files, book? }` | -| `400` | `invalid library id` | +| `400` | `invalid library id`; `path is too long` (over 4096 bytes) | | `404` | `library not found`; `code: "no_book"` - no indexed book at that path (or none in the library) | Without a path, a podcast library always returns `404 no_book`, so an older @@ -10664,7 +10672,7 @@ one, an id that doesn't match is `404 unknown_account`. | `policy_since` | when its automatic policy began to act: a policy backs up only titles first seen after it. Set at the account's creation, when an automatic policy is turned on, and moved to the account's first complete poll (Phase 4b) | | `titles` | `GET /admin/audible/accounts` only: the titles counted by state, and how many are matched to a book here (`matched` leaves out a match marked "Not this book") | | `last_sync_at` | when its last `audible.sync` finished having read Audible's places, `null` before the first (and after a mapping to another person) (Phase 4c) | -| `refused_at` | when a sync failed `audible_refused` (Audible refused the account's device): no sync is queued while it is set. Any `PUT /bridge/audible/accounts/{account}` (a sign-in again maps the account) or a complete poll clears it (Phase 4c) | +| `refused_at` | when a sync failed `audible_refused` (Audible refused the account's device): no sync is queued while it is set. Any `PUT /bridge/audible/accounts/{account}` (a sign-in again maps the account) or a complete poll clears it (Phase 4c). A refused sync the Bridge claimed in an earlier second than that clearing (it ran before the fresh sign-in, but reported after) doesn't set it again | | `backup_estimate` | `GET /admin/audible/accounts` only: what [Back up everything](#post-apiv1adminaudibleaccountsaccountbackup-all) would queue now, `count` titles and about `bytes` (each title's `runtime_s` x 8000, 64 kbps AAXC: approximate) (Phase 4b) | ### `GET /api/v1/bridge/users` @@ -10937,7 +10945,7 @@ keyset-paged. | `no_sync` | marked "Don't sync this book" (Phase 4c): neither direction of the account's sync touches its book. Separate from `rejected`; backups never read it | | `dest_path` | the console's destination for the title's backup, `""` for the account's placement | | `import_job_id` | the title's latest backup job, `null` for none | -| `import` | the title's backup: `{job_id, state, code}` with `state` the job's ([job states](#the-job-object)) and `code` a failed or parked job's code (`not_released`, `no_rights`, `audible_refused`, `audible_sign_in`, `not_writable`, `exists`, `disk_full`, `offline_url_expired`, `unknown_account`, or `""`); `{job_id: null, state: "pending"}` for a Back up on a preorder waiting for its release; `null` for none | +| `import` | the title's backup: `{job_id, state, code}` with `state` the job's ([job states](#the-job-object)) and `code` a failed or parked job's code (`not_released`, `no_rights`, `audible_refused`, `not_writable`, `exists`, `disk_full`, `offline_url_expired`, `unknown_account`, `drm_free` (Audible offers the title only as a DRM-free MP3, which the Bridge doesn't back up yet), or `""`); `{job_id: null, state: "pending"}` for a Back up on a preorder waiting for its release; `null` for none | | `needs_attention` | a preorder still not available 14 days after its `release_date`: the account is no longer polled hourly for it | | `planned` | where a backup would go, as the Bridge planned it at its last poll (the object the [titles PUT](#put-apiv1bridgeaudibleaccountsaccounttitles) takes); `null` when matched, not planned, or the account is no longer mapped to the plan's library (paused or mapped again: none until its next poll) | | `contested` | for a title left unmatched as `contested`, `{library_id, path, with: [{asin, title}]}`: the book it tied for and the account's other titles (not `gone`) contested over that book or matched to it, by title, at most 10; `null` otherwise, or while the account isn't mapped to that library | @@ -11104,8 +11112,12 @@ account's **person**, both ways, furthest wins. Audible's places come in through the progress feed, which the Bridge's push loop reads every minute. An account syncs while the module is on, its `sync` is on, it isn't paused, `refused_at` is unset and its person may still be synced for (not disabled, a kid or a demo -account). A title syncs when it is matched, not `rejected`, `ignored`, `no_sync` or -`gone`. Only places move: no listening session, `listening_daily` or history row +account). A title syncs when it is matched, not `rejected`, `ignored`, `no_sync`, +`gone` or a `preorder` (a poll may match a preorder to another edition on disk, +and Audible holds no place for an unreleased title). A preorder is in neither +direction: not in the job's params, `not_synced` on the listening route, and not +in the progress feed. Once it is released (`preorder` to `available`), its book's +place is offered to Audible again, as for a title back from `gone`. Only places move: no listening session, `listening_daily` or history row is ever written for Audible's listening, and nothing is recorded against the bridge token. @@ -11125,7 +11137,9 @@ kept, with server-built strict params `{account, titles}`. The scheduler queues one every 15 minutes for each account that syncs; turning sync on, and a title's `rejected` or `no_sync` change, queue it at once. Done having read Audible's places sets the account's `last_sync_at`; failed `audible_refused` sets `refused_at`, and -none is queued until a Bridge PUT or a complete poll clears it. Its params, +none is queued until a Bridge PUT or a complete poll clears it (a refused sync +claimed before that clearing marks nothing). Failed `audible_sign_in` (Audible +didn't accept the sign-in just now) is an ordinary failure. Its params, queueing, steps, result and failure codes are in [Position sync](../../bridge/audible.md#audible-to-the-server-audiblesync). @@ -11165,7 +11179,9 @@ book is resolved under the **person's** access (outside it: `rejected` `no_access`) and must be the match of one of the account's synced titles (else `rejected` `not_synced`). It is `rejected` `duration_mismatch` when none of the book's titles has an Audible `runtime_s` within 5% of the longer plus 2 minutes of -the indexed book's duration (an unknown length on either side agrees). Then: +the book's length: the indexed duration, else the `duration` on the person's own +progress row for the book (the progress feed's rule). An unknown length on either +side agrees. Then: - a `listen` more than 1 s **ahead** of the stored place applies over any device's row, whatever its time: the row's `updated_at` never moves back, and its @@ -11305,29 +11321,24 @@ console's [first-run setup flow](#admin-first-run-setup) does the rest. See ### `POST /setup` -*Public (token-guarded).* Creates the first admin. The page sends only the -token and the account; the library fields are still accepted so a caller that -scripts this route (an older page, or the archived desktop manager) keeps working. +*Public (token-guarded).* Creates the first admin account, nothing else: the +library is added in the console's flow. | Body field | Type | Required | Notes | |---|---|---|---| | `token` | string | yes | the one-time setup token | | `username` | string | no | defaults to `"admin"` | | `password` | string | yes | admins must have a password | -| `library_name` | string | with `library_root` | optional; send both or neither | -| `library_root` | string | with `library_name` | must be an existing directory on the server | -Without the library fields it creates only the admin: `201 { "user": { … } }`. -With both, it checks the folder exists before making the admin, then creates -the library with the defaults, starts its background scan and records the -console flow's `folder` step as done (its `naming` step stays open): -`201 { "user": { … }, "library": { … } }`. +The body is decoded strictly: any other field is `400` and creates nothing. +That includes the old `library_name`/`library_root` pair, which only the +archived desktop manager sent. Success is `201 { "user": { … } }`. | Status | Meaning | |---|---| -| `400` | validation (only one of the library fields, the folder doesn't exist, password rules) | +| `400` | an unknown field (the old library fields included), or the password rules | | `403` | invalid setup token | -| `409` | setup not available (already completed or never enabled); with the library fields, also a library create refusal (`name_taken`, `root_overlaps`) after the admin was made, which then exists | +| `409` | setup not available (already completed or never enabled) | --- diff --git a/docs-developers/server/auth-and-security.md b/docs-developers/server/auth-and-security.md index be0f3a65..3442a340 100644 --- a/docs-developers/server/auth-and-security.md +++ b/docs-developers/server/auth-and-security.md @@ -461,7 +461,11 @@ and that the person can neither see nor revoke: member's from their owner's `POST /me/household/members/{id}/pair`), each of which would make a session after the change; - their **recovery code** (its linked pairing tokens cascade), so `has_recovery` - turns `false`. + turns `false`; +- their **household members** where the person put them: a member's session made + on any of the person's devices but the kept one (`created_on_token_id`, the + device revoke's `also_members` rule, a device signed out earlier included), and + every member's open pairing code. A member's own paired device stays signed in. What stays: **API keys** (listed and revoked on their own, in the device lists and `/auth/tokens`), the [calendar feed token](#calendar-feed-tokens) (the person's to @@ -483,7 +487,8 @@ its writer transaction; otherwise it answers `ErrInvalidCreds` (`401`). same `endSessions` without a password change, keeping the admin's own session when the account is theirs, and forgets the person's browsers (`forgetBrowsers`), so each one's next password sign-in is announced as a new device. It answers -`{"signed_out": n}` (the sessions it ended) and is audited as +`{"signed_out": n}` (the person's own sessions it ended; members' sessions aren't +counted) and is audited as `user.sign_out_everywhere`. A new password forgets the browsers too, as it always has. ## Admin guards diff --git a/docs-developers/server/community-chapters.md b/docs-developers/server/community-chapters.md index b602965f..eb106fdc 100644 --- a/docs-developers/server/community-chapters.md +++ b/docs-developers/server/community-chapters.md @@ -74,7 +74,7 @@ its pauses). A `Kick` follows every committed book change: the API registers it with `Catalog.OnBookChange`, which `UpsertBooks` and `UpsertBook` (a scan), `EditBook` and `EditBooks` (an edit, an accepted match, a [bulk match run](api/reference.md#bulk-community-matching) -applying), `SetEnrichment`, clearing community matches and setting or removing a +applying), `BackfillASIN`, clearing community matches and setting or removing a custom cover fire after their transaction (to every listener: the [cover colour pass](media.md#the-background-colour-pass) hears them too). A kick waits 5 s for the kicks behind it (a scan kicks once per book), so a burst is one pass, and the pass only checks what is due. diff --git a/docs-developers/server/configuration.md b/docs-developers/server/configuration.md index 04fe9eeb..72ee3e17 100644 --- a/docs-developers/server/configuration.md +++ b/docs-developers/server/configuration.md @@ -1,6 +1,6 @@ --- title: Configuration reference -description: "Every config.yaml key, AUDIOSILO_* environment variable and CLI flag; which settings the admin console can change, which apply live and which need a restart; validation and secure defaults; launcher Options for embedders; and the ffmpeg/ffprobe auto-download." +description: "Every config.yaml key, AUDIOSILO_* environment variable and CLI flag; which settings the admin console can change, which apply live and which need a restart; validation and secure defaults; and the ffmpeg/ffprobe auto-download." --- Configuration is loaded by `internal/config` and layered in a fixed order. @@ -10,24 +10,23 @@ Later layers override earlier ones, and the result is validated before use: flowchart LR A["config.Default()
secure defaults"] --> B["<data>/config.yaml
(if present)"] B --> C["AUDIOSILO_* env vars
(applyEnv)"] - C --> D["launcher Options overrides
(embedders only)"] - D --> E["Validate()"] + C --> E["Validate()"] ``` Most keys can also be changed while the server runs, from the admin console's **Server > Settings** (`PATCH /admin/settings`), which writes them back to -`config.yaml`. A key set by an environment variable or a launcher override is -**locked** there instead. See +`config.yaml`. A key set by an environment variable is **locked** there +instead. See [Changing settings at runtime](#changing-settings-at-runtime). ## The config file - **Location**: `/config.yaml` (`config.Path`). The data directory - itself comes from the `--data` flag (or `Options.DataDir` for embedders) - - there is **no environment variable for the data dir**. + itself comes from the `--data` flag. There is **no environment variable for + the data dir**. - **First run**: when the file does not exist, `config.Load` returns secure - defaults with `firstRun=true`, and `pkg/launcher` persists them (with any - Options overrides applied) via `Config.Save`. The file is written `0600`, + defaults with `firstRun=true`, and `pkg/launcher` persists them via + `Config.Save`. The file is written `0600`, the data dir created `0700`. - **Annotated template**: `config.example.yaml` in the repo root mirrors this reference. @@ -52,7 +51,7 @@ updates only a declared library's root and default view). | Flag | Default | Meaning | |---|---|---| | `--data` | `./data` | Data directory: config, SQLite database, generated certs, downloaded tools, the Matroska remux cache (`cache/remux/`), the watch rendition cache (`cache/renditions/watch/`) | -| `--ffprobe` | `ffprobe` | Path to ffprobe (durations, chapters, codec detection). With FFmpeg 7.1 or 8.1 the scanner reads most files in-process by that release's rules and runs it only for the rest; with any other release ffprobe reads every file (see [The in-process probe](scanner.md#the-in-process-probe)). **`""` disables it**, the in-process probe included - the scanner degrades to path-derived metadata and `direct_playable` defaults to true | +| `--ffprobe` | `ffprobe` | Path to ffprobe (durations, chapters, codec detection). With FFmpeg 7.1 or 8.1 the scanner reads most files in-process by that release's rules and runs it only for the rest; with any other release ffprobe reads every file (see [The in-process probe](scanner.md#the-in-process-probe)). **`""` disables it**; the in-process probe still runs, by FFmpeg 8.1's rules, so most files keep their duration, codec and chapters, and a file it is unsure of gets none (its `codec` empty, so `direct_playable` defaults to true; see [With no ffprobe](scanner.md#with-no-ffprobe)) | | `--ffmpeg` | `ffmpeg` | Path to ffmpeg for on-the-fly transcoding, the MP3-in-Matroska remux and the watch renditions. **`""` disables it** - `?transcode=1` returns 503, the `transcode` capability reports false, an `.mka` is served as stored (no remux, `direct_playable: false`), and the `renditions` capability is false (`rendition=` answers `404 rendition_unavailable`) | | `--setup` | `false` | First-run **setup page**: instead of auto-creating the admin and printing credentials once, mint a one-time token and enable the guarded `/setup` page, where the admin's username and password are chosen in the browser (see [Built-in web UI](web-ui.md#first-run-setup-page-setup)). Either way the admin console's [first-run setup flow](web-ui.md#the-first-run-setup-flow) then adds the books folder and the rest on the first `/admin` visit | @@ -82,7 +81,7 @@ is enabled but not found locally, the auto-download kicks in (below). | Key | Type / default | Meaning | |---|---|---| | `bind` | string, `"0.0.0.0:8080"` | `host:port` to listen on. Must parse with `net.SplitHostPort` | -| `public_url` | string, `""` | Externally reachable base URL, used in QR pairing payloads, invite links and the launcher's "open this URL" output. Empty → derived per-request from scheme + `Host` header. Also the **away** address the apps use outside the home network (see [Home address](#home-address-lan_url)) | +| `public_url` | string, `""` | Externally reachable base URL, used in QR pairing payloads, invite links and the first-run banner's setup URL. Empty → derived per-request from scheme + `Host` header. Also the **away** address the apps use outside the home network (see [Home address](#home-address-lan_url)) | | `lan_url` | string, `""` | The server's **home** address on the household network, e.g. `http://192.168.1.20:8080`: see [Home address](#home-address-lan_url). Empty → derived per request when the request's `Host` is a home-network host | | `trusted_proxies` | []string (CIDRs), `[]` | Networks whose `X-Forwarded-For` is trusted when deriving the client IP (which keys the per-address [rate limit](#rate-limits-rate_limit) and the lockouts). Set when running behind a reverse proxy. Each entry must be a valid CIDR | | `cors_origins` | []string, `[]` | Browser origins granted CORS (methods `GET, POST, PUT, PATCH, DELETE, OPTIONS`). Empty = no cross-origin headers at all (native apps and same-origin web still work); `"*"` disables the check entirely. Needed for a hot-reload frontend dev server, e.g. `http://localhost:8081`. Once set, every authenticated media answer (covers, streams) is `Cross-Origin-Resource-Policy: cross-origin`, so a player on another site shows covers and plays audio (from 2.4.1; before, the browser blocked its covers): see [Transport hardening](auth-and-security.md#transport-hardening) | @@ -886,11 +885,9 @@ database). The ffmpeg/ffprobe paths are CLI flags; the console's Transcoding topic only shows what was found. **Locked settings.** A setting is locked when an `AUDIOSILO_*` variable set it -(see [Environment variables](#environment-variables)) or a launcher override -pinned it (`Config.Pin`, called by `applyOverrides` for `bind`, `tls.mode` and -`public_url`). `Config.Locked` maps each locked setting id to the variable's -name or `"launcher"`; the console disables locked fields and shows why they -can't be changed. +(see [Environment variables](#environment-variables)). `Config.Locked` maps +each locked setting id to the variable's name; the console disables locked +fields and shows why they can't be changed. **Live versus restart.** `api.API` keeps `boot`, the config the server started with (never changed), and `live`, an `atomic.Pointer[liveConfig]` that @@ -957,12 +954,8 @@ Every list is trimmed, with empty entries and repeats dropped, and holds at most 50 entries. `Config.Save` writes the saved config, but keeps `config.yaml`'s own value for -every key the environment set (as above) and every key a launcher pinned -(`bind`, `tls.mode`, `public_url` and the launcher's library list). The one -exception is the save that creates `config.yaml` (first run), which records the -pinned values; later saves leave them as the file has them. So a console save -never writes an environment variable's or a launcher's override into the -file. +every key the environment set (as above), so a console save never writes an +environment variable's value into the file. ## Update check @@ -989,8 +982,7 @@ console's Server > About and Health > System. `pkg/launcher` starts one ## Validation & secure defaults -`Config.Validate` runs after all layers (and **again** after launcher -overrides). It rejects: +`Config.Validate` runs after all layers. It rejects: - an empty data dir; a `bind` that isn't `host:port`; - an unknown `tls.mode`; `autocert` without `tls.hosts`; @@ -1015,40 +1007,23 @@ and printed once, or set via the guarded setup page), config written [Auth & security](auth-and-security.md) for the app-layer hardening these feed into. -## Launcher `Options` (for embedders) +## `pkg/launcher` is not an embedding API {/* #launcher-options-for-embedders */} -`pkg/launcher.Run(ctx, opts)` is the run loop used by the headless `audiosilo` -command and is also available for running the server in-process. The full -`Options` surface: +`pkg/launcher.Run(ctx, opts)` is the run loop behind the headless `audiosilo` +command. Its `Options` are only what the command's flags set: `DataDir` +(`--data`), `FFprobePath` / `FFmpegPath` (`--ffprobe` / `--ffmpeg`, `""` turns +the tool off), `Log` and `Setup` (`--setup`: the token-guarded `/setup` page +for the admin account instead of the printed credentials banner). -| Field | Type | Meaning | -|---|---|---| -| `DataDir` | string | Config/database/certs directory (as `--data`) | -| `FFprobePath` / `FFmpegPath` | string | Tool paths; `""` disables the tool (as the flags) | -| `Log` | `*slog.Logger` | Logger; nil → default stderr text logger | -| `Setup` | bool | Select the first-run flow: `false` = auto-admin + printed banner; `true` = token-guarded `/setup` page (the admin account only; the console's setup flow follows) | -| `OnURL` | `func(url string)` | Called once at startup with the URL to open: the token-carrying `/setup#token=…` while first-run setup is pending, else `/web` when a player is available, else `/admin`. A GUI launcher uses it to open a browser | -| `Bind` | string | Config override for `bind` (e.g. `"127.0.0.1:8080"`) | -| `TLSMode` | string | Config override for `tls.mode` (`"off"`/`"selfsigned"`/`"autocert"`) | -| `PublicURL` | string | Config override for `public_url` (e.g. a Cloudflare Tunnel URL) | -| `Libraries` | `[]launcher.Library` (`{Name, Root}`) | When **non-nil**, replaces the configured libraries wholesale (`launcher.Library` mirrors `config.Library` so external modules don't import an internal type) | - -Override semantics (`applyOverrides`): empty/zero fields are ignored, so the -headless command - which sets none - gets the file's configuration verbatim. -Overrides are layered on top of the loaded `config.yaml`, then the config is -**re-validated** (a malformed override fails startup rather than baking an -unbootable file). Each override is also pinned (`Config.Pin`, `libraries` -included), so the console disables `bind`, `tls.mode` and `public_url` rather -than offering a change the next start would undo. -On first run, the config **including overrides** is what gets persisted to -`config.yaml`; on later runs overrides apply in memory only, and a settings save -from the admin console keeps the file's own values for the pinned keys (see -[Changing settings at runtime](#changing-settings-at-runtime)). - -The base URL that `OnURL` (and the startup banners) use is `public_url` when -set; otherwise it is derived from `tls.mode` (scheme) and `bind` - a wildcard -bind (`0.0.0.0`/`::`) becomes `localhost`, which is both reachable on the -host and a secure context for the admin PWA. +It takes no configuration overrides and pins no setting: `config.yaml` and the +`AUDIOSILO_*` variables are the only configuration. (The bind, TLS, public URL, +library and URL-callback overrides were for the archived desktop manager and are +gone.) + +The startup banners' base URL is `public_url` when set; otherwise it is derived +from `tls.mode` (scheme) and `bind`. A wildcard bind (`0.0.0.0`/`::`) becomes +`localhost`, which is both reachable on the host and a secure context for the +admin PWA. ## ffmpeg/ffprobe resolution & auto-download @@ -1081,28 +1056,34 @@ tools): moves only to a branch `metadata`'s `knownRules` has. evermeet's macOS download is the latest release when it was fetched (and is kept), so the probe runs with it only while that is 7.1 or 8.1. A local ffprobe of a known - release found first avoids both. + release found first avoids both. If the pinned 8.1 asset is gone (a 404) in + the foreground download, with no tools cached, it falls back to BtbN's master + build: the tools work, the probe stays off, and a warning says so. - **A cached master build is replaced in the background** (`replaceMaster`): when the cached ffprobe's version starts `N-` (an older server's download), that pair serves this run, and the 8.1 build is fetched in a goroutine into `/tools/next` and self-checked; the next start's `promoteStaged` moves it - into `/tools/` (an incomplete or failing one is removed). A failed fetch + into `/tools/`, ffprobe first (an incomplete or failing one is removed; + a rename that fails keeps `next/`, so the following start finishes it). A failed fetch touches `/tools/.release-fetch-failed` (an empty file whose modification time is the failure's) and is retried no sooner than `releaseRetry` (24 hours) after that time; a shutdown mid-fetch leaves the marker as it was, and a - success removes it. The start never waits for it. + success removes it. The start never waits for it. A download unpacks into a + `.download-*` folder inside the one it fills (never the system temp folder), + and a start removes one an exit left behind. - **The start-up log** says which way the probe went (`pkg/launcher.resolveTools`, `metadata.ProbeRelease`): "in-process probe on" - with the rules' release, or "in-process probe off: ffprobe is not a release it - knows, so ffprobe reads every file". The probe's per-file fallback reason is + with the rules' release, "in-process probe off: ffprobe is not a release it + knows, so ffprobe reads every file", or, with no ffprobe, that the probe is on + by 8.1's rules with no ffprobe for the rest. The probe's per-file fallback reason is logged at debug (`metadata.SetLogger`). - **Hardening**: extraction only takes files named `ffmpeg`/`ffprobe` (basename-only, so no zip-slip), capped at 300 MiB per binary (decompression-bomb guard). Pinning per-asset SHA-256 is noted in the package as a future hardening step. - **Graceful degradation**: offline, an unsupported platform, or a failed - check just means running without the tool - chapters/durations and codec - detection off without ffprobe, transcoding off without ffmpeg - with a + check just means running without the tool - durations, chapters and codecs + only from the in-process probe without ffprobe, transcoding off without ffmpeg - with a warning logged and a retry on the next start. Nothing is fatal. ## What ends up in the data directory diff --git a/docs-developers/server/data-model.md b/docs-developers/server/data-model.md index 5f46bbdc..ee1c9de2 100644 --- a/docs-developers/server/data-model.md +++ b/docs-developers/server/data-model.md @@ -413,14 +413,20 @@ row itself (as migration 0016 did) instead of blanking the fields, and A scan also writes an `@rev` key (`catalog.scannedRevKey`): the revision of the name-reading rules (the path baseline, `metadata.DeriveFromPath`, plus the tags -over it) the snapshot used, `3` today (`catalog.scannedRev`). Revision 2 (2.6.0) +over it) the snapshot used, `4` today (`catalog.scannedRev`). Revision 2 (2.6.0) is the one where an album tag that only names the book's series stopped being its title; the next scan checks a snapshot of revision 1 or none once, as [the scanner](scanner.md#an-album-that-is-only-the-series) describes. Revision 3 (2.6.0, server#319) is the one where a leaf named `Author; Series; Book NN[; Title]` came to state its series and position; the next scan reads a snapshot older than 3 with such a leaf once, and stamps an unflagged revision 1 or 2 -snapshot, as [the scanner](scanner.md#a-leaf-that-states-its-series) describes. A snapshot +snapshot, as [the scanner](scanner.md#a-leaf-that-states-its-series) describes. +Revision 4 (2.7.0, server#175) is the tag reading of 2.4.0 and 2.5.0 (a series +tag carrying its position, generic titles in every script, the MP4 narrator item +and a narrator tag over composer); the next scan reads once a snapshot older than +4 that may read otherwise (an MP4, a narrator, a splittable series tag, a +non-ASCII letter in its title or path) and stamps the rest, as +[the scanner](scanner.md#books-already-indexed) describes. A snapshot with no revision (any server before revision 1) was read when the lone folder above a book (`Charles Dickens/Great Expectations`) was its series rather than its author. For a book one folder deep, `Catalog.Signatures` sorts such a snapshot out in SQL @@ -782,9 +788,15 @@ id, since a span synced late opens a session dated in the past); a session opens the span's start and `from_pos`; listened time is `listenedBetween` of the span's own wall time and how far it moved, so a seek ahead or an inflated span is not listening; and only a span ending within 60 s of now is live or can be marked -transcoded. The player cuts its span every 15 s while it plays online, so the live +transcoded. A span older than the session's last record (one synced late) never +moves the session's place, length or speed back; only a span at or after it does. +The player cuts its span every 15 s while it plays online, so the live list behaves as with saves. A span's recording is part of its batch's transaction, -not best effort. Sync `finish` and `unfinish` events record no session. +not best effort. A sync `finish` (applied, or `not_applied` `older`) marks the +token's session on the book finished when that session's last span ended within +`SessionGap` before the finish (`finishSession`); it never opens a session, and a +furthest-only batch (the Bridge's Audible route) marks none. `unfinish` records +nothing. - **Server time only** (for a save). Every time here is when the server received the save; the save's own `updated_at` is not used (a device clock can be wrong). @@ -1021,6 +1033,11 @@ Its keys: before the listener; a failure stops the start, as a failed migration does, since every series filter reads only the keys (see `books.series_key` and `book_more_series` in [The rebuildable index](#the-rebuildable-index)). +- `search_text`: the version of the search-text rule the `books_fts` rows were + written with (`searchTextVersion`: the rule's number and the Unicode version). + `Catalog.IndexSearchText` rewrites the rows the rule changes once, at start + after `KeySeries`, while it is missing or different (see + [FTS design](#fts-design-books_fts)). - `setup_state`: the admin console's [first-run setup flow](web-ui.md#the-first-run-setup-flow), `{"finished":false,"dismissed":false,"steps":{"folder":"done","naming":"skipped"}}` @@ -1091,8 +1108,9 @@ hold only what it reports: `authors` and `series` (JSON), `release_date`, `runtime_s`, `state` (`available` | `preorder` | `unavailable` | `gone`), `reason`, the match (`match_library_id`, `match_path`, `match_how` `asin` | `fuzzy` | `import`, the last set by a done backup): **path-keyed with no FK to the - index**, carried by `MoveDurableState` (`moveBookState`) and by a join - (`joinBookState`), cleared by `DeleteLibrary` in the library's own transaction; + index**, carried by a move (`MoveDurableState`) and by a join + (`JoinDurableState`) in the per-user transaction after the places + (`moveAudibleTitles`), cleared by `DeleteLibrary` in the library's own transaction; `first_seen`, `updated_at`, `poll_id`, and Phase 4b's `rejected`, `ignored`, `dest_path`, `import_job_id`, which a titles call never overwrites. Indexed by match, by title and by account then title (the console's orders). @@ -1137,6 +1155,13 @@ and `''` for none), the book a contested title tied for, with a partial index. Like a match's path, the contest is path-keyed with no FK, carried by a move or a join and cleared with its library. All three are the poll's, and a stored match clears them. +`0071_audible_sync_followups` recreates the `audible_title_match_update` trigger +so a `preorder` title takes no part in its account's sync (as a `gone` one), and a +preorder becoming available offers its book's place to Audible again. It adds +`audible_accounts.refusal_ended_at` (when the account's sync refusal last ended: +a Bridge PUT or a complete poll), so a refused `audible.sync` claimed before then +doesn't mark the account refused again (`FoldAudibleSync`). + ### Podcasts Migration `0052_podcasts` adds the library kind and initial tables; @@ -1147,7 +1172,7 @@ The current tables are: |---|---| | `podcast_shows` | rebuildable show metadata from `feed.json`, keyed by `(library_id, show_path)`; show id, title/author/description/link/language/type/explicit/private, public feed URL and host/hash, podcast GUID, cover path, episode count/latest publication, signature and index time | | `podcast_feed_hashes` | current/previous URL hashes; `(library_id, sha256, show_path)`; cascades with its indexed show | -| `podcast_episodes` | rebuildable one-to-one row for `books.id` with cascade; library/show path, episode id/GUID, publication, season/episode/type/explicit, feed duration, file `present`, generation, removed time and duration source; 0054 adds `sha256`, `timeline_changed`, `fetch_json` and `feed_sig` | +| `podcast_episodes` | rebuildable one-to-one row for `books.id` with cascade; library/show path, episode id/GUID, publication, season/episode/type/explicit, feed duration, file `present`, generation, removed time and duration source; 0054 adds `sha256`, `timeline_changed`, `fetch_json` and `feed_sig`; 0073 adds `indexed_at`, the `books.indexed_at` the podcast index wrote the row with: a books row an older server rewrote in place (a rollback scans a podcast library as books) no longer matches it, and the next scan re-indexes the show (`PodcastEpisodeRows` counts only rows whose stamps agree; `NULL` counts as matching) | | `podcast_follows` | durable `(user_id, library_id, show_path)`, created time and the followed `show_id`; no FK to the show or `books`, so pruning keeps it, but a different show at the same path does not inherit private access | | `podcast_feed_state` | server-owned `(library_id, show_path, show_id)` validators, last check/success/status, error code/text, `secret_missing`, failure count and next poll; not written to disk per poll | | `podcast_policies` | durable `(library_id, show_path, show_id)` policy: `mode` (`keep`, `on_demand`, `archive`), `keep` (1..1000), `days` (0..365), `updated_at`; no override means Keep 3/days 7 | @@ -1281,8 +1306,9 @@ the `unknown_asin_report` key of [`server_state`](#server-state). folder's detection when the scanner's heuristic gets it wrong: `book` forces folder-is-one-book, `collection` forces one book per file. - **`book_enrichment`** *(0012)* - PK `(library_id, path)`, `asin`, `isbn`, - `updated_at`. Stores external identifiers attached through - `PUT /admin/libraries/{id}/enrichment`. Writing it never blanks a + `updated_at`. Stores external identifiers a Bridge attaches through + `PUT /bridge/libraries/{id}/enrichment` (`BackfillASIN`); rows the retired + admin enrichment route stored, ISBNs among them, still apply. Writing it never blanks a stored field (a blank `asin` or `isbn` keeps the existing one). It survives rebuilds because it is layered onto the book row on every re-index (see below); an admin's own `asin`/`isbn` edit wins over it. @@ -1380,7 +1406,7 @@ book's `book_more_series` rows) and chapter titles and refreshes the book's `books_fts` row from the result (and its `book_labels`); the admin book page shows the same resolution as provenance, so the two can't disagree. Every write path calls it inside its own transaction - `UpsertBook` (each scan or on-demand index), -`SetEnrichment`, and `EditBook`/`EditBooks` (the admin edits, the bulk form +`BackfillASIN` (the Bridge's enrichment), and `EditBook`/`EditBooks` (the admin edits, the bulk form all-or-nothing). Two consequences: - **No read-time join.** Players, search, the `/fs` annotations, `/meta` @@ -1596,6 +1622,8 @@ The migration history so far: | 0068 | `unknown_asin_reports` | the [unknown-ASIN report](#unknown-asin-reports)'s `unknown_asin_reports` table (`asin` primary key, `checked_at`, `unknown`, `sent_at`; `WITHOUT ROWID`), derived and naming no book. No data moves | | 0069 | `book_matches` | the [recorded community match](#durable-per-library-config-path-keyed-no-fk-to-books) `book_matches` (`library_id`, `path`, `work_id`, `recording_id`, `how` `auto`\|`accepted`, `run_id`, `set_by`, `set_at`; primary key `(library_id, path)`, `library_id` `ON DELETE CASCADE`). No data moves | | 0070 | `book_match_offered` | `book_matches.offered_asin` and `offered_isbn` (`TEXT NOT NULL DEFAULT ''`, server#330: the identifiers the [recorded community match](#durable-per-library-config-path-keyed-no-fk-to-books) offered) and `match_run_items.unchanged` (`TEXT NOT NULL DEFAULT ''`, server#332: the comma-separated scopes under which a refresh item had nothing to change when recorded; an apply under one leaves it out without reading the book again). No backfill: `''` behaves as before. No data moves | +| 0071 | `audible_sync_followups` | the `audible_title_match_update` trigger recreated so a `preorder` title is left out of the sync and its release re-offers the book's place; `audible_accounts.refusal_ended_at` (see [The Bridge](#the-bridge)). No data moves | +| 0073 | `podcast_episode_indexed` | `podcast_episodes.indexed_at`, backfilled from each episode's `books.indexed_at` (see [Podcasts](#podcasts)) | ## SQLite choices @@ -1622,7 +1650,7 @@ The migration history so far: | The Docker image (`ghcr.io/kodestar/audiosilo-server`) | native, a static binary linked against glibc on the Alpine base; the build arg `SQLITE_ENGINE=modernc` builds modernc instead | | GoReleaser Linux binaries (tarballs, `.deb`/`.rpm`), and so the LinuxServer.io image, which downloads them | native, static against glibc as in the Docker image | | GoReleaser macOS and Windows binaries | modernc | -| An embedder running `pkg/launcher` in-process, a plain `go build` | modernc | +| A plain `go build` | modernc | On a native build, `AUDIOSILO_SQLITE=modernc` runs that process on modernc instead (see [The SQLite engine](configuration.md#the-sqlite-engine-audiosilo_sqlite)). @@ -1760,7 +1788,7 @@ It is kept in sync **by the application**, not by triggers, keyed by - `catalog.refreshEffective` deletes then re-inserts the FTS row from the book's **effective** values, inside the same transaction as the write that - called it (`UpsertBook`, `SetEnrichment`, a metadata edit), so index and FTS + called it (`UpsertBook`, `BackfillASIN`, a metadata edit), so index and FTS can't diverge and search finds edited titles. - `catalog.DeleteBooksNotIn` (the scanner's prune step) deletes the FTS row alongside each stale book. @@ -1768,7 +1796,22 @@ It is kept in sync **by the application**, not by triggers, keyed by `catalog.Search` joins `books_fts` to `books` on `rowid`, ranked by FTS relevance (`ORDER BY rank`). User input never reaches FTS5 syntax directly: `buildMatchQuery` reduces it to quoted, AND-ed prefix terms (`"foo"* AND -"bar"*`), giving type-ahead behavior without exposing FTS operators. Results +"bar"*`), giving type-ahead behavior without exposing FTS operators. + +**Chinese and Japanese.** FTS5's `unicode61` tokenizer splits only on spaces and +punctuation, so an unspaced CJK title would be one token. Each column is stored +in `ftsText`'s form instead: Han, Hiragana and Katakana (with the prolonged sound +and iteration marks) a character a token. `buildMatchQuery` splits a query the +same way, a CJK run becoming an adjacent-token phrase whose last token is a +prefix, and `ftsPhrase` writes the author and narrator filters' phrase alike, so +`猫` finds `吾輩は猫である` and `吾輩` still does. A trigram tokenizer was rejected: +it can't match the one- and two-character words CJK search mostly is. Hangul is +spaced, so it stays as it is. Rows an older server wrote are rewritten once at +start-up (`Catalog.IndexSearchText`, run after `KeySeries`), gated by +`server_state` `search_text` (the rule version and the Unicode version), in one +transaction and with no migration. + +Results are scoped per-library by the caller's share rules and de-duplicated across libraries (see [Data model → libraries](#libraries--shares) for the `sort_order` tiebreak). diff --git a/docs-developers/server/media.md b/docs-developers/server/media.md index 855c881f..7db0a5e2 100644 --- a/docs-developers/server/media.md +++ b/docs-developers/server/media.md @@ -493,10 +493,14 @@ ffmpeg -nostdin -loglevel error -xerror -protocol_whitelist fd -f matroska -i fd exactly one MP3 audio stream, which also covers folder parts whose codec the index doesn't hold. That refusal is `ErrRefused`, remembered in memory per cache key for an hour (`cacheRefusalTTL`, up to 1,024 keys: what looks like - the file's fault may be the environment's, an I/O error at open or an ffmpeg - missing an encoder; a changed file is a new key, tried again at once) and - answered with the stored bytes. An exit after ffmpeg wrote output (killed partway, a - full disk, a read error) is not a refusal. + the file's fault may be the environment's, such as an ffmpeg missing an + encoder; a changed file is a new key, tried again at once) and answered with + the stored bytes. An exit after ffmpeg wrote output (killed partway, a full + disk, a read error) is not a refusal, nor is one whose log names an I/O error + before any output (`readFailed`: a flaky share, tried again on the next + request). A run its own timeout killed **is** one, in either cache + (`timedOut`), so a copy that can't finish in time isn't started from zero on + every request; the server shutting down mid-run is not. **The cache.** `/cache/remux/` (`0700`, made again if it went), one file per source named by the first 32 hex digits of a SHA-256 of the absolute path, @@ -615,8 +619,8 @@ remux's (above) with its own bounds: The encode version (`renditionVersion`) is part of the key and the ETag, so a change to the encode is a new key and old copies age out of the LRU. A source that changes while it is read is never cached under the old key. A refusal -(ffmpeg's exit before writing anything, a trackless or short copy) is remembered -for an hour or until the file changes. A failed run's log carries the first and +(ffmpeg's exit before writing anything, a trackless or short copy, or an encode +its 2-hour timeout killed) is remembered for an hour or until the file changes. A failed run's log carries the first and last 2 KiB of ffmpeg's output. Without ffmpeg (`--ffmpeg ""`) or without a data directory the cache is off: the `renditions` flag is `false`, the top-level `renditions` list is `[]`, and a `watch` rendition request outside a podcast @@ -641,8 +645,9 @@ audio and a regular file (`404`). Then `FileCache.Open` waits up to transcode: its bytes aren't rangeable, so a watch couldn't resume it. - **Refused:** `422 rendition_failed` (remembered, as above), with `Retry-After` the seconds the remembered refusal has left (`FileCache.RefusedFor`, 1 to 3600). -- **Failed otherwise** (the disk guard, a run killed partway, the source changed, - the 256-file queue full): `503 rendition_error` with `Retry-After: 60`. +- **Failed otherwise** (the disk guard, a run killed partway by a shutdown, the + source changed, the 256-file queue full): `503 rendition_error` with + `Retry-After: 60`. **Preparing a book.** `POST /libraries/{id}/renditions?path=` with `{"kind":"watch"}` resolves the book as `/item` does (a part path names its book) diff --git a/docs-developers/server/overview.md b/docs-developers/server/overview.md index f5b19de6..1860c8d7 100644 --- a/docs-developers/server/overview.md +++ b/docs-developers/server/overview.md @@ -73,10 +73,9 @@ startup logic belongs in `pkg/launcher`. The run loop: load config → open the store → wire services → first-run bootstrap (auto-admin banner, or the token-guarded `/setup` page in setup mode; the console's first-run flow takes it from there) → sync config-declared libraries → kick off the initial background scan → serve -until the context is cancelled. It is public so the server can also run -in-process (see [Configuration](configuration.md#launcher-options-for-embedders)); -`Options` carries embedding-friendly overrides (`Bind`, `TLSMode`, `PublicURL`, `Libraries`, -`OnURL`) that are re-validated after being layered onto the loaded config. +until the context is cancelled. It is not an embedding API: `Options` holds +only what the command's flags set, and it overrides no configuration (see +[Configuration](configuration.md#launcher-options-for-embedders)). Before the database opens, `Run` applies a restore an admin asked for (`backup.ApplyPendingRestore`); it then starts the backup schedule, the notification workers and the daily retention (sessions, audit log, event diff --git a/docs-developers/server/scanner.md b/docs-developers/server/scanner.md index 06a87398..8e1f5647 100644 --- a/docs-developers/server/scanner.md +++ b/docs-developers/server/scanner.md @@ -20,7 +20,11 @@ catches up. ## Podcast libraries -`Scanner.Scan` branches on `libraries.kind` before book detection. A podcast +`Scanner.Scan` branches on `libraries.kind` before book detection. A kind this +server doesn't know (`catalog.KnownKind`: a newer server's, left by a rollback) +is never scanned as books: the scan fails with `library.ErrUnknownKind`, recorded +as a failed run, and a targeted index refuses any library that isn't `books` +(`resolveBook`, `ErrNotIndexable`). A podcast scan lists real, non-hidden top-level show directories, reads each `feed.json` (regular file, up to 32 MiB, version 1), and indexes its episodes with or without audio files. Unsupported versions are skipped and logged; @@ -35,10 +39,11 @@ files are noticed even with unchanged feed stat. Refresh observes regular file presence, sizes, artwork, barriers and matching CHAP, then atomically updates the show/episode index. An unchanged signature skips the show only while its episodes are still indexed: a show with fewer `podcast_episodes` rows than its -`feed.json` listed is indexed again (`PodcastEpisodeRows`, from 2.4.0), since an -older server run on the same database (a rollback to 2.3.1, say) scans the +`feed.json` listed is indexed again (`PodcastEpisodeRows`), since an +older server run on the same database (a rollback) scans the library as books and drops the episode rows while the show keeps its -signature. Each podcast scan then deletes the books rows that are not episodes +signature. A row counts only while its `indexed_at` (`0073`) is its books row's, +so one that older scan rewrote in place counts as missing too. Each podcast scan then deletes the books rows that are not episodes (`PruneNonEpisodeBooks`: what such a books scan left, a folder book at a show path for one). A targeted `POST /bridge/rescan` forces that refresh for a show path or an indexed episode @@ -833,9 +838,10 @@ path-derived metadata still works, durations fall back to chapter ends or remain 0, and `codec` stays empty - which the API treats as directly playable (`direct_playable: true`, so the web player streams the file as it is rather than through the transcoder). See -[Media & streaming](media.md). With ffprobe disabled (`--ffprobe ""`) the -[in-process probe](#the-in-process-probe) is off too, so the server reads tags -and paths only (probing in-process with no ffprobe is a possible follow-up). +[Media & streaming](media.md). With ffprobe disabled (`--ffprobe ""`) or not +found, the [in-process probe](#the-in-process-probe) still runs, by FFmpeg 8.1's +rules: most files get their duration, codec and chapters, and only a file it is +unsure of goes without (see [With no ffprobe](#with-no-ffprobe)). ### A leaf that states its series @@ -919,20 +925,30 @@ part), which the console's provenance labels `path` (above). #### Books already indexed Books already indexed are put right without a re-index or a migration: the -snapshot's `@rev` went from 1 to 2 for this rule, and from 2 to 3 for -[a leaf that states its series](#a-leaf-that-states-its-series) (see -[Data model](data-model.md#the-rebuildable-index)). +snapshot's `@rev` went from 1 to 2 for this rule, from 2 to 3 for +[a leaf that states its series](#a-leaf-that-states-its-series), and from 3 to 4 +for the tag reading of 2.4.0 and 2.5.0 (a series tag carrying its position, +`metadata.SplitSeriesTag`; a title in any script not generic, `IsGenericTitle`; +the MP4 `\xa9nrt` narrator item and a narrator tag over composer; see +[Data model](data-model.md#the-rebuildable-index)). The snapshot carries the +narrator too, so a backfill rewrites it. - **Flagged:** `Catalog.Signatures` sets `Signature.NamesCheck` on a snapshot of revision 1 or none whose stored title and series could be such a reading - (`metadata.MaybeSeriesTitle`, about 2 µs a row), and on a snapshot older than - revision 3 whose leaf `metadata.PathLeafSeries` reads. The next scan reads that book's + (`metadata.MaybeSeriesTitle`, about 2 µs a row), on a snapshot older than + revision 3 whose leaf `metadata.PathLeafSeries` reads, and on a snapshot older + than revision 4 whose tag reading may differ (`tagReadingMayDiffer`: an MP4-family + book, a narrator, a series `SplitSeriesTag` splits, or a non-ASCII letter in the + title or path). The next scan reads that book's first file once, with the same read as the path backfill, and `Catalog.SetPathReading` rewrites its snapshot (it now skips only a snapshot already at the current revision). A book the snapshot alone could settle for the lone-folder check waits for this read too, so an unreadable file never - records today's revision with its title or the leaf's series unread. -- **Not flagged:** a current revision-1 or revision-2 snapshot + records today's revision with its title or the leaf's series unread. These + reads run `readSlots` wide, in batches (`tagRead`, `tagReadsPerSlot`), so a large + revision-4 backfill (every narrated book, once) costs one parallel tag read + each; a read a Stop cuts short is left for the next scan. +- **Not flagged:** a current revision 1 to 3 snapshot (`Signature.RevStamp`) is stamped at today's revision by that scan (`Catalog.StampScannedRev` with `revOlderOwn`, a compare-and-set that changes only the revision), so later scans don't test it again. A snapshot of no revision that isn't flagged is not stamped: it keeps no @@ -975,7 +991,8 @@ Which releases it runs for: - **7.1 and 8.1** (`knownRules`; they differ in MP3 gapless durations and in empty UTF-16 ID3 strings). The official and LinuxServer.io images are both Alpine 3.24, whose ffprobe is 8.1.3. -- **Any other release turns the probe off**, and ffprobe reads every file: +- **Any other release turns the probe off**, and ffprobe reads every file (with + no ffprobe at all it runs by 8.1's rules, see [With no ffprobe](#with-no-ffprobe)): Ubuntu 24.04's 6.1, a master or nightly build (its version is no release: the BtbN master builds a server before 2.5.1 [downloaded](configuration.md#ffmpegffprobe-resolution--auto-download) for @@ -1004,6 +1021,13 @@ with nothing logged: - a crafted table past a bound (a chapter track over 65,536 samples or with counts past its tables, Matroska SimpleTags over 4,096 or 8 deep, a compressed ID3 frame over 16 MiB); +- an MP4 track FFmpeg's `mov_read_trak` sanity check calls a broken header + (chunks but no `stts`/`stsc` entries or no sample size or count, or samples but + no chunks): FFmpeg then reads its `mdhd` duration in 1/90000, so a 13,163 s + book prints as 3,510 s, and only ffprobe gives that same answer; +- the edges where FFmpeg's own arithmetic misbehaves: an `mvhd`/`mdhd` timescale + past `INT32_MAX`, a rescale that overflows int64, and an ID3v2.2 `CHA` frame + (FFmpeg's three-letter match reads it as `CHAP`); - a read error, a reader's panic, no duration, or a probe limit (below). **Limits.** A file's probe stops after 5 s or 64 MiB read, and that file goes @@ -1015,8 +1039,37 @@ Vorbis key grows (`blockFile.check`), and every 256 ticks in the walks that loop without reading (`blockFile.tick`, a `probeLimit` panic `fastProbe` recovers). -**A file neither reads** keeps ffprobe's error as its `ProbeErr`, so Health -shows the same `probe_failed` text as before (`ErrProbeRejected` included). +**Bounded reads on a hung share.** The tag read and the narrator walk before the +probe are bounded too (`readFile`): by the ffprobe run's timeout, the caller's +context and the probe's byte budget. A tag read out of time skips the probe and +leaves the file to ffprobe (killable, with its own timeout); one past the budget +leaves the tags unread and the probe still runs. While armed, every read from the +file runs on its own goroutine and is waited for only until the deadline +(`blockFile.waitRead`), so a `pread` on an NFS or SMB share that stopped answering +costs a goroutine, never a stuck scan reader. Once 16 such abandoned reads are +outstanding process-wide (`maxAbandonedReads`), a new read fails fast without +opening the file, logged once until they have all returned. `os.Open` and `Stat` +are not bounded; the scan's root check catches a dead mount before discovery. +What a cut-short read does to the stored book is in [Read problems](#read-problems). + +**A file neither reads** keeps ffprobe's error as its `ProbeErr` +(`ErrProbeRejected` included), with FFmpeg's context address dropped +(`[mp3 @ 0xcbcc3c000]` is stored as `[mp3]`). + +#### With no ffprobe + +With `--ffprobe ""`, or no ffprobe found, the probe still runs, by FFmpeg 8.1's +rules (`noFFprobeRules`, the release the Docker images ship). A file it is unsure +of gets no duration and no `ProbeErr`. A read out of time says so +(`Metadata.ProbeTimedOut`) and is read again alone, as an ffprobe time-out is. +The start-up log says the probe is on with no ffprobe. + +A scan reads an unchanged book with no stored duration or codec again once per +server run (`Scanner.rereadUnprobed`, remembered in memory). So a server that ran +without ffprobe gets its books' lengths on the first scan after the upgrade, and +a file the probe is unsure of isn't re-indexed on every scan. +`metadata.ExtractTags` reads tags alone, for the one tags-only caller +(`suspectFromTags`). **Keeping parity.** A new FFmpeg release that changes a rule would change a value, so the probe is checked against ffprobe mechanically: a fixture corpus @@ -1206,18 +1259,20 @@ reading a book's files, in three columns: | `books.scan_error` | When | |---|---| -| `unreadable` | a file couldn't be opened (`metadata.Metadata.OpenErr`) | +| `unreadable` | a file couldn't be opened (`metadata.Metadata.OpenErr`), or its read was cut short on a hung share (below) | | `empty_file` | a file is 0 bytes | | `probe_failed` | ffprobe is configured and couldn't read a file (`ProbeErr`): the in-process probe left it to ffprobe, which failed | `scan_error_file` is the library-relative file and `scan_error_detail` the OS's or ffprobe's own message, without the absolute path (the console shows the file -beside it). To keep that message, ffprobe now runs with `-v error` instead of +beside it). To keep that message, ffprobe runs with `-v error` instead of `-v quiet`: on failure, every distinct line it wrote to stderr, each without the -input path, is joined with `"; "` and trimmed to 300 bytes (for example -`[mov,mp4,m4a,3gp,3g2,mj2 @ 0x…] moov atom not found; Invalid data found when -processing input` - the specific cause usually comes before the generic last -line). Every re-index clears and re-checks the columns. A fixed permission or a +input path and without FFmpeg's context address, is joined with `"; "` and +trimmed to 300 bytes (for example `[mov,mp4,m4a,3gp,3g2,mj2] moov atom not found; +Invalid data found when processing input` - the specific cause usually comes +before the generic last line). The console shows a `probe_failed` book as one +plain line ("couldn't be read as audio") with this detail behind **Details**, +stripping an address from a row stored before. Every re-index clears and re-checks the columns. A fixed permission or a share that came back changes neither the file's mtime nor its size, so the unchanged-skip would keep the stale problem; instead each scan re-reads just the recorded file of an `unreadable` or `probe_failed` book (`problemCleared`) and @@ -1243,6 +1298,18 @@ Only a file that also times out when read alone is recorded as a problem. On a v slow share where many files time out, the retries make the scan slower than reading one file at a time would; lower `scan.concurrency` there. +**A read cut short on a hung share.** When any file of an already-indexed book +comes back with its read cut short or refused at the abandoned-read cap +(`Metadata.TagsUnread`, `ReadRefused`; see +[The in-process probe](#the-in-process-probe)), the book's row is not rewritten at +all (`catalog.recordReadProblem`): names, length, codec, chapters, cover flag, +mtime and size stay as the last full read left them. Only the problem is +recorded, `unreadable` with "reading it timed out" (or, for a refused read, "too +many earlier reads are stuck (on this or another library), so it wasn't tried": +the cap is process-wide), with ffprobe or without, +so the next scan reads it again. A new book read so is written with its path's +names. + Extraction stays best-effort: a problem never stops a scan, and the book is still indexed with what could be read. Each book with a problem adds a `problem` line to the scan's log and counts in its `errors`. diff --git a/docs-developers/server/web-ui.md b/docs-developers/server/web-ui.md index bd648ea7..570d0e9c 100644 --- a/docs-developers/server/web-ui.md +++ b/docs-developers/server/web-ui.md @@ -827,8 +827,8 @@ CSP-sensitive work against a real build served by Go. The console's own gate is (`settings-form.tsx`): a draft of its fields, Reset and Save changes, only the changed fields sent (`sectionPatch`); list settings are edited as one entry per line. A refusal's `field` puts the server's message under that - field. `SettingBadges` reads the envelope: the source of an environment or - launcher override from `locked` (the field is disabled), "Restart + field. `SettingBadges` reads the envelope: the environment variable that sets a + field from `locked` (the field is disabled), "Restart to apply" from `restart_settings`, "Waiting for a restart" from `restart_pending`, which also drives the notice at the top of every topic. The Network & HTTPS cards pass `confirmRestart`, so saving a restart setting @@ -966,7 +966,11 @@ CSP-sensitive work against a real build served by Go. The console's own gate is "Checking an Audible library", an `audible.import` "Backing up an Audible book" with its ASIN (`backupAsin`), an `audible.sync` "Syncing places with Audible", and one the server queued "Scheduled by the server" (`serverScheduled`, which counts - `audible.sync`); a sync cancelled by turning sync off shows the server's reason + `audible.sync`); a failed Audible job's code reads as a sentence + (`audibleJobReason`: an `audible.import`'s from its result's `code` or an error + starting `:`, as `audible.code.`; an `audible.poll` or + `audible.sync` failed `audible_refused` or `audible_sign_in` as + `audible.pollCode.`; an unknown code keeps the Bridge's own text); a sync cancelled by turning sync off shows the server's reason ("sync with Audible was turned off for the account", `SERVER_CANCEL_REASONS`). - **Settings > Audible** (`features/audible/audible-topic.tsx`, Bridge Phase 4a, a `MODULE_PAGES` page listed only while the `audible` module is on) - over @@ -1179,8 +1183,8 @@ asks. The headless first-run banner (`writeBanner` in `pkg/launcher`) prints the admin username and password, the auth code and the config file's path, then -sends the reader to this flow and to the connect page (from 2.4.0; before, it -named only the API routes), from the same base URL as `OnURL`: +sends the reader to this flow and to the connect page, from the configured base +URL (`public_url`, else one derived from `tls.mode` and `bind`): ``` Finish setting up: open https://audiosilo.example.com/admin and sign in as admin. @@ -1537,21 +1541,12 @@ that admins have a password) and answers `201 {user}`; the page then shows "Your account is ready" with a **Continue setup** link to `/admin`, whose first visit opens the console's flow. -`POST /setup` still accepts the old library fields, so a caller that scripts -it (an older page, or the archived desktop manager) keeps working: with both -`library_name` and `library_root` it checks the folder exists **before** making -the admin, then creates the library with the defaults, enqueues a manual scan, -records the flow's `folder` step as done (its `naming` step stays open, so the -admin still picks the library's source) and answers `201 {user, library}`. One -of the two alone is `400`. If library creation fails after the admin was -created, the page still closes (an admin now exists) and the admin finishes in -the console - intentionally fail-safe; failing to record the `folder` step is -only logged. - -The launcher prints the token-carrying URL in a startup banner and also -reports it through `Options.OnURL`, so an in-process caller can open it in a -browser - see [Configuration](configuration.md#launcher-options-for-embedders). -The banner's last line reads "You'll choose an admin password, then the console +`POST /setup` creates the account only. Its body is decoded strictly: any +other field, the old `library_name`/`library_root` pair included (only the +archived desktop manager sent it), is `400 invalid request` and creates +nothing. The library is added in the console's flow. + +The launcher prints the token-carrying URL in a startup banner. The banner's last line reads "You'll choose an admin password, then the console walks you through the rest." (`setupBanner` in `pkg/launcher`). ## Well-known app-association files diff --git a/docs-users/admin/audible.md b/docs-users/admin/audible.md index fd682684..8f382ab8 100644 --- a/docs-users/admin/audible.md +++ b/docs-users/admin/audible.md @@ -130,6 +130,7 @@ The reason under the title says what to do. Press **Back up** again to retry. | **There isn't enough free space.** | The Bridge needs about twice the book's size in its data folder, and the book's size in the library. | | **The download link ran out twice.** | Back it up again. | | **The Bridge no longer has this account.** | Sign in again on the Bridge. | +| **Audible offers this title only as a DRM-free MP3** | Nothing to do for now: the Bridge can't back these up yet. | #### Sign in again after updating the Bridge @@ -239,7 +240,11 @@ are on this server. Only the place syncs: listening on Audible adds nothing to your history or stats. A title whose length on Audible is quite different from the book here doesn't -sync. +sync. A preorder starts syncing once it is released. + +Sometimes the Audible app uploads an older place it still held over the one the +Bridge wrote. The next sync writes your place to Audible again, once. If you then +go back in the Audible app on purpose, that place is left alone. ### When it stops @@ -280,5 +285,3 @@ server stay. - Turning the Audible module off stops checks, backups and syncing. Accounts and titles are kept. Turn it back on to remove an account in the console. - Error messages never include your Amazon account's details. -- A book Audible offers only as an MP3 without copy protection can't be backed up - yet. diff --git a/docs-users/admin/bridge.md b/docs-users/admin/bridge.md index 8b1b8232..325d5cd5 100644 --- a/docs-users/admin/bridge.md +++ b/docs-users/admin/bridge.md @@ -59,7 +59,7 @@ Bridge**, which opens the right tab of the Bridge's page. | **Paired with this server** | the Bridge is paired | | **Libraries mapped and writable** | the Bridge can find and write to every library | | **Speech recognition ready** | the Bridge or a [worker](../getting-started/running-a-worker.md) can transcribe now | -| **Agent credential set** | the agent is ready. Only characters, recaps and descriptions need it | +| **Agent credential set** (Optional) | the agent is ready. Only characters, recaps and descriptions need it | | **Monthly budget reviewed** | someone saved the monthly budget on the Bridge's page | | **GitHub signed in** (Optional) | the Bridge can share with the community | | **Audible signed in** (Optional) | at least one Audible account is signed in and working | diff --git a/docs-users/admin/generating-characters-and-recaps.md b/docs-users/admin/generating-characters-and-recaps.md index 350aec45..8229cdf4 100644 --- a/docs-users/admin/generating-characters-and-recaps.md +++ b/docs-users/admin/generating-characters-and-recaps.md @@ -178,7 +178,9 @@ offers **Detect chapters** on each row, and for the books you tick. ![The Detect chapters confirmation](/img/screenshots/admin/detect-chapters-dialog.png) -Generating characters and recaps later reuses that transcription. +Generating characters and recaps later reuses that transcription. If you cancel +a Chapters job and detect chapters again, it carries on from what was already +transcribed. ### When no chapters are found diff --git a/docs-users/admin/health.md b/docs-users/admin/health.md index a4675a23..e4299580 100644 --- a/docs-users/admin/health.md +++ b/docs-users/admin/health.md @@ -31,7 +31,7 @@ Click a card to open its list. | Card | What it means | How to fix it | |---|---|---| -| **Files that couldn't be read** | A file couldn't be opened, is empty, or is damaged. The row gives the reason. | Fix the file or its permissions. **Read again** checks it straight away; otherwise the next scan does. | +| **Files that couldn't be read** | A file couldn't be read, is empty, or isn't audio. The row gives the reason in one line; for a file that isn't audio, **Details** shows the full message. | Fix the file or its permissions. **Read again** checks it straight away; otherwise the next scan does. | | **Folder may hold several books** | A folder reads as one book but looks like it holds several. | **Choose detection** opens the folder in [Library > Folders](libraries.md#the-folders-page): choose **Separate books** or **Always one book**. | | **One book split into disc folders** | A book ripped from CDs reads as one book per disc. | **Join into one book**. See [Books split into disc folders](libraries.md#books-split-into-disc-folders). | | **Likely duplicates** | Copies of one book in the same library. | Compare them side by side. See [Duplicates](#duplicates). | @@ -42,6 +42,9 @@ Click a card to open its list. | **More detailed chapters available** | The community has finer chapters that fit this copy. | **Use detailed chapters**. | | **Converted to play in browsers** | The format doesn't play in web browsers, so the server converts it for the web player. The apps play it as it is. | Nothing needed: it works. | +If a network share stops answering during a scan, its books show **reading it +timed out**. They keep their length and chapters, and the next scan tries again. + ### Working through a list - **Ignore** hides a book from this list, even through rescans. **Show @@ -210,7 +213,7 @@ a status such as **Healthy**, **Needs attention**, **Missing** or **Off**. | Row | What it tells you | |---|---| | **ffmpeg** | Converts formats browsers can't play, and places [community chapters](books.md#community-chapters) exactly. | -| **ffprobe** | Reads lengths, chapters and codecs from the files. | +| **ffprobe** | Reads lengths, chapters and codecs for the few files AudioSilo can't read itself. Without it, those files have no length or chapters. | | **Community metadata** | Whether the metadata service is answering. | | **HTTPS certificate** | Days left. **Needs attention** within two weeks of expiry. | | **Database** | Its version. Accounts and progress live here, so the server makes [backups](backups.md). | diff --git a/docs-users/admin/server.md b/docs-users/admin/server.md index b08ba1b8..7bb6702f 100644 --- a/docs-users/admin/server.md +++ b/docs-users/admin/server.md @@ -237,7 +237,8 @@ Nothing here can be changed in the console. makes the small copies the [Apple Watch app](../listening/apple-watch.md) downloads. Without it, some books won't play in a browser and watches get no copies. -- **ffprobe** reads lengths, chapters and codecs during a scan. See +- **ffprobe** reads lengths, chapters and codecs during a scan, for the files + AudioSilo can't read itself. See [ffmpeg and ffprobe](../getting-started/install-binary.md#ffmpeg-and-ffprobe). The server finds them next to its own program, on the system's PATH, or diff --git a/docs-users/admin/users-and-invites.md b/docs-users/admin/users-and-invites.md index 20133bcc..82609d4c 100644 --- a/docs-users/admin/users-and-invites.md +++ b/docs-users/admin/users-and-invites.md @@ -95,7 +95,8 @@ Their signed-in devices, each with **Sign out**, and their personal API keys, each with **Revoke**. **Sign out everywhere** signs them out of every phone, tablet and browser at -once. Their password and API keys don't change. To get back in, they sign in +once. Household members signed in on those devices are signed out too. Their +password and API keys don't change. To get back in, they sign in with their password, or you send them a new invite. ### Invites diff --git a/docs-users/faq.md b/docs-users/faq.md index 815a1799..702d147c 100644 --- a/docs-users/faq.md +++ b/docs-users/faq.md @@ -17,7 +17,7 @@ Audible's protected `.aax` and `.aaxc` files are not supported. See [Supported f No, but you'll want it. The Docker image includes it, and the standalone program downloads it if your system lacks it. -- **ffprobe** reads durations and chapters. Without it, books still play, but with no durations or chapter lists. +- **ffprobe** reads lengths and chapters for the few files AudioSilo can't read itself. Without it, those files have no length or chapter list. - **ffmpeg** converts formats a browser can't play, lets every player play an `.mka` book with MP3 inside, and places [community chapters](./admin/books.md#community-chapters) precisely. Without it, books a browser can't play work only in the apps, where the phone can decode them. [Health > System](./admin/health.md#system) shows whether each was found. See [ffmpeg and ffprobe](./getting-started/install-binary.md#ffmpeg-and-ffprobe). diff --git a/docs-users/getting-started/install-binary.md b/docs-users/getting-started/install-binary.md index c8c83b84..a4ec0632 100644 --- a/docs-users/getting-started/install-binary.md +++ b/docs-users/getting-started/install-binary.md @@ -41,7 +41,7 @@ Useful options: | `--data ` | Where settings, the database and certificates live (default `./data`). | | `--setup` | On first run, choose the admin password in the browser. | | `--ffmpeg ` | Path to ffmpeg; `""` turns off converting audio. | -| `--ffprobe ` | Path to ffprobe; `""` turns off reading durations and chapters. | +| `--ffprobe ` | Path to ffprobe; `""` turns it off. AudioSilo still reads most files' lengths and chapters itself. | ## Linux packages and systemd @@ -70,13 +70,13 @@ Your data folder and your books are not changed. When an update changes the data ## ffmpeg and ffprobe -AudioSilo uses ffprobe to read durations and chapters, and ffmpeg to convert audio a browser can't play. They are not in the download, but you usually have nothing to do: +AudioSilo uses ffprobe to read lengths and chapters, and ffmpeg to convert audio a browser can't play. They are not in the download, but you usually have nothing to do: 1. The server looks for them next to `audiosilo`, then on your `PATH`. 2. If it finds neither, it downloads a trusted copy **once** into `tools/` in your data folder. -3. If that can't happen (offline, say), the server still runs without durations, chapters or converting. It tries again at the next start. +3. If that can't happen (offline, say), the server still runs. It doesn't convert audio, and the few files it can't read itself have no length or chapters. It tries again at the next start. -**FFmpeg 7.1 or 8.1 makes first scans much faster.** With those versions the server reads most files itself instead of running ffprobe on each one. The copy the server downloads for Linux and Windows is 8.1. Run `ffprobe -version` to see which you have. +**FFmpeg 7.1 or 8.1 makes first scans much faster.** With those versions, or with no ffprobe at all, the server reads most files itself. With any other version, ffprobe reads every file. The copy the server downloads for Linux and Windows is 8.1. Run `ffprobe -version` to see which you have. ## Good to know diff --git a/docs-users/getting-started/organizing-your-library.md b/docs-users/getting-started/organizing-your-library.md index f84086d2..53efd8de 100644 --- a/docs-users/getting-started/organizing-your-library.md +++ b/docs-users/getting-started/organizing-your-library.md @@ -65,7 +65,7 @@ AudioSilo takes each book's details from these sources: - **Folder names.** The book's folder (or file) is the title, its parent folder the series, and the folder above that the author. A leading number like `03 - Title` is the book's place in the series (`02.5` works too). `Author/Title/` with no series folder works as well. - **The files' tags.** Title (usually the album tag), author, narrator, series and genres. Junk titles like "Track 01" are ignored. - **Audiobookshelf's `metadata.json`**, when the book's folder has one. See [below](#coming-from-audiobookshelf). -- **The audio itself**, via ffprobe: durations and any chapters in the file. +- **The audio itself**: durations and any chapters in the file. - **The community database**, for chapters only: a book with no chapters of its own gets them once it is [matched](../admin/books.md#community-chapters). By default tags win over folder names. If your folders are named well and your tags aren't, set the library to [folder names first](../admin/libraries.md#where-book-details-come-from). Anything still wrong can be fixed in the admin console without touching the files: see [Books and metadata](../admin/books.md). diff --git a/docs-users/getting-started/running-a-bridge.md b/docs-users/getting-started/running-a-bridge.md index b7de640e..a915d401 100644 --- a/docs-users/getting-started/running-a-bridge.md +++ b/docs-users/getting-started/running-a-bridge.md @@ -153,7 +153,8 @@ settings stay as they are. ## 3. Open the Bridge's page -Open `http://:8091` and sign in with the password. +Open `http://:8091` and sign in with the password. The page is +light or dark to match your device. ![The Bridge's sign-in page](/img/screenshots/bridge/sign-in.png) @@ -199,8 +200,9 @@ For a **podcast** library, mount its own folder the same way. An empty podcast library can be ready too: the Bridge writes a small hidden marker there. Leave it in place. -If you only want podcasts, you can skip steps 6 to 9: podcasts use no speech -recognition and no AI agent. +Steps 7 and 9 are optional: only characters, recaps and descriptions need the +agent. If you only want podcasts, you can skip steps 6 to 9: podcasts use no +speech recognition and no AI agent. ### Feeds on your local network @@ -226,7 +228,10 @@ To make it faster, or to give a `:latest-slim` Bridge a way to transcribe: Mac with Apple Silicon, or - use the `:latest-cuda` image on a computer with an NVIDIA graphics card. -## 7. The agent's credential +## 7. The agent's credential (optional) {/* #7-the-agents-credential */} + +Skip this step if you don't want characters, recaps or descriptions. Chapters, +read-alongs, podcasts and Audible work without the agent. The agent needs a credential for an AI account. It runs **Claude Code** unless you switch it to Codex (below). You can set the credential on the Bridge's diff --git a/docs-users/listening/account.md b/docs-users/listening/account.md index e2802706..ce123a1a 100644 --- a/docs-users/listening/account.md +++ b/docs-users/listening/account.md @@ -23,7 +23,7 @@ Accounts made by invite often have no password. That's fine day to day, but a pa Setting a first password signs nothing out. -**Changing a password signs out your other devices.** This device stays signed in. Every other device needs to sign in again. Your [API keys](#personal-api-keys) and [calendar link](upcoming.md#add-to-your-calendar) keep working. If someone else knew your old password, this locks them out. +**Changing a password signs out your other devices.** This device stays signed in. Every other device needs to sign in again, and so do household members signed in on those devices. Your [API keys](#personal-api-keys) and [calendar link](upcoming.md#add-to-your-calendar) keep working. If someone else knew your old password, this locks them out. ## Pair another device diff --git a/docs-users/listening/browsing.md b/docs-users/listening/browsing.md index 2e06b6b6..fccd0c40 100644 --- a/docs-users/listening/browsing.md +++ b/docs-users/listening/browsing.md @@ -169,7 +169,7 @@ The **Search** tab finds books, series, authors, narrators and characters, acros Results appear as you type, grouped into: -- **Books** - matching titles, authors, narrators, series, genres and tags. A book on more than one server shows once. +- **Books** - matching titles, authors, narrators, series, genres and tags. A book on more than one server shows once. A Chinese or Japanese title is found by any character or word in it. - **Series** - each opens its [series page](#series-pages). - **People** - authors and narrators. - **Characters** - from books you've started. diff --git a/docs-users/listening/end-of-book.md b/docs-users/listening/end-of-book.md index a9da213e..b51d2f53 100644 --- a/docs-users/listening/end-of-book.md +++ b/docs-users/listening/end-of-book.md @@ -32,6 +32,8 @@ AudioSilo picks the next book the same way everywhere: A book in more than one series follows its main series first, then its others. After an omnibus of books 1-3, the next book is book 4. +If the server is still scanning a series folder, the next book's folder plays even before it's scanned. + If nothing comes next, the card says **End of the series**. If the community knows a next book that your server doesn't have, it shows as a pale cover with **View on AudioSilo Meta**. AudioSilo never tries to play a book you don't have, and skips any gap in the series. ## Playing on automatically diff --git a/screenshots/capture-admin.mjs b/screenshots/capture-admin.mjs index 6216b747..d06610f8 100644 --- a/screenshots/capture-admin.mjs +++ b/screenshots/capture-admin.mjs @@ -1114,7 +1114,7 @@ const standInSetup = (libraries, {budget, github}) => { detail: `${libraries.length - missing.length} of ${libraries.length} libraries writable`, ...(missing.length ? {library_ids: missing} : {})}, {id: 'asr', done: true, required: true, detail: 'worker'}, - {id: 'agent', done: true, required: true, detail: 'claude'}, + {id: 'agent', done: true, required: false, detail: 'claude'}, {id: 'budget', done: budget, required: true, detail: '$100 a month'}, {id: 'github', done: github, required: false, ...(github ? {detail: BRIDGE_GITHUB_LOGIN} : {})}, ], @@ -1228,7 +1228,7 @@ await step('settings: bridge', async () => { await tryShot(async () => { await open(page, '/server?topic=bridge'); const setupCard = page.locator('[aria-labelledby="bridge-setup-title"]'); - await setupCard.getByText('3 of 5 required steps done').waitFor({timeout: 15000}); + await setupCard.getByText('2 of 4 required steps done').waitFor({timeout: 15000}); await setupCard.scrollIntoViewIfNeeded(); await sleep(800); await shoot(page, 'admin/bridge-setup.png', {clip: around([await setupCard.boundingBox()])}); diff --git a/screenshots/manifest.mjs b/screenshots/manifest.mjs index cd97575f..ac0c6819 100644 --- a/screenshots/manifest.mjs +++ b/screenshots/manifest.mjs @@ -149,8 +149,8 @@ export const SHOTS = [ {file: 'admin/settings-modules.png', capture: 'admin', title: 'Admin - Modules', hint: 'Server > Settings ?topic=modules with no Bridge paired: the intro line, the Contributions card (its blurb on sending corrections, characters and recaps to the community, Learn more, the Enable Contributions switch off, Needs a Bridge with the Bridge settings link) and the Podcasts card beside it (its blurb, the Enable Podcasts switch off, Needs a Bridge) (1440x1100)'}, {file: 'admin/settings-bridge.png', capture: 'admin', title: 'Admin - Bridge, not paired', hint: 'Server > Settings ?topic=bridge with no Bridge paired: the Bridge card (what it is) and Connect a bridge'}, {file: 'admin/bridge-pair.png', capture: 'admin', title: 'Admin - Connect a bridge', hint: 'the Connect a bridge dialog: a fresh pairing code with its countdown, the server address, the two steps and the two environment variables, Waiting for the bridge... (the code is exchanged straight after by a stand-in Bridge)'}, - {file: 'admin/bridge-setup.png', capture: 'admin', title: 'Admin - Bridge setup card', hint: 'Server > Settings ?topic=bridge, clipped to the Setup card at the top: the stand-in Bridge\'s setup checklist (Get set up, 3 of 5 required steps done: paired, speech recognition (worker) and the agent (claude) done, libraries 1 of 2 writable and the monthly budget still to do with Fix on the Bridge links, GitHub optional and not done), then GitHub account Not signed in, Agent credential Set, Contributing Not ready, and Open Accounts on the Bridge'}, - {file: 'admin/settings-bridge-paired.png', capture: 'admin', title: 'Admin - Bridge, paired', hint: 'Server > Settings ?topic=bridge with a stand-in Bridge paired through the API (heartbeat sent with its kinds, the sidecars estimates and a setup checklist with the budget reviewed and GitHub signed in, page link set): the Get set up card first (4 of 5: one library read-only), Online, the status facts with the Agent, Paid by, This month and Estimates from rows, Libraries (the seeded library Ready, any other one mapped but read-only), Workers (one ASR worker, a Mac on Metal), Jobs with a finished Verify, the Unpair danger zone (viewport as tall as the page)'}, + {file: 'admin/bridge-setup.png', capture: 'admin', title: 'Admin - Bridge setup card', hint: 'Server > Settings ?topic=bridge, clipped to the Setup card at the top: the stand-in Bridge\'s setup checklist (Get set up, 2 of 4 required steps done: paired and speech recognition (worker) done, libraries 1 of 2 writable and the monthly budget still to do with Fix on the Bridge links; the agent (claude) optional and done, GitHub optional and not done), then GitHub account Not signed in, Agent credential Set, Contributing Not ready, and Open Accounts on the Bridge'}, + {file: 'admin/settings-bridge-paired.png', capture: 'admin', title: 'Admin - Bridge, paired', hint: 'Server > Settings ?topic=bridge with a stand-in Bridge paired through the API (heartbeat sent with its kinds, the sidecars estimates and a setup checklist with the budget reviewed and GitHub signed in, page link set): the Get set up card first (3 of 4: one library read-only), Online, the status facts with the Agent, Paid by, This month and Estimates from rows, Libraries (the seeded library Ready, any other one mapped but read-only), Workers (one ASR worker, a Mac on Metal), Jobs with a finished Verify, the Unpair danger zone (viewport as tall as the page)'}, {file: 'admin/book-generate.png', capture: 'admin', title: 'Admin - Generate characters and recaps', hint: 'the first book\'s Community metadata card (clipped) with its Characters and recaps part and Generate characters and recaps, while the stand-in Bridge offers sidecars.generate'}, {file: 'admin/generate-dialog.png', capture: 'admin', title: 'Admin - Generate dialog', hint: 'the Generate characters and recaps confirmation over the first book\'s page: Reads, Length, Transcription, Agent cost (a subscription), Results, the subscription note and Also contribute to the community, switched on (the stand-in can contribute and the Contributions module is on); closed, never submitted'}, {file: 'admin/book-story.png', capture: 'admin', title: 'Admin - Characters and recaps card', hint: "The Call of the Wild's Characters and recaps card (clipped) on its book page: the Characters tab with its count, the Generated on this server badge and date, Contributed, waiting for review (a generate job for it, run to done by the stand-in Bridge, contributed both), the first 10 characters (name, role, aliases, description, From chapter N) and Show all 12 characters; the characters and recaps are the ones run.sh seeds into its .audiosilo folder"}, diff --git a/static/img/screenshots/admin/bridge-setup.png b/static/img/screenshots/admin/bridge-setup.png index 88396fed..0233c102 100644 Binary files a/static/img/screenshots/admin/bridge-setup.png and b/static/img/screenshots/admin/bridge-setup.png differ diff --git a/static/img/screenshots/admin/health-issues.png b/static/img/screenshots/admin/health-issues.png index b1266973..b7da1035 100644 Binary files a/static/img/screenshots/admin/health-issues.png and b/static/img/screenshots/admin/health-issues.png differ diff --git a/static/img/screenshots/admin/settings-bridge-paired.png b/static/img/screenshots/admin/settings-bridge-paired.png index c18bbe23..9013fbd2 100644 Binary files a/static/img/screenshots/admin/settings-bridge-paired.png and b/static/img/screenshots/admin/settings-bridge-paired.png differ