Skip to content

1878 api cms hls media data model - #1910

Open
johan-bell wants to merge 66 commits into
mainfrom
1878-api-cms-hls-media-data-model
Open

1878 api cms hls media data model#1910
johan-bell wants to merge 66 commits into
mainfrom
1878-api-cms-hls-media-data-model

Conversation

@johan-bell

@johan-bell johan-bell commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

The CMS ⇄ encoder media integration: the HLS data model, the storage lifecycle, key delivery, and the player that plays what the encoder produces. ~52 commits; the companion encoder branch is bccsa/luminary-media-convert#200 (review both together — the app consumes the encoder's player packages via the submodule).

What lands

  • Data model & storage lifecycleMediaDto on the content parent, S3 migration on a bucket change, delete-files-on-request at document deletion, and the encoder-config handshake that hands the encoder its credentials (GET /storage/encoderconfig, gated on Assign).
  • Relative media URLshlsUrl stored relative to the document's own bucket so the two cannot drift; external URLs (YouTube, another CDN) keep their full form and (late fix) need no bucket at all — only a URL inside our own storage must name one.
  • Key delivery as sidecars (ADR 0019) — the HLS AES-128 key lives in a non-replicated sidecar document, masked at rest, served one parent at a time by GET /sidecar: uniform 404s so parents can't be probed, read + probe rate limits, publish-state gating with the CMS exempt, Cache-Control: no-store. Sidecars are excluded from /query, socket fan-out, ACL grants, and change requests (including id-prefix squatting).
  • The player swap — the app's VideoPlayer.vue renders the shared player instead of assembling video.js itself (−778 lines of hls.js workarounds); the CMS previews through the same player in a modal.
  • The CMS Media section, redone — one card: bucket → encode status → video → audio list. Encode progress from the local encoder app, availability polling that notices the encoder opening/closing on its own, resume of an encode after a reload, and an encoder-update notice.
  • Bucket-owned encode settings — encryption / byte-range / chunk size live on the media bucket (StorageDto.mediaSettings) and travel to the encoder through the encoderconfig response; encryption defaults on.
  • v21 schema upgrade — the legacy per-language Content.video URL collapses onto the parent's media.hlsUrl (many-to-one, logged when values are dropped).
  • The encoder as a submodule — its player packages consumed by path; CI builds them before the app.
  • Main is merged in — NestJS 11 (middie CVE), slug-change rendering, theme pre-paint.

Closes

#1900 (all six items), and the media half of #1878.

Verified

  • Encrypted playback end to end against a real encoder session writing LMCENC playlists to MinIO — playlist + segment decryption, angle switching, audio tracks, audio-only, YouTube (after the techOrder fix on the encoder branch).
  • Sidecar endpoint against real CouchDB with real ACLs: allowed, denied, absent, draft — denial byte-identical to absence.
  • Suites: api 982, app 992, cms 1177, shared 1367/1368 (the one failure pre-exists on main); app + cms type-checks at 0 errors.

Before merging

  1. CI is red for one reason: the submodule repo is private and the MEDIA_CONVERT_SUBMODULE_TOKEN secret isn't set yet. Either add the secret (fine-grained PAT, read-only Contents on luminary-media-convert) or make that repo public first.
  2. Merge APP: Add query caching for heavy database queries #200 → make the encoder repo public → bump the submodule pointer to the resulting commit on its main (and flip .gitmodules to branch = main). The current pointer references a branch commit that a squash-merge of APP: Add query caching for heavy database queries #200 would eventually orphan.

ivanslabbert and others added 8 commits August 4, 2026 16:38
- Refactored MediaEditor component to remove legacy media upload functionality.
- Added useMediaEncoder composable to manage encoding sessions with Luminary Media Convert.
- Implemented health checks and session management for the encoder.
- Created utility functions for handling encoder session keys and events.
- Updated API to fetch encoder configuration for media buckets.
- Enhanced media DTO to include HLS URL and key for encoded media.
- Added tests for media encoder utilities and session management.
processMediaDto no longer processes uploads, and MediaDto has no
fileCollections or uploadData, so two validateChangeRequest tests were
asserting against a shape the DTO rejects. Replaced with the current
contract: a collection validates with an hlsUrl (plus optional hlsKey),
and a key with no URL fails on hlsUrl.

Also drop a now-unused MediaType import that failed lint.

fix(app): skip player setup when there is no <video> element

videojs() throws on an element it does not recognise, and inside the
mounted hook that surfaces as an unhandled rejection rather than
something the caller can catch. The ref is empty whenever the element
was not rendered, which is why mounting the player for content whose
video comes from parentMedia.hlsUrl failed the test run while every
assertion still passed.
Deleting a post left its whole HLS collection in the bucket permanently:
hundreds of objects, the playlists, the chunk chains, the sprites and the
sidecars. Images have always been cleaned up by deleteImage three lines away;
media was the only asset type that leaked.

Opt-in, carried on the document. A delete *is* a change request — the whole
document arrives with deleteReq set — so `media.deleteFiles` travels with the
thing it applies to and cannot be separated from it in flight. Write-only and
never persisted, the same shape as `media.hlsKey`.

The location comes from the *saved* document and the intent from the incoming
one, so an hlsUrl edited in the same breath as the delete cannot redirect the
deletion somewhere else.

Resolving the prefix is where the safety lives, and the guard falls out of the
arithmetic rather than being bolted on: the only way to get an object key from a
public URL is to strip the bucket's own public base, so a URL that does not start
with that base cannot be resolved at all. That is the "never delete a prefix we
did not create" rule, and unlike a marker object it also protects every
collection already in a bucket. On top of it: the key must end in
/master.m3u8, must not be the bucket root, must contain no traversal, and its
last folder must be a session id — because hlsUrl is an editable field, and
without that last check someone could paste a URL naming a shared folder and turn
"delete everything under it" into a tick box.

A refusal is a warning, not an error, and so is a failure part-way: the caller is
deleting a document, and refusing to do that because a bucket was unreachable
would be worse than leaving objects behind. Every key is logged before removal,
so a deletion that turns out to be wrong can be reconstructed rather than guessed
at.

Adds S3Service.listObjectsUnder: the existing listObjects returns the whole
bucket as a stream, which cannot answer "what belongs to this collection".

The resolver's tests need no database — deliberately, since the guards are the
part that must never regress and CouchDB is not always up.
A checkbox in the delete confirmation, shown only when the document actually has
a collection, with the URL underneath so it is clear what goes.

A checkbox rather than a toggle: a toggle reads as a setting applied the moment
it moves, and this is a choice being confirmed. Unticked by default, and reset
every time the dialog opens — an irreversible option that remembers a previous
"yes" is one somebody eventually triggers without meaning to. Matches the
existing "Duplicate image" checkbox in the duplicate dialog two blocks below.

The answer rides on the document as `media.deleteFiles`, which the API reads and
never stores, because a delete already sends the whole document with `deleteReq`
set. Nothing to clean up afterwards either way.
The Vite SSG work (#1686) rewrote SingleContent.vue and reworked ContentTile.vue
around it. Both conflicts were in the import blocks alone — the functional hunks
merged on their own — so main's versions stand and the videoSource changes are
re-applied on top: hasVideoSource/videoSourceFor pick the encoder's
parentMedia.hlsUrl over a typed-in `video` URL, in the tile's media icons and
progress bar and in the article page's VideoPlayer guard. Main's one-line
readingTime import is kept.
Changing `mediaBucketId` used to move nothing: `processMedia` only stored the
key, so the document ended up naming one bucket while `hlsUrl` still pointed at
another. That is worse than untidy — `resolveCollectionPrefix` derives the
storage prefix by stripping the bucket's own public base off the URL, so once the
two disagree the collection can no longer be resolved, and deleting the document
with "delete the files" ticked refuses and leaves the objects behind for good.
Images have migrated between buckets since they were added; media never did.

The order is the design: copy the whole collection, check each object's size
against the source, only then rewrite `hlsUrl`, and only then delete the
originals. A collection is not a set of independent files — a master playlist
without its segments is a broken video — so a per-file "upload then delete" like
`migrateImagesBetweenBuckets` is not enough here, where a partial result costs a
video rather than one thumbnail. Any failure leaves the source whole and
untouched and reverts `mediaBucketId`, so the document keeps pointing at files
that exist.

Copies stream. `uploadFile` takes a Buffer, which is right for a thumbnail and
wrong for a byte-range chunk chain capped at 500 MB by default, so `putStream`
was added alongside it and takes the size from the source's own `statObject`.

Refusals are inherited rather than restated: the same session-id proof that
guards deletion decides whether a collection may be moved. Changing the URL and
the bucket in one save is read as repointing the document by hand and moves
nothing, which keeps a deliberate edit from being overwritten.
@johan-bell johan-bell linked an issue Aug 18, 2026 that may be closed by this pull request
johan-bell and others added 2 commits August 18, 2026 07:46
…acing a key (#1912)

`hlsUrl` is stored relative to the bucket the document already names, so the two
cannot disagree later — which removes the divergence that left files unreachable
when a bucket changed. Absolute URLs are normalised on save and resolved back at
playback; media hosted elsewhere is recognised as external and left alone, since
it is not this bucket's to move or delete.

Re-submitting a key that is already stored is no longer a change: the CMS cannot
tell (a saved key is only ever an id there), so it confirms a replacement it
cannot rule out, and the API compares the two and keeps the existing crypto
object rather than orphaning it.
johan-bell and others added 18 commits August 18, 2026 16:32
The encoder's player libraries are consumed as a git submodule rather than
published to a registry: npm cannot install a subdirectory of a git repo, and
the packages move together with the encoder they are cut from.

`app` depends on `player-web-legacy` by path. It is Video.js 8 over the same
`player-core` contract as `player-web`, and it replicates the chrome the app
already draws — so pointing `VideoPlayer.vue` at it is an integration rather
than a chrome rewrite.

The Docker build builds the submodule's libraries before the app installs,
because they ship only `dist/`.
… needs (#1913)

VideoPlayer.vue renders LuminaryPlayer from the encoder's player-web-legacy
instead of assembling video.js itself. Same Video.js 8 chrome, now maintained
next to the encoder that produces the streams — 661 lines become 181. The app
stops owning control-bar layout, auto-hide, the iOS keep-alive element, rotation
and fullscreen handling, audio-track selection, the stall nudge, audio-only mode
and the whole YouTube branch. It keeps what is Luminary's: which URL to play,
where the key comes from, the resume point, and the engagement a finished video
records.

Encrypted media plays for the first time here. The key is stored as a crypto
object and never replicated — documents carry only hlsKey_id — so nothing served
it to a client and every encrypted collection was undecryptable by every player.
GET /media/key closes that, gated on exactly the permission that decides whether
the document is visible.

Verified by hand against a real encoder session: LMCENC playlists, segment
decryption, angle switching, audio tracks, audio-only and YouTube playback.
EditContentVideo had a URL field and a key field and no way to find out whether
they work — a wrong key, a path that 404s and angles that never reached the bucket
look identical in a text input, and the editor learned about it when a reader did.

The preview plays through the same LuminaryPlayer the app uses, so what an editor
checks is what a viewer gets. It loads on request rather than on sight, and takes
the key from whichever source has it: one just typed is used directly, and a saved
one — which the document can never show again — is fetched.

Which required widening who may fetch a key: the endpoint checked View, the app's
permission, and an editor previewing media they are about to publish holds CmsView
without necessarily holding it. Either now suffices, and no request parameter picks
between them.

Also stops both images installing the encoder's desktop app to build its
libraries — around 500 MB each for electron and electron-builder, to produce five
small libraries neither image runs.
Ten files, none of them imported by anything since VideoPlayer.vue started
rendering LuminaryPlayer. Six sources and the four specs whose only subject is
the file beside them.

  extractAndBuildAudioMaster.ts  the app built a data-URI audio master by hand;
                                 the library switches to the audio-only angle
  audioTrackLanguage.ts          superseded by the player's preferredLanguage,
                                 which matches across all three ISO code sets
  form/AudioVideoToggle.vue      the player draws its own
  util/youtube.ts                detection, lazy tech import and URL normalising
                                 are the player's YouTube mode
  content/VideoPlayer.css        ported into the library's styles.css, compared
                                 rule by rule; the two .audio-mode rules it had
                                 that the port lacked are there as of #195
  content/px.png                 replaced by TRANSPARENT_POSTER — and this file
                                 was the correct pixel the base64 got wrong

Kept until playback had been verified by hand, because VideoPlayer.css was the
only written record of the old skin and a swap you cannot yet compare is a swap
you might have to undo. That is done, so it goes.

cms/src/util/youtubeUtils.ts is a different file and is untouched.
Every other test of this component stubs `getRest`, which proves the component
and proves nothing about the wiring. This runs the whole chain — component,
shared REST client, real HTTP, the endpoint, CouchDB and the crypto document —
with only the player stubbed, because video.js cannot play in jsdom.

The case worth having is not that a key comes back. It is that a refusal does
not: the endpoint 404s for unencrypted media and for a caller who may not have
the key, `handleResponse` turns both into `undefined`, and the component plays on
without one. A throw anywhere in that path would leave an unhandled rejection in
a watcher and no player on screen, and nothing else covers it.

Skipped unless LUMINARY_LIVE_API points at a running API, so it costs nothing in
CI and is a real check when someone stands one up.

Found while running it: the group ACL belongs on the group the documents are
memberOf, naming the group allowed to view them — `parentGroup.upsertMap(this.id,
…)` in permissions.service.ts. Seeded the other way round, every request 404s
indistinguishably from a missing document, because this endpoint deliberately
answers both the same way.
The app workflows asked for the private encoder submodule and could not have it —
a workflow's GITHUB_TOKEN is scoped to this repository, so a private sibling
answers "Repository not found" and checkout aborts before any test runs. The CMS
workflows never asked for it at all, so ten suites died on importing
player-web-legacy.

All four now check it out with secrets.MEDIA_CONVERT_TOKEN. That secret does not
exist yet: until it is set these fail exactly as they do now, and pass the moment
it is. If the encoder is ever made public the token line can go.
#1919)

actions/checkout rejects an empty token: outright, so pointing at an uncreated secret broke this repository's own checkout rather than leaving only the submodule to fail. The fallback restores the intended degradation.
Introduces a `Sidecar` doc type for Post/Tag-scoped payloads (starting
with masked HLS AES-128 keys) that are never replicated to clients,
never bulk-queryable, and never ACL-grantable. Locks the type down at
every boundary: change-request validation, ACL validation, /query's
internal-type gate, socket.io broadcast, and the delete-cmd path -
plus keeps `memberOf` synced from the parent on every save.
Add api/src/util/maskKey.ts — the XOR mask (SHA-256(seed)[0..15]) that
keeps raw AES keys out of logs, proxy caches, and DB dumps. Self-inverse,
so mask and unmask are one function; the seed is the sidecar _id.

Deliberate duplication of cms/src/util/mediaEncoder.ts unmaskKeyHex (the
API cannot import shared/cms). A shared (seed, key) -> masked literal is
asserted in both api/src/util/maskKey.spec.ts and
cms/src/util/mediaEncoder.spec.ts so a divergence fails a test rather
than a video in the player.

Also lands the docs/sidecar/ design docs for the encryption-key-service
feature and marks step 1 done in the implementation-order tracker.

Co-Authored-By: Claude <noreply@anthropic.com>
…(step 3)

processMedia now masks the submitted HLS key and writes it as a sidecar
(via upsertHlsKeySidecar) instead of AES-256-CBC-encrypting it into a
CryptoDto. The sidecar carries the parent's memberOf so the existing
permission system gates it; the key is masked, not encrypted at rest —
see ADR 0018.

A failed key store now fails the change request instead of being
swallowed into a warning. The previous behaviour caused silent,
unrecoverable key loss: the plaintext existed only for the duration of
the request, so saving the Post with an hlsUrl and no hlsKey_id left an
unplayable collection. processPostTagDto no longer catches the
processMedia error, so it propagates through processChangeRequest.

processMedia takes the parent doc (needed for the deterministic sidecar
_id, memberOf, parentType, updatedBy); processPostTagDto passes it.

CryptoDto/storeCryptoData/retrieveCryptoData/encryption.ts stay — still
used for S3 credentials. No migration (1878 unshipped). Specs inverted
to assert sidecar storage, idempotency per parent, and fail-on-store.

Co-Authored-By: Claude <noreply@anthropic.com>
Sidecars now go with their parent (deleteReq branch, warn-not-throw to
match deleteImage's precedent) and are dropped when the HLS key
reference disappears, whether the editor clears just the key field or
the whole media object. Covers docs/sidecar/03.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Serves sidecar payloads one parent at a time: parent-doc permission
check (never the sidecar's copied memberOf), an availability check
that refuses draft/scheduled/expired parents even to a caller holding
View, and no batch parameter so bulk extraction costs one authorised
request per parent. Covers docs/sidecar/02, 09, 10.

Rate limiting and audit logging (doc 02) are intentionally deferred —
the doc leaves the concrete mechanism as an implementation-time
decision rather than a spec.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…llow-up)

docs/sidecar/02 calls rate limiting a requirement, not a footnote, for
the key-harvesting risk a per-parent GET endpoint has without a batch
parameter. Adds SidecarRateLimiterService with two independently
bucketed limiters: `read` bounds successful key fetches (default on,
~30/min/identity) and `probe` bounds repeated 403/404s at a lower
ceiling (default on) — both default on, unlike the query limiter,
since this endpoint hands out secrets.

Extracts the config-gated StrikeLimiter wrapper QueryRateLimiterService
already had into a shared RateLimiterService rather than duplicating it
a second and third time for sidecar's two limiters. Concrete limits and
rationale recorded in ADR 0018.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… silently

maskKeyHex truncates invalid hex to an empty buffer, so a bad key was
accepted, stored as a broken sidecar, and only surfaced as a 409 from
GET /sidecar at playback time. Enforce the same 32-hex-char AES-128
format that isHlsEncryptionKeyData already expects on read.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s them

The design docs were still marked "Status: proposal, not implemented"
against a stale branch base, yet ~14 code comments cited them as if
durable. The substance now lives in ADR 0018; repoint every citation
there and delete the temp design-doc directory.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The comment still described the retired crypto-object storage path
(e26c5b6) and, more fundamentally, shouldn't describe API-side
storage at all — this is client code, and the API's implementation
isn't its concern. State only the observable contract: submit a key,
get back an id, never the key itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Provisioned by the Orca worktree setup so `api`/`app`/`cms` each run on
dedicated ports (avoids collisions when multiple worktrees run in parallel).
Encode settings move onto the media bucket, where they apply to every
encode written into it instead of being decided per encode or hard-coded:

- `StorageDto.mediaSettings` (shared + api mirror): one object holding
  `encrypted`, `byteRange` and `chunkSizeMB`. All optional — an absent
  field means the encoder's own default, and encryption defaults to on,
  which was the behaviour before it became a setting.
- The bucket form shows the section for media buckets only: two toggles
  and a chunk-size field, patched onto the document as one object.
- `GET /storage/encoderconfig` translates the settings into the encoder's
  session-body field names (`encryption.required`, `byteRange`,
  `byteRangeMaxFileSizeMB` + audio twin — one chunk size covers both),
  and `useMediaEncoder` forwards the whole response without knowing what
  is in it. The hard-coded `encryption: { required: true }` is gone.

segmentDuration stays unsent on purpose: the encoder defaults it (6s).

The update check: the CMS already reads the encoder's version off its
health response. `MIN_ENCODER_VERSION` now names the oldest encoder this
CMS can talk to, and an older one gets a warning notice in the Media
section pointing at the GitHub releases page. Outdated outranks the
availability notices — the encoder answered, so "not running" would be
wrong. An unparsable version is not called outdated: a false alarm would
nag every working install.

No encoder-side changes: its CMS session DTO already accepts every field
sent here.
The focus ring around a just-clicked toggle was pressed against the
modal's edge; the media-settings rows now keep a little right padding so
it has room.
… broken pin (#1991)

The root package.json was the encoder's app-electron manifest committed
into this repo by accident — luminary deliberately has no root package
(no workspace tooling), and this one declared a `file:../api` dependency
that resolves to nothing here.

The submodule was pinned to 8ff10ae, a commit that neither compiles nor
passes the encoder's tests, and which predates the player fixes the app
depends on — most importantly the YouTube techOrder fix (#224), without
which YouTube playback shows a broken-media placeholder for readers.
Now pinned to ff0f6c9, the current tip of luminary-1897-encrypted-hls.
johan-bell and others added 2 commits September 2, 2026 19:45
… CI actually run (#1993)

* fix(api): migrate NestJS 10 to 11 to clear the @fastify/middie auth-bypass CVE (#1950)

* fix(SingleContent): ensure correct content rendering on slug change and add navigation tests (#1968)

* fix(SingleContent): ensure correct content rendering on slug change and add navigation tests

* test(post-navigation): add tests to ensure previous post content is not rendered when navigating between posts

* fix(SingleContent): prevent navigation to previous post's translation when opening a new post

* fix(HybridQuery): implement keepPreviousResult option to control output persistence across query rebuilds

* feat(HybridQuery): add keepPreviousResult option to maintain output during query rebuilds

* feat(theme): centralize theme storage key and enhance pre-paint theme handling (#1984)

* feat(theme): centralize theme storage key and enhance pre-paint theme handling

* refactor(theme): replace THEME_STORAGE_KEY import with hard-coded value and remove themeStorage module

* fix(ci): reach the private submodule, build the player libs for E2E, and clear the type errors CI never got far enough to see

The four workflows that clone the submodule assumed the encoder
repository is public; it is not yet, so every run died at checkout with
"repository not found". They now pass
`secrets.MEDIA_CONVERT_SUBMODULE_TOKEN || github.token` — a fine-grained
read PAT until the repository goes public, the default token after,
when the secret can simply be deleted.

E2E (local stack) had two gaps of its own: no submodule in its checkout
and no player-library build, so the app's production build could not
resolve @luminary-media-converter/player-web-legacy. It now checks out
and builds them the same way the unit-test workflows do.

With checkout fixed, type-check runs — and fails on errors the dead
checkout had been masking: the sync specs' accessible-groups literals
predate DocType.Sidecar, EditContentMedia's spec reads props("parent")
without narrowing, and one hoisted mock's call tuple is typed empty.
All fixed; app and cms type-checks are clean.

---------

Co-authored-by: Christian Touoyim <156022993+ChrisTouo@users.noreply.github.com>
Co-authored-by: Dirk <dirkdevbusiness@gmail.com>
The content of main's three commits arrived in the squash of #1993; the
squash kept the tree and discarded the merge parent, so GitHub still
reported #1910 as conflicting. This merge adds the parentage and changes
no files.
johan-bell and others added 3 commits September 3, 2026 22:18
#200 is merged, so the player packages the app builds against live on the
encoder's main branch. The pin was ff0f6c9 and `branch` still named the
integration branch — which the merge deleted, leaving the pinned commit
unreachable and every fresh checkout one GitHub garbage collection away
from failing to clone it.

Both now point at main: 33e0ecf, the squash of #200.
CI could not clone the submodule:

    fatal: remote error: upload-pack: not our ref fe8b0102…
    Fetched in submodule path 'luminary-media-convert', but it did not
    contain fe8b0102…

fe8b010 is a local commit in the nested checkout — it enables the
encoder's workflows by renaming .github.disabled to .github — and it was
never pushed, so nothing but that one working copy can resolve it. The
commit before it moved the pin back to ff0f6c9, which the #200 squash
orphaned when GitHub deleted the branch it was on.

Pinned to 6f4fcd9, the tip of the encoder's main, which reaches the same
end by a pushed route: #231 enables the test workflow there, and #230 is
in it too.
…#2002)

The only download link sat in the "your encoder is outdated" notice —
which by definition an editor only sees once they already have the app.
Someone who has never installed it got "Luminary Media Convert is not
running. Open it, then try again", where "Open it" is a luminary-convert://
link nothing on their machine has registered. The click did nothing, and
the CMS offered no way to get it.

Both unreachable notices now offer the download beside the launch link,
and the Media help text names it too, so it is findable when no notice is
showing.

Links to releases/latest rather than a named asset: asset filenames carry
the version, so a direct link would break on the next release, while the
page is where a person picks the right one for their machine anyway.
… the editor (#1985)

* feat(cms): notice when the encoder appears, without asking the editor to retry

The notice told an editor to open Luminary Media Convert and try again, and
meant it: nothing on the page would ever find out on its own. The launch
link scheduled a single re-check two seconds later, which is shorter than
the app takes to boot Nest and probe the machine's encoders — so the usual
outcome was a notice that still said the app was not running when it was.
Opening it from the Dock instead of the link re-checked nothing at all.

The section now polls while the encoder is missing and stops the moment it
answers. Only while the tab is visible: a background tab is not somebody
waiting for a window to appear, and this is a request per interval to a
port that may have nothing on it. Coming back to the tab checks
immediately, which is the likeliest moment for the app to have been started
in the meantime.

The single delayed re-check goes with it, and EncodeStatus loses its one
emit — one mechanism rather than two doing the same thing badly.

Both notices say so instead of asking for another attempt: "Open it — this
updates on its own once it is."

* fix(cms): notice the encoder going away, not only arriving

The watch was one-directional. Polling stopped the moment the encoder
answered and the visibility check bailed out while it was available, so an
editor who quit the encoder was left with an Encode button that still
looked usable on an app that was no longer there. Clicking it did report
the truth — start() re-checks before opening a session — but only after
they had clicked something that looked ready.

Returning to the tab now checks whichever way the answer goes. Polling
still only runs while the encoder is missing; a request every few seconds
for the life of an open document is not worth keeping a button greyed out,
and quitting the encoder means leaving the browser and coming back, which
is the moment that catches it for nothing.

A check that finds it gone starts the watch again, so the notice recovers
on its own when the app is reopened.

* fix(cms): watch the encoder continuously, because switching apps is not a tab change

Reported: closing the encoder still left the CMS thinking it was there.

The previous commit claimed the reverse direction was covered by checking
on `visibilitychange`, reasoning that quitting the encoder means leaving
the browser and coming back. That is wrong about the event.
`visibilitychange` fires when a *tab* is hidden — switched away from, or
the window minimised — and not when another application takes focus. So
quitting the encoder from its own menu, with the CMS tab still the visible
one, fired nothing at all: exactly the case it was added for.

Polling now runs the whole time the section is mounted rather than only
while the encoder is missing, which makes the answer true in both
directions without depending on what the browser considers a visibility
change. Slower once it has answered — ten seconds against three — because
then it is confirming rather than waiting, and either way it is a loopback
request that fails immediately when nothing is listening on the port. A
hidden tab is still skipped; nobody there is waiting for an answer.

`focus` on the window is added beside `visibilitychange`. That one does
fire on an application switch, so coming back from the encoder asks at once
rather than at the next interval.

* fix(cms): one decimal on the encode percentage, and none over the video

The encoder reports progress as a raw float, so the bar was labelled
"Encoding 1.7666666666666668%" — arithmetic rather than progress. Rounded
to one decimal, through Number() so a whole percentage still reads as one
rather than "23.0".

The preview's waiting panel loses its second line entirely. "Encoding is at
23.6%. Checking again automatically." put the same number a second time,
unrounded, across the middle of a frame the editor is trying to judge —
and progress belongs in the Media section, where there is a bar for it.
What is left is the thing the panel is for: "Nothing at this URL yet."

The two props that fed it, and their drilling down from EditContentMedia
through EditContentVideo, go with it.
* ci: drop the submodule token, the encoder repository is public

The checkout steps passed
`secrets.MEDIA_CONVERT_SUBMODULE_TOKEN || github.token` because the
submodule was private and the built-in token could not read it. It is
public now, so the fallback is the only branch that ever runs and the
secret it names does not exist.

`submodules: true` alone is enough. The comments say why the submodule is
there rather than what a secret was for.

* ci: drop the comment with the token

`submodules: true` says what it does. Why the submodule exists belongs
in the README, not repeated across five checkout steps.
* Second review pass over the HLS media epic

Races: a slow encode start could write another document's hlsUrl and
key onto the one now open, and leaked its EventSource; the Encode
button went "checking" on every poll tick and window focus; the poller
could re-arm after unmount; encrypted video loaded twice in the app;
the CMS preview could take a stale key after a document switch.

Duplication: one unmaskKeyHex, one mediaUrl resolver and one
fetchHlsKey in shared; one effective-bucket rule; one loadBucket and
one MASTER in documentProcessing, using isInOurStorage.

Dead code: the app's video.js/videojs-*/m3u8-parser/iso-639-2
dependencies, their shims and stale mocks; LDialog's unused props;
a stale coverage snapshot.

Also: no-store on the credential endpoint, a relative enums import,
two `any`s typed, and the flagged comment paragraphs cut to a why.

* v21: stamp parentMedia on migrated children

Only a change request stamps parentMedia, so after the upgrade the
app read neither parentMedia.hlsUrl nor video and every migrated post
showed no video until its parent was next saved; the idempotency
guards meant a re-run could not repair it.
Comment thread shared/src/util/mediaUrl.ts Fixed
johan-bell and others added 2 commits September 4, 2026 13:44
`/\/+$/` is quadratic when the run of slashes does not end the string —
2.8s for 100k on this machine — and `shared` is a published library, so
the argument is the caller's (CodeQL js/polynomial-redos, alert 51).

Adds the spec the helper never had: it moved into shared from two copies
that had none, covered only indirectly through the app's resolveVideoSource.
Three conflicts:

- app/package.json — main added inapp-spy for the Telegram in-app browser
  check; the epic had removed iso-639-2 as dead. Both hold: inapp-spy is
  imported by util/inAppBrowser.ts, iso-639-2 still has no importer.
- app/package-lock.json — regenerated from the resolved manifest.
- LModal — main (#1957) lifted the size classes into an isFullscreen
  computed; the epic added a `wide` size for the video preview. Kept
  main's computed with the epic's branch added, and restored the
  `isMobileScreen && stickToEdges` test the computed had dropped: as
  merged on main the condition reads `stickToEdges || !stickToEdges`,
  which is always true, so every CMS modal renders full-screen. That is
  a live regression on main and wants its own fix there.
Segments upload as they are packed, so the encoder's session status stays
`encoding` until only the playlists are left. The Media section showed
"Encoding" throughout, while the encoder's own window showed an upload bar
moving — the same job described two different ways.

The frame already carried what was needed: `pipelineProgress` rides on
every SSE event and on the status endpoint, and the CMS was dropping it.
Once encoding reaches 100% and the upload is still moving, the label and
the bar follow the upload instead.

While the two overlap, encoding stays the answer — it is the one still
producing work. An encoder that sends no pipeline detail reads exactly as
it did before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API, CMS: HLS media data model

4 participants