Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 11 additions & 17 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -252,25 +252,19 @@ jobs:
node-version: 24
cache: npm

# build-ytdlp-bin.mjs freezes yt-dlp into a standalone binary via a
# local venv + PyInstaller -- needs a real Python on PATH on every
# platform, not just whatever the runner image happens to ship.
- uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
cache-dependency-path: src/python/requirements-build.txt

- run: npm ci

# Build only -- no network/GitHub calls happen during this step, it's
# purely local packaging. Deliberately NOT electron-builder's own
# `--publish` flow: its GitHub publisher has its own internal retry
# behavior around creating/uploading, and a retry there (e.g. after a
# transient network hiccup) has been the suspected cause of a single
# run ending up with duplicate release state before. Publishing is
# done explicitly below instead, as plain `gh` calls this workflow
# fully controls.
# Build only -- this step no longer publishes anything itself. It does
# make one read-only network call now (build:ytdlp:bin fetches and
# GPG-verifies yt-dlp's own signed release asset from GitHub, see
# scripts/fetch-ytdlp-bin.mjs), but that's an idempotent GET against an
# already-published release, unrelated to the actual risk this
# separation guards against: electron-builder's own `--publish` flow
# has internal retry behavior around *creating/uploading* a release,
# and a retry there (e.g. after a transient network hiccup) has been
# the suspected cause of a single run ending up with duplicate release
# state before. Publishing is done explicitly below instead, as plain
# `gh` calls this workflow fully controls.
- name: Build
run: npm run dist

Expand Down
20 changes: 5 additions & 15 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,6 @@ actually contains.
## Prerequisites

- **Node.js 24** (matches what CI builds with — see `.github/workflows/build.yml`)
- **Python 3.11**, with `venv` available — used to freeze `yt-dlp` into a
standalone binary as part of the build (see [What the build scripts
do](#what-the-build-scripts-do) below)
- `npm`

## Setup
Expand All @@ -36,10 +33,7 @@ npm run dev

`build:all` needs to be re-run whenever you change renderer source
(`src/ui/`) **or** main-process source (`src/electron/`) — there's no hot
reload here, `dev` just loads whatever was most recently built. `build:all`
is fast for a code-only change; it does *not* re-freeze `yt-dlp` or
re-copy `ffmpeg` unless you explicitly ask it to (see below) — those are
the slow, rarely-needed steps.
reload here, `dev` just loads whatever was most recently built.

> `npm run dev:vite` also exists, but it's not a way to preview the real
> app — it's an isolated Vite dev server for hot-swapping visual/theme/design
Expand Down Expand Up @@ -80,16 +74,12 @@ The full dependency chain, from `package.json`'s `scripts`:
| `dev:vite` | Isolated visual/theme/design work only — not connected to the real app (see above). |
| `build:renderer` | Compiles the React UI with Vite into `dist/renderer/`. |
| `build:copy:electron` | Copies `src/electron/` (plain JS, no compile step) into `dist/electron/`. |
| `build:ytdlp:bin` | The slow one: creates a local Python venv, installs the exact `yt-dlp` version pinned in `src/python/requirements-build.txt`, and freezes `src/python/ytdlp_entrypoint.py` into a standalone native executable via PyInstaller. This is a real build from source, not a downloaded prebuilt binary — it's also what the app's own in-app "update yt-dlp" feature re-runs later on a user's machine. |
| `build:ytdlp:bin` | Downloads yt-dlp's own official prebuilt release binary and GPG-verifies it against yt-dlp's published signature and checksums before unpacking it into `dist/ytdlp-bin/` — not a build from source. This is also what the app's own in-app "update yt-dlp" feature re-runs later on a user's machine. |
| `build:copy:ffmpeg` | Copies the `ffmpeg`/`ffprobe` binaries already fetched by the `ffmpeg-static`/`ffprobe-static` npm packages into `dist/ffmpeg/`. |
| `clean:venv` | Deletes the local Python venv `build:ytdlp:bin` creates, so the next build starts from a clean environment instead of a possibly-stale one. |
| `build:electron` | Runs `build:copy:electron` → `build:ytdlp:bin` → `build:copy:ffmpeg`, in order. |
| `build:all` | `build:renderer` + `build:electron` — everything needed to run the app locally via `dev`. |
| `build:all:deep` | `clean:venv` first, then `build:all` — a genuinely from-scratch build. This is what CI/`dist` uses; you shouldn't normally need it locally unless something in the venv is stuck. |
| `build:all` | `build:renderer` + `build:electron` — everything needed to run the app locally via `dev`, and also what CI/`dist` uses. |
| `electron-build` | Runs `electron-builder` (packages `dist/` into a real installer for your current OS) without publishing anywhere. |
| `dist` | `build:all:deep` + `electron-build` — produces an actual installer in `dist/`, the same thing a release build does. See `docs/RELEASING.md` for how that connects to an actual GitHub release. |
| `dist` | `build:all` + `electron-build` — produces an actual installer in `dist/`, the same thing a release build does. See `docs/RELEASING.md` for how that connects to an actual GitHub release. |
| `prepare` | Runs `husky` to install the pre-commit hook. `npm` runs this automatically after `npm install`/`npm ci` — you shouldn't need to run it yourself. |

If you only need to run the app locally, `build:all` is what you want — the
heavier `build:all:deep`/`dist` scripts exist for producing a real
installer, not for day-to-day development.
If you only need to run the app locally, `build:all` is what you want.
7 changes: 4 additions & 3 deletions THIRD-PARTY-NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ redistributed here.
unrestricted; no attribution is legally required, and this notice is
included as a courtesy.
- **Project:** <https://github.com/yt-dlp/yt-dlp>
- **How it's bundled:** built from source at release time (not the official
prebuilt binary) via PyInstaller, from the exact version pinned in
`src/python/requirements-build.txt`.
- **How it's bundled:** yt-dlp's own official prebuilt release binary,
fetched and GPG-verified against yt-dlp's published signature at build
time (see `scripts/fetch-ytdlp-bin.mjs`), from the exact release tag
pinned there.

## FFmpeg / FFprobe

Expand Down
57 changes: 32 additions & 25 deletions docs/PROJECT_STRUCTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ see [ARCHITECTURE.md](ARCHITECTURE.md).
|---|---|
| `src/ui/` | The renderer — the React interface. |
| `src/electron/` | The Electron main process — all privileged logic (filesystem, subprocesses, IPC). |
| `src/python/` | The `yt-dlp` entrypoint script and pinned Python requirements used to build the bundled `yt-dlp` binary. |
| `src/types/` | Shared TypeScript type declarations for the renderer, notably the full type contract for the preload bridge. |
| `src/utils/` | Small renderer-side helper modules with no Electron/Node dependency (URL parsing, formatting, a debounce hook). |
| `scripts/` | Node scripts that assemble the pieces needed for a working build/install (see [below](#bundled-low-level-dependencies-and-the-build-pipeline)). |
Expand Down Expand Up @@ -51,7 +50,7 @@ subprocess access at all — its only way to do anything real is through

## The Electron main process (`src/electron`)

This is where essentially all of the app's actual logic lives. Five files:
This is where essentially all of the app's actual logic lives. Six files:

### `main.mjs`

Expand All @@ -61,7 +60,7 @@ external tools (`yt-dlp`, `ffmpeg`, `ffprobe`). It's easiest to think of it as
several loosely-grouped areas rather than one thing:

- **Startup/runtime plumbing** — resolving where the bundled `yt-dlp`/`ffmpeg`/
`ffprobe`/`deno` binaries actually live (this differs between a local dev run
`ffprobe` binaries actually live (this differs between a local dev run
and a packaged install — see [below](#bundled-low-level-dependencies-and-the-build-pipeline)),
starting a small local HTTP server the browser window loads its UI from instead
of a `file://` load, and registering the custom `app-video://` protocol used to
Expand Down Expand Up @@ -117,13 +116,23 @@ pure filesystem logic, called *by* `main.mjs`'s handlers.
### `updater.mjs`

The self-updater for the bundled `yt-dlp` binary, kept separate from `main.mjs`
because it's a genuinely distinct pipeline: checking the latest available
`yt-dlp` version, fetching a portable Python runtime if one isn't already staged,
installing PyInstaller into it, re-freezing `yt-dlp` from source with that
runtime, sanity-checking the result actually runs, and atomically swapping it
into place. This is effectively a smaller, on-device rerun of the same freeze
process `scripts/build-ytdlp-bin.mjs` does at build time (see below) — the two
are intentionally kept in sync.
because it's a genuinely distinct pipeline: resolving the latest release,
fetching and verifying it (via `ytdlpRelease.mjs`, below), sanity-checking the
result actually runs, and atomically swapping it into place. This is
effectively a smaller, on-device rerun of the same fetch process
`scripts/fetch-ytdlp-bin.mjs` does at build time (see below) — the two share
`ytdlpRelease.mjs` directly rather than being kept manually in sync.

### `ytdlpRelease.mjs`

The actual fetch/verify logic `updater.mjs` and `scripts/fetch-ytdlp-bin.mjs`
both call into: resolving a release from GitHub, downloading the right
platform asset, checking its SHA-256 against yt-dlp's published
`SHA2-256SUMS`, verifying that file's own GPG signature against a vendored
copy of yt-dlp's public key, and unzipping the result. Deliberately zero third-party dependencies — the packaged app ships with no
`node_modules` at all — so this hand-rolls just enough of a ZIP reader and an
OpenPGP-signature parser to do the job, delegating all actual cryptography to
Node's built-in `crypto` module.

### `utils/constants.mjs`

Expand Down Expand Up @@ -176,26 +185,24 @@ A few things worth internalizing from this:

## Bundled low-level dependencies and the build pipeline

The app depends on three external tools it never assumes the user already has
installed: `yt-dlp`, `ffmpeg`/`ffprobe`, and `deno` (a small JS runtime `yt-dlp`
needs to solve certain sites' anti-bot challenges). All three are bundled inside
the packaged app rather than downloaded at runtime, assembled by scripts run as
part of the build:
The app depends on two external tools it never assumes the user already has
installed: `yt-dlp` and `ffmpeg`/`ffprobe`. The JS runtime `yt-dlp` needs to
solve certain sites' anti-bot challenges is Electron's own bundled Node
runtime (`--js-runtimes node:<process.execPath>`, see `main.mjs`'s
`jsRuntimeArgs`) rather than a separately bundled binary. Both remaining
tools are bundled inside the packaged app rather than downloaded at runtime,
assembled by scripts run as part of the build:

| Script | What it produces |
|---|---|
| `scripts/build-ytdlp-bin.mjs` | Builds `yt-dlp` itself, from source — creates a local Python virtual environment, installs the exact `yt-dlp` version pinned in `src/python/requirements-build.txt` (plus its own JS-challenge-solver package and PyInstaller), and freezes `src/python/ytdlp_entrypoint.py` into a standalone native executable. This is a real local build step, not a download of a prebuilt binary — it's what lets the app avoid depending on a system Python install, and it's the same freeze process `updater.mjs` reruns later for in-app self-updates. |
| `scripts/fetch-ytdlp-bin.mjs` | Downloads yt-dlp's own official prebuilt release binary and GPG-verifies it (via `ytdlpRelease.mjs`, see above) before unpacking it into `dist/ytdlp-bin/` — not a build from source. This is the same fetch/verify path `updater.mjs` reruns later for in-app self-updates. |
| `scripts/copy-ffmpeg.mjs` | Copies the `ffmpeg`/`ffprobe` binaries already fetched by their own npm packages into the build output, preserving the right executable name/extension per platform. |
| `scripts/copy-deno.mjs` | Same idea, for the `deno` binary. |
| `scripts/copy-electron.mjs` | Copies the main-process source (`src/electron/`) into the build output as-is — it's plain JS, so this is a copy, not a compile. |
| `scripts/clean-venv.mjs` | Deletes the local Python virtual environment `build-ytdlp-bin.mjs` creates, so a full rebuild starts from a clean environment rather than reusing a possibly stale or (on Windows) account-locked one. |

These are chained together by `package.json`'s own `build:*` npm scripts (renderer
build, then each of the copy/build steps above, in order), and the final packaging
step (`electron-builder`) bundles the resulting `dist/ytdlp-bin/`, `dist/ffmpeg/`,
and `dist/deno/` folders into the installed app as extra, read-only resources —
alongside a copy of the Python entrypoint script and requirements file (used
later, entirely on the user's own machine, if they ever trigger a self-update).
build, then each of the fetch/copy steps above, in order), and the final packaging
step (`electron-builder`) bundles the resulting `dist/ytdlp-bin/` and `dist/ffmpeg/`
folders into the installed app as extra, read-only resources.

At runtime, `main.mjs` resolves the real path to each of these differently
depending on whether it's running from a local dev checkout or an installed,
Expand All @@ -215,6 +222,6 @@ elevated permissions.
| Adding a brand-new capability the UI needs from the main process | `main.mjs` (add the handler) → `preload.cjs` (expose it) → `src/types/electron-api.d.ts` (type it) |
| Changing a screen or adding a new one | `src/ui/screens/` |
| Changing shared, stateful UI logic (the bulk queue, download progress, search) | `src/ui/hooks/` |
| Touching the self-update flow | `updater.mjs` |
| Changing how `yt-dlp`/`ffmpeg`/`deno` get bundled at build time | the relevant `scripts/*.mjs` file |
| Touching the self-update flow | `updater.mjs`, or `ytdlpRelease.mjs` for the fetch/verify logic itself |
| Changing how `yt-dlp`/`ffmpeg` get bundled at build time | the relevant `scripts/*.mjs` file |
| Adjusting persisted settings (a new Options-tab toggle, etc.) | the settings handlers in `main.mjs`, plus wherever it's read in `src/ui/screens/OptionsScreen.tsx` |
21 changes: 12 additions & 9 deletions docs/RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,8 +165,8 @@ Not currently supported as a separate path — the workflow either releases
push and release creation both fail with a permissions error.
- **Settings → Actions → General → Actions permissions** should allow the
standard actions this workflow uses (`actions/checkout`,
`actions/setup-node`, `actions/setup-python`) to run — the default "Allow
all actions and reusable workflows" is fine.
`actions/setup-node`) to run — the default "Allow all actions and reusable
workflows" is fine.
- No code-signing secrets exist or are required today — every installer ships
unsigned (`mac.identity: null`, no Windows certificate configured). Windows
shows a SmartScreen "Unknown Publisher" warning and macOS shows a Gatekeeper
Expand All @@ -180,20 +180,23 @@ Not currently supported as a separate path — the workflow either releases
a full build on every merge (without publishing) specifically to catch
build breakage immediately. That was deliberately removed to guarantee a
no-version-bump merge costs nothing — the tradeoff is that a merge which
breaks the *packaging* build (electron-builder, the PyInstaller freeze,
etc.) silently doesn't get caught until the next real version bump tries
to release. `.github/workflows/ci.yml` covers a different, cheaper gap —
breaks the *packaging* build (electron-builder, fetching/verifying the
`yt-dlp` release binary, etc.) silently doesn't get caught until the next
real version bump tries to release. `.github/workflows/ci.yml` covers a
different, cheaper gap —
lint + the Vitest suite on every pull request — which catches broken code
well before merge, but it never runs `npm run dist`, so a packaging-only
failure can still slip through.
- **Portable Windows build is slow to start** — NSIS's portable target
re-extracts the entire app on every launch, not just once at install time.
Not a priority today.
- **Apple Silicon vs. Intel mac builds are two separate native jobs**, not
one universal2 binary — a true universal binary needs more work than this
(the PyInstaller-frozen `yt-dlp` binary can't cross-compile between
architectures, so producing one means running that build twice and merging
the results with `lipo`).
one universal2 binary. `yt-dlp`'s own official macOS release is already a
universal2 binary, so that's no longer what's blocking this — the
remaining work is auditing `ffmpeg-static`/`ffprobe-static`'s per-arch
bundling and switching electron-builder over to its own universal-build
mode. Tracked as a must-fix item before this project stops using separate
mac jobs.
- **Linux support is new** — the `AppImage` target and the `ubuntu-latest`
build job were both added the same day as this document; treat it as
less battle-tested than the Windows/macOS paths until it's been through a
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 3 additions & 13 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "sloth-archiver",
"private": true,
"version": "0.25.6",
"version": "0.26.0",
"description": "A desktop app for downloading and archiving videos, with a built-in library and playlist tracking.",
"author": "Sloth <sloth.software.dev@protonmail.com>",
"license": "GPL-3.0-or-later",
Expand Down Expand Up @@ -29,14 +29,6 @@
"filter": [
"**/*"
]
},
{
"from": "src/python/",
"to": "python-src",
"filter": [
"ytdlp_entrypoint.py",
"requirements-build.txt"
]
}
],
"directories": {
Expand Down Expand Up @@ -67,15 +59,13 @@
"lint": "eslint .",
"preview": "vite preview",
"electron-build": "electron-builder --publish never",
"dist": "npm run build:all:deep && npm run electron-build",
"dist": "npm run build:all && npm run electron-build",
"build:renderer": "vite build",
"build:copy:electron": "node scripts/copy-electron.mjs",
"build:ytdlp:bin": "node scripts/build-ytdlp-bin.mjs",
"build:ytdlp:bin": "node scripts/fetch-ytdlp-bin.mjs",
"build:copy:ffmpeg": "node scripts/copy-ffmpeg.mjs",
"build:electron": "npm run build:copy:electron && npm run build:ytdlp:bin && npm run build:copy:ffmpeg",
"build:all": "npm run build:renderer && npm run build:electron",
"clean:venv": "node scripts/clean-venv.mjs",
"build:all:deep": "npm run clean:venv && npm run build:all",
"test": "vitest run",
"test:watch": "vitest",
"test:coverage": "vitest run --coverage",
Expand Down
Loading
Loading