Signal Forge

Signal Forge is a backend-focused Flask and Socket.IO application for isolated asynchronous media-processing jobs. A user imports a CSV track list, selects any or all imported tracks, and follows each job from admission through background processing to authorized file delivery.

The project began as personal automation and is presented publicly for its engineering architecture: session-scoped authorization, atomic resource admission, bounded concurrency, path-confined artifacts, real-time progress, cleanup/recovery semantics, and automated reliability/security tests.

Use Signal Forge only for media you are legally permitted to download and process. Source-provider terms and copyright rules remain the user's responsibility. Search-based source selection may return a different recording from the one intended. Signal Forge is not affiliated with Spotify, YouTube, Apple, or any other media platform.

Illustrated from the verified local interface with synthetic track data. No live-provider request, download, or copyrighted media is part of this visual.

Engineering highlights

• Opaque isolated jobs. Each browser session owns one random job ID. Song data, statuses, and file paths remain server-side.
• Ownership-scoped access. Job status, completed files, ZIP creation, cleanup, and Socket.IO room membership require ownership of the current job.
• CSRF-protected mutations. Mutating HTTP routes require a per-job HMAC CSRF token.
• Atomic resource admission. Task slots and conservative byte reservations are committed before background threads or output directories are exposed to new work.
• Bounded resource usage. The service applies request, CSV, selection, source-download, artifact, per-job, process-wide, ZIP, job-count, and concurrency ceilings.
• Thread-safe lifecycle management. Jobs move through explicit accepting/closing states, retain capacity while storage deletion occurs, and can be restored after cleanup failures.
• Confined artifact delivery. Filenames are generated by the server, validated against an allowlist, and constrained to the owning job directory.
• Real-time progress. Socket.IO sends job-scoped queue, download, conversion, success, and failure updates.
• Failure cleanup. Oversized files, source-limit failures, failed thread starts, and processing exceptions release reservations and remove partial outputs.
• Testable external I/O. The pytest suite uses temporary job roots and mocks media/network operations rather than downloading live content.

Architecture

flowchart LR
    B[Browser session] -->|CSV + CSRF-protected mutations| F[Flask routes]
    B <-->|job-scoped progress| S[Socket.IO]
    F --> R[Thread-safe job registry]
    R -->|atomic task + byte reservation| Q[Bounded background queue]
    Q --> C[Concurrency semaphore]
    C --> M[Media source processing]
    M --> Y[yt-dlp]
    M --> X[FFmpeg]
    M --> A[Artwork lookup]
    M --> T[Metadata tagging]
    T --> D[Confined per-job directory]
    D -->|ownership + allowlist check| F
    R -->|idle expiry / cleanup / recovery| D

Loading

The source-processing implementation currently uses yt-dlp, FFmpeg, and an HTTPS iTunes artwork lookup that is embedded into the MP3 after download, with the YouTube thumbnail as a fallback. Search quotes the artist and title. Ranking keeps results whose channel matches the artist, rejects movie clips and extra-title mismatches, and can fall back to a trusted-channel video or a well-known label upload if no official audio is available. A 403 on one result retries the next acceptable match. Those dependencies are deliberately kept behind the job/resource boundary rather than trusted with unbounded filesystem or request behavior. If automatic download fails, the failed row can offer up to three leftover YouTube hits to choose from; the pick still goes through the same job reservation and size limits. On local runs, finished MP3s are moved into Music/SignalForge instead of staying in the job directory.

Runtime model

Jobs, ownership records, progress, rate limits, reservations, and concurrency controls are held in process memory. Production must use exactly one worker. Multiple application workers would maintain independent registries and would break ownership, progress, and capacity accounting.

The committed Procfile uses one Gunicorn gthread worker with four threads:

web: gunicorn --worker-class gthread --workers 1 --threads 4 --bind 0.0.0.0:$PORT app:app

Job files live below DATA_ROOT in random per-job directories. Uploaded CSV bytes are parsed in memory and are not written to disk. Completed artifacts remain temporary and are not durable/private cloud storage.

Idle accepting jobs are opportunistically reaped after one hour by default. Queued, active, reserved, closing, and path-invalid jobs are not reaped. Storage deletion happens outside registry/job locks while the closing job continues to consume its slot and retained-byte capacity. If deletion fails, surviving allowlisted files are reconciled and the same job is restored for a later retry.

Resource admission model

The defaults are intentionally finite:

Limit	Default
Request body	2 MB
CSV bytes	1 MB
CSV rows	2,000
Consumed field length	500 characters
Selected tracks per request	Entire imported list (up to 2,000 CSV rows)
Concurrent media operations	2
Reserved/active tasks per job	8
Pending queued tracks per job	Remainder of the imported list
Source media bytes	100 MB
Final artifact bytes	100 MB
Conservative reservation per admitted task	100 MB
Retained artifacts per job	500 MB
Process-local jobs	100
Process-wide retained + reserved bytes	2 GB
Idle expiry	1 hour
ZIP input	250 MB
Artwork response	5 MB

A request that would exceed a per-job or process-wide ceiling is rejected as a whole before work is spawned when none of the selected tracks can be reserved or pending-queued. Larger selections are queued: up to eight tracks per job receive a conservative reservation immediately, and the rest wait in order until a reservation is released. After processing, the conservative reservation is reconciled against the authoritative artifact size.

Requirements

• Python 3.14.6 as declared in .python-version
• FFmpeg, either supplied by the pinned imageio-ffmpeg package or explicitly configured with FFMPEG_PATH
• Deno 2.3 or newer for YouTube downloads (yt-dlp JavaScript challenge solver). The app looks on PATH and in common install locations, including a WinGet package folder on Windows. After a winget install, restart the app; if Deno is still missing, set YTDLP_JS_RUNTIME_PATH. Node.js is an optional runtime if YTDLP_JS_RUNTIME=node is set; it is also used for the optional JavaScript syntax check in CI
• Render and other production hosts must provide that JS runtime themselves. build.sh does not install Deno

Runtime and development dependencies are directly version-pinned in requirements.txt and requirements-dev.txt.

Local setup

python -m venv .venv

Activate it:

# Windows
.venv\Scripts\activate

# macOS/Linux
source .venv/bin/activate

Install development dependencies:

python -m pip install --requirement requirements-dev.txt

Optionally copy the safe environment template:

cp .env.example .env

On Windows PowerShell:

Copy-Item .env.example .env

Start the local development server:

python app.py

The direct development server binds to 127.0.0.1 by default.

Secret configuration

Development may omit SECRET_KEY; the application will generate an ephemeral key for that process and log a warning. Sessions will reset after restart.

Persistent or production deployments require a strong secret of at least 32 characters. Generate one locally:

python -c "import secrets; print(secrets.token_urlsafe(48))"

Store it in deployment/environment configuration. Never commit it.

Configuration

Variable	Purpose	Default
SECRET_KEY	Signs the opaque job cookie and per-job CSRF tokens	Ephemeral in development; required in production
APP_ENV=production	Enables production secret validation and secure cookies	Development
DATA_ROOT	Parent directory for isolated job folders	OS temp directory + application folder
FFMPEG_PATH	Optional executable override	Executable supplied by imageio-ffmpeg
YTDLP_JS_RUNTIME	yt-dlp JavaScript runtime name (deno or node)	deno
YTDLP_JS_RUNTIME_PATH	Optional path to that runtime executable	First deno or node on PATH
PORT	Development/deployment port	5000

Additional task, storage, TTL, and capacity limits can be changed through Flask configuration when embedding or testing the app. All capacity and TTL settings must remain positive.

CSV format

Files must be UTF-8 with unique headers. Song and Artist are required. Album and Genres are optional. Other columns are ignored.

Song,Artist,Album,Genres
Midnight Drive,Nova Lines,Afterglow,"Electronic, Synthwave"
Paper Moons,The Low Signals,,Indie

Blank required fields, malformed quoting, excessive rows/bytes/field lengths, and extra values without headers are rejected before a replacement job is created.

Verification

The test suite uses temporary job roots and mocks live network/media operations.

python -m pytest -q
python -m compileall -q app.py tests tools
node --check static/app.js
python tools/publication_guard.py

GitHub Actions also runs a dependency vulnerability audit with pip-audit before the test suite. CI is intentionally part of the publication contract rather than decorative green confetti.

At the 2026-09-03 portfolio finalization baseline, the complete local suite passes 118 tests. Live provider, FFmpeg, populated-browser accessibility, and deployed one-worker behavior remain separate release checks.

The tests cover, among other things:

• production secret validation and secure cookies;
• CSV validation and upload limits;
• CSRF enforcement;
• selection validation;
• immutable background-task inputs;
• job-room isolation and cross-session access denial;
• file allowlisting and path confinement;
• POST-only cleanup and cleanup recovery;
• ZIP size limits;
• FFmpeg resolution;
• rate limiting;
• admission/cleanup race behavior;
• per-job and process-wide reservation accounting;
• failure cleanup and reservation release.

Security and privacy boundary

• The signed cookie contains only the current opaque job ID. Track data and filesystem paths do not enter the cookie.
• Every mutating HTTP route requires a per-job CSRF token.
• File, ZIP, Socket.IO room, and cleanup access require ownership of the current job.
• Filenames are deterministic server-generated names and are validated to remain directly below the owning job directory.
• Cleanup closes admission before deleting an idle job. Active/reserved work blocks cleanup.
• Cover-art responses are checked for HTTPS, status, content type, and byte size.
• Job counts, byte budgets, activity clocks, expiry, and rate limits are process-local. The one-worker deployment constraint is therefore mandatory.
• This is not an account system, durable private store, or horizontally scalable multi-worker service.

See SECURITY.md for the publication/security policy.

Known limitations

• Job and progress state is lost on process restart.
• Horizontal scaling requires replacing the process-local registry, rate-limit storage, Socket.IO coordination, and capacity accounting with shared infrastructure.
• Temporary media artifacts depend on host storage and may disappear independently on ephemeral platforms.
• Search-based source matching is heuristic and may select the wrong recording.
• External media/artwork availability and provider terms can change independently of this repository.
• Browser-session possession is the ownership boundary; there are no user accounts or durable multi-tenant identities.
• The project is a personal engineering application, not a commercial service or an official integration with any media platform.

Production command

The documented deployment shape matches the committed Procfile:

gunicorn --worker-class gthread --workers 1 --threads 4 --bind 0.0.0.0:${PORT:-5000} app:app

Build environments need only install the pinned runtime requirements:

python -m pip install --requirement requirements.txt

Publication notes

See PUBLICATION.md for supported portfolio claims, the history-preserving publication decision, and remaining release checks. SignalForge remains the canonical public repository with its valid development history. A rotated historical credential exists only in an already-unreferenced GitHub object; its cached-view removal is a GitHub Support action and is not represented as complete here.

License

Signal Forge is licensed under the Apache License 2.0.