From 4bf605a62f21b749284bc7e2c4b8d3783c2e076b Mon Sep 17 00:00:00 2001 From: Nic Date: Mon, 17 Aug 2026 08:54:20 +0100 Subject: [PATCH] =?UTF-8?q?chore(release):=20v0.12.0=20=E2=80=94=20bundle?= =?UTF-8?q?=20antd=20v0.12.0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Changelog + upgrading notes for the 0.12.0 cut. The antd pins already moved on master (#157); this is the docs half: download-cache epic, download admission control, RealIP removal + resolved client IPs, already_stored fix, migrations 013/014, security bumps. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 33 ++++++++++++++++++++++++++++++++- UPGRADING.md | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fa28703..b686687 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,11 +2,42 @@ All notable changes to Indelible are documented in this file. This project adheres to [Semantic Versioning](https://semver.org/). -## [Unreleased] +## [0.12.0] - 2026-08-17 + +This release tracks **antd / ant-sdk v0.12.0** (bundled daemon, Go client module, and image). The matching version numbers are a coincidence — the two projects are versioned independently. + +### Download cache (read acceleration) + +A new per-instance, content-addressed download cache turns the read path into a three-layer degradation: `304` (ETag match) → **cache hit** (local bytes, Range preserved, no antd fetch, no gate slot) → gated miss (fetch, then promote the temp file the download already paid to materialise). **Off by default** — set `download_cache_max_bytes` to enable; see the new deployment guide at `docs/guides/download-cache.md`. + +- **Cache core** — atomic rename promotion (no partially-written file is ever visible), boot scan rebuilds the index with no DB writes (reader-fleet discipline), per-key fill coalescing so concurrent misses on one object produce a single antd fetch. +- **Disk policy sweeper** — a per-minute pass in strict precedence order: disk pressure (the cache is sacrificed, toward empty if needed, before disk usage can pause uploads), optional inactivity window (`download_cache_inactive_secs` — also the privacy dial bounding how long cached plaintext lingers), then LRU eviction to the byte budget with admission headroom. Runs on every role, readers included. +- **Write-through seeding** (`download_cache_seed_on_upload`, default on) — after a public upload completes, the staged temp file is renamed into the cache instead of deleted, so publish-then-read is served from local disk with zero extra I/O. Warms the writer's cache only; readers warm by read-through. +- **Private-content caching opt-in** (`download_cache_private`, default off) with **fleet-wide delete purge**: deletes append the upload's cache keys to a `cache_purge_log` (migration 013) consumed by every instance's sweep tick, with boot reconciliation covering deletes that happen while an instance is down. The purge-window contract: the deleting instance purges synchronously (a reported deletion never leaves pre-existing cached plaintext readable — the API fails closed instead), every other instance purges within ~one 60s sweep tick. +- **Observability** — lifecycle counters across serve/admission/coalescing/eviction/purge, emitted as a periodic `download cache stats` log line only when something changed, with a telemetry table in the guide mapping each stat to the action it should trigger. +- Settings: `download_cache_max_bytes` (master switch; `INDELIBLE_DOWNLOAD_CACHE_MAX_BYTES` overrides per instance for heterogeneous fleets), `download_cache_max_object_bytes`, `download_cache_min_uses`, `download_cache_inactive_secs`, `download_cache_seed_on_upload`, `download_cache_private` — all live-tunable, all in the admin UI. + +### Download admission control + +- **`max_concurrent_downloads`** (default 8) bounds concurrent downloads for the whole request, so download temp disk is deterministically bounded and bulk-download bursts can no longer flip the disk-pressure uploads pause or collectively starve the per-file fetch timeout. Excess requests wait up to **`download_queue_wait_secs`** (default 30) for a slot, then get `503` with `{"code":"downloads_saturated"}` and a `Retry-After` hint. The `304` conditional path and cache hits stay outside the gate. + +### Security & hardening + +- **Removed chi's `RealIP` middleware** (GO-2026-5777, Critical): it rewrote `RemoteAddr` from the forgeable `X-Forwarded-For` before the trusted-proxy logic ran, making per-IP rate limits bypassable and logged IPs spoofable by any direct client. `X-Forwarded-For` is now honored exclusively by the trusted-proxy-gated resolver (`trusted_proxies`); chi bumped to v5.3.0. +- **Audit, file-access, and settings-change logs record the resolved client IP** — `X-Forwarded-For` honored only from peers inside `trusted_proxies` — restoring real client attribution behind a reverse proxy, safely. +- Toolchain and dependency advisories cleared: Go 1.25.13 (six stdlib advisories incl. `crypto/tls`, `net/http`, `net/url`), `golang.org/x/net` v0.55.0, `google.golang.org/grpc` 1.82.1, chi 5.3.0, vite 8 (esbuild gone from the tree entirely), axios 1.18.1, nanoid, brace-expansion. ### Fixed + - **Duplicate re-uploads no longer fail at the final step.** The `already_stored` status (content-addressed dedup — re-uploading a file whose chunks are all already on the network) was written by the upload worker but missing from the database schema's status constraint, so the transition was rejected after the network store succeeded and the upload surfaced as "Failed to save upload record". Migration 014 fixes the constraint in both dialects, and such uploads are now also deletable like any other stored upload. - **Operator note:** uploads that hit this bug before the fix sit in `failed` state; migration 014 does not rewrite them. Retry them once (the re-Prepare is a zero-cost dedup) and they will complete as `already_stored`. +- **Deleting an upload purges its cached bytes** on the handling instance before the row is removed (fails closed if the unlink fails), with resurrection guards closing the promote-vs-delete race on every instance. + +### Internal + +- Bundled antd moved v0.10.0 → v0.12.0 across all three lockstep pins (`antd-go` module, `.antd-version`, Docker `ANTD_IMAGE`). +- CI: heavy jobs skip on documentation-only changes (required Test contexts still reported); release artifact download scoped to the binaries. +- Docs: runtime-settings table reconciled with the actual settings surface; new download-cache deployment guide; reader-fleet scaling guide updated for cache warm-up semantics. ## [0.11.0] - 2026-06-18 diff --git a/UPGRADING.md b/UPGRADING.md index d4db0e4..3f32ea6 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -4,6 +4,44 @@ Operator-facing notes for upgrades that need an action. Most upgrades are a plain `docker compose pull && docker compose up -d` (data volumes persist) and need nothing from this file — only the entries below call for a manual step. +## v0.12.0 — 2026-08-17 + +### Bundled antd updated to v0.12.0 + +The bundled antd daemon (and the `antd-go` client module) move to **v0.12.0**. Container +deployments get the new daemon automatically on `docker compose pull`. **If you run antd +yourself**, update your antd daemon to v0.12.0 so it stays in lockstep with the binary +Indelible was built against. + +### Reverse-proxy deployments: set `trusted_proxies` (log/rate-limit attribution change) + +The spoofable `RealIP` middleware was removed (GO-2026-5777). `X-Forwarded-For` is now +honored **only** when the connecting peer is listed in `trusted_proxies`. Behind nginx/caddy: + +- **With `trusted_proxies` configured** (as the deployment guide recommends): audit and + file-access logs now record the true client IP, and per-IP rate limiting works as designed. + No action beyond confirming the setting. +- **Without it**: logs record the proxy's address and rate limiting keys on the proxy — + add your proxy addresses to `trusted_proxies` to restore client attribution. + +Direct (no-proxy) deployments need nothing; forged `X-Forwarded-For` headers from clients +were never trusted by the rate limiter and are now ignored everywhere else too. + +### Database migrations 013 + 014 (automatic) + +Both apply on first boot as usual (no manual step): 013 adds the download-cache purge log +and `uploads.cache_key`; 014 admits the `already_stored` status. One post-upgrade action: +**uploads that previously failed with "Failed to save upload record"** (a duplicate-content +re-upload hitting the pre-014 constraint) sit in `failed` state — retry each once; the +re-Prepare is a zero-cost dedup and they will complete as `already_stored`. + +### Download cache is available but off by default + +Nothing changes unless you opt in. To enable the read cache, set `download_cache_max_bytes` +(admin UI → Transfer Limits, or the `INDELIBLE_DOWNLOAD_CACHE_MAX_BYTES` per-instance +override) and read `docs/guides/download-cache.md` for sizing, fleet scope of each dial, +and the privacy posture — especially before enabling `download_cache_private`. + ## v0.11.0 — 2026-06-18 ### Bundled antd updated to v0.10.0