[PERF] Docker CI jobs rebuild every dependency on any lockfile change

[Verdenroz]

Summary

The two Docker jobs in ci.yml cost 15-21 minutes whenever Cargo.lock or a
manifest changes, and about a minute otherwise. They are the slowest thing on
a PR when they are slow, and the trigger that makes them slow — a lockfile
change — is exactly what a dependency bump or a cargo update produces.

Two causes, one structural and one a build setting.

Measurement

Job durations across the last 8 ci.yml runs:

job	typical	worst
Docker Build	0-1m	21m
MCP Docker Build	0-1m	15m

The fast runs are full GHA layer-cache hits. The slow runs are the ones where
the COPY Cargo.toml Cargo.lock* layer (server/Dockerfile:34) invalidated.

Expected vs actual

Expected: a lockfile change recompiles the crates that actually moved.
Actual: it recompiles every dependency from scratch, twice — once per image.

Cause 1 — the dependency cache never survives a run

Both Dockerfiles build under BuildKit cache mounts
(server/Dockerfile:56-58, finance-query-mcp/Dockerfile:56-58):

RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
    --mount=type=cache,target=/build/target,sharing=locked \
    cargo build --profile ${CARGO_PROFILE} ...

cache-to: type=gha,mode=max (ci.yml:437-438, :490-491) exports layers.
It does not export cache-mount contents. So once the layer above it
invalidates, the mount is empty and the full dependency build runs.

There is a second-order effect worth noting before anyone "fixes" this by
leaning harder on the layer cache: when the dummy-source dependency layer
(server/Dockerfile:39-59) is served from cache, its writes into the mount
never happen either, so the real build at :70 starts with an empty target
directory and compiles everything regardless. That layer only pays off inside
a single cold build.

Cause 2 — CI builds with fat LTO

Both jobs pass only build-args: CARGO_FEATURES=translation (ci.yml:436,
:489), leaving ARG CARGO_PROFILE=release. That resolves to the root
[profile.release]: lto = true, codegen-units = 1, opt-level = 3.

These jobs exist to prove the image builds, smoke-test /v2/health, and run
Trivy. None of that needs a fully optimized binary, and Build Release (binaries) already validates the real release build separately.

Ideas

Roughly in order of payoff per unit of risk.

1. Pass a cheaper profile to the CI image builds. A release-ci profile
   inheriting release with lto = false, codegen-units = 16, passed as a
   second build-arg. cargo auditable still embeds the dependency manifest
   on any profile, so Trivy's binary scan is unaffected. Two lines.
2. Persist the cache mount, via reproducible-containers/buildkit-cache-dance,
   or restructure onto cargo-chef so dependencies land in a real layer the
   GHA cache can export. cargo-chef also removes the dummy-source scaffolding
   at :39-50.
3. Build the workspace once. The two images compile the same dependency
   graph independently under separate cache scopes (docker-server,
   docker-mcp). A shared builder stage emitting both binaries roughly halves
   the cold cost.
4. Reconsider mode=max. It stores every intermediate layer across two
   scopes, against a 10 GB per-repo cache shared with the Rust caches.
   Evictions are what turn a warm build back into a 20-minute one; mode=min
   lowers that pressure.
5. Narrow the trigger. server_docker / mcp_docker fire on any src/**
   change, so nearly every PR builds images. Restricting PR-time builds to
   Dockerfile / Cargo.lock / workflow changes and doing the full build on
   master would save the most — at the cost of catching an image-only
   breakage after merge rather than before. Build Check (server) and
   Build Check (mcp) already compile the crates, so the uncovered surface is
   the Dockerfile itself.

Items 1 and 4 are independent of the rest and could land on their own.