Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,28 @@
# Environment name: local | staging | production
APP_ENV=local

# ── Running without a local AI runtime (#962) ────────────────────────────────
# The local LLM runtime (Ollama) is ON by default and nothing here changes that:
# `docker compose up -d` from a fresh clone starts it, with or without this file.
#
# To run HOSTED-ONLY — hosted chat and hosted embeddings, no Ollama at all — use
# the opt-out overlay, which removes the container *and* blanks OLLAMA_URL (the
# blank is what tells the core there is no local runtime rather than leaving it
# probing a host that is gone):
#
# task hosted-only-up
# # == docker compose -f compose.yaml -f infra/ollama/compose.hosted-only.yaml up -d
#
# Then set both model defaults to hosted ids below — a bare name routes to the
# local runtime, so leaving the shipped defaults gives working chat and a failing
# embed on every memory recall and module index — and add each provider's API key
# on the Models page. With no local runtime the core refuses pull / delete /
# unload / the KV-cache setting with 409 and a reason, the Models page collapses
# its local half, and chat readiness reports the model as n/a rather than
# "warming" forever.
# LLM_DEFAULT_MODEL=claude/claude-sonnet-4-6
# MEMORY_EMBED_MODEL=gpt/text-embedding-3-small

# ── Image tag pinning (#56) ──────────────────────────────────────────────────
# All service compose fragments default to `${EPICURUS_VERSION:-latest}`.
# Omitting this var (the default) pulls :latest — convenient for local dev
Expand Down
71 changes: 71 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,42 @@ jobs:
- name: Validate the assembled compose stack (observability profile)
run: docker compose -f compose.yaml --profile observability config -q

# The documented install — `git clone` && `docker compose up -d`, no `.env`, no flags
# (README.md, docs/user/installation.md) — must keep starting the local AI runtime.
# #962 made "no local runtime" a supported mode, and the first cut expressed it as a
# compose profile on `ollama`: profiles are opt-in, so that silently removed Ollama
# from every fresh clone while the shipped `llama3.2` / `nomic-embed-text` defaults
# still pointed at it. This step is the guard against that being reintroduced.
- name: Validate the assembled compose stack (default install starts the local runtime)
run: |
rendered="$(docker compose -f compose.yaml config)"
for svc in ollama ollama-init; do
if ! printf '%s' "$rendered" | grep -qE "^ ${svc}:"; then
echo "the default stack no longer starts '${svc}' — see #962"; exit 1
fi
done
if ! printf '%s' "$rendered" | grep -q 'OLLAMA_URL: http://ollama:11434'; then
echo "the default stack does not point the core at the local runtime"; exit 1
fi
echo "the documented install still brings up the local AI runtime"

# And the opt-out overlay (#962, ADR-0144): no Ollama, blank OLLAMA_URL. The assertion
# is not just that it parses — it is that core-app's dependency on the now-absent
# ollama service is `required: false`, which is what lets the whole `up` succeed on a
# service that is missing on purpose.
- name: Validate the assembled compose stack (hosted-only overlay — no local runtime)
run: |
set -- -f compose.yaml -f infra/ollama/compose.hosted-only.yaml
docker compose "$@" config -q
rendered="$(docker compose "$@" config)"
if ! printf '%s' "$rendered" | grep -q 'OLLAMA_URL: ""'; then
echo "the overlay did not blank OLLAMA_URL"; exit 1
fi
if printf '%s' "$rendered" | grep -qE '^ ollama:'; then
echo "the overlay did not remove the ollama service"; exit 1
fi
echo "the hosted-only overlay resolves with no local runtime"

# The Helm chart gets the same treatment the compose stack does: rendered, then
# checked. `helm lint` catches template errors, `helm template` proves the branches
# render at all, and `kubeconform -strict` validates the result against the real
Expand Down Expand Up @@ -234,6 +270,41 @@ jobs:
--set searxng.enabled=false,searxng.external.url=http://searx.example.com:8080 \
| kubeconform -strict -summary -kubernetes-version "$KUBE_VERSION" -

# No local LLM runtime at all (#962, ADR-0144) — the third Ollama state, which the
# chart refused to render until now. Hosted model ids are mandatory here, so this
# renders the escape hatch the guard's message names.
- name: Render and validate — hosted-only (no local runtime)
run: |
rendered="$(helm template epicurus "$CHART" \
--set ollama.enabled=false \
--set core.llm.defaultModel=claude/claude-sonnet-4-6 \
--set core.memoryEmbedModel=gpt/text-embedding-3-small)"
printf '%s' "$rendered" | kubeconform -strict -summary -kubernetes-version "$KUBE_VERSION" -
if ! printf '%s' "$rendered" | grep -A1 'name: OLLAMA_URL' | grep -q 'value: ""'; then
echo "OLLAMA_URL is not empty on a hosted-only release"; exit 1
fi
if ! printf '%s' "$rendered" | grep -A1 'name: LLM_BOOTSTRAP_MODELS' | grep -q 'value: ""'; then
echo "LLM_BOOTSTRAP_MODELS was not blanked on a hosted-only release"; exit 1
fi
if printf '%s' "$rendered" | grep -q 'app.kubernetes.io/component: ollama'; then
echo "an Ollama workload was rendered for a hosted-only release"; exit 1
fi
echo "hosted-only release renders with no local runtime"

# And the guard that makes the mode safe: a hosted-only release whose models are
# still bare local names must FAIL to render, naming the fix. A guard nothing proves
# is a guard that quietly stops guarding.
- name: Render — hosted-only with a local model name must be refused
run: |
if helm template epicurus "$CHART" --set ollama.enabled=false >/dev/null 2>err.txt; then
echo "the chart rendered a hosted-only release with local model defaults"; exit 1
fi
grep -q 'no local LLM runtime' err.txt \
|| { echo "the refusal did not name the mode:"; cat err.txt; exit 1; }
grep -q 'core.memoryEmbedModel' err.txt \
|| { echo "the refusal did not name the fix:"; cat err.txt; exit 1; }
echo "the guard refuses a half-working hosted-only release"

# Validates Prometheus and Alertmanager configs without booting the
# observability stack. The runtime-smoke gate skips that stack, so a bad rule
# (e.g. an undefined template function) would otherwise reach production
Expand Down
31 changes: 31 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,37 @@ images to GHCR.

## [Unreleased]

- **A hosted-only deployment runs no local runtime** (#962) — running with hosted chat and
hosted embeddings and *no* Ollama was a documented capability that nothing actually supported.
The Helm chart refused to render it (`ollama.enabled: false` demanded an external URL), Compose
could not express it (an unconditional fragment and a hard `depends_on`), and an operator who
forced it with a placeholder URL met a core that collapsed three different facts into one:
**absent** (no runtime configured — a deliberate choice), **unreachable** (one is configured
and does not answer) and **ok**. `GET /platform/v1/llm/models` **500ed**, on a page that polls
it every ten seconds; pull and delete 500ed the same way; the chat warm-up indicator reported
the model "warming" forever, on every turn, for the life of the deployment; and startup spent
180 seconds polling an address that was never going to answer. Absence is now a first-class
mode, spelled `OLLAMA_URL=""`: the model list answers **200 and an empty array** in both
non-serving states, the new `GET /platform/v1/llm/local-runtime` says which one applies, the
local-only actions (pull, delete, unload, the KV-cache setting) refuse with **409** and a
sentence naming the mode — and pull and delete answer **502** when a configured runtime is
unreachable, never a bare 500 — readiness reports the model as **n/a** instead of warming,
and the bootstrap logs one line
and returns. A local model id asked to serve is refused before any provider call with the
capability error the rest of the gateway already speaks (ADR-0140), so a hosted embedding model
keeps working while a bare one fails with the fix in the message instead of a connection error.
On **Kubernetes** the chart renders an empty `OLLAMA_URL`, blanks `LLM_BOOTSTRAP_MODELS` on its
own, and gains the guard actually worth having — it refuses to render a runtime-less release
whose chat or embedding default is still a bare local name, naming the hosted alias to set. On
**Compose** it is an opt-out overlay — `task hosted-only-up` — which removes the Ollama
services *and* blanks `OLLAMA_URL` in one step, the same idiom as the Docker-socket and
external-mount opt-ins; the documented install (`git clone`, `docker compose up -d`, no
`.env`) is byte-identical and still starts the local runtime, which a compose profile on
those services would have quietly stopped doing. Three gates cover it: `compose-validate`
proves both the default stack and the overlay resolve, `chart-validate` renders the
hosted-only release *and* proves the guard refuses the half-working one, and `k8s-smoke`
upgrades a live release into the mode and asserts the core answers. `core-app`
0.128.0→0.129.0 (MINOR), chart 0.1.2→0.2.0 (MINOR).
- **The bound on a turn is the operator's; runaway is caught by behaviour** (#925) — the
**Agent cycles** setting stopped at 12, and the route enforced it *silently*: type 40 and 12 was
stored. A genuinely long task — search → read → read → summarize → write — ran out of rounds and
Expand Down
12 changes: 11 additions & 1 deletion Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,10 +62,20 @@ tasks:
- uv run python scripts/migrate.py check {{.CLI_ARGS}}

up:
desc: Start the full stack (data plane + enabled modules; Docker control is on by default through docker-proxy-core, observability is opt-in — see `obs-up` / `docker-socket-up`)
desc: Start the full stack (data plane + enabled modules; Docker control is on by default through docker-proxy-core, observability is opt-in — see `obs-up` / `docker-socket-up`, and `hosted-only-up` to run without the local AI runtime)
cmds:
- docker compose up -d

hosted-only-up:
desc: 'Start the stack with NO local LLM runtime (#962) — no Ollama container, and OLLAMA_URL blank so the core says so rather than probing a host that is not there. Set LLM_DEFAULT_MODEL + MEMORY_EMBED_MODEL to hosted ids in .env first, or chat and every embedding fail at call time'
cmds:
- docker compose -f compose.yaml -f infra/ollama/compose.hosted-only.yaml up -d

hosted-only-down:
desc: Stop the stack including the hosted-only override (append `-- -v` to drop volumes)
cmds:
- docker compose -f compose.yaml -f infra/ollama/compose.hosted-only.yaml down {{.CLI_ARGS}}

down:
desc: Stop the full stack (append `-- -v` to also drop volumes)
cmds:
Expand Down
11 changes: 10 additions & 1 deletion compose.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Top-level stack (ADR-0006): the data plane plus enabled modules, assembled from
# fragments. `docker compose up -d` (or `task up`) brings up everything.
# fragments. `docker compose up -d` (or `task up`) brings up everything — the local AI
# runtime included, with no `.env`, no profile and no flags. That is the install the README
# and docs/user/installation.md publish, and nothing added here may quietly ask more of it:
# anything optional is layered *over* this file as an overlay the operator opts into.
#
# services/core-app/compose.docker-socket.yaml the raw Docker socket (#708)
# services/core-app/compose.external-mounts.yaml extra Files roots from the host (#731)
# infra/ollama/compose.hosted-only.yaml no local AI runtime at all (#962)
#
# (`--profile observability` is the one profile the stack uses, and it *adds* services.)
#
# Each module contributes its own services/<name>/compose.yaml fragment; add a
# module by including its fragment here (the installer does this dynamically
Expand Down
46 changes: 46 additions & 0 deletions docs/infrastructure/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,52 @@ hit `PermissionError` and the choice would save but never apply (#392). `ollama-
only**: the core's write is lazy — it happens when the operator changes the KV-cache type, long
after boot — so there is no startup race regardless.

### Hosted-only: no local LLM runtime

A deployment can run **no local runtime at all** — hosted chat and hosted embeddings, nothing
local (#962, ADR-0144). It is an **opt-out overlay**,
[`infra/ollama/compose.hosted-only.yaml`](../../infra/ollama/compose.hosted-only.yaml), the same
idiom as the Docker-socket and external-mount opt-ins:

```bash
task hosted-only-up
# == docker compose -f compose.yaml -f infra/ollama/compose.hosted-only.yaml up -d
```

The overlay does **both** halves, because either alone is wrong: it removes `ollama` and
`ollama-init` from the stack (which `core-app`'s `required: false` dependency on `ollama` is what
makes survivable), *and* it blanks `OLLAMA_URL`, which is how the core is told the absence is
deliberate. Remove only the container and the core is left probing a host that is gone — that is
the `unreachable` state, not `absent`.

**Nothing about the default install changes.** `docker compose up -d` from a fresh clone, with no
`.env` and no flags, still starts the local runtime exactly as before — which is why this is an
overlay and not a compose profile on the `ollama` services. A profile is opt-in by construction,
so it would have taken the runtime out of every fresh clone while the shipped `llama3.2` /
`nomic-embed-text` defaults still pointed at it, which is the half-working stack the Kubernetes
chart now refuses to render.

Set both model defaults to hosted ids in `.env` as well — a bare name routes to the local
runtime, so leaving them is that same half-working stack:

```dotenv
LLM_DEFAULT_MODEL=claude/claude-sonnet-4-6
MEMORY_EMBED_MODEL=gpt/text-embedding-3-small
```

Add each provider's API key on the **Models** page before the first turn. With no local runtime
the core refuses every local-only action with a reason — `409` from pull, delete, unload and the
KV-cache setting — `GET /platform/v1/llm/models` answers `200` with an empty list rather than
500ing, `GET /platform/v1/llm/local-runtime` reports `absent`, and chat readiness reports the
model as `n/a` instead of "warming" forever. A local model id asked to answer is refused with a
sentence naming the fix, before any provider call. The Kubernetes equivalent is
[`ollama.enabled: false` with a blank `ollama.external.url`](kubernetes.md#hosted-only-no-local-llm-runtime).

On the deploy box, `EPICURUS_HOSTED_ONLY=1` makes `infra/cd/reconcile.sh` apply the same overlay;
unset (the default) it keeps the local runtime, so a box that never asked for a hosted-only stack
cannot lose Ollama to a reconcile. To go back, drop the overlay (`task up`): the `ollama-models`
volume is untouched by any of this, so the models are still there.

## Log retention

Every service in every compose fragment sets a bounded `json-file` logging driver
Expand Down
51 changes: 47 additions & 4 deletions docs/infrastructure/kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -398,7 +398,7 @@ cluster points at managed services.
| `ollama.resources` | `requests: 500m / 4Gi` (add `limits` — memory especially) |
| `ollama.env` | `OLLAMA_KEEP_ALIVE: 5m`, `OLLAMA_FLASH_ATTENTION: "0"`, `OLLAMA_KV_CACHE_TYPE: f16` |
| `ollama.gpu.enabled` / `.count` / `.resourceName` / `.runtimeClassName` | `false` / `1` / `nvidia.com/gpu` / `""` |
| `ollama.external.url` | `""` |
| `ollama.external.url` | `""` — blank with `ollama.enabled: false` means **no local runtime at all** (see below) |
| `searxng.enabled` | `true` |
| `searxng.image.repository` / `.tag` | `searxng/searxng` / `2026.6.10-de03f4eb1` |
| `searxng.settings` | `""` (blank = the compose `settings.yml`) |
Expand All @@ -411,9 +411,52 @@ defaults set `requests` only, and `limits` are yours to add. Each data-plane blo
also takes `nodeSelector`, `tolerations` and `affinity`.
Every `external.url` is required once its `enabled` is `false` — Helm fails the
render with a named message rather than deploying something that cannot connect.
Postgres is the exception in shape: an external server is addressed by
`external.host`/`.port` and the credentials still come from the Secret, so no DSN
with a password in it ever sits in a values file.
**Ollama and MinIO are the two exceptions**, because unlike Postgres, NATS, Qdrant
and OpenBao the core can genuinely run without either: a blank `ollama.external.url`
with `ollama.enabled: false` means *there is no local LLM runtime*, which is a
supported deployment and not an omission (see below). Postgres is the exception in
shape: an external server is addressed by `external.host`/`.port` and the
credentials still come from the Secret, so no DSN with a password in it ever sits in
a values file.

### Hosted-only: no local LLM runtime

Three Ollama deployments, not two (#962, ADR-0144):

| | `ollama.enabled` | `ollama.external.url` | `OLLAMA_URL` in the pod |
| --- | --- | --- | --- |
| the chart runs Ollama (default) | `true` | ignored | `http://ollama:11434` |
| an Ollama you run elsewhere | `false` | your URL | your URL |
| **no local runtime at all** | `false` | `""` | `""` |

The third is hosted chat and hosted embeddings with nothing local. It is a real mode, not a
degraded one: with an empty `OLLAMA_URL` the core reports `absent` at
`GET /platform/v1/llm/local-runtime`, answers `200` and `[]` from `GET /platform/v1/llm/models`
(rather than the 500 it used to, on a page that polls every ten seconds), refuses pull / delete
/ unload / the KV-cache setting with **409** and a sentence naming the mode, reports the chat
model as `n/a` in readiness instead of "warming" forever, and skips the first-boot model
bootstrap in one log line. The chart also blanks `LLM_BOOTSTRAP_MODELS` for you when it is left
at `auto` — there is nothing to pull into — while leaving an explicit list you set alone.

**Both model defaults must name hosted models**, and the chart enforces it at render time:

```bash
helm install epicurus oci://ghcr.io/baakhoff/charts/epicurus \
--namespace epicurus --create-namespace \
--set ollama.enabled=false \
--set core.llm.defaultModel=claude/claude-sonnet-4-6 \
--set core.memoryEmbedModel=gpt/text-embedding-3-small
```

Leave the chart's defaults (`llama3.2`, `nomic-embed-text`) in place and the render **fails**,
naming the key and an example value. That is deliberate: a bare model name routes to the local
runtime, so a runtime-less release with bare defaults is the half-working stack where chat works
through the hosted provider while memory recall and every module index fail at call time. The
old guard — `required` on `ollama.external.url` — refused a *legitimate* deployment; this one
refuses a broken one. Add each provider's API key on the Models page before the first turn.

The Compose equivalent is the
[hosted-only overlay](index.md#hosted-only-no-local-llm-runtime) (`task hosted-only-up`).

**MinIO is on by default**, matching the Compose stack. It backs two different
things: the `storage` module's object store (chat uploads, agent-written objects,
Expand Down
Loading
Loading