Skip to content

feat(selfhost): wire native Docker Compose secrets for the gittensory service#5121

Merged
JSONbored merged 2 commits into
mainfrom
feat/4937-docker-compose-secrets
Jul 11, 2026
Merged

feat(selfhost): wire native Docker Compose secrets for the gittensory service#5121
JSONbored merged 2 commits into
mainfrom
feat/4937-docker-compose-secrets

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Adds a top-level secrets: block to docker-compose.yml for the 10 highest-value secrets the always-on gittensory service reads (GitHub App private key, webhook secret, API/MCP/internal-job tokens, the setup token, the two token-encryption master keys, the Orb enrollment secret, and the PagerDuty routing key). Each is file-mounted at /run/secrets/<name> instead of sitting as a plain environment:/env_file value visible via docker inspect/docker compose config.
  • Zero application code changes: the app already has a fully generic <NAME>_FILE resolver (src/selfhost/load-file-secrets.ts, previously only demonstrated for GITHUB_APP_PRIVATE_KEY_FILE) — this PR just wires the Compose-level mount + sets the matching <NAME>_FILE env default for the rest of the list.
  • Purely additive and backward-compatible: an inline .env value always takes priority over the file (existing loader behavior, unchanged), so this cannot break any current self-host operator (including the live edge-nl-01 instance, which keeps using inline .env values unless/until it migrates a secret at a time).
  • scripts/selfhost-init-secrets.sh idempotently creates empty placeholder files for every declared secret so docker compose build/up never fails on a missing secrets: source file, even on a fresh checkout that has never touched this convention. Wired into both deploy-selfhost-prebuilt.sh and deploy-selfhost-image.sh before they touch docker compose, so every future deploy self-heals.
  • secrets/README.md documents the convention; .env.selfhost.example, .env.example, and the /docs/self-hosting-security page point at it, making the pre-existing (but previously only prose) "Prefer secret files" guidance concrete and turnkey.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • Maintainer/own-branch PR — no linked issue filed; this came directly out of a same-session operator request to harden self-host secrets handling, following the security doc's own pre-existing "Prefer secret files" guidance.

Validation

  • git diff --check
  • npm run test:ci — full local gate green, including ui:openapi:check, ui:lint, ui:typecheck, ui:test, ui:build.
  • npm audit --audit-level=moderate — 0 vulnerabilities.
  • docker compose config validated with every profile active (postgres, observability, rees, qdrant, ollama, visual-review, caddy, tailscale, pgbouncer, runner, backup) — all 22 services resolve cleanly, no YAML errors.
  • Empirically verified the secrets mount at actual container runtime (not just config-parse time): built a throwaway override swapping the gittensory service for busybox, confirmed /run/secrets/github_app_private_key and /run/secrets/pagerduty_routing_key exist and are readable inside the running container.
  • scripts/selfhost-init-secrets.sh tested directly: creates all 10 placeholder files on first run, correctly no-ops (never overwrites) on a second run.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed. (New secrets/* files are gitignored except the README; verified git add secrets/ only stages README.md.)
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (N/A — no app-level auth/session code changed, this is deploy-time secret delivery only.)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (N/A — no API surface changed.)
  • No visible UI change beyond a docs-content addition on /docs/self-hosting-security — no UI Evidence table needed (prose/code-block only, no new interactive states).
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

Notes

Scoped to the always-on gittensory service's own secrets for this pass. The same convention (one secrets: entry + one <NAME>_FILE default) extends trivially to the optional-profile services' own secrets (Postgres password, Grafana admin password, Litestream S3 keys, Tailscale auth key, runner tokens, Qdrant/Browserless API keys) as a natural follow-up — left out here to keep this PR narrow and fully testable against the one service I can validate end-to-end.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 11, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui 00cf22f Commit Preview URL

Branch Preview URL
Jul 11 2026, 04:48 PM

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.32%. Comparing base (46d63b9) to head (28a44df).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5121   +/-   ##
=======================================
  Coverage   94.32%   94.32%           
=======================================
  Files         471      471           
  Lines       39821    39821           
  Branches    14533    14533           
=======================================
  Hits        37560    37560           
  Misses       1583     1583           
  Partials      678      678           
Flag Coverage Δ
shard-1 46.34% <ø> (ø)
shard-2 34.66% <ø> (ø)
shard-3 32.09% <ø> (ø)
shard-4 32.01% <ø> (ø)
shard-5 33.58% <ø> (ø)
shard-6 44.87% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gittensory-orb gittensory-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 11, 2026
@gittensory-orb

gittensory-orb Bot commented Jul 11, 2026

Copy link
Copy Markdown

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-11 16:56:36 UTC

9 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI green · unstable

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
This PR wires a top-level Docker Compose `secrets:` block for 10 high-value secrets on the always-on `gittensory` service, using file mounts instead of plain env vars, with an idempotent placeholder-creation script and matching docs. The application-side reader is unchanged and the `<NAME>_FILE` precedence logic is sound and backward-compatible, but the `build + boot smoke test` CI check is failing on this commit, and the diff itself explains why: the new `secrets:` block requires each `file:` source to exist on disk before Compose will even parse/build, yet `scripts/selfhost-init-secrets.sh` is only invoked from `deploy-selfhost-prebuilt.sh` and `deploy-selfhost-image.sh` — not from whatever CI job runs the smoke test directly against `docker-compose.yml`.

Blockers

  • docker-compose.yml's new top-level `secrets:` block (e.g. `github_app_private_key: file: ./secrets/github_app_private_key.pem`) requires every listed file to exist before `docker compose build`/`up`/`config` will succeed, but the placeholder-creation step (`scripts/selfhost-init-secrets.sh`) is wired only into `deploy-selfhost-prebuilt.sh` and `deploy-selfhost-image.sh`, not into whatever workflow runs the failing 'build + boot smoke test' check — a fresh checkout/CI runner that invokes `docker compose` directly (without going through those two deploy scripts) will fail exactly as the failed check shows.
  • Per repo policy, this PR must close or clearly link an eligible open issue; the description only drops an inline `Wire on-call/paging for the central hosted service #4937-secrets-hardening` tag rather than a real issue reference, so as written this looks like unsolicited feature/hardening work rather than issue-tied work.
  • docker-compose.yml:965 makes the base Compose file depend on `./secrets/*.txt` and `./secrets/github_app_private_key.pem`, but `.gitignore:18` prevents those files from being present in a fresh checkout and `scripts/selfhost-init-secrets.sh` only runs from the deploy wrappers, so any existing direct `docker compose config`/`up` path fails before the app can prefer inline `.env` values; either keep these secrets in a generated/opt-in override or make the base Compose source files exist before Compose is invoked.
Nits — 7 non-blocking
  • docker-compose.yml is now ~1000 lines; consider splitting the secrets wiring or per-service env blocks into an include/override file per Compose's `include:` support if it keeps growing.
  • secrets/README.md and the docs page both use 'master key' terminology (`secrets/README.md:53`, `docs.self-hosting-security.tsx` copy) — consider 'primary key' for consistency with more neutral naming used elsewhere.
  • No automated test (e.g. a `docker compose config` or `config --quiet` CI step) validates that all `secrets:` file sources resolve before a real deploy attempt — exactly the class of failure now showing red on this PR.
  • `scripts/selfhost-init-secrets.sh` and the two deploy scripts have no test coverage; consider at least a shellcheck/CI smoke invocation for the new script given it now gates every deploy.
  • Add a `docker compose config` (or `--dry-run`) step to whatever CI job runs 'build + boot smoke test', invoking `scripts/selfhost-init-secrets.sh` first, so the secrets file requirement is satisfied the same way the two production deploy scripts already do it.
  • Diff looks like trivial or whitespace-only churn — Reduce whitespace-only or formatting-only churn and keep the diff focused on substantive changes.
  • Code changes lack test evidence — Add focused regression tests or explain why existing coverage is sufficient.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 45 registered-repo PR(s), 37 merged, 414 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 45 PR(s), 414 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Improvement ⚠️ ℹ️ None detected risk: elevated · value: none — No structural-improvement signals were detected for this PR.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 45 PR(s), 414 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

Visual preview
Route Viewport Before (production) After (this PR's preview) Diff
/docs/self-hosting-security desktop before /docs/self-hosting-security after /docs/self-hosting-security
/docs/self-hosting-security mobile before /docs/self-hosting-security (mobile) after /docs/self-hosting-security (mobile)

Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@gittensory-orb gittensory-orb Bot added the manual-review Gittensor contributor context label Jul 11, 2026
… service

Adds a `secrets:` block for the 10 highest-value secrets (GitHub App
private key, webhook secret, API/MCP/internal-job tokens, the setup
token, the two token-encryption master keys, the Orb enrollment secret,
and the PagerDuty routing key), file-mounted at /run/secrets/<name>
instead of a plain environment/env_file value visible via `docker
inspect`/`docker compose config`.

Zero application code changes: reuses the existing generic <NAME>_FILE
loader (src/selfhost/load-file-secrets.ts). Purely additive and
backward-compatible -- an inline .env value always wins over the file,
so migrating is optional and can be done one secret at a time.
scripts/selfhost-init-secrets.sh idempotently creates empty placeholder
files so a deploy never breaks for an operator who hasn't opted in;
wired into both deploy scripts before they touch docker compose.
#4937 is the unrelated PagerDuty on-call issue (closed via #5116) --
this feature has no filed issue, so referencing it as
"#4937-secrets-hardening" was actively misleading, not just unlinked.
@JSONbored JSONbored merged commit 9a1020a into main Jul 11, 2026
20 checks passed
@JSONbored JSONbored deleted the feat/4937-docker-compose-secrets branch July 11, 2026 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

1 participant