From 1b419881cd1db47ace63d822aac6a041f8f02fd7 Mon Sep 17 00:00:00 2001 From: Teodor Calin Date: Mon, 15 Jun 2026 15:03:04 +0000 Subject: [PATCH] docs(app-store): document hardening + signed catalogue; whole-site sitemap - docs/app-store: add a Security model & hardening section covering the signed catalogue (fail-closed verification + rotation), broker authorization (exposes + ipc.call grants), supervisor hardening (TOCTOU re-verify, exponential backoff, RLIMIT_AS/NOFILE, audit log rotation), and bounded extension hooks; document sign-catalogue in the publishing flow and the three integrity layers. - sitemap: rewrite as a filesystem-glob over all pages so the whole site is covered automatically (was a hand-maintained list missing /app-store and the dynamic /for/setups/* pages). Excludes error pages, dynamic templates, and the /plain text mirror; expands setups slugs; keeps blog lastmod. 207 URLs, verified via astro build. --- src/pages/docs/app-store.astro | 43 ++++++++++++- src/pages/sitemap.xml.ts | 107 +++++++++++++++++++++------------ 2 files changed, 111 insertions(+), 39 deletions(-) diff --git a/src/pages/docs/app-store.astro b/src/pages/docs/app-store.astro index db337a6..4ff9b2d 100644 --- a/src/pages/docs/app-store.astro +++ b/src/pages/docs/app-store.astro @@ -14,6 +14,7 @@ const bodyContent = `

App Store

  • Building an app
  • Publishing an app
  • Catalogue vs sideload
    • +
  • Security model & hardening
  • Worked example: io.pilot.cosift
    • @@ -135,7 +136,13 @@ ipc.Serve(ctx, conn, d) // on the --socket the daemon su "bundle_sha256": "<sha256 of the tarball>" } -

    Two integrity layers protect every install, both re-checked at each spawn: the catalogue pins the tarball sha256 (a swapped CDN byte fails), and the manifest pins the binary sha256 under an ed25519 signature.

    +

    The catalogue itself is signed. After editing catalogue.json, re-sign it so the daemon and pilotctl will trust it:

    + + 
    pilotctl appstore sign-catalogue --key catalog-signing.key catalogue/catalogue.json
    + +

    This writes a detached catalogue.json.sig (commit both). pilotctl verifies it against the embedded catalogue key before trusting any entry - see Security model.

    + +

    Three integrity layers protect every install: the catalogue carries a detached ed25519 signature (a substituted app list fails), the catalogue pins each tarball sha256 (a swapped CDN byte fails), and the manifest pins the binary sha256 under an ed25519 signature - the last two re-checked at every spawn.

    Catalogue vs sideload

    @@ -150,6 +157,40 @@ ipc.Serve(ctx, conn, d) // on the --socket the daemon su

    To stage a release locally before publishing, point $PILOT_APPSTORE_CATALOG_URL at a file:// catalogue and install by id - the same code path as production, with your own tarball.

    +

    Security model & hardening

    + +

    The app store is deny-by-default at every layer. Trust flows from a signed catalogue, through a signed manifest, to a sandboxed and continuously-verified child process.

    + +

    Signed catalogue (fail-closed)

    + +

    The catalogue is signed with a dedicated ed25519 key whose public half is compiled into pilotctl and the daemon. pilotctl fetches both catalogue.json and a detached catalogue.json.sig and verifies the signature before trusting any entry. An unsigned, missing-signature, or tampered catalogue is refused - a compromised host or CDN cannot point installs at hostile bundle URLs without forging the signature. The signing key can be rotated at build time:

    + + 
    go build -ldflags \
+  "-X .../internal/catalogtrust.publicKeyB64=<new-b64-pubkey>" \
+  ./cmd/pilotctl ./cmd/daemon
    + +

    Broker authorization (exposes + grants)

    + +

    Apps never dial each other's sockets directly - every call goes through the daemon's broker, which enforces two gates before any dispatch:

    +
      +
    • Exposes gate - a method is dispatchable only if the target app lists it in its manifest exposes set. That list is the app's entire callable surface; anything else is refused, even for the daemon itself.
      • +
    • Grant gate - when one app calls another, the caller must declare a matching ipc.call grant targeting <app>.<method> (exact, <app>.*, or *). No grant, no call.
      • +
    + +

    Supervisor hardening

    + +

    The supervisor that spawns and watches each app applies defence-in-depth:

    +
      +
    • Launch-time re-verification (TOCTOU) - immediately before exec, the binary is re-checked: rejected if it became a symlink, and its sha256 must still match the pinned hash. A binary swapped between install-scan and launch is caught before it runs.
      • +
    • Exponential backoff - a binary that fails verification is retried with capped exponential backoff (not a fixed interval), and a crash-looping app is suspended after too many failures in a window until an operator restarts it.
      • +
    • Resource limits - on Linux each app is spawned with RLIMIT_NOFILE and an RLIMIT_AS address-space cap, bounding fd and memory abuse.
      • +
    • Audit log rotation - every lifecycle event (spawn, exit, verify-fail, suspend, resume) is written to a per-app JSONL audit log that rotates across a bounded number of generations, so a crash-looping app can't fill the disk while recent forensics are retained.
      • +
    + +

    Extension hooks

    + +

    Apps may register hooks on daemon primitives (declared in the manifest), but the hook surface is bounded: per-app rate limiting caps how often the daemon will dispatch into an app's hooks, and the number of dynamic hook registrations per app is capped - so a hostile hook can't become a DoS amplifier.

    +

    Worked example: io.pilot.cosift

    The cosift app is a stateless adapter to a search / answer / research API over a multi-million-document web corpus. It exposes three utility methods and several status/discovery ones:

    diff --git a/src/pages/sitemap.xml.ts b/src/pages/sitemap.xml.ts index df07328..c393bf2 100644 --- a/src/pages/sitemap.xml.ts +++ b/src/pages/sitemap.xml.ts @@ -1,8 +1,13 @@ import { blogPosts } from '../data/blogPosts'; -import { docsNav } from '../data/docsNav'; const site = 'https://pilotprotocol.network'; +// Enumerate every page module so the sitemap reflects the real route +// tree — no hand-maintained list to drift out of date. New pages appear +// automatically; only error pages, dynamic templates, and the /plain +// text-mirror are filtered out. +const pageGlob = import.meta.glob('./**/*.{astro,md,mdx}'); + function url(loc: string, lastmod: string, priority: number, changefreq = 'monthly') { return ` ${site}${loc}${lastmod}${changefreq}${priority}`; } @@ -10,54 +15,80 @@ function url(loc: string, lastmod: string, priority: number, changefreq = 'month function blogDate(date: string, year?: number): string { const y = year || new Date().getFullYear(); const d = new Date(`${date}, ${y}`); - return d.toISOString().split('T')[0]; + const iso = d.toISOString(); + return iso.split('T')[0]; } -export async function GET() { - const today = new Date().toISOString().split('T')[0]; - - const urls: string[] = []; +// Map a glob key (e.g. './docs/app-store.astro') to a route. +function routeFromKey(key: string): string { + const rel = key.replace(/^\.\//, ''); + if (rel === 'index.astro' || rel === 'index.md' || rel === 'index.mdx') return '/'; + const m = rel.match(/^(.*\/)index\.(astro|mdx?|md)$/); + if (m) return '/' + m[1]; // directory index → trailing slash + return '/' + rel.replace(/\.(astro|mdx?|md)$/, ''); +} - // Static pages - urls.push(url('/', today, 1.0, 'weekly')); - urls.push(url('/plans', today, 0.9)); - urls.push(url('/blog/', today, 0.9, 'weekly')); - urls.push(url('/llms.txt', '2026-02-28', 0.5)); +const REFERENCE = new Set(['error-codes', 'troubleshooting', 'diagnostics', 'configuration']); +const LEGAL = new Set(['/privacy', '/cookies', '/terms', '/aup']); - // Press / brand - urls.push(url('/press', today, 0.7)); - urls.push(url('/brand/', today, 0.6)); +function priorityFor(loc: string): { p: number; freq: string } { + if (loc === '/') return { p: 1.0, freq: 'weekly' }; + if (loc === '/blog/') return { p: 0.9, freq: 'weekly' }; + if (loc === '/docs/' || loc === '/docs/getting-started') return { p: 0.9, freq: 'monthly' }; + if (loc === '/plans' || loc === '/app-store') return { p: 0.9, freq: 'monthly' }; + if (loc.startsWith('/blog/')) return { p: 0.8, freq: 'monthly' }; + if (loc.startsWith('/docs/')) { + const slug = loc.replace('/docs/', '').replace(/\/$/, ''); + return { p: REFERENCE.has(slug) ? 0.6 : 0.8, freq: 'monthly' }; + } + if (LEGAL.has(loc) || loc === '/press') return { p: 0.7, freq: 'monthly' }; + if (loc.startsWith('/for/')) return { p: 0.8, freq: 'monthly' }; + return { p: 0.7, freq: 'monthly' }; +} - // Legal pages - urls.push(url('/privacy', today, 0.7)); - urls.push(url('/cookies', today, 0.7)); - urls.push(url('/terms', today, 0.7)); - urls.push(url('/aup', today, 0.7)); +export async function GET() { + const today = new Date().toISOString().split('T')[0]; + const blogDates = new Map(blogPosts.map((b) => [b.slug, blogDate(b.date, b.year)])); - // Solution / "for" pages - urls.push(url('/for/mcp', today, 0.8)); - urls.push(url('/for/p2p', today, 0.8)); - urls.push(url('/for/networks', today, 0.8)); - urls.push(url('/for/setups', today, 0.8)); - urls.push(url('/for/skills', today, 0.8)); + const seen = new Set(); + const urls: string[] = []; + const add = (loc: string, lastmod: string, priority: number, freq = 'monthly') => { + if (seen.has(loc)) return; + seen.add(loc); + urls.push(url(loc, lastmod, priority, freq)); + }; - // Doc pages - for (const nav of docsNav) { - const isIndex = nav.href === '/docs/' || nav.href === '/docs/getting-started'; - const isReference = nav.slug === 'error-codes' || nav.slug === 'troubleshooting' || nav.slug === 'diagnostics' || nav.slug === 'configuration'; - const priority = isIndex ? 0.9 : isReference ? 0.6 : 0.8; - urls.push(url(nav.href, today, priority)); + // 1. Every static page route discovered from the filesystem. + for (const key of Object.keys(pageGlob)) { + const loc = routeFromKey(key); + if (loc === '/404' || loc === '/500') continue; // error pages + if (loc.includes('[')) continue; // dynamic template — expanded below + if (loc.startsWith('/plain/')) continue; // non-canonical text mirror + const blogSlug = loc.startsWith('/blog/') ? loc.replace('/blog/', '').replace(/\/$/, '') : ''; + const lastmod = blogSlug && blogDates.has(blogSlug) ? blogDates.get(blogSlug)! : today; + const { p, freq } = priorityFor(loc); + add(loc, lastmod, p, freq); } - // Research pages - urls.push(url('/research/ietf/draft-teodor-pilot-problem-statement-01.html', '2026-04-06', 0.7)); - urls.push(url('/research/ietf/draft-teodor-pilot-protocol-01.html', '2026-04-06', 0.7)); - - // Blog posts - for (const post of blogPosts) { - urls.push(url(`/blog/${post.slug}`, blogDate(post.date, post.year), 0.8)); + // 2. Dynamic /for/setups/ — expanded from the same source the + // page's getStaticPaths uses. Best-effort: a fetch failure just + // omits these rather than breaking the build. + try { + const res = await fetch('https://raw.githubusercontent.com/TeoSlayer/pilot-skills/main/setups.json'); + if (res.ok) { + const catalog: { setups?: Array<{ slug: string }> } = await res.json(); + for (const s of catalog.setups ?? []) add(`/for/setups/${s.slug}`, today, 0.7); + } + } catch { + // offline build — skip dynamic setup pages } + // 3. Static assets served from public/ that aren't .astro routes. + add('/llms.txt', '2026-02-28', 0.5); + add('/brand/', today, 0.6); + add('/research/ietf/draft-teodor-pilot-problem-statement-01.html', '2026-04-06', 0.7); + add('/research/ietf/draft-teodor-pilot-protocol-01.html', '2026-04-06', 0.7); + const xml = ` ${urls.join('\n')}