docs(app-store): hardening + signed catalogue; whole-site sitemap

src/pages/docs/app-store.astro

@@ -14,6 +14,7 @@ const bodyContent = `<h1>App Store</h1>
       <li><a href="#building">Building an app</a></li>
       <li><a href="#publishing">Publishing an app</a></li>
       <li><a href="#install-models">Catalogue vs sideload</a></li>
+      <li><a href="#security">Security model &amp; hardening</a></li>
       <li><a href="#example">Worked example: io.pilot.cosift</a></li>
     </ul>
   </div>
@@ -135,7 +136,13 @@ ipc.Serve(ctx, conn, d)   <span class="comment">// on the --socket the daemon su
   "bundle_sha256": "&lt;sha256 of the tarball&gt;"
 }</code></pre>
 
-  <p>Two integrity layers protect every install, both re-checked at each spawn: the catalogue pins the <em>tarball</em> sha256 (a swapped CDN byte fails), and the manifest pins the <em>binary</em> sha256 under an ed25519 signature.</p>
+  <p>The catalogue itself is signed. After editing <code>catalogue.json</code>, re-sign it so the daemon and <code>pilotctl</code> will trust it:</p>
+
+  <pre><code><span class="cmd">pilotctl</span> appstore sign-catalogue --key catalog-signing.key catalogue/catalogue.json</code></pre>
+
+  <p>This writes a detached <code>catalogue.json.sig</code> (commit both). <code>pilotctl</code> verifies it against the embedded catalogue key before trusting any entry - see <a href="#security">Security model</a>.</p>
+
+  <p>Three integrity layers protect every install: the <em>catalogue</em> carries a detached ed25519 signature (a substituted app list fails), the catalogue pins each <em>tarball</em> sha256 (a swapped CDN byte fails), and the manifest pins the <em>binary</em> sha256 under an ed25519 signature - the last two re-checked at every spawn.</p>
 
   <h2 id="install-models">Catalogue vs sideload</h2>
 
@@ -150,6 +157,40 @@ ipc.Serve(ctx, conn, d)   <span class="comment">// on the --socket the daemon su
     <p>To stage a release locally before publishing, point <code>$PILOT_APPSTORE_CATALOG_URL</code> at a <code>file://</code> catalogue and install by id - the same code path as production, with your own tarball.</p>
   </div>
 
+  <h2 id="security">Security model &amp; hardening</h2>
+
+  <p>The app store is deny-by-default at every layer. Trust flows from a signed catalogue, through a signed manifest, to a sandboxed and continuously-verified child process.</p>
+
+  <h3>Signed catalogue (fail-closed)</h3>
+
+  <p>The catalogue is signed with a dedicated ed25519 key whose public half is compiled into <code>pilotctl</code> and the daemon. <code>pilotctl</code> fetches both <code>catalogue.json</code> and a detached <code>catalogue.json.sig</code> and verifies the signature <strong>before trusting any entry</strong>. An unsigned, missing-signature, or tampered catalogue is refused - a compromised host or CDN cannot point installs at hostile bundle URLs without forging the signature. The signing key can be rotated at build time:</p>
+
+  <pre><code><span class="cmd">go</span> build -ldflags \
+  "-X .../internal/catalogtrust.publicKeyB64=&lt;new-b64-pubkey&gt;" \
+  ./cmd/pilotctl ./cmd/daemon</code></pre>
+
+  <h3>Broker authorization (exposes + grants)</h3>
+
+  <p>Apps never dial each other's sockets directly - every call goes through the daemon's broker, which enforces two gates before any dispatch:</p>
+  <ul>
+    <li><strong>Exposes gate</strong> - a method is dispatchable only if the target app lists it in its manifest <code>exposes</code> set. That list is the app's entire callable surface; anything else is refused, even for the daemon itself.</li>
+    <li><strong>Grant gate</strong> - when one app calls another, the caller must declare a matching <code>ipc.call</code> grant targeting <code>&lt;app&gt;.&lt;method&gt;</code> (exact, <code>&lt;app&gt;.*</code>, or <code>*</code>). No grant, no call.</li>
+  </ul>
+
+  <h3>Supervisor hardening</h3>
+
+  <p>The supervisor that spawns and watches each app applies defence-in-depth:</p>
+  <ul>
+    <li><strong>Launch-time re-verification (TOCTOU)</strong> - immediately before <code>exec</code>, the binary is re-checked: rejected if it became a symlink, and its sha256 must still match the pinned hash. A binary swapped between install-scan and launch is caught before it runs.</li>
+    <li><strong>Exponential backoff</strong> - a binary that fails verification is retried with capped exponential backoff (not a fixed interval), and a crash-looping app is suspended after too many failures in a window until an operator restarts it.</li>
+    <li><strong>Resource limits</strong> - on Linux each app is spawned with <code>RLIMIT_NOFILE</code> and an <code>RLIMIT_AS</code> address-space cap, bounding fd and memory abuse.</li>
+    <li><strong>Audit log rotation</strong> - every lifecycle event (spawn, exit, verify-fail, suspend, resume) is written to a per-app JSONL audit log that rotates across a bounded number of generations, so a crash-looping app can't fill the disk while recent forensics are retained.</li>
+  </ul>
+
+  <h3>Extension hooks</h3>
+
+  <p>Apps may register hooks on daemon primitives (declared in the manifest), but the hook surface is bounded: per-app rate limiting caps how often the daemon will dispatch into an app's hooks, and the number of dynamic hook registrations per app is capped - so a hostile hook can't become a DoS amplifier.</p>
+
   <h2 id="example">Worked example: io.pilot.cosift</h2>
 
   <p>The cosift app is a stateless adapter to a search / answer / research API over a multi-million-document web corpus. It exposes three utility methods and several status/discovery ones:</p>

src/pages/sitemap.xml.ts

@@ -1,63 +1,94 @@
 import { blogPosts } from '../data/blogPosts';
-import { docsNav } from '../data/docsNav';
 
 const site = 'https://pilotprotocol.network';
 
+// Enumerate every page module so the sitemap reflects the real route
+// tree — no hand-maintained list to drift out of date. New pages appear
+// automatically; only error pages, dynamic templates, and the /plain
+// text-mirror are filtered out.
+const pageGlob = import.meta.glob('./**/*.{astro,md,mdx}');
+
 function url(loc: string, lastmod: string, priority: number, changefreq = 'monthly') {
   return `  <url><loc>${site}${loc}</loc><lastmod>${lastmod}</lastmod><changefreq>${changefreq}</changefreq><priority>${priority}</priority></url>`;
 }
 
 function blogDate(date: string, year?: number): string {
   const y = year || new Date().getFullYear();
   const d = new Date(`${date}, ${y}`);
-  return d.toISOString().split('T')[0];
+  const iso = d.toISOString();
+  return iso.split('T')[0];
 }
 
-export async function GET() {
-  const today = new Date().toISOString().split('T')[0];
-
-  const urls: string[] = [];
+// Map a glob key (e.g. './docs/app-store.astro') to a route.
+function routeFromKey(key: string): string {
+  const rel = key.replace(/^\.\//, '');
+  if (rel === 'index.astro' || rel === 'index.md' || rel === 'index.mdx') return '/';
+  const m = rel.match(/^(.*\/)index\.(astro|mdx?|md)$/);
+  if (m) return '/' + m[1]; // directory index → trailing slash
+  return '/' + rel.replace(/\.(astro|mdx?|md)$/, '');
+}
 
-  // Static pages
-  urls.push(url('/', today, 1.0, 'weekly'));
-  urls.push(url('/plans', today, 0.9));
-  urls.push(url('/blog/', today, 0.9, 'weekly'));
-  urls.push(url('/llms.txt', '2026-02-28', 0.5));
+const REFERENCE = new Set(['error-codes', 'troubleshooting', 'diagnostics', 'configuration']);
+const LEGAL = new Set(['/privacy', '/cookies', '/terms', '/aup']);
 
-  // Press / brand
-  urls.push(url('/press', today, 0.7));
-  urls.push(url('/brand/', today, 0.6));
+function priorityFor(loc: string): { p: number; freq: string } {
+  if (loc === '/') return { p: 1.0, freq: 'weekly' };
+  if (loc === '/blog/') return { p: 0.9, freq: 'weekly' };
+  if (loc === '/docs/' || loc === '/docs/getting-started') return { p: 0.9, freq: 'monthly' };
+  if (loc === '/plans' || loc === '/app-store') return { p: 0.9, freq: 'monthly' };
+  if (loc.startsWith('/blog/')) return { p: 0.8, freq: 'monthly' };
+  if (loc.startsWith('/docs/')) {
+    const slug = loc.replace('/docs/', '').replace(/\/$/, '');
+    return { p: REFERENCE.has(slug) ? 0.6 : 0.8, freq: 'monthly' };
+  }
+  if (LEGAL.has(loc) || loc === '/press') return { p: 0.7, freq: 'monthly' };
+  if (loc.startsWith('/for/')) return { p: 0.8, freq: 'monthly' };
+  return { p: 0.7, freq: 'monthly' };
+}
 
-  // Legal pages
-  urls.push(url('/privacy', today, 0.7));
-  urls.push(url('/cookies', today, 0.7));
-  urls.push(url('/terms', today, 0.7));
-  urls.push(url('/aup', today, 0.7));
+export async function GET() {
+  const today = new Date().toISOString().split('T')[0];
+  const blogDates = new Map(blogPosts.map((b) => [b.slug, blogDate(b.date, b.year)]));
 
-  // Solution / "for" pages
-  urls.push(url('/for/mcp', today, 0.8));
-  urls.push(url('/for/p2p', today, 0.8));
-  urls.push(url('/for/networks', today, 0.8));
-  urls.push(url('/for/setups', today, 0.8));
-  urls.push(url('/for/skills', today, 0.8));
+  const seen = new Set<string>();
+  const urls: string[] = [];
+  const add = (loc: string, lastmod: string, priority: number, freq = 'monthly') => {
+    if (seen.has(loc)) return;
+    seen.add(loc);
+    urls.push(url(loc, lastmod, priority, freq));
+  };
 
-  // Doc pages
-  for (const nav of docsNav) {
-    const isIndex = nav.href === '/docs/' || nav.href === '/docs/getting-started';
-    const isReference = nav.slug === 'error-codes' || nav.slug === 'troubleshooting' || nav.slug === 'diagnostics' || nav.slug === 'configuration';
-    const priority = isIndex ? 0.9 : isReference ? 0.6 : 0.8;
-    urls.push(url(nav.href, today, priority));
+  // 1. Every static page route discovered from the filesystem.
+  for (const key of Object.keys(pageGlob)) {
+    const loc = routeFromKey(key);
+    if (loc === '/404' || loc === '/500') continue; // error pages
+    if (loc.includes('[')) continue;                // dynamic template — expanded below
+    if (loc.startsWith('/plain/')) continue;        // non-canonical text mirror
+    const blogSlug = loc.startsWith('/blog/') ? loc.replace('/blog/', '').replace(/\/$/, '') : '';
+    const lastmod = blogSlug && blogDates.has(blogSlug) ? blogDates.get(blogSlug)! : today;
+    const { p, freq } = priorityFor(loc);
+    add(loc, lastmod, p, freq);
   }
 
-  // Research pages
-  urls.push(url('/research/ietf/draft-teodor-pilot-problem-statement-01.html', '2026-04-06', 0.7));
-  urls.push(url('/research/ietf/draft-teodor-pilot-protocol-01.html', '2026-04-06', 0.7));
-
-  // Blog posts
-  for (const post of blogPosts) {
-    urls.push(url(`/blog/${post.slug}`, blogDate(post.date, post.year), 0.8));
+  // 2. Dynamic /for/setups/<slug> — expanded from the same source the
+  //    page's getStaticPaths uses. Best-effort: a fetch failure just
+  //    omits these rather than breaking the build.
+  try {
+    const res = await fetch('https://raw.githubusercontent.com/TeoSlayer/pilot-skills/main/setups.json');
+    if (res.ok) {
+      const catalog: { setups?: Array<{ slug: string }> } = await res.json();
+      for (const s of catalog.setups ?? []) add(`/for/setups/${s.slug}`, today, 0.7);
+    }
+  } catch {
+    // offline build — skip dynamic setup pages
   }
 
+  // 3. Static assets served from public/ that aren't .astro routes.
+  add('/llms.txt', '2026-02-28', 0.5);
+  add('/brand/', today, 0.6);
+  add('/research/ietf/draft-teodor-pilot-problem-statement-01.html', '2026-04-06', 0.7);
+  add('/research/ietf/draft-teodor-pilot-protocol-01.html', '2026-04-06', 0.7);
+
   const xml = `<?xml version="1.0" encoding="UTF-8"?>
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
 ${urls.join('\n')}