Full user-facing documentation: getting started, user guide, CLI/flag reference, and whatever else this project needs

[scttfrdmn]

Why

calque now has real, live-verified capability (v0.3.0: two AI-Almanac functions running unmodified on real AWS hardware with byte-identical results vs. local reference runs; secrets injection, volume sync, multi-arg positional payloads via --arg-file/--arg-json, --function selection, --stage-file, --pip/--python-version, fleet pooling, spot/multi-region acquisition, institutional GPU sharing via MIG/MPS). None of this is discoverable by a new user without reading source or --help output flag-by-flag.

Existing docs are strong on design rationale (docs/*.md — an "index of what each design doc answers", docs/porting-modal-to-aws.md, examples/README.md, README's quick start) but there is no single onboarding path: no getting-started walkthrough for a first real (billable) run, no consolidated user guide covering the full command surface (analyze, run, smoke, real, ramp, pool, spawn-run, session) and how they relate, and no flag/CLI reference generated or maintained outside scattered --help strings in cmd/calque/main.go.

Scope (to be refined by the exploration pass below)

At minimum, this should produce:

• Getting started: install → calque analyze on a sample script → first real (billable) run, with the exact cost/confirmation gates explained up front.
• User guide: task-oriented walkthroughs per verb (real, ramp, pool, fleetrun/shards, spawn-run, session), including the newer primitives from this session (--secret, --item-file, --function, --arg-file/--arg-json, --stage-file, --pip/--python-version).
• Reference: a complete, accurate CLI/flag reference (ideally generated from or checked against the actual flag.NewFlagSet definitions so it can't silently drift).
• Anything else a project like this should have: e.g. a troubleshooting/FAQ page (reading leak reports), a "what calque does NOT do" pointer to docs/behind-the-seam-register.md, a migration checklist tying back to docs/porting-modal-to-aws.md, and a clear map of how docs/README.md's existing design-doc index relates to the new user-facing docs (design docs stay design docs; this is new, separate, user-facing material).

Ask

Run an exploration pass (agent) to:

1. Inventory the current command surface, flags, and existing docs (README, docs/*.md, examples/README.md) to find the real gaps vs. what's already covered.
2. Propose a concrete information architecture (what pages/files, where they live — e.g. a new docs/guide/ tree vs. flat files — and how they cross-link with the existing design-doc index).
3. Identify anything that should be auto-generated or auto-checked (e.g. flag reference vs. flag.NewFlagSet) so docs don't silently drift from the CLI the way docs/README.md had to correct for elsewhere.
4. Report back a plan; do not write the documentation itself in that pass.

Labels: documentation, kind:design (this issue tracks a design/planning pass first, not a direct doc-writing task).

[scttfrdmn]

Exploration pass results

An exploration agent inventoried the full command/flag surface, existing docs, and drift risk. Full report below.

1. Full current command/flag surface

calque analyze <script.py> [...] — no flags, positional script path(s). Zero cost, zero AWS calls.

calque run [flags] <script.py> (dry-run only, never billable): --n (100000), --region (us-west-2), --dry-run (true), --rates (config/rates.json), --entrypoint ("").

calque smoke [flags] (billable — first billable action, acquire-only de-risking test): --bucket (req), --region (us-west-2), --run-id (req), --ttl (30m), --deadline-min (20), --instance (""→g7e.2xlarge), --ami (""), --spot/--spot-max-price, --i-understand-this-spends-money (req).

calque real [flags] (billable — real GPU inference): --bucket/--region/--run-id/--instance(g6.2xlarge)/--ami/--model/--n(1)/--shards(1)/--ttl(40m)/--deadline-min(40)/--rates/--pool/--spot+--spot-max-price/--script/--entrypoint, plus the recent additions from this session: --function, --secret (repeatable NAME=VALUE), --item-file, --arg-file (repeatable IDX=PATH), --arg-json (repeatable IDX=JSON), --pip (repeatable), --python-version, --stage-file (repeatable URL=PATH), --i-understand-this-spends-money (req).

calque ramp [flags] (billable — acquire-once/hold/run-many): --bucket/--region/--run-id/--instance(g7e.2xlarge)/--ami/--model/--rungs(1,100,1000)/--ttl(3h)/--acquire-deadline-min(180)/--rates/--spot+--spot-max-price/--fallback-regions/--prep-timeout-min(30)/--concurrency(1)/--batch-size(1)/--script/--i-understand-this-spends-money(req).

calque pool <subcommand> (billable for create/scale/delete): create (--model/--instance-type(req)/--workers(1)/--min-viable(1)/--spot+--spot-max-price/--ttl(12h)/--idle-timeout(30m)/--manifest-bucket(req)/--results-bucket(req)/--runner-path(req)/--ami/--i-understand-this-spends-money(req)), scale (same shape, --add-workers), delete (--model/--region/--i-understand-this-terminates-instances(req, distinct destructive-confirm)), status/list (read-only, --model/--region).

calque spawn-run [flags] <script.py> (billable — block-and-wait .spawn()/.get() fan-out): --bucket/--region/--run-id/--instance(m7i.large)/--ami(req)/--ttl(20m)/--deadline-min(15)/--rates(unused today)/--i-understand-this-spends-money(req).

calque session <subcommand> (institutional MIG/MPS check-out on an already-running instance — never touches EC2 lifecycle): checkout (--instance-id/--user/--backend mig|mps(req)/--ttl(2h)/--instance-type(g7e.2xlarge, MIG only)/--slots(4, MPS only)/--i-understand-shared-gpu-has-no-isolation(required only for --backend mps)/--spot), checkin (--slice/--session-token), status/list (--instance-id).

Notable convention: every billable command uses --i-understand-this-spends-money except pool delete (--i-understand-this-terminates-instances) and session checkout --backend mps (--i-understand-shared-gpu-has-no-isolation) — a consistent 3-tier gating taxonomy (spend / terminate / no-isolation) worth documenting once, explicitly.

2. Already well-covered — don't duplicate

docs/modal-compatibility-matrix.md (support census), docs/porting-modal-to-aws.md (analyze-driven porting), docs/behind-the-seam-register.md (non-goals), docs/serve-architecture.md, docs/tenancy-vs-session.md, docs/gpu-sharing-support-matrix.md, docs/pool-queue-contract.md, docs/m12-m13-boundary.md, docs/substrate-offline-test-tier.md, examples/README.md + examples/*.py (zero-spend walkthroughs, guarded byte-identical to testdata/scripts/), README's zero-spend quick start, CONTRIBUTING.md, CHANGELOG.md.

3. Genuine gaps

1. No first-billable-run walkthrough — README stops at run --dry-run; nothing connects smoke → real --n 1 or explains why smoke exists as the de-risking step first. Biggest gap given v0.3.0 is now a real, live, billable capability.
2. No verb-selection guide — nowhere says "use real for one-off, ramp for repeated-N on a held instance, pool for shared warm models, spawn-run for .spawn() fanout, session for slicing a running instance, --shards for .map() fleet fanout." Requires synthesizing from 4+ separate docs today.
3. No CLI/flag reference at all — only source string literals and main.go's own usage(), which is already stale (missing --function/--secret/--item-file/--arg-file/--arg-json/--pip/--python-version/--stage-file and pool's subcommand flags) — live proof of the drift risk below.
4. No troubleshooting/FAQ — real fixes (IAM instance profile, uv venv binary path, AL2023 missing git, vLLM stdout vs JSON protocol collision, AMI auto-select brokenness) are scattered across CHANGELOG/code comments only.
5. No migration/AWS-setup checklist distinct from the analyze-driven porting narrative.
6. No flag-interaction/mutual-exclusion table (--item-file vs --arg-file/--arg-json vs --n; --pool vs --shards>1) — currently implicit in error messages only.

4. Drift risk — concrete finding, not hypothetical

No mechanism anywhere cross-checks any doc against actual flag.FlagSet definitions. main.go's own hand-written usage() is already stale — demonstrated proof this will keep happening. docs/modal-compatibility-matrix.md's hand-classified/unverified convention is fine for design docs (judgment content) but wrong for a flag reference (zero judgment content, pure enumeration).

Recommendation: generate the CLI reference, don't hand-write it.

• Option A (preferred): refactor each flag-set-construction function (parseSmokeArgs, parseRealArgs, poolCreateCmd, sessionCheckoutCmd, etc.) to expose its flag.FlagSet separately from parse+validate (heaviest lift: rampCmd, which currently inlines all three), then a go generate-invoked program walks each via FlagSet.VisitAll and renders docs/guide/cli-reference.md. CI step (go generate ./... && git diff --exit-code docs/guide/cli-reference.md) enforces it — mirrors the existing gofmt -l . and examples_test.go byte-identical-copy guard patterns already used in this repo.
• Option B (lighter): a Go test that walks each FlagSet and diffs against a golden file — less ceremony, but the "doc" is really a test fixture.
• Split: generated facts (flag name/default/one-line help) vs. hand-written narrative (examples, "when to use this," cross-links) — don't ask one file to be both.

5. Proposed architecture

docs/
  README.md              # existing index — gains ONE new bullet pointing at guide/
  guide/
    README.md             # short index, mirrors docs/README.md's own style
    getting-started.md    # clone → build → analyze → run --dry-run → smoke → real --n 1
    which-verb.md          # decision table: real vs ramp vs pool vs spawn-run vs session vs --shards
    cli-reference.md       # GENERATED — one section per verb/subcommand
    troubleshooting.md     # symptom → cause → fix, mined from CHANGELOG's real fixes

Rationale: keeps docs/README.md's existing "one design question per file" index uncontaminated by task-oriented HOW-TO content (different genre); root README.md gains links in rather than absorbing full content (already ~265 lines).

Cross-links: README's ## CLI section gets one pointer line to cli-reference.md; docs/README.md gets one new bullet; examples/README.md's "where to go next" links to getting-started.md; CONTRIBUTING.md gains one line about regenerating the reference on flag changes (if Option A adopted).

6. Open questions for a human decision

1. docs/guide/ subtree vs. top-level GETTING_STARTED.md (discoverability) vs. folding into README — recommend the subtree for consistency with this repo's stated "docs go in docs/" convention, but flagging the tradeoff.
2. Generator (A) vs. golden-file test (B) for the CLI reference — A is more idiomatic and produces a nicer artifact, but requires refactoring rampCmd and a few others to separate FlagSet construction from parse+validate. Real but bounded cost.
3. Should troubleshooting.md cover AWS quota/capacity/spot-interruption behavior too, or stay scoped to calque-CLI-specific gotchas only (that broader territory belongs to spawn/lagotto per README's acquisition-seam note)?
4. Fold aws-setup-checklist into getting-started.md's "Prerequisites" section, or keep it separate? Leaning toward folding.
5. Priority if this is staged: cli-reference.md's real section + getting-started.md close the most real pain first, since the newest (session's) flags are completely undocumented anywhere outside code.

---

Next step: a decision on the open questions above, then this can move from kind:design to an implementation pass.

[scttfrdmn]

Closing this out. The design pass in the earlier comment proposed the docs/guide/ subtree, and those files were written separately — but they'd never been fact-checked against the actual CLI/parser source, and this issue stayed open. Did that audit now (#183, merged as c31ac7f):

Fixed (real drift found):

• docs/guide/cli-reference.md's real flags table was missing --allow-card-swap (calque#178) entirely, and its --instance row claimed a literal g6.2xlarge default when the actual default is "" (falling back to g6.2xlarge unless --allow-card-swap substitutes a different card).
• docs/modal-compatibility-matrix.md's §H modal.Cron/modal.Period rows said the object forms were "not recognized as a distinct construct" (⬜) — stale. pyast.py's _schedule_marker + parse.go's decodeScheduleMarker already recognize both structurally (calque#91), same recorded-but-not-honored posture as the bare-string schedule= row. Fixed both rows plus the backlog item and provenance note.

Verified accurate, no changes needed: getting-started.md's IAM policy/tagging/role-reuse claims, which-verb.md's flag cross-refs, troubleshooting.md's bug citations, every other flag across run/smoke/ramp/pool/spawn-run/session, and all cross-links between README.md / docs/README.md / docs/guide/ / examples/README.md (all already wired up).

Deliberately deferred, not a blocker here: the original design comment's idea of auto-generating cli-reference.md from flag.FlagSet introspection (instead of hand-writing it) is a real idea worth doing, but it's its own separate, smaller follow-up — not required to close this issue, since the hand-written content has now been verified accurate against the current source.