Summary
Documentation audit across all 23 tracked markdown files. Several docs reference files that no longer exist, describe removed features, or will go stale when #32 (Lakebase Autoscaling migration) merges. Filing as a separate task so #32 stays scoped to code.
Findings are ordered by whether they actively mislead a user following instructions.
Tier 1 — broken instructions (a user following these hits a dead end)
1. plugin/skills/agentops-stacks/SKILL.md:149-151 links to three examples that don't exist
The component-choice table links all three:
| line |
link |
| 149 |
examples/simple-rag/ |
| 150 |
examples/multi-agent/ |
| 151 |
examples/hello-agent/ |
examples/ contains only README.md. hello-agent was deleted in e66c5ca ("drop stale hello-agent example"); the other two were never added. examples/README.md itself correctly says "Examples are coming" — so the skill contradicts the directory it points into.
Fix: drop the links, or point at examples/README.md until examples land.
2. Generated README.md:49 tells users to run an installer that isn't in their project
Install with ./plugin/skills/install_skills.sh or run the install_genie_code_skills.py notebook.
Neither path exists in a generated project — plugin/ is not part of the template output (verified by generating a project and checking). Both files live in this repo, not the scaffold.
Fix: in template/{{.input_root_dir}}/README.md.tmpl, reference the plugin by repo URL rather than a relative path.
3. plugin/skills/agentops-lifecycle/SKILL.md references three docs that are never created
docs/monitoring-runbook.md, docs/production_baseline.md, docs/sme_calibration.md — none exist in this repo or in generated output. If these are outputs the lifecycle skill is supposed to produce, the phrasing should say so.
Tier 2 — stale after #32 merges
4. plugin/skills/agentops-lifecycle/SKILL.md:234 still shows the old model endpoint
LLM_ENDPOINT = os.environ.get("LLM_ENDPOINT", "databricks-claude-sonnet-4")
#32 bumps the template to databricks-claude-sonnet-4-5 in graph.py.tmpl, app.yaml.tmpl, and .env.example.tmpl, but not here. This is the only remaining sonnet-4 reference after #32.
5. docs/setup.md.tmpl + lakebase-ops/SKILL.md claim staging/prod are "always-on"
Both state the template "leaves staging/prod always-on". Nothing in #32 sets no_suspension: true — it appears only in prose. Staging/prod inherit the project default suspension.
Fix: either add no_suspension: true to those targets (code change) or soften the docs to describe the actual default.
6. plugin/skills/agentops-stacks/reference/common-issues.md has no Lakebase entries
#32 documents two genuinely non-obvious failure modes — the CREATE ON SCHEMA public grant, and the 7-day soft-delete slug reservation blocking redeploy. Both currently live only in the generated docs/setup.md and lakebase-ops. common-issues.md is where the assistant looks first during scaffolding, so at minimum it should cross-reference them.
7. lakebase-ops/SKILL.md step numbering skips
Sections run 1, 2, 3, 4, 5a, 6, 7, 8 — there's no 5. Pre-existing (main also jumps 4 → 6); #32 adds 5a into the gap rather than renumbering. Cosmetic, but the file is otherwise carefully structured.
Tier 3 — coverage gaps
8. README.md and agentops-lifecycle/SKILL.md don't mention input_memory_type
Compared every doc's input_* list against databricks_template_schema.json (12 inputs). Missing:
README.md — omits memory_type, has_chunked_table, uc_functions_exist
agentops-lifecycle/SKILL.md — omits the same three plus root_dir; its example config block lists only 8 of 12 inputs
memory_type matters most: after #32 it selects between three materially different scaffolds. Only agentops-stacks/SKILL.md documents it today. (The partial config block still works — defaults fill the gaps — so this is completeness, not breakage.)
9. Governance-doc references are aspirational, phrased as present tense
README.md:46, AGENTS.md:77, AGENTS.md:98, docs/README.md:11 (all generated) reference governance/posture.md and governance/data_flows.md, which the scaffold does not create. The prod-promotion workflow does conditionally check for governance/posture.md (-bundle-cd-prod.yml:44), so the mechanism is real and the files are opt-in — README.md:46 gets this right ("apply by adding"), but AGENTS.md:77 reads as though they ship.
Docs verified clean
AGENTS.md (root), plugin/README.md, plugin/commands/*.md, add-agent, uc-functions-ops, vector-search-ops, genie-code.md, post-scaffold.md, workflows/README.md, the three CI/CD README.md.tmpls, and generated docs/README.md.
Method
- Enumerated all 23 tracked
.md/.md.tmpl files
- Generated projects for every
memory_type variant and checked each documented path against real output
- Diffed each doc's
input_* references against the schema
- Extracted backticked repo paths and resolved them against
git ls-tree
Summary
Documentation audit across all 23 tracked markdown files. Several docs reference files that no longer exist, describe removed features, or will go stale when #32 (Lakebase Autoscaling migration) merges. Filing as a separate task so #32 stays scoped to code.
Findings are ordered by whether they actively mislead a user following instructions.
Tier 1 — broken instructions (a user following these hits a dead end)
1.
plugin/skills/agentops-stacks/SKILL.md:149-151links to three examples that don't existThe component-choice table links all three:
examples/simple-rag/examples/multi-agent/examples/hello-agent/examples/contains onlyREADME.md.hello-agentwas deleted in e66c5ca ("drop stale hello-agent example"); the other two were never added.examples/README.mditself correctly says "Examples are coming" — so the skill contradicts the directory it points into.Fix: drop the links, or point at
examples/README.mduntil examples land.2. Generated
README.md:49tells users to run an installer that isn't in their projectNeither path exists in a generated project —
plugin/is not part of the template output (verified by generating a project and checking). Both files live in this repo, not the scaffold.Fix: in
template/{{.input_root_dir}}/README.md.tmpl, reference the plugin by repo URL rather than a relative path.3.
plugin/skills/agentops-lifecycle/SKILL.mdreferences three docs that are never createddocs/monitoring-runbook.md,docs/production_baseline.md,docs/sme_calibration.md— none exist in this repo or in generated output. If these are outputs the lifecycle skill is supposed to produce, the phrasing should say so.Tier 2 — stale after #32 merges
4.
plugin/skills/agentops-lifecycle/SKILL.md:234still shows the old model endpoint#32 bumps the template to
databricks-claude-sonnet-4-5ingraph.py.tmpl,app.yaml.tmpl, and.env.example.tmpl, but not here. This is the only remainingsonnet-4reference after #32.5.
docs/setup.md.tmpl+lakebase-ops/SKILL.mdclaim staging/prod are "always-on"Both state the template "leaves staging/prod always-on". Nothing in #32 sets
no_suspension: true— it appears only in prose. Staging/prod inherit the project default suspension.Fix: either add
no_suspension: trueto those targets (code change) or soften the docs to describe the actual default.6.
plugin/skills/agentops-stacks/reference/common-issues.mdhas no Lakebase entries#32 documents two genuinely non-obvious failure modes — the
CREATE ON SCHEMA publicgrant, and the 7-day soft-delete slug reservation blocking redeploy. Both currently live only in the generateddocs/setup.mdandlakebase-ops.common-issues.mdis where the assistant looks first during scaffolding, so at minimum it should cross-reference them.7.
lakebase-ops/SKILL.mdstep numbering skipsSections run 1, 2, 3, 4, 5a, 6, 7, 8 — there's no
5. Pre-existing (mainalso jumps 4 → 6); #32 adds5ainto the gap rather than renumbering. Cosmetic, but the file is otherwise carefully structured.Tier 3 — coverage gaps
8.
README.mdandagentops-lifecycle/SKILL.mddon't mentioninput_memory_typeCompared every doc's
input_*list againstdatabricks_template_schema.json(12 inputs). Missing:README.md— omitsmemory_type,has_chunked_table,uc_functions_existagentops-lifecycle/SKILL.md— omits the same three plusroot_dir; its example config block lists only 8 of 12 inputsmemory_typematters most: after #32 it selects between three materially different scaffolds. Onlyagentops-stacks/SKILL.mddocuments it today. (The partial config block still works — defaults fill the gaps — so this is completeness, not breakage.)9. Governance-doc references are aspirational, phrased as present tense
README.md:46,AGENTS.md:77,AGENTS.md:98,docs/README.md:11(all generated) referencegovernance/posture.mdandgovernance/data_flows.md, which the scaffold does not create. The prod-promotion workflow does conditionally check forgovernance/posture.md(-bundle-cd-prod.yml:44), so the mechanism is real and the files are opt-in —README.md:46gets this right ("apply by adding"), butAGENTS.md:77reads as though they ship.Docs verified clean
AGENTS.md(root),plugin/README.md,plugin/commands/*.md,add-agent,uc-functions-ops,vector-search-ops,genie-code.md,post-scaffold.md,workflows/README.md, the three CI/CDREADME.md.tmpls, and generateddocs/README.md.Method
.md/.md.tmplfilesmemory_typevariant and checked each documented path against real outputinput_*references against the schemagit ls-tree