diff --git a/.changeset/teach-spec-inventory-verb.md b/.changeset/teach-spec-inventory-verb.md new file mode 100644 index 0000000000..aad490682c --- /dev/null +++ b/.changeset/teach-spec-inventory-verb.md @@ -0,0 +1,7 @@ +--- +"@fission-ai/openspec": patch +--- + +Teach the generated guidance how to find and read a project's specs. `openspec list --specs` appeared in no generated skill, command, or artifact instruction, while `openspec list --json` (the in-flight *change* list) appeared throughout, so an agent asked to read the existing specs first enumerated changes instead and reported the step complete against the wrong object. The explore skill and command now list the spec inventory alongside the change list and say which is which, and the spec-driven `proposal` and `specs` instructions name the command where they ask for existing capabilities to be researched and for a delta's path to match an existing one. Both steps carry `--store ""`, and capabilities are read with `openspec show "" --type spec --json --no-scenarios` so the read resolves against the same root the listing came from. Fixes #1689. + +The filtered read is only an overview. Agents read relevant specs in full, including scenarios, before deciding what is already covered or what should change. diff --git a/schemas/spec-driven/schema.yaml b/schemas/spec-driven/schema.yaml index 1431c5ebb0..e03102ee0d 100644 --- a/schemas/spec-driven/schema.yaml +++ b/schemas/spec-driven/schema.yaml @@ -18,7 +18,19 @@ artifacts: - **Impact**: Affected code, APIs, dependencies, or systems. IMPORTANT: The Capabilities section is critical. It creates the contract between - proposal and specs phases. Research existing specs before filling this in. + proposal and specs phases. Research existing specs before filling this in: + run `openspec list --specs` for the project's capability inventory, then + `openspec show "" --type spec --json --no-scenarios` for any that + look related - that returns a capability's purpose and requirement texts + without pulling whole spec files into context. Append `--store ""` to + both commands only for a registered standalone store, and keep `--type + spec`: a change and a spec sharing a name is otherwise an ambiguous-item + error. `openspec list` without `--specs` lists in-flight changes, not + specs - it never shows what the project already covers. Reuse an existing + capability's exact path instead of introducing a near-duplicate name. + The filtered read is only an overview. Before deciding what is already + covered or what should change, read each relevant spec in full, including + scenarios, with `openspec show "" --type spec` (same `--store` rule). Each capability listed here will need a corresponding spec file. Every change must either declare at least one capability (new or @@ -63,7 +75,7 @@ artifacts: `` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path: - New capabilities: use the exact path from the proposal at `specs//spec.md`. Any path segment newly introduced in the proposal must be kebab-case. Follow the project's existing organization; do not add a new domain level when the project uses a flat layout. - - Modified capabilities: use the exact existing path from `openspec/specs//` when creating the delta at `specs//spec.md`. Do not move or rename the capability. + - Modified capabilities: use the exact existing path from `openspec/specs//` when creating the delta at `specs//spec.md`. Run `openspec list --specs` to confirm that path before writing the delta, appending `--store ""` only for a registered standalone store - a mistyped or invented path targets a capability that does not exist rather than the one you meant. Do not move or rename the capability. There must be at least one spec file unless the change's `.openspec.yaml` sets `skip_specs: true` (no spec-level behavior change) - `openspec validate` diff --git a/skills/openspec-explore/SKILL.md b/skills/openspec-explore/SKILL.md index b597796fb1..2ff15bfed1 100644 --- a/skills/openspec-explore/SKILL.md +++ b/skills/openspec-explore/SKILL.md @@ -120,6 +120,14 @@ This tells you: - Their names, schemas, and status - What the user might be working on +That is the *change* list - work in flight. It does not include the project's durable capabilities, so list those too: +```bash +openspec list --specs +``` +Add `--json` for ids and requirement counts, and append `--store ""` only for a registered standalone store. This is the inventory of what the project already claims to do, and `openspec list` on its own never shows it. To look at one, run `openspec show "" --type spec --json --no-scenarios` (same `--store` rule) - it returns that capability's purpose and requirement texts without pulling the whole spec file into context, and `--type spec` stops a change of the same name from making it ambiguous. + +The filtered read is only an overview. Before deciding what is already covered or what should change, read each relevant spec in full, including scenarios, with `openspec show "" --type spec` (same `--store` rule). + Then read the project's own context from the resolved root - `/openspec/config.yaml` (or `config.yml`). Use the `root.path` returned above, and skip this if neither file exists: - `context`: project background - tech stack, conventions, constraints - `rules`: keyed by artifact id - the entries for an artifact apply only when you write that artifact diff --git a/src/core/templates/workflows/explore.ts b/src/core/templates/workflows/explore.ts index 8954a2cce4..20db318293 100644 --- a/src/core/templates/workflows/explore.ts +++ b/src/core/templates/workflows/explore.ts @@ -124,6 +124,14 @@ This tells you: - Their names, schemas, and status - What the user might be working on +That is the *change* list - work in flight. It does not include the project's durable capabilities, so list those too: +\`\`\`bash +openspec list --specs +\`\`\` +Add \`--json\` for ids and requirement counts, and append \`--store ""\` only for a registered standalone store. This is the inventory of what the project already claims to do, and \`openspec list\` on its own never shows it. To look at one, run \`openspec show "" --type spec --json --no-scenarios\` (same \`--store\` rule) - it returns that capability's purpose and requirement texts without pulling the whole spec file into context, and \`--type spec\` stops a change of the same name from making it ambiguous. + +The filtered read is only an overview. Before deciding what is already covered or what should change, read each relevant spec in full, including scenarios, with \`openspec show "" --type spec\` (same \`--store\` rule). + Then read the project's own context from the resolved root - \`/openspec/config.yaml\` (or \`config.yml\`). Use the \`root.path\` returned above, and skip this if neither file exists: - \`context\`: project background - tech stack, conventions, constraints - \`rules\`: keyed by artifact id - the entries for an artifact apply only when you write that artifact @@ -446,6 +454,14 @@ This tells you: - Their names, schemas, and status - What the user might be working on +That is the *change* list - work in flight. It does not include the project's durable capabilities, so list those too: +\`\`\`bash +openspec list --specs +\`\`\` +Add \`--json\` for ids and requirement counts, and append \`--store ""\` only for a registered standalone store. This is the inventory of what the project already claims to do, and \`openspec list\` on its own never shows it. To look at one, run \`openspec show "" --type spec --json --no-scenarios\` (same \`--store\` rule) - it returns that capability's purpose and requirement texts without pulling the whole spec file into context, and \`--type spec\` stops a change of the same name from making it ambiguous. + +The filtered read is only an overview. Before deciding what is already covered or what should change, read each relevant spec in full, including scenarios, with \`openspec show "" --type spec\` (same \`--store\` rule). + Then read the project's own context from the resolved root - \`/openspec/config.yaml\` (or \`config.yml\`). Use the \`root.path\` returned above, and skip this if neither file exists: - \`context\`: project background - tech stack, conventions, constraints - \`rules\`: keyed by artifact id - the entries for an artifact apply only when you write that artifact diff --git a/test/commands/store-root-selection.test.ts b/test/commands/store-root-selection.test.ts index 314ce44da8..b601cc7aa5 100644 --- a/test/commands/store-root-selection.test.ts +++ b/test/commands/store-root-selection.test.ts @@ -10,6 +10,7 @@ import { import { writeStoreMetadataState } from '../../src/core/store/foundation.js'; import { runCLI, type RunCLIResult } from '../helpers/run-cli.js'; import { cleanupTempPath } from '../helpers/temp-cleanup.js'; +import { writeSpec } from '../helpers/openspec-fixtures.js'; const VALID_DELTA_SPEC = `## ADDED Requirements @@ -123,6 +124,72 @@ describe('store root selection for normal commands', () => { expect(fs.existsSync(path.join(appRepo, 'openspec'))).toBe(false); } + it.each(['local', 'store', 'declared', 'global_default'] as const)( + 'discovers and reads capabilities in the %s root using the generated guidance (#1689)', + async (source) => { + const selectedRoot = source === 'local' ? appRepo : storeRoot; + const storeArgs = source === 'store' ? ['--store', 'team-context'] : []; + if (source === 'local' || source === 'store') { + createOpenSpecRoot(appRepo); + } else if (source === 'declared') { + fs.mkdirSync(path.join(appRepo, 'openspec'), { recursive: true }); + fs.writeFileSync(path.join(appRepo, 'openspec', 'config.yaml'), 'store: team-context\n'); + } else { + const configDir = path.join(tempDir, 'config', 'openspec'); + fs.mkdirSync(configDir, { recursive: true }); + fs.writeFileSync(path.join(configDir, 'config.json'), JSON.stringify({ defaultStore: 'team-context' })); + } + + const spec = '# Billing\n\n## Purpose\nBills from the selected root.\n\n## Requirements\n\n### Requirement: Billing\nThe system SHALL bill.\n\n#### Scenario: Bills\n- **WHEN** due\n- **THEN** billed\n'; + writeSpec(selectedRoot, 'billing', spec); + writeSpec(selectedRoot, 'billing/invoices', spec); + createChange(selectedRoot, 'billing'); + if (source === 'store') { + // A missing --store on the read must not silently return local content. + writeSpec(appRepo, 'billing', spec.replace('SHALL bill', 'SHALL use local billing')); + writeSpec(appRepo, 'local-only', spec); + } + + const changes = await runCLI(['list', '--json', ...storeArgs], { cwd: appRepo, env }); + expect(changes.exitCode).toBe(0); + expect(parseJson(changes).changes.map((change: any) => change.name)).toEqual(['billing']); + + const inventory = await runCLI(['list', '--specs', '--json', ...storeArgs], { cwd: appRepo, env }); + expect(inventory.exitCode).toBe(0); + const json = parseJson(inventory); + expect(json.specs).toEqual([ + { id: 'billing', requirementCount: 1 }, + { id: 'billing/invoices', requirementCount: 1 }, + ]); + expect(json.root).toEqual({ + path: selectedRoot, + source: source === 'local' ? 'nearest' : source, + ...(source === 'local' ? {} : { store_id: 'team-context' }), + }); + + for (const { id } of json.specs) { + const shown = await runCLI( + ['show', id, '--type', 'spec', '--json', '--no-scenarios', ...storeArgs], + { cwd: appRepo, env } + ); + expect(shown.exitCode).toBe(0); + expect(parseJson(shown)).toMatchObject({ + id, + overview: 'Bills from the selected root.', + requirementCount: 1, + requirements: [{ text: 'The system SHALL bill.', scenarios: [] }], + root: json.root, + }); + + // The overview omits scenarios; decisions use the complete spec. + const full = await runCLI(['show', id, '--type', 'spec', ...storeArgs], { cwd: appRepo, env }); + expect(full.exitCode).toBe(0); + expect(full.stdout.trim()).toBe(spec.trim()); + } + }, + 30_000 + ); + describe('selecting a registered store by id', () => { it('creates a change only in the store and names the root on stderr', async () => { const result = await runCLI(['new', 'change', 'add-billing', '--store', 'team-context'], { @@ -315,24 +382,6 @@ operations: expectNoLocalOpenSpec(); }); - it('lists specs from the store with minimal JSON support', async () => { - const specDir = path.join(storeRoot, 'openspec', 'specs', 'billing'); - fs.mkdirSync(specDir, { recursive: true }); - fs.writeFileSync( - path.join(specDir, 'spec.md'), - '# billing\n\n## Purpose\nBills.\n\n## Requirements\n\n### Requirement: Billing SHALL work\nThe system SHALL bill.\n\n#### Scenario: Bills\n- **WHEN** due\n- **THEN** billed\n' - ); - - const result = await runCLI(['list', '--specs', '--json', '--store', 'team-context'], { - cwd: appRepo, - env, - }); - expect(result.exitCode).toBe(0); - const json = parseJson(result); - expect(json.specs).toEqual([{ id: 'billing', requirementCount: 1 }]); - expect(json.root.store_id).toBe('team-context'); - }); - it('runs bulk validation against the selected store', async () => { createChange(storeRoot, 'store-change'); diff --git a/test/core/templates/skill-templates-parity.test.ts b/test/core/templates/skill-templates-parity.test.ts index 3f309bdaa3..4abe4b020a 100644 --- a/test/core/templates/skill-templates-parity.test.ts +++ b/test/core/templates/skill-templates-parity.test.ts @@ -38,14 +38,14 @@ import { import { STORE_SELECTION_GUIDANCE } from '../../../src/core/templates/workflows/store-selection.js'; const EXPECTED_FUNCTION_HASHES: Record = { - getExploreSkillTemplate: '6315fcc5c2eb848963bc8bca4c23e657412a99608e610daee59fb4e58cd21fd4', + getExploreSkillTemplate: '06aba775c621e61f00995a9ebc3a02fe873ddcc9bf024e416c4adaf91ccce115', getNewChangeSkillTemplate: 'eabd1e895c5881dcb17dcbaa3fb26098dd59e8eacb318e400820b4dc811ef781', getContinueChangeSkillTemplate: '012136f6411a99c8fa228e2f9444cb64b0a89e0f56fdeac2fe03b2f5bee0c5d7', getApplyChangeSkillTemplate: 'd1e7d5ceb85193c0964057dbb88e9651526754bd33f84020e2440ff0621d5dbb', getFfChangeSkillTemplate: 'efa6a70c111b18b61a7720250b9622afa9a212fb64edf609cf80e2182a9bdf8c', getSyncSpecsSkillTemplate: 'b099e2ff31859c9b10d928066e662524f9aad9ecf2be12fceacb732d718c4146', getOnboardSkillTemplate: '3a836faae463d88c289a1c129cb7ee556a563b7e53e1a52a4711ff152a3b51f7', - getOpsxExploreCommandTemplate: 'b4706a5b8fd280f7929eea610ecc9d41676b2d2dd6653d259cbbc2bfe01813d9', + getOpsxExploreCommandTemplate: '8046003e97d885a86ed392d4fb522bb78544a02872b042e51347a5021cc10523', getOpsxNewCommandTemplate: 'f2d30e569798a4c92ba932859d6ba4e0ad10e18feccbade1cfee0957597b3463', getOpsxContinueCommandTemplate: 'e50e50266efa1b8e64ff9b6274ee8254f0a240d6adc1b862d126e2f1c9d3a559', getOpsxApplyCommandTemplate: 'e3579ac78f2e2c75fa3d3a7ac7dc3e49c395e96f7323398f0f041d94f8de9bb0', @@ -66,7 +66,7 @@ const EXPECTED_FUNCTION_HASHES: Record = { }; const EXPECTED_GENERATED_SKILL_CONTENT_HASHES: Record = { - 'openspec-explore': 'dd84af68d3c93b40659dcdd8d383423b25b443cacdc4b514cd70614ae10c5cac', + 'openspec-explore': '32b20cfbcc7d51ff526bb19571ff3dc3d0c616a5911b8de74cf6d9b15650cf3e', 'openspec-new-change': 'ec4529beef978e34634a6f7286fab55d68fad8fb374dceb45691d52caab33fbb', 'openspec-continue-change': 'bb6194a16c54891cdb253678e8f70ce53b2af86735243980f366ce551d37e42e', 'openspec-apply-change': '81ea96d9fa6ec8536cd23c1fe561ed28e1cc1cad0a8ceb700588e08974cc0e49', diff --git a/test/core/templates/spec-inventory.test.ts b/test/core/templates/spec-inventory.test.ts new file mode 100644 index 0000000000..b1dcdf890a --- /dev/null +++ b/test/core/templates/spec-inventory.test.ts @@ -0,0 +1,157 @@ +import path from 'path'; +import { fileURLToPath } from 'url'; +import { describe, expect, it } from 'vitest'; + +import { + getSkillTemplates, + getCommandTemplates, +} from '../../../src/core/shared/skill-generation.js'; +import { + getExploreSkillTemplate, + getOpsxExploreCommandTemplate, +} from '../../../src/core/templates/skill-templates.js'; +import { loadSchema } from '../../../src/core/artifact-graph/schema.js'; + +// #1689: 1.9.0 removed openspec/AGENTS.md, which carried the spec index, and +// nothing that replaced it ever named the verb that lists specs. Measured +// across one repo's generated surfaces: `openspec list --json` (the CHANGE +// list) appeared 10 times, `openspec list --specs` zero times. An agent told +// to "read the existing specs first" reaches for the one enumeration verb it +// was taught, gets the in-flight change list, and reports the step complete +// against the wrong object. +const SPEC_INVENTORY = 'openspec list --specs'; +const SPEC_READ = 'openspec show "" --type spec --json --no-scenarios'; + +// Assertions about the guidance attached to the command are scoped to a window +// after it rather than to the whole body, so an unrelated occurrence elsewhere +// in a long template cannot stand in for the passage under test. +const PASSAGE_WINDOW = 700; + +const repoRoot = path.resolve(fileURLToPath(new URL('.', import.meta.url)), '../../..'); +const defaultSchema = loadSchema(path.join(repoRoot, 'schemas', 'spec-driven', 'schema.yaml')); + +function instructionFor(artifactId: string): string { + const artifact = defaultSchema.artifacts.find(entry => entry.id === artifactId); + expect(artifact, `spec-driven has no "${artifactId}" artifact`).toBeDefined(); + const instruction = artifact?.instruction; + expect(instruction, `spec-driven "${artifactId}" has no instruction`).toBeDefined(); + return instruction as string; +} + +const exploreBodies: Array<[string, string]> = [ + ['explore skill', getExploreSkillTemplate().instructions], + ['explore command', getOpsxExploreCommandTemplate().content], +]; + +describe('spec inventory vocabulary (#1689)', () => { + it('teaches the spec-inventory verb somewhere in the generated surfaces', () => { + const bodies = [ + ...getSkillTemplates().map(entry => entry.template.instructions), + ...getCommandTemplates().map(entry => entry.template.content), + ]; + + const carriers = bodies.filter(body => body.includes(SPEC_INVENTORY)); + expect( + carriers.length, + `no generated skill or command names "${SPEC_INVENTORY}", so the spec inventory is unreachable by any path the tool teaches` + ).toBeGreaterThan(0); + }); + + it('names the spec inventory in explore, where the agent orients', () => { + for (const [label, body] of exploreBodies) { + expect(body, label).toContain(SPEC_INVENTORY); + } + }); + + it('distinguishes the change list from the spec inventory in explore', () => { + // Naming the command is not enough on its own: `openspec list` defaults to + // changes, so the two enumerations have to be told apart explicitly. + for (const [label, body] of exploreBodies) { + expect(body, label).toContain('openspec list --json'); + expect(body, label).toContain('`openspec list` on its own never shows it'); + } + }); + + it('names the spec inventory where the proposal picks capabilities', () => { + // "Research existing specs before filling this in" named no command, which + // is how the Capabilities section ends up inventing a near-duplicate + // capability instead of reusing the existing one. + expect(instructionFor('proposal')).toContain(SPEC_INVENTORY); + }); + + it('names the spec inventory where a delta must match an existing path', () => { + expect(instructionFor('specs')).toContain(SPEC_INVENTORY); + }); + + // A bare `openspec list --specs` reads the local inventory, so under a + // selected store it confirms a capability path against the wrong root. + // Every site that names the command must carry the store qualifier with it. + it('carries the store qualifier everywhere it names the command', () => { + const sites: Array<[string, string]> = [ + ...exploreBodies, + ['proposal instruction', instructionFor('proposal')], + ['specs instruction', instructionFor('specs')], + ]; + + for (const [label, body] of sites) { + const start = body.indexOf(SPEC_INVENTORY); + expect(start, label).toBeGreaterThanOrEqual(0); + + // Scoped to the passage that names the command: every explore body + // already carries the store qualifier in its unrelated capture steps, + // so a whole-body match would pass even with the qualifier dropped here. + const passage = body.slice(start, start + PASSAGE_WINDOW); + expect(passage, `${label} names the command without its store qualifier`).toContain( + 'registered standalone store' + ); + expect(passage, label).toContain('--store ""'); + } + }); + + // Reading the inventory back by raw path defeats the fix under a store: the + // ids `list --specs --store ` returns are not present under the local + // `openspec/specs/`, so the read either fails or silently lands on a + // same-named local capability - the wrong-object failure #1689 is about. + // `openspec show` resolves against the same root the listing came from. + it('reads a listed capability with the store-aware command', () => { + const sites: Array<[string, string]> = [ + ...exploreBodies, + ['proposal instruction', instructionFor('proposal')], + ]; + + for (const [label, body] of sites) { + const start = body.indexOf(SPEC_INVENTORY); + const passage = body.slice(start, start + PASSAGE_WINDOW); + // Pin the complete low-context read. Each flag is load-bearing: --type + // disambiguates a same-named change, JSON makes the result structured, + // and --no-scenarios avoids pulling every scenario into context. + expect(passage, `${label} does not name the complete store-aware read`).toContain( + SPEC_READ + ); + + // Tie the conditional store qualifier to the read itself. A separate + // --store mention for the inventory list must not let a local-root read + // pass this guard. + const readStart = passage.indexOf(SPEC_READ); + const readContext = passage.slice(readStart, readStart + 350); + expect(readContext, `${label} does not apply the store rule to the read`).toMatch( + /(?:same `--store` rule|Append `--store ""` to\s+both commands only for a registered standalone store)/ + ); + } + }); + + it('reads full relevant specs before deciding coverage or changes', () => { + const sites: Array<[string, string]> = [ + ...exploreBodies, + ['proposal instruction', instructionFor('proposal')], + ]; + + for (const [label, body] of sites) { + const normalized = body.replace(/\s+/g, ' '); + expect(normalized, label).toContain('The filtered read is only an overview.'); + expect(normalized, label).toContain( + 'Before deciding what is already covered or what should change, read each relevant spec in full, including scenarios, with `openspec show "" --type spec` (same `--store` rule).' + ); + } + }); +});