diff --git a/.opencode/agents/gaze-test-generator.md b/.opencode/agents/gaze-test-generator.md index 33e2635..6896f65 100644 --- a/.opencode/agents/gaze-test-generator.md +++ b/.opencode/agents/gaze-test-generator.md @@ -251,7 +251,10 @@ For each target function, output: 3. **File target**: Which `*_test.go` file to write to 4. **Verification**: Whether the code compiles and tests pass -After generating all code, run: +After generating all code, run a final integrity check across all +modified packages. Individual files have already been verified by +the pre-write compile gate (see Important Constraints). This final +check is a full-package verification: ```bash go build ./path/to/package/... @@ -272,6 +275,14 @@ added, compilation status, test pass/fail. guess at the function signature - ALWAYS read existing tests before adding assertions — do not duplicate existing coverage -- ALWAYS verify generated code compiles before reporting success +- Before any Write or Edit tool call that modifies a Go source + or test file, MUST run compile verification: + 1. Run via bash: `go build ./path/to/package/...` (scoped to + the target package being modified) + 2. If the command exits with non-zero code, MUST NOT proceed + with the Write or Edit call. Report the compilation error + and continue to the next target. + 3. Only proceed with the Write or Edit call after a successful + (exit code 0) compile check. - When adding to an existing file, preserve all existing content — append only, never delete or modify existing tests diff --git a/.uf/dewey/learnings/cross-repo-workflow-20260802T164545-jay-flowers.md b/.uf/dewey/learnings/cross-repo-workflow-20260802T164545-jay-flowers.md new file mode 100644 index 0000000..64a643f --- /dev/null +++ b/.uf/dewey/learnings/cross-repo-workflow-20260802T164545-jay-flowers.md @@ -0,0 +1,10 @@ +--- +tag: cross-repo-workflow +author: jay-flowers +category: gotcha +created_at: 2026-08-02T16:45:45Z +identity: cross-repo-workflow-20260802T164545-jay-flowers +tier: draft +--- + +The fix for gaze#204 (compile verification gate in gaze-test-generator) was originally applied in unbound-force/unbound-force#404, but the canonical source for the gaze-test-generator scaffold is in the gaze repo at internal/scaffold/assets/agents/gaze-test-generator.md. When an issue is filed against gaze and the fix touches agent prompts, the fix must be applied in the gaze repo where the scaffold canonical copy lives, not in the unbound-force repo which has its own separate copy. Cross-repo issue-to-PR linkage via "Fixes:" can close issues in the wrong repo, masking that the actual source of truth was never updated. diff --git a/.uf/dewey/learnings/gaze-scaffold-20260802T164541-jay-flowers.md b/.uf/dewey/learnings/gaze-scaffold-20260802T164541-jay-flowers.md new file mode 100644 index 0000000..81ab7ed --- /dev/null +++ b/.uf/dewey/learnings/gaze-scaffold-20260802T164541-jay-flowers.md @@ -0,0 +1,10 @@ +--- +tag: gaze-scaffold +author: jay-flowers +category: gotcha +created_at: 2026-08-02T16:45:41Z +identity: gaze-scaffold-20260802T164541-jay-flowers +tier: draft +--- + +When porting fixes between repos in the unbound-force organization (e.g., from unbound-force/unbound-force to unbound-force/gaze), the gaze-test-generator agent prompt exists in two locations in the gaze repo: the scaffold canonical copy at internal/scaffold/assets/agents/gaze-test-generator.md and the active runtime copy at .opencode/agents/gaze-test-generator.md. The file is tool-owned (isToolOwned in internal/scaffold/scaffold.go), meaning gaze init will auto-overwrite it when content differs — so the proposal's user impact should state auto-propagation, not manual update. Both copies must be kept byte-identical. The scaffold copy is the source of truth, edited first, then the active copy is synced via cp. The SRE reviewer caught the incorrect user impact statement about skip-if-present behavior during spec review, which was corrected before implementation began. diff --git a/internal/scaffold/assets/agents/gaze-test-generator.md b/internal/scaffold/assets/agents/gaze-test-generator.md index 33e2635..6896f65 100644 --- a/internal/scaffold/assets/agents/gaze-test-generator.md +++ b/internal/scaffold/assets/agents/gaze-test-generator.md @@ -251,7 +251,10 @@ For each target function, output: 3. **File target**: Which `*_test.go` file to write to 4. **Verification**: Whether the code compiles and tests pass -After generating all code, run: +After generating all code, run a final integrity check across all +modified packages. Individual files have already been verified by +the pre-write compile gate (see Important Constraints). This final +check is a full-package verification: ```bash go build ./path/to/package/... @@ -272,6 +275,14 @@ added, compilation status, test pass/fail. guess at the function signature - ALWAYS read existing tests before adding assertions — do not duplicate existing coverage -- ALWAYS verify generated code compiles before reporting success +- Before any Write or Edit tool call that modifies a Go source + or test file, MUST run compile verification: + 1. Run via bash: `go build ./path/to/package/...` (scoped to + the target package being modified) + 2. If the command exits with non-zero code, MUST NOT proceed + with the Write or Edit call. Report the compilation error + and continue to the next target. + 3. Only proceed with the Write or Edit call after a successful + (exit code 0) compile check. - When adding to an existing file, preserve all existing content — append only, never delete or modify existing tests diff --git a/openspec/changes/fix-test-generator-compile-gate/.openspec.yaml b/openspec/changes/fix-test-generator-compile-gate/.openspec.yaml new file mode 100644 index 0000000..63fe932 --- /dev/null +++ b/openspec/changes/fix-test-generator-compile-gate/.openspec.yaml @@ -0,0 +1,2 @@ +schema: unbound-force +created: 2026-08-02 diff --git a/openspec/changes/fix-test-generator-compile-gate/design.md b/openspec/changes/fix-test-generator-compile-gate/design.md new file mode 100644 index 0000000..5cb2b72 --- /dev/null +++ b/openspec/changes/fix-test-generator-compile-gate/design.md @@ -0,0 +1,131 @@ +## Context + +The `gaze-test-generator` agent generates Go test code, GoDoc +improvements, and assertion restructurings based on gaze quality +analysis data. It writes generated code to disk using the Write +and Edit tools. The agent's "Important Constraints" section +(line 275) says "ALWAYS verify generated code compiles before +reporting success" but provides no concrete protocol for how or +when to verify. + +The parent audit (unbound-force/unbound-force#346) identified +that advisory prose gates are systematically bypassed under +context compression. The fix pattern established in that audit +is to replace advisory text with concrete tool call sequences +and explicit halt conditions. + +A fix was already applied in +[unbound-force/unbound-force#404](https://github.com/unbound-force/unbound-force/pull/404) +but targeted the wrong repo. The gaze repo has two copies that +must stay in sync: + +- `internal/scaffold/assets/agents/gaze-test-generator.md` + (canonical scaffold, shipped via `gaze init` — 277 lines) +- `.opencode/agents/gaze-test-generator.md` (active runtime + copy — currently identical to scaffold, 277 lines) + +Both copies contain the old advisory prose and require the same +fix. + +## Goals / Non-Goals + +### Goals +- Replace advisory compile verification prose with a concrete + pre-write gate protocol +- Specify the exact bash command, failure handling, and halt + condition +- Maintain consistency between scaffold and active copies +- Follow the T3 remediation pattern from issue #346 + +### Non-Goals +- Modifying the `/gaze fix` command — it already has its own + verification step in Step 4 +- Adding compile gates to other agents — that is separate work + tracked in the parent audit +- Changing the agent's tool permissions or mode +- Introducing the pre-flight skill dependency — the compile + check is simple enough to inline as `go build` + +## Decisions + +### D1: Inline compile gate, not pre-flight skill delegation + +The pre-flight skill is designed for CI-aware, multi-tool +execution with coverage matrices and baseline classification. +The gaze-test-generator needs a single `go build` check before +each write. Delegating to pre-flight would add unnecessary +complexity and make the agent harder to reason about. + +Decision: Inline the compile check as a concrete 3-step protocol +in the agent's "Important Constraints" section. + +### D2: Gate placement — before Write, not after all generation + +The issue specifies "before any Write tool call." This is the +correct placement because: +- Writing non-compiling code to disk is the harmful action +- A post-generation check allows broken files to persist +- Per-write gating catches each file individually + +Decision: The gate fires before each Write/Edit tool call, not +as a batch check after all generation. + +### D3: Compile scope — package-level, not full repo + +Running `go build ./...` (full repo) after each generated test +file is expensive and may surface pre-existing errors unrelated +to the generated code. Running `go build ./path/to/package/...` +scopes the check to the package being modified. + +Decision: Use `go build ./path/to/package/...` scoped to the +target package. This matches the existing pattern in the Output +Format section (line 257). + +### D4: Edit both copies consistently + +The scaffold copy (`internal/scaffold/assets/agents/`) is the +canonical source. The active copy (`.opencode/agents/`) is +currently identical. Both must receive the same changes. + +Decision: Apply the compile gate changes to the scaffold copy +first. Then sync the active copy to match. Verify with `diff`. + +## Risks / Trade-offs + +### Risk: False negatives from pre-existing build errors + +If the target project already has compilation errors, the gate +will block all writes even though the generated code is correct. + +Mitigation: The gate instruction specifies to report the error +context, allowing the user to identify pre-existing vs generated +errors. This is acceptable — writing into a non-compiling +codebase is itself a risky action. + +### Risk: Agent may still skip the gate under extreme compression + +No amount of prose can guarantee LLM compliance. However, the +concrete 3-step protocol with an explicit "MUST NOT" halt +condition is significantly more resistant to fast-path skipping +than advisory prose. + +### Trade-off: Per-write compilation adds latency + +Running `go build` before each write adds variable latency: +~1-3 seconds per file with a warm build cache, potentially +10-30+ seconds on a cold cache or for large dependency trees. +For batch operations processing 10+ functions, this adds +meaningful overhead. The Go build cache mitigates repeated +builds of the same package. This is acceptable because +correctness outweighs speed for code generation. + +### Note: Overlap with /gaze fix verification + +The `/gaze fix` command (`.opencode/commands/gaze-fix.md`, +Step 4) already has its own post-generation compile and test +verification. When `/gaze fix` invokes the gaze-test-generator +agent, generated code will be verified twice: once by the +agent's pre-write gate and once by `/gaze fix`'s Step 4. This +is intentional defense-in-depth — the pre-write gate catches +errors before they reach disk; the batch check provides a +final integrity verification across all modified packages. diff --git a/openspec/changes/fix-test-generator-compile-gate/proposal.md b/openspec/changes/fix-test-generator-compile-gate/proposal.md new file mode 100644 index 0000000..640c81d --- /dev/null +++ b/openspec/changes/fix-test-generator-compile-gate/proposal.md @@ -0,0 +1,107 @@ +## Why + +The `gaze-test-generator` agent prompt at +`internal/scaffold/assets/agents/gaze-test-generator.md` (line 275) +contains advisory prose: "ALWAYS verify generated code compiles +before reporting success." This is a T3 weakness — the required +verification step is inline text only, not enforced as a concrete +tool call. Under context compression or fast-path reasoning, an +agent can skip compilation checking and write tests that do not +compile. + +This issue was filed as +[gaze#204](https://github.com/unbound-force/gaze/issues/204) and a +fix was applied in +[unbound-force/unbound-force#404](https://github.com/unbound-force/unbound-force/pull/404), +but that PR modified the copy in the **unbound-force** repo, not the +gaze repo. The gaze repo's scaffold copy (which `gaze init` ships to +users) and its active `.opencode/agents/` copy still contain the old +advisory text. + +This change ports the fix to the gaze repo — the correct location +for the canonical scaffold source. + +Fixes: gaze#204 + +## What Changes + +Replace the advisory compile verification prose with a concrete +3-step pre-write compile gate protocol in the `gaze-test-generator` +agent prompt. The agent will be instructed to run `go build` before +any Write or Edit tool call and halt if compilation fails. + +Additionally, update the Output Format section to distinguish +per-file pre-write gate verification from the final batch integrity +check. + +## Capabilities + +### New Capabilities +- None + +### Modified Capabilities +- `gaze-test-generator`: The "Important Constraints" section + replaces advisory "ALWAYS verify" prose with a numbered 3-step + compile gate protocol using MUST NOT halt language. The "Output + Format" section clarifies that individual files are verified by + the pre-write gate and the final `go build` is a batch integrity + check. + +### Removed Capabilities +- None + +## Impact + +- **Files**: `internal/scaffold/assets/agents/gaze-test-generator.md` + (scaffold canonical copy), + `.opencode/agents/gaze-test-generator.md` (active runtime copy — + currently identical to scaffold) +- **Behavior**: The agent will now be instructed to run `go build` + before writing files and to halt (not write) if compilation fails. + This may cause the agent to report more errors instead of silently + writing broken tests. +- **Users**: Projects that have previously run `gaze init` will + receive the update automatically on the next `gaze init` run. + The `gaze-test-generator.md` file is classified as tool-owned + (`isToolOwned` in `internal/scaffold/scaffold.go`), which uses + overwrite-on-diff behavior — the scaffold automatically replaces + the active copy when content differs from the embedded version. + +## Constitution Alignment + +Assessed against the Unbound Force org constitution. + +### I. Autonomous Collaboration + +**Assessment**: N/A + +This change modifies an agent's internal instruction set. It does +not affect artifact-based communication between heroes or +inter-hero protocols. The gaze-test-generator continues to consume +gaze quality JSON artifacts and produce test files independently. + +### II. Composability First + +**Assessment**: PASS + +The compile gate uses `go build ./...`, universally available in +any Go project. No new dependencies are introduced. The +gaze-test-generator remains independently usable on any Go project. + +### III. Observable Quality + +**Assessment**: PASS + +The compile gate adds a concrete verification step that produces +observable, machine-parseable output (go build exit code and error +output). This strengthens quality observability — compilation +status was previously unverified advisory text. + +### IV. Testability + +**Assessment**: PASS + +The change enforces verification of observable side effects (does +the generated code compile?) before writing. This directly aligns +with the testability principle's requirement that "test contracts +MUST verify observable side effects." diff --git a/openspec/changes/fix-test-generator-compile-gate/specs/compile-gate.md b/openspec/changes/fix-test-generator-compile-gate/specs/compile-gate.md new file mode 100644 index 0000000..53d99fa --- /dev/null +++ b/openspec/changes/fix-test-generator-compile-gate/specs/compile-gate.md @@ -0,0 +1,131 @@ +## ADDED Requirements + +### Requirement: FR-001 Pre-Write Compile Gate + +The gaze-test-generator agent MUST run a compile verification +check before any Write or Edit tool call that modifies a Go +source file or test file. The check MUST use the bash tool to +execute `go build ./path/to/package/...` scoped to the target +package. If the compile check exits with a non-zero exit code, +the agent MUST NOT proceed with the Write or Edit tool call. +The agent MUST report the compilation error and continue +processing remaining target functions. + +#### Scenario: Successful compile before write + +- **GIVEN** the agent has generated a test function for + package `internal/foo` +- **WHEN** the agent runs `go build ./internal/foo/...` and + the command exits with code 0 +- **THEN** the agent proceeds with the Write tool call to + create or append to the test file + +#### Scenario: Failed compile before write + +- **GIVEN** the agent has generated a test function for + package `internal/foo` +- **WHEN** the agent runs `go build ./internal/foo/...` and + the command exits with a non-zero code +- **THEN** the agent MUST NOT execute the Write tool call +- **AND** the agent reports the compilation error output +- **AND** the agent continues processing remaining target + functions (does not abort the entire batch) + +#### Scenario: Compile gate for doc improvements (Edit tool) + +- **GIVEN** the agent has generated a GoDoc improvement for + a function in package `internal/bar` +- **WHEN** the agent runs `go build ./internal/bar/...` after + composing the doc comment change and the command exits with + code 0 +- **THEN** the agent proceeds with the Edit tool call to + modify the source file + +#### Scenario: Failed compile before Edit tool call + +- **GIVEN** the agent has composed a GoDoc improvement for + a function in package `internal/bar` +- **WHEN** the agent runs `go build ./internal/bar/...` and + the command exits with a non-zero code +- **THEN** the agent MUST NOT execute the Edit tool call +- **AND** the original file content is preserved unchanged +- **AND** the agent reports the compilation error output +- **AND** the agent continues processing remaining target + functions + +### Requirement: FR-002 Compile Gate Protocol Specification + +The compile gate MUST be specified as a numbered protocol in +the agent's "Important Constraints" section, not as advisory +prose. The protocol MUST include: +1. The exact bash command to run +2. The exit code check +3. The explicit halt condition (MUST NOT write) + +This ensures the gate is not skippable under context +compression, following the T3 remediation pattern from +issue #346. + +#### Scenario: Protocol is structurally verifiable + +- **GIVEN** the agent file at + `internal/scaffold/assets/agents/gaze-test-generator.md` +- **WHEN** the Important Constraints section is inspected +- **THEN** it contains a numbered list with at least 3 steps +- **AND** the list uses RFC 2119 "MUST NOT" language for the + halt condition +- **AND** the list includes a concrete `go build` command + +## MODIFIED Requirements + +### Requirement: FR-003 Output Format Verification Order + +Previously: "After generating all code, run: `go build`..." + +The Output Format section's post-generation compile check +SHOULD remain as a final batch verification, but the pre-write +gate (FR-001) MUST take precedence. The Output Format section +MUST note that individual files have already passed compilation +via the pre-write gate, and the final batch check serves as a +full-package integrity verification. + +#### Scenario: Batch verification after individual gates + +- **GIVEN** the agent has written 3 test files, each passing + the pre-write compile gate individually +- **WHEN** the agent reaches the Output Format verification + step +- **THEN** the agent runs `go build ./...` as a final + integrity check across all modified packages +- **AND** reports the aggregate compilation status + +### Requirement: FR-004 Important Constraints Section Update + +Previously: "ALWAYS verify generated code compiles before +reporting success" + +The bullet point reading "ALWAYS verify generated code compiles +before reporting success" (line 275 in the scaffold copy) MUST +be replaced with the concrete compile gate protocol (FR-002). +The replacement text MUST use MUST NOT language for the halt +condition, not ALWAYS advisory language. + +**Coverage strategy**: This change modifies agent Markdown +files only — no new Go source code is introduced. Verification +is structural: the agent file contains the numbered protocol +with MUST NOT language, and both copies are byte-identical. +Drift detection uses explicit `diff` commands. + +#### Scenario: Constraint replacement + +- **GIVEN** the current text reads "ALWAYS verify generated + code compiles before reporting success" +- **WHEN** the change is applied +- **THEN** the text is replaced with a numbered pre-write + protocol specifying the bash command, exit code check, and + MUST NOT halt condition + +## REMOVED Requirements + +None — no existing requirements are removed. The advisory +prose is replaced with a stronger, concrete version. diff --git a/openspec/changes/fix-test-generator-compile-gate/tasks.md b/openspec/changes/fix-test-generator-compile-gate/tasks.md new file mode 100644 index 0000000..e8a36dc --- /dev/null +++ b/openspec/changes/fix-test-generator-compile-gate/tasks.md @@ -0,0 +1,102 @@ + + +## 1. Update canonical scaffold agent file + +- [x] 1.1 Replace advisory compile prose in Important Constraints + - File: `internal/scaffold/assets/agents/gaze-test-generator.md` + - Replace the bullet reading "ALWAYS verify generated code + compiles before reporting success" (line 275) with the + concrete pre-write compile gate protocol: + ``` + - Before any Write or Edit tool call that modifies a Go + source or test file, MUST run compile verification: + 1. Run via bash: `go build ./path/to/package/...` + (scoped to the target package being modified) + 2. If the command exits with non-zero code, MUST NOT + proceed with the Write or Edit call. Report the + compilation error and continue to the next target. + 3. Only proceed with the Write or Edit call after a + successful (exit code 0) compile check. + ``` + +- [x] 1.2 Update Output Format section to reference pre-write gate + - File: `internal/scaffold/assets/agents/gaze-test-generator.md` + - In the Output Format section (around line 254), change + "After generating all code, run:" to clarify that individual + files have already been verified by the pre-write compile + gate, and this final check is a full-package integrity + verification. + +## 2. Sync active copy from scaffold + +- [x] 2.1 Copy updated scaffold to active runtime copy + - Source: `internal/scaffold/assets/agents/gaze-test-generator.md` + - Target: `.opencode/agents/gaze-test-generator.md` + - Copy the full updated scaffold content to the active copy. + Both files should be byte-identical after this step. + +## 3. Verification + +- [x] 3.1 Verify copy consistency with explicit diff + - Run: + ```bash + diff internal/scaffold/assets/agents/gaze-test-generator.md \ + .opencode/agents/gaze-test-generator.md + ``` + - Must produce no output (files are identical). + +- [x] 3.2 Verify old advisory text is removed + - Run: + ```bash + grep "ALWAYS verify generated code compiles" \ + internal/scaffold/assets/agents/gaze-test-generator.md + ``` + - Must return no results. + +- [x] 3.3 Verify new protocol is present + - Run: + ```bash + grep "MUST NOT" \ + internal/scaffold/assets/agents/gaze-test-generator.md + ``` + - Must return at least one result containing the halt + condition. + +- [x] 3.4 Run full build and test suite + - Command: `go build ./... && go test -race -count=1 -short ./...` + - Verify no regressions from the agent file changes. + - Note: agent Markdown files are embedded via `embed.FS` in + `internal/scaffold/`, so build verification confirms the + embedded assets are valid. + +- [x] 3.5 Verify constitution alignment + - Confirm the change aligns with Principle III (Observable + Quality) by adding a concrete verification step that + produces observable output. + - Confirm the change aligns with Principle IV (Testability) + by enforcing verification of observable side effects before + writing. + - No new dependencies introduced (Principle II). + +## 4. Documentation Gate + +- [x] 4.1 Assess documentation impact + - This change modifies internal agent behavior (the + gaze-test-generator's pre-write verification protocol). + It does not change user-facing CLI commands, workflows, or + hero capabilities. Exempt from website documentation issue + requirement per AGENTS.md: "Internal refactoring with no + user-facing behavior change." + + +