Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions .opencode/agents/gaze-test-generator.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,10 @@ For each target function, output:
3. **File target**: Which `*_test.go` file to write to
4. **Verification**: Whether the code compiles and tests pass

After generating all code, run:
After generating all code, run a final integrity check across all
modified packages. Individual files have already been verified by
the pre-write compile gate (see Important Constraints). This final
check is a full-package verification:

```bash
go build ./path/to/package/...
Expand All @@ -272,6 +275,14 @@ added, compilation status, test pass/fail.
guess at the function signature
- ALWAYS read existing tests before adding assertions — do not
duplicate existing coverage
- ALWAYS verify generated code compiles before reporting success
- Before any Write or Edit tool call that modifies a Go source
or test file, MUST run compile verification:
1. Run via bash: `go build ./path/to/package/...` (scoped to
the target package being modified)
2. If the command exits with non-zero code, MUST NOT proceed
with the Write or Edit call. Report the compilation error
and continue to the next target.
3. Only proceed with the Write or Edit call after a successful
(exit code 0) compile check.
- When adding to an existing file, preserve all existing content —
append only, never delete or modify existing tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
tag: cross-repo-workflow
author: jay-flowers
category: gotcha
created_at: 2026-08-02T16:45:45Z
identity: cross-repo-workflow-20260802T164545-jay-flowers
tier: draft
---

The fix for gaze#204 (compile verification gate in gaze-test-generator) was originally applied in unbound-force/unbound-force#404, but the canonical source for the gaze-test-generator scaffold is in the gaze repo at internal/scaffold/assets/agents/gaze-test-generator.md. When an issue is filed against gaze and the fix touches agent prompts, the fix must be applied in the gaze repo where the scaffold canonical copy lives, not in the unbound-force repo which has its own separate copy. Cross-repo issue-to-PR linkage via "Fixes:" can close issues in the wrong repo, masking that the actual source of truth was never updated.
10 changes: 10 additions & 0 deletions .uf/dewey/learnings/gaze-scaffold-20260802T164541-jay-flowers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
tag: gaze-scaffold
author: jay-flowers
category: gotcha
created_at: 2026-08-02T16:45:41Z
identity: gaze-scaffold-20260802T164541-jay-flowers
tier: draft
---

When porting fixes between repos in the unbound-force organization (e.g., from unbound-force/unbound-force to unbound-force/gaze), the gaze-test-generator agent prompt exists in two locations in the gaze repo: the scaffold canonical copy at internal/scaffold/assets/agents/gaze-test-generator.md and the active runtime copy at .opencode/agents/gaze-test-generator.md. The file is tool-owned (isToolOwned in internal/scaffold/scaffold.go), meaning gaze init will auto-overwrite it when content differs — so the proposal's user impact should state auto-propagation, not manual update. Both copies must be kept byte-identical. The scaffold copy is the source of truth, edited first, then the active copy is synced via cp. The SRE reviewer caught the incorrect user impact statement about skip-if-present behavior during spec review, which was corrected before implementation began.
15 changes: 13 additions & 2 deletions internal/scaffold/assets/agents/gaze-test-generator.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,10 @@ For each target function, output:
3. **File target**: Which `*_test.go` file to write to
4. **Verification**: Whether the code compiles and tests pass

After generating all code, run:
After generating all code, run a final integrity check across all
modified packages. Individual files have already been verified by
the pre-write compile gate (see Important Constraints). This final
check is a full-package verification:

```bash
go build ./path/to/package/...
Expand All @@ -272,6 +275,14 @@ added, compilation status, test pass/fail.
guess at the function signature
- ALWAYS read existing tests before adding assertions — do not
duplicate existing coverage
- ALWAYS verify generated code compiles before reporting success
- Before any Write or Edit tool call that modifies a Go source
or test file, MUST run compile verification:
1. Run via bash: `go build ./path/to/package/...` (scoped to
the target package being modified)
2. If the command exits with non-zero code, MUST NOT proceed
with the Write or Edit call. Report the compilation error
and continue to the next target.
3. Only proceed with the Write or Edit call after a successful
(exit code 0) compile check.
- When adding to an existing file, preserve all existing content —
append only, never delete or modify existing tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: unbound-force
created: 2026-08-02
131 changes: 131 additions & 0 deletions openspec/changes/fix-test-generator-compile-gate/design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
## Context

The `gaze-test-generator` agent generates Go test code, GoDoc
improvements, and assertion restructurings based on gaze quality
analysis data. It writes generated code to disk using the Write
and Edit tools. The agent's "Important Constraints" section
(line 275) says "ALWAYS verify generated code compiles before
reporting success" but provides no concrete protocol for how or
when to verify.

The parent audit (unbound-force/unbound-force#346) identified
that advisory prose gates are systematically bypassed under
context compression. The fix pattern established in that audit
is to replace advisory text with concrete tool call sequences
and explicit halt conditions.

A fix was already applied in
[unbound-force/unbound-force#404](https://github.com/unbound-force/unbound-force/pull/404)
but targeted the wrong repo. The gaze repo has two copies that
must stay in sync:

- `internal/scaffold/assets/agents/gaze-test-generator.md`
(canonical scaffold, shipped via `gaze init` — 277 lines)
- `.opencode/agents/gaze-test-generator.md` (active runtime
copy — currently identical to scaffold, 277 lines)

Both copies contain the old advisory prose and require the same
fix.

## Goals / Non-Goals

### Goals
- Replace advisory compile verification prose with a concrete
pre-write gate protocol
- Specify the exact bash command, failure handling, and halt
condition
- Maintain consistency between scaffold and active copies
- Follow the T3 remediation pattern from issue #346

### Non-Goals
- Modifying the `/gaze fix` command — it already has its own
verification step in Step 4
- Adding compile gates to other agents — that is separate work
tracked in the parent audit
- Changing the agent's tool permissions or mode
- Introducing the pre-flight skill dependency — the compile
check is simple enough to inline as `go build`

## Decisions

### D1: Inline compile gate, not pre-flight skill delegation

The pre-flight skill is designed for CI-aware, multi-tool
execution with coverage matrices and baseline classification.
The gaze-test-generator needs a single `go build` check before
each write. Delegating to pre-flight would add unnecessary
complexity and make the agent harder to reason about.

Decision: Inline the compile check as a concrete 3-step protocol
in the agent's "Important Constraints" section.

### D2: Gate placement — before Write, not after all generation

The issue specifies "before any Write tool call." This is the
correct placement because:
- Writing non-compiling code to disk is the harmful action
- A post-generation check allows broken files to persist
- Per-write gating catches each file individually

Decision: The gate fires before each Write/Edit tool call, not
as a batch check after all generation.

### D3: Compile scope — package-level, not full repo

Running `go build ./...` (full repo) after each generated test
file is expensive and may surface pre-existing errors unrelated
to the generated code. Running `go build ./path/to/package/...`
scopes the check to the package being modified.

Decision: Use `go build ./path/to/package/...` scoped to the
target package. This matches the existing pattern in the Output
Format section (line 257).

### D4: Edit both copies consistently

The scaffold copy (`internal/scaffold/assets/agents/`) is the
canonical source. The active copy (`.opencode/agents/`) is
currently identical. Both must receive the same changes.

Decision: Apply the compile gate changes to the scaffold copy
first. Then sync the active copy to match. Verify with `diff`.

## Risks / Trade-offs

### Risk: False negatives from pre-existing build errors

If the target project already has compilation errors, the gate
will block all writes even though the generated code is correct.

Mitigation: The gate instruction specifies to report the error
context, allowing the user to identify pre-existing vs generated
errors. This is acceptable — writing into a non-compiling
codebase is itself a risky action.

### Risk: Agent may still skip the gate under extreme compression

No amount of prose can guarantee LLM compliance. However, the
concrete 3-step protocol with an explicit "MUST NOT" halt
condition is significantly more resistant to fast-path skipping
than advisory prose.

### Trade-off: Per-write compilation adds latency

Running `go build` before each write adds variable latency:
~1-3 seconds per file with a warm build cache, potentially
10-30+ seconds on a cold cache or for large dependency trees.
For batch operations processing 10+ functions, this adds
meaningful overhead. The Go build cache mitigates repeated
builds of the same package. This is acceptable because
correctness outweighs speed for code generation.

### Note: Overlap with /gaze fix verification

The `/gaze fix` command (`.opencode/commands/gaze-fix.md`,
Step 4) already has its own post-generation compile and test
verification. When `/gaze fix` invokes the gaze-test-generator
agent, generated code will be verified twice: once by the
agent's pre-write gate and once by `/gaze fix`'s Step 4. This
is intentional defense-in-depth — the pre-write gate catches
errors before they reach disk; the batch check provides a
final integrity verification across all modified packages.
107 changes: 107 additions & 0 deletions openspec/changes/fix-test-generator-compile-gate/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
## Why

The `gaze-test-generator` agent prompt at
`internal/scaffold/assets/agents/gaze-test-generator.md` (line 275)
contains advisory prose: "ALWAYS verify generated code compiles
before reporting success." This is a T3 weakness — the required
verification step is inline text only, not enforced as a concrete
tool call. Under context compression or fast-path reasoning, an
agent can skip compilation checking and write tests that do not
compile.

This issue was filed as
[gaze#204](https://github.com/unbound-force/gaze/issues/204) and a
fix was applied in
[unbound-force/unbound-force#404](https://github.com/unbound-force/unbound-force/pull/404),
but that PR modified the copy in the **unbound-force** repo, not the
gaze repo. The gaze repo's scaffold copy (which `gaze init` ships to
users) and its active `.opencode/agents/` copy still contain the old
advisory text.

This change ports the fix to the gaze repo — the correct location
for the canonical scaffold source.

Fixes: gaze#204

## What Changes

Replace the advisory compile verification prose with a concrete
3-step pre-write compile gate protocol in the `gaze-test-generator`
agent prompt. The agent will be instructed to run `go build` before
any Write or Edit tool call and halt if compilation fails.

Additionally, update the Output Format section to distinguish
per-file pre-write gate verification from the final batch integrity
check.

## Capabilities

### New Capabilities
- None

### Modified Capabilities
- `gaze-test-generator`: The "Important Constraints" section
replaces advisory "ALWAYS verify" prose with a numbered 3-step
compile gate protocol using MUST NOT halt language. The "Output
Format" section clarifies that individual files are verified by
the pre-write gate and the final `go build` is a batch integrity
check.

### Removed Capabilities
- None

## Impact

- **Files**: `internal/scaffold/assets/agents/gaze-test-generator.md`
(scaffold canonical copy),
`.opencode/agents/gaze-test-generator.md` (active runtime copy —
currently identical to scaffold)
- **Behavior**: The agent will now be instructed to run `go build`
before writing files and to halt (not write) if compilation fails.
This may cause the agent to report more errors instead of silently
writing broken tests.
- **Users**: Projects that have previously run `gaze init` will
receive the update automatically on the next `gaze init` run.
The `gaze-test-generator.md` file is classified as tool-owned
(`isToolOwned` in `internal/scaffold/scaffold.go`), which uses
overwrite-on-diff behavior — the scaffold automatically replaces
the active copy when content differs from the embedded version.

## Constitution Alignment

Assessed against the Unbound Force org constitution.

### I. Autonomous Collaboration

**Assessment**: N/A

This change modifies an agent's internal instruction set. It does
not affect artifact-based communication between heroes or
inter-hero protocols. The gaze-test-generator continues to consume
gaze quality JSON artifacts and produce test files independently.

### II. Composability First

**Assessment**: PASS

The compile gate uses `go build ./...`, universally available in
any Go project. No new dependencies are introduced. The
gaze-test-generator remains independently usable on any Go project.

### III. Observable Quality

**Assessment**: PASS

The compile gate adds a concrete verification step that produces
observable, machine-parseable output (go build exit code and error
output). This strengthens quality observability — compilation
status was previously unverified advisory text.

### IV. Testability

**Assessment**: PASS

The change enforces verification of observable side effects (does
the generated code compile?) before writing. This directly aligns
with the testability principle's requirement that "test contracts
MUST verify observable side effects."
Loading
Loading