Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
971a6cf
Add logic spec for quirk:full-code-review
zpyoung Oct 15, 2026
1a57029
Mark full-code-review logic spec approved, tech spec requested
zpyoung Oct 15, 2026
1e46552
Add tech spec for quirk:full-code-review
zpyoung Oct 15, 2026
84ebda4
Add full-code-review implementation plan; apply plan-review fixes to …
zpyoung Oct 15, 2026
7c82096
feat(full-code-review): port review-mr-combined verbatim
zpyoung Oct 15, 2026
08880a2
refactor(full-code-review): add forge package and move prepare_diff g…
zpyoung Oct 15, 2026
f83c942
refactor(full-code-review): move borrow_review_source glab clone and …
zpyoung Oct 15, 2026
42ce72f
refactor(full-code-review): move reviewer-posting and memory glab cal…
zpyoung Oct 15, 2026
6f06647
feat(full-code-review): detect the forge from the remote and dispatch…
zpyoung Oct 15, 2026
0638e07
feat(full-code-review): neutral change-request vocabulary in context,…
zpyoung Oct 15, 2026
d56c11c
feat(full-code-review): generic lenses with front matter, split test …
zpyoung Oct 15, 2026
3f824d3
feat(full-code-review): quirk identity for markers, storage keys, led…
zpyoung Oct 15, 2026
7546716
feat(full-code-review): guidance CLIs without the sakura guidance store
zpyoung Oct 15, 2026
58246ad
feat(full-code-review): rewrite SKILL.md for the generic skill and ad…
zpyoung Oct 15, 2026
dea7e4f
feat(full-code-review): resolve review policy from packs and REVIEW.md
zpyoung Oct 15, 2026
687b00d
feat(full-code-review): report contract for incomplete runs, models, …
zpyoung Oct 15, 2026
19c4ad9
feat(full-code-review): mixed-authorship seats and the mutation-probe…
zpyoung Oct 15, 2026
952e66a
docs(full-code-review): document policy packs and wire the policy ste…
zpyoung Oct 15, 2026
c026a6a
docs(full-code-review): record the verified GitHub pending-review rep…
zpyoung Oct 15, 2026
206cd1f
feat(full-code-review): GitHub adapter and forge-neutral reviewer del…
zpyoung Oct 15, 2026
6930634
test(full-code-review): FakeGitHub coverage for the GitHub adapter an…
zpyoung Oct 15, 2026
b99a40f
feat(full-code-review): scripted author posting for both forges; docu…
zpyoung Oct 15, 2026
c393ca4
feat(full-code-review): per-lens keep/dismiss stats from the decision…
zpyoung Oct 15, 2026
d869f84
docs(full-code-review): cross-family verification procedure
zpyoung Oct 15, 2026
a85f057
test(full-code-review): smoke corpus of five planted bugs and smoke_c…
zpyoung Oct 15, 2026
a668f85
feat(full-code-review): GitLab JSON content type and overview routing…
zpyoung Oct 15, 2026
ac4d964
fix(full-code-review): issues found by the live smoke runs
zpyoung Oct 15, 2026
84ea481
docs(full-code-review): mark the plan complete with smoke-run results
zpyoung Oct 15, 2026
a224ea9
fix(full-code-review): address PR #50 review findings, per-area fixes
zpyoung Oct 15, 2026
5f930ba
refactor(full-code-review): shared JSON writer, forge boundary, test …
zpyoung Oct 15, 2026
3605fd8
fix(full-code-review): address the second PR #50 review
zpyoung Oct 16, 2026
30c29ff
docs(full-code-review): logic spec for deep mode
zpyoung Oct 16, 2026
2896550
docs(full-code-review): approve logic spec and author tech spec for d…
zpyoung Oct 16, 2026
ffa5c9a
feat(full-code-review): add deep_sandbox.py for isolated throwaway co…
zpyoung Oct 16, 2026
c1aa59a
feat(full-code-review): add build_area_map.py for coupled area partit…
zpyoung Oct 16, 2026
0f6634b
feat(full-code-review): add merge_deep_findings.py for checker gating…
zpyoung Oct 16, 2026
3343248
feat(full-code-review): render deep report fields and block plausible…
zpyoung Oct 16, 2026
1cd99c3
feat(full-code-review): add PR #50 answer key and score_deep_report.py
zpyoung Oct 16, 2026
bd0e550
Merge branch 'sdd-deep-T1' into zpyoung/fcr-5-deep
zpyoung Oct 16, 2026
bca519c
Merge branch 'sdd-deep-T2' into zpyoung/fcr-5-deep
zpyoung Oct 16, 2026
2fade16
Merge branch 'sdd-deep-T3' into zpyoung/fcr-5-deep
zpyoung Oct 16, 2026
6d45df0
Merge branch 'sdd-deep-T4' into zpyoung/fcr-5-deep
zpyoung Oct 16, 2026
98e1664
docs(full-code-review): add --deep runbook, finder/contract/checker p…
zpyoung Oct 16, 2026
feecdd0
Merge branch 'sdd-deep-T5' into zpyoung/fcr-5-deep
zpyoung Oct 16, 2026
9d1f711
Merge branch 'sdd-deep-T6' into zpyoung/fcr-5-deep
zpyoung Oct 16, 2026
d6e1d0e
docs(full-code-review): sync tech.md non-defect count with answer key
zpyoung Oct 16, 2026
7e1380e
fix(full-code-review): harden deep execution and evaluation evidence
zpyoung Oct 16, 2026
4da4621
fix(full-code-review): retain non-UTF-8 reproduction output
zpyoung Oct 16, 2026
9d30355
docs(full-code-review): record verified implementation and acceptance…
zpyoung Oct 16, 2026
401104c
fix(full-code-review): address deep review round 1 findings
zpyoung Oct 16, 2026
db8a73f
fix(full-code-review): address deep review round 2 findings
zpyoung Oct 16, 2026
b7264ab
fix(full-code-review): address deep review round 3 findings
zpyoung Oct 17, 2026
f84ba3e
feat(full-code-review): drop the change-size gate
zpyoung Oct 17, 2026
fa2b6ad
Merge deep review fixes (rounds 1-3) into zpyoung/fcr-4-policy
zpyoung Oct 17, 2026
3ef8378
docs(full-code-review): drop the last size-gate reference in policy.md
zpyoung Oct 17, 2026
d5813e6
feat(full-code-review): plan deep areas by changed lines with per-hun…
zpyoung Oct 17, 2026
d730e2a
feat(full-code-review): enforce hunk coverage, soften over-budget, sh…
zpyoung Oct 17, 2026
65759c9
Merge branch 'sdd-tune-A' into zpyoung/fcr-4-policy
zpyoung Oct 17, 2026
fce097d
feat(full-code-review): add --since delta reviews, report only
zpyoung Oct 17, 2026
d2206a5
Merge branch 'sdd-tune-B' into zpyoung/fcr-4-policy
zpyoung Oct 17, 2026
5acd5cf
Merge branch 'sdd-tune-C' into zpyoung/fcr-4-policy
zpyoung Oct 17, 2026
008e773
docs(full-code-review): document deep coverage planning, --since, and…
zpyoung Oct 17, 2026
3b5a2b6
docs(full-code-review): update specs for coverage planning, --since, …
zpyoung Oct 17, 2026
e19ba7e
Merge branch 'sdd-tune-D1' into zpyoung/fcr-4-policy
zpyoung Oct 17, 2026
496e8e4
Merge branch 'sdd-tune-D2' into zpyoung/fcr-4-policy
zpyoung Oct 17, 2026
4a20c93
fix(full-code-review): reject empty --since refs and capacity overflow
zpyoung Oct 17, 2026
880e346
fix(full-code-review): suggest a full review when --since ancestry ch…
zpyoung Oct 17, 2026
a0550ec
fix(full-code-review): close the recall gaps the MR !722 re-review ex…
zpyoung Oct 18, 2026
c0a87d4
fix(full-code-review): run the full lens fan-out at any diff size
zpyoung Oct 19, 2026
07d9093
Merge branch 'main' of https://github.com/zpyoung/quirk into zpyoung/…
zpyoung Oct 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ permissions:
jobs:
pytest:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.12"]
# several tests make throwaway git commits, which fail on a runner with no identity
env:
GIT_AUTHOR_NAME: ci
Expand All @@ -21,8 +24,16 @@ jobs:

- uses: actions/setup-python@v5
with:
python-version: "3.12"
python-version: ${{ matrix.python-version }}

- run: pip install pytest
- run: pip install pytest pygments pyyaml

# the kernel-sandbox tests skip when no sandbox works, so fail here rather than skip silently
- name: Install bubblewrap and verify it can isolate
run: |
sudo apt-get update
sudo apt-get install -y bubblewrap
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 || true
bwrap --unshare-pid --die-with-parent --ro-bind / / --dev /dev --proc /proc /bin/true

- run: pytest -q
22 changes: 22 additions & 0 deletions BUGS.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,25 @@ entries' IDs; manual edits to fix typos are fine.
- **Severity**: medium
- **Proposed fix**: Skip link destinations and autolinks when applying the underscore guard, as scenario-example.tsx now does.

## BUG-15: SKILL.md and orchestration.md disagree on which dispatches carry the mutation-probe line
- **Observed**: 2026-10-18
- **File**: skills/full-code-review/SKILL.md:318
- **Description**: SKILL.md Step 4 says to state the mutation-probe line in every dispatch, but orchestration.md's 'Every specialist prompt also says' paragraph says the coordinator adds 'mutation probe: enabled' only to the three test lenses, and only with consent. A coordinator can't tell whether every dispatch needs a mutation-probe line (disabled for the non-test lenses) or only the test lenses do. Doc-only. Found while fixing the gaps the MR !722 re-review exposed.
- **Severity**: low
- **Proposed fix**: Pick one rule: either every dispatch states the probe status (enabled or disabled), or only the test lenses get the line. Then make both files say the same thing.

## BUG-16: full-code-review lets the coordinator skip the lens fan-out on small diffs
- **Observed**: 2026-10-19
- **File**: skills/full-code-review/SKILL.md:292
- **Description**: In a headless smoke-corpus run (claude -p "/quirk:full-code-review --local --no-open", Sonnet 5.5 default, 2026-10-19), all 5 smoke cases reviewed the 2-9 line diff inline instead of dispatching lenses and wrote no report.json (one used the ReportFindings tool, one wrote a Markdown report). One also overrode a pre-exported REVIEW_OUT_DIR with its own mktemp despite the ${REVIEW_OUT_DIR:-...} default in Setup. SKILL.md never says the fan-out runs regardless of diff size, so the smoke corpus can't run without a prompt override and real tiny-diff reviews silently drop to one inline pass.
- **Severity**: medium
- **Proposed fix**: State in SKILL.md (Setup + Step 4) that the lens fan-out and report.json are mandatory at any diff size with no inline shortcut, and that a pre-set REVIEW_OUT_DIR is kept; add a test pinning that wording.
- **Resolved**: 2026-10-19. Wording alone (fan-out at any size, keep REVIEW_OUT_DIR) fixed the out-dir override but still left 3 of 5 plain-prompt runs skipping, now citing "headless run on a one-line diff". Closing those excuses by name (one Agent per lens; skip only on run_when, policy, or failed dispatch; diff size, headless, no forge, or cost is never a skip reason) brought all 5 to a full fan-out, report.json in the caller's dir, and a passing smoke_check. Residual lens-packing is BUG-17.

## BUG-17: full-code-review coordinator occasionally packs several lenses into one agent
- **Observed**: 2026-10-19
- **File**: skills/full-code-review/SKILL.md:298
- **Description**: In the 2026-10-19 plain-prompt smoke run after the BUG-16 fix, 1 of 54 lens dispatches (correctness case) combined comments, db and observability into one agent despite SKILL.md Step 4 saying one Agent per lens. A packed agent shares one context across checklists, so a lens can get a shallower pass than its own dispatch would give, and run.models then reports per-lens models for a single dispatch. Earlier rounds also packed lenses (test-adequacy case, 2 multi-lens agents).
- **Severity**: low
- **Proposed fix**: Re-run the smoke corpus a few times to measure the rate; if it persists, have the coordinator check its dispatch count against policy.json's runnable lenses before Step 5, or record per-lens agent ids in run so a packed dispatch is visible in the report.

13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,24 @@ Version: **2026.7.9**

## What ships

- **25 skills** under `skills/` covering TDD, systematic debugging, brainstorming (which produces a logic spec), deep-research/brainstorming exploration (with a `--involve` user-inclusion dial: checkpoints, mid-run steering, and idea-gate co-creation), divergent ideation, the two-document spec rubric (logic spec + tech spec) and plan writing, scannable-prose revision, code review, adversarial review (typed profiles, a promote/refute two-stage, and an evidence gate), parallel agent dispatch, guided bug/feature/code-change filing with provenance-marked artifacts, gating simplification behind a correctness check, and more.
- **26 skills** under `skills/` covering TDD, systematic debugging, brainstorming (which produces a logic spec), deep-research/brainstorming exploration (with a `--involve` user-inclusion dial: checkpoints, mid-run steering, and idea-gate co-creation), divergent ideation, the two-document spec rubric (logic spec + tech spec) and plan writing, scannable-prose revision, code review, a full multi-lens pre-merge review for GitHub and GitLab, adversarial review (typed profiles, a promote/refute two-stage, and an evidence gate), parallel agent dispatch, guided bug/feature/code-change filing with provenance-marked artifacts, gating simplification behind a correctness check, and more.
- **Typed artifacts** (added in 5.1.0) — surface-routing for observations Claude cannot act on. See below.

## Installation

Install via the quirk-dev marketplace (`.claude-plugin/marketplace.json`).

## Deep code review

`/quirk:full-code-review --deep [minutes]` adds bounded area finders, cross-file
contract checking, and fresh candidate verification to the standard review.
The default budget is 60 minutes. Eligible runs ask before executing repros in
throwaway copies; plausible findings stay visible but cannot be posted.
See the [deep-mode runbook](skills/full-code-review/references/deep.md).

`--since <ref>` reviews only the changes after `<ref>`, such as the fix commits made since
the last review's head SHA. A delta review is report only: it loads no memory and posts nothing.

## Typed artifacts

A discipline for routing deferred observations into structured markdown files instead of burying them in prose ("pre-existing", "out of scope", "skipped for brevity").
Expand Down
5 changes: 5 additions & 0 deletions REVIEW.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
skip_globs: ["tests/fixtures/full-code-review/answer-keys/**"]
---

The answer keys are evaluation fixtures; reviewing them would leak expected defects to review agents.
7 changes: 7 additions & 0 deletions TEST_BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,10 @@ Reviewed every 2 weeks. Use `/quirk:artifacts:test-skip` to append.
- **Edge cases to cover**: brainstorming step 9 reads logic-spec.md and writes logic.md with all six required sections; brainstorming halts at the user review gate; subagent-driven-development Step 2 resolves quirk:writing-specs and reads tech-spec.md when the complexity gate fires; executing-plans Step 0 does the same on the no-subagent path
- **Priority**: P2

## TEST-3: Two complete Sonnet deep-mode recall acceptance runs
- **File under test**: skills/full-code-review/references/deep.md
- **Test type**: e2e
- **Reason skipped**: Sonnet runtime dispatch failed Anthropic OAuth refresh with invalid_grant. Historical eval-run1/eval-run2 were invalidated because they reused old lenses, supplied literal timings, seeded prompts, and claimed unvisited areas complete. No qualifying recall run has been established.
- **Edge cases to cover**: After reauthenticating Anthropic, run every applicable lens and every area twice at 3605fd8777b60646bcad11aba08f83c2c858e6b9 with unseeded prompts, measured whole-run budgets, fresh per-claim checkers, and independent spot checks; score against the corrected answer key.
- **Priority**: P1

Loading
Loading