Skip to content

Allow follow-up questions using only existing evidence - #64

Merged
kubbot merged 4 commits into
mainfrom
codex/get-325-existing-evidence-followups-20261031
Oct 31, 2026
Merged

kubbot merged 4 commits into
mainfrom
codex/get-325-existing-evidence-followups-20261031

Conversation

@kubbot

@kubbot kubbot commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

The two first-step follow-up APIs now accept explicit evidenceMode: "existing_evidence". Children reuse the owner’s currently permitted active material and revocation lineage. Source tools are refused before reservation or dispatch; model reasoning remains metered. Omitted mode and retry retain their existing behavior.

Inheritance enforces the live per-URL scope and preserves unresolved original-read/billing gaps using completed original-source provenance. Child creation and inheritance are atomic, and the mode participates in idempotency identity and persisted checkpoints.

Validation: tests first demonstrated the missing behavior on the unchanged baseline. The actual new-head offline suite passes 1,781 of 1,782 tests, with the existing maintainer-cache audit skipped, plus all 7 separate PDF tests. Both supplemental scope/provenance regressions pass. Full typecheck/build, container runtime, design and compatibility checks pass on this same head; independent full-diff review has no unresolved P0/P1/P2. The existing single-instance manager has deployed merge 3b0ee672fcd30213bbef52e05293c718f9b76758: independent loopback and Tailnet checks confirm health, exact release identity, home and entry-asset bytes, and a complete rollback snapshot. The obsolete startup job pinned to the previous revision was retired with its original configuration retained. Existing project documentation was updated. Live provider and semantic-quality acceptance were not rerun.

Related to GET-325.

Related to GET-325. Test-only stage; production changes await correct semantic RED.
@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Security Evidence

Commit: 46dd588f18e12e5369a3a150059a188081578f8a

Security evidence gate passed (success)

No security-sensitive scanner-evidence gap detected.

Mode: enforce

Scanned 1 changed file(s). No missing scanner-evidence signal was detected.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / PR Risk Taxonomy

Commit: 46dd588f18e12e5369a3a150059a188081578f8a

PR taxonomy review recommended (neutral)

Detected 1 PR taxonomy bucket(s): CI/CD Recommendation.

Scanned 1 changed file(s).

Roadmap taxonomy buckets:

CI/CD Recommendation

CI, dependency, coverage, and contract signals should be routed into follow-up checks or verification work.

Signals:

  • 1 CI or workflow path(s) changed

Paths:

  • apps/web/src/tests/first-step-existing-evidence-optin.test.ts

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Reference Set Readiness

Commit: 46dd588f18e12e5369a3a150059a188081578f8a

Reference set readiness gaps detected (neutral)

Reference evidence present for 0/7 areas (0%) across 1 changed file(s).

This check is based on files changed in this PR. Repository-level readiness is still reported by /ecc-tools analyze comments and generated manifests.

Area Status Evidence / Next Step
Deep analyzer corpus Missing Add analyzer fixture, golden, benchmark, or reference-set files that can catch analyzer regressions.
RAG/evaluator comparison Missing Add retrieval or evaluator reference-set comparison fixtures with expected ranking behavior.
PR salvage/review corpus Missing Add stale-PR, review-thread, reopen-flow, or salvage reference cases for queue cleanup automation.
Discussion triage corpus Missing Add public discussion triage fixtures, golden cases, or reference sets for informational, answered, and no-response classifications.
Harness compatibility Missing Add cross-harness, adapter-compliance, or harness-audit evidence for Claude, Codex, OpenCode, Zed, dmux, and agent surfaces.
Security evidence Missing Attach security evidence such as SBOMs, SARIF, audit reports, or AgentShield evidence packs.
CI failure-mode evidence Missing Add captured CI failure logs, dry-run fixtures, or troubleshooting docs for common workflow failure modes.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Hosted Promotion Readiness

Commit: 46dd588f18e12e5369a3a150059a188081578f8a

Hosted promotion readiness passed (success)

No hosted promotion evidence gaps detected across 1 changed file(s); 0 corpus scenarios had matching evidence.

This check compares PR file changes against the evaluator/RAG promotion corpus in src/analyzers/fixtures/evaluator-rag-corpus.ts.
Hosted output scoring inspected 0 completed cached hosted job results.

No evaluator corpus scenarios matched this PR.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Security Evidence

Commit: 5d0313174b53bb2f1426c903bf72dd4c9960dc88

Security evidence gate passed (success)

No security-sensitive scanner-evidence gap detected.

Mode: enforce

Scanned 1 changed file(s). No missing scanner-evidence signal was detected.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / PR Risk Taxonomy

Commit: 5d0313174b53bb2f1426c903bf72dd4c9960dc88

PR taxonomy review recommended (neutral)

Detected 1 PR taxonomy bucket(s): CI/CD Recommendation.

Scanned 1 changed file(s).

Roadmap taxonomy buckets:

CI/CD Recommendation

CI, dependency, coverage, and contract signals should be routed into follow-up checks or verification work.

Signals:

  • 1 CI or workflow path(s) changed

Paths:

  • apps/web/src/tests/first-step-existing-evidence-optin.test.ts

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Reference Set Readiness

Commit: 5d0313174b53bb2f1426c903bf72dd4c9960dc88

Reference set readiness gaps detected (neutral)

Reference evidence present for 0/7 areas (0%) across 1 changed file(s).

This check is based on files changed in this PR. Repository-level readiness is still reported by /ecc-tools analyze comments and generated manifests.

Area Status Evidence / Next Step
Deep analyzer corpus Missing Add analyzer fixture, golden, benchmark, or reference-set files that can catch analyzer regressions.
RAG/evaluator comparison Missing Add retrieval or evaluator reference-set comparison fixtures with expected ranking behavior.
PR salvage/review corpus Missing Add stale-PR, review-thread, reopen-flow, or salvage reference cases for queue cleanup automation.
Discussion triage corpus Missing Add public discussion triage fixtures, golden cases, or reference sets for informational, answered, and no-response classifications.
Harness compatibility Missing Add cross-harness, adapter-compliance, or harness-audit evidence for Claude, Codex, OpenCode, Zed, dmux, and agent surfaces.
Security evidence Missing Attach security evidence such as SBOMs, SARIF, audit reports, or AgentShield evidence packs.
CI failure-mode evidence Missing Add captured CI failure logs, dry-run fixtures, or troubleshooting docs for common workflow failure modes.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Hosted Promotion Readiness

Commit: 5d0313174b53bb2f1426c903bf72dd4c9960dc88

Hosted promotion readiness passed (success)

No hosted promotion evidence gaps detected across 1 changed file(s); 0 corpus scenarios had matching evidence.

This check compares PR file changes against the evaluator/RAG promotion corpus in src/analyzers/fixtures/evaluator-rag-corpus.ts.
Hosted output scoring inspected 0 completed cached hosted job results.

No evaluator corpus scenarios matched this PR.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Security Evidence

Commit: f0ea85f3107c0c766fa1744d6466defa100ad0ab

Security evidence gate passed (success)

No security-sensitive scanner-evidence gap detected.

Mode: enforce

Scanned 1 changed file(s). No missing scanner-evidence signal was detected.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / PR Risk Taxonomy

Commit: f0ea85f3107c0c766fa1744d6466defa100ad0ab

PR taxonomy review recommended (neutral)

Detected 1 PR taxonomy bucket(s): CI/CD Recommendation.

Scanned 1 changed file(s).

Roadmap taxonomy buckets:

CI/CD Recommendation

CI, dependency, coverage, and contract signals should be routed into follow-up checks or verification work.

Signals:

  • 1 CI or workflow path(s) changed

Paths:

  • apps/web/src/tests/first-step-existing-evidence-optin.test.ts

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Reference Set Readiness

Commit: f0ea85f3107c0c766fa1744d6466defa100ad0ab

Reference set readiness gaps detected (neutral)

Reference evidence present for 0/7 areas (0%) across 1 changed file(s).

This check is based on files changed in this PR. Repository-level readiness is still reported by /ecc-tools analyze comments and generated manifests.

Area Status Evidence / Next Step
Deep analyzer corpus Missing Add analyzer fixture, golden, benchmark, or reference-set files that can catch analyzer regressions.
RAG/evaluator comparison Missing Add retrieval or evaluator reference-set comparison fixtures with expected ranking behavior.
PR salvage/review corpus Missing Add stale-PR, review-thread, reopen-flow, or salvage reference cases for queue cleanup automation.
Discussion triage corpus Missing Add public discussion triage fixtures, golden cases, or reference sets for informational, answered, and no-response classifications.
Harness compatibility Missing Add cross-harness, adapter-compliance, or harness-audit evidence for Claude, Codex, OpenCode, Zed, dmux, and agent surfaces.
Security evidence Missing Attach security evidence such as SBOMs, SARIF, audit reports, or AgentShield evidence packs.
CI failure-mode evidence Missing Add captured CI failure logs, dry-run fixtures, or troubleshooting docs for common workflow failure modes.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Hosted Promotion Readiness

Commit: f0ea85f3107c0c766fa1744d6466defa100ad0ab

Hosted promotion readiness passed (success)

No hosted promotion evidence gaps detected across 1 changed file(s); 0 corpus scenarios had matching evidence.

This check compares PR file changes against the evaluator/RAG promotion corpus in src/analyzers/fixtures/evaluator-rag-corpus.ts.
Hosted output scoring inspected 0 completed cached hosted job results.

No evaluator corpus scenarios matched this PR.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

Preserve default and retry semantics, inherit live permitted evidence and revocation lineage, and reject new source tools before reservation. Related to GET-325.
@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Security Evidence

Commit: 7f3d3aa261a42ebcfceb8032e202d2527241d496

Security evidence gate passed (success)

No security-sensitive scanner-evidence gap detected.

Mode: enforce

Scanned 9 changed file(s). No missing scanner-evidence signal was detected.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / PR Risk Taxonomy

Commit: 7f3d3aa261a42ebcfceb8032e202d2527241d496

PR taxonomy review recommended (neutral)

Detected 1 PR taxonomy bucket(s): CI/CD Recommendation.

Scanned 9 changed file(s).

Roadmap taxonomy buckets:

CI/CD Recommendation

CI, dependency, coverage, and contract signals should be routed into follow-up checks or verification work.

Signals:

  • API contract changes may ship without integration coverage
  • API implementation changes may ship without contract artifact updates
  • 1 CI or workflow path(s) changed

Paths:

  • apps/web/src/tests/first-step-existing-evidence-optin.test.ts
  • apps/web/src/server/research/controller.ts
  • apps/web/src/server/research/first-step-store.ts
  • apps/web/src/server/research/planner.ts
  • apps/web/src/server/research/research-store.ts
  • apps/web/src/server/routes/runs.ts
  • apps/web/src/server/services/runner.ts
  • apps/web/src/server/util/fingerprint.ts

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Reference Set Readiness

Commit: 7f3d3aa261a42ebcfceb8032e202d2527241d496

Reference set readiness gaps detected (neutral)

Reference evidence present for 0/7 areas (0%) across 9 changed file(s).

This check is based on files changed in this PR. Repository-level readiness is still reported by /ecc-tools analyze comments and generated manifests.

Area Status Evidence / Next Step
Deep analyzer corpus Missing Add analyzer fixture, golden, benchmark, or reference-set files that can catch analyzer regressions.
RAG/evaluator comparison Missing Add retrieval or evaluator reference-set comparison fixtures with expected ranking behavior.
PR salvage/review corpus Missing Add stale-PR, review-thread, reopen-flow, or salvage reference cases for queue cleanup automation.
Discussion triage corpus Missing Add public discussion triage fixtures, golden cases, or reference sets for informational, answered, and no-response classifications.
Harness compatibility Missing Add cross-harness, adapter-compliance, or harness-audit evidence for Claude, Codex, OpenCode, Zed, dmux, and agent surfaces.
Security evidence Missing Attach security evidence such as SBOMs, SARIF, audit reports, or AgentShield evidence packs.
CI failure-mode evidence Missing Add captured CI failure logs, dry-run fixtures, or troubleshooting docs for common workflow failure modes.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@ecc-tools

ecc-tools Bot commented Oct 31, 2026

Copy link
Copy Markdown

ECC Tools / Hosted Promotion Readiness

Commit: 7f3d3aa261a42ebcfceb8032e202d2527241d496

Hosted promotion readiness passed (success)

No hosted promotion evidence gaps detected across 9 changed file(s); 0 corpus scenarios had matching evidence.

This check compares PR file changes against the evaluator/RAG promotion corpus in src/analyzers/fixtures/evaluator-rag-corpus.ts.
Hosted output scoring inspected 0 completed cached hosted job results.

No evaluator corpus scenarios matched this PR.

Check publication was denied or unavailable. An app owner must enable Checks: read and write, and the installation owner must approve the updated permission.

@kubbot kubbot left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 7f3d3aa / tree 1382c14053ba660dca56e9a586bb69636e837a8b with an independent whole-diff review. Both identified P2 issues are resolved: inheritance checks the current URL guard, and discovery snippets do not erase original-read failures. Default fingerprint/retry, current cancellation/late-write/source-event controls and provider accounting are preserved. Same-head offline tests and all applicable CI pass; the existing maintainer-source-cache audit remains explicitly skipped. No unresolved P0/P1/P2.

@kubbot
kubbot marked this pull request as ready for review October 31, 2026 09:52
@kubbot
kubbot merged commit 3b0ee67 into main Oct 31, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant