docs(comments): give the a57 findings doc a findable path - #1709
Merged
Merged
Conversation
Contributor
|
Claude Code Review — skipped: PR author 'arkash20' is not a public member of the 'caura-ai' org |
Contributor
Author
|
@Eldad-Caura please approve |
Eldad-Caura
approved these changes
Sep 23, 2026
arkash20
force-pushed
the
docs/a57-findings-path
branch
from
September 23, 2026 17:24
47d8eea to
70753f9
Compare
Contributor
|
Claude Code Review — skipped: PR author 'arkash20' is not a public member of the 'caura-ai' org |
Three source comments cite `benchmark/a57-recall-experiments-findings.md` as though it were in this repo. It is not, and never has been on any ref. It lives at that path relative to the MULTI-REPO WORKSPACE ROOT, one level above this checkout -- a local analysis artifact that is not published. The citation resolves only if your working directory happens to be the workspace root, so from inside a clone it is a dead pointer. That cost a reviewer of #1706 a search across every ref before they asked, which is the whole reason a comment-only change is worth making: the comments name a convention ("change only with a fresh control pair on the 67-q regression sample") and then point at nothing. Each site now says where the file actually is and that a checkout alone will not find it. The recall_service one also points at the `regression-bench` skill, because the second thing that reviewer was blocked on was believing the control pair needed hosted credentials -- it needs only the local docker stack and `dev-admin-key`, and takes about eight minutes. No behaviour change; comments only. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Arkady Mankovsky <arkash20@gmail.com>
arkash20
force-pushed
the
docs/a57-findings-path
branch
from
September 23, 2026 20:27
70753f9 to
5f0dc88
Compare
Contributor
|
Claude Code Review — skipped: PR author 'arkash20' is not a public member of the 'caura-ai' org |
Merged
Eldad-Caura
pushed a commit
that referenced
this pull request
Sep 24, 2026
🤖 I have created a release *beep* *boop* --- <details><summary>backend: 3.19.0</summary> ## [3.19.0](backend-v3.18.1...backend-v3.19.0) (2026-09-24) ### Features * **search:** let a caller or a tenant exclude fan-out children (09/18 c-03) ([#1708](#1708)) ([1c98ce1](1c98ce1)) ### Bug Fixes * **api:** make the 504 name the layer that ate the request budget (ax-0917-h-01/h-02) ([#1707](#1707)) ([68d49fa](68d49fa)) * **installer:** verify TLS unless asked to trust on first use ([#1711](#1711)) ([cab9eed](cab9eed)) ### Documentation * **c-01:** measure what a deferred title does to FTS rank and ordering ([#1704](#1704)) ([2e7937a](2e7937a)) * **comments:** give the a57 findings doc a findable path ([#1709](#1709)) ([ea4d066](ea4d066)) * **search:** measure the include_derived blast radius (09/18 c-03) ([#1703](#1703)) ([ee482a9](ee482a9)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Signed-off-by: release-please[bot] <release-please[bot]@users.noreply.github.com> Co-authored-by: caura-deploy-bot[bot] <265395343+caura-deploy-bot[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three source comments —
type_ii_materializer.py:9,organization_settings.py:76,recall_service.py:58— citebenchmark/a57-recall-experiments-findings.mdas if it were in this repo. It is not, and never has been on any ref.It exists, at that path relative to the multi-repo workspace root, one level above this checkout. It is a local analysis artifact and is not published, so the citation resolves only if your working directory happens to be the workspace root — and is a dead pointer from inside a clone.
Why a comment-only change is worth making
Those comments name a convention and then point at nothing:
A reviewer on #1706 searched every ref for that file before asking where it was. The doc does carry what the comments promise — §"A64 control run — true-premise cost (2026-08-27, 67-q regression sample, d=0 fast)" has the method line and the per-category table, including the −14.3pp knowledge-update cost that is the reason the convention exists.
What each site now says
That the path is relative to the workspace root, and that a checkout alone will not find it.
recall_service.pyadditionally points at theregression-benchskill, because the second thing that reviewer was blocked on was believing the control pair needed hosted Caura credentials and a live instance. It does not: it runs against the local docker stack withdev-admin-keyand takes about eight minutes atd=0. That misconception was about to gate a security fix behind infrastructure nobody needed.(Also corrects the line number: the recall citation is at
:58, not:105.)Not done here
Publishing the doc itself. The honest options are to vendor the control-pair procedure into
BENCHMARKS.md, or to move the file in after a scrub — parts of it reference internal benchmark scores, so it cannot be dropped into a public repo as-is. Worth its own decision rather than smuggling into a comment fix.Checks
Comments only; no behaviour change.
ruff check+format --checkclean.-k "recall or premise or materializ or settings": 7 failed / 301 passed on this branch and the same 7 on a pristineorigin/mainworktree — identical sets, zero regressions. All 7 are FTS-dependent and share one cause: the test DB is built from the ORM models, so migration 001'ssearch_vectortrigger never exists and the column is NULL on every row. Pre-existing, filed separately, not touched here.🤖 Generated with Claude Code