CR-README-02: make organisational memory lead, and stop a third confidence leak - #259
Merged
Merged
Conversation
…dence leak
Public port of private PR #343 (merged 2026-09-15, merge commit 2ac78d2b).
All four files are byte-identical to the merged private/master (hash-verified).
The 0.84.0 README asserted "organisational intelligence" in the hero and then
reverted to code in every proof beneath it. A word count hid this: the first 70
lines scored 15 code-words to 19 org-words, apparently balanced, because the
org words sat in the HERO while the STRUCTURE underneath was code-only:
- "Why GraQle exists" opened "Your AI coding tool..."
- the 90-second proof was `scan repo` -> a code question
- documents lived in a sub-section titled "the knowledge that isn't code",
framing them as the exception rather than the category
- "How it works" step 1 said documents are "auto-linked to the code they
describe", subordinating them
- "The compounding advantage" opened "it knows your codebase"
- "What teams use it for" was 6 code rows to 2 org rows
RESTRUCTURE (both surfaces, kept in sync)
- The 90-second proof now leads with documents and runs on a folder with NO
CODE IN IT AT ALL. Every number was measured, not estimated: 3 policy/ADR
files -> 12 nodes (3 DOCUMENT + 9 SECTION) + 9 SECTION_OF edges. Then
`graq learn knowledge "vendor DPA must be signed before any data access"
--domain policy` extracted the entity "DPA" and created 2
SEMANTICALLY_RELATED edges to the vendor-onboarding document AND to its
"Due diligence" section — it found where the rule belonged unprompted.
That is persistent, compounding organisational memory demonstrated with no
repository present.
- Code is now an explicit wedge, not the boundary: "### The same graph, for
code" follows the documents proof and says so plainly.
- "How it works" step 1 leads with documents/policies/ADRs; step 2 lists
SECTION_OF and SEMANTICALLY_RELATED first.
- "What teams use it for" reordered: policies, institutional memory and
cross-source questions first.
- After: first 70 lines score code 8 / org 34 (README) and 11 / 27 (PyPI);
the first scan command on both surfaces is `graq scan docs`.
THIRD ATTEMPT AT THE PLACEHOLDER CONFIDENCE LEAK
`graq run` with no LLM configured printed "Confidence: 62%" directly beneath
text reading "[NO LLM CONFIGURED - this is a placeholder response]".
Attempt 1 (shipped in 0.84.0) fixed backends/mock.py - the ANSWER TEXT only.
Attempt 2 guarded `def reason()`. But `graq run` dispatches to `def run()`
at main.py:401, a DIFFERENT function with its own print site. The number
kept printing, and the source-grep test written alongside PASSED anyway -
a false green, asserting on strings present in the wrong function.
Attempt 3 (this commit) guards run(), safety_check_command() and both
reason() sites, verified by RUNNING the commands.
backend_status is NOT usable here: it is only ever set to "failed" on an
exception (core/graph.py), so the fallback path leaves it "ok".
MockBackend.is_fallback is the signal that tracks the condition.
The regression test asserts per-function and scopes its sweep to
result.confidence / r.confidence. profile.confidence, env.confidence and
rec.confidence are deliberately excluded - those are computed locally from the
graph and are meaningful with no LLM configured. Narrowing the sweep rather
than weakening the guard is the point.
VERIFICATION ON THIS PUBLIC TREE
- All 4 files hash-identical to merged private/master.
- README lock gate (snapshot + eu_ai_act_docs_present): 62 passed.
- tests/test_cli/test_run_placeholder_no_confidence.py: 8 passed.
- Every graq command in both READMEs machine-diffed against `graq --help`:
0 invalid.
- Forbidden-word scan: 0 violations in both files.
NOT DONE HERE
No version bump. 0.84.0 is merged to master but still UNTAGGED and PyPI still
serves 0.83.0, so this copy and this fix ride into the same 0.84.0 release -
PyPI never sees the weaker version.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
🛡️ GraQle PR Guardian💥 Blast Radius: 4 modules affected
Total blast radius: 4 🏛️ Governance Verdict
|
| Metric | Value |
|---|---|
| Blast Radius | 4 |
| Files Analyzed | 4 |
| Blocked | 0 |
| SHACL Violations | 0 |
| Verdict | WARN |
🔬 Powered by GraQle PR Guardian v0.1.0 · Scan completed 2026-09-15T06:29:43.755458+00:00
harishquantamix
approved these changes
Sep 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
The 0.84.0 README asserted "organisational intelligence" in the hero and then reverted to code in every proof beneath it.
A word count hid it. The first 70 lines scored 15 code-words to 19 org-words — apparently balanced — because the org words sat in the hero while the structure underneath was code-only:
scan repo→ a code questionPublic port of private PR #343 (merged 2026-09-15, merge commit
2ac78d2b). All four files are hash-identical to the mergedprivate/master.The restructure
The 90-second proof now leads with documents and runs on a folder with no code in it at all. Every number was measured, not estimated:
Then
graq learn knowledge "vendor DPA must be signed before any data access" --domain policyextracted the entity DPA and created 2 SEMANTICALLY_RELATED edges — to the vendor-onboarding document and to its "Due diligence" section. It found where the rule belonged unprompted.That is persistent, compounding organisational memory, demonstrated with no repository present.
### The same graph, for codefollows the documents proof and says so plainly.SECTION_OFandSEMANTICALLY_RELATEDfirst.Result: first 70 lines now score code 8 / org 34 (README) and 11 / 27 (PyPI). The first scan command on both surfaces is
graq scan docs.Third attempt at the placeholder confidence leak
With no LLM configured,
graq runprintedConfidence: 62%directly beneath text reading[NO LLM CONFIGURED — this is a placeholder response]. In a screenshot that is indistinguishable from a governed answer.backends/mock.py— the answer textdef reason()graq rundispatches todef run()(main.py:401) — a different function. The source-grep test written alongside passed anyway: a false green, asserting on strings in the wrong functionrun(),safety_check_command()and bothreason()sitesbackend_statusis not usable here: it is only ever set to"failed"on an exception (core/graph.py), so the fallback path leaves it"ok".MockBackend.is_fallbackis the signal that tracks the condition.The regression test asserts per-function and scopes its sweep to
result.confidence/r.confidence.profile.confidence,env.confidenceandrec.confidenceare deliberately excluded — computed locally from the graph, meaningful with no LLM configured. Narrowing the sweep rather than weakening the guard is the point.Verification on this public tree
private/mastereu_ai_act_docs_present)test_run_placeholder_no_confidence.pygraqcommand vsgraq --helpRelease Gate (PyPI)will fail as it does on every PR touchinggraqle/**— not a required check; the only required context on master isip-protection-gate.Not done here
No version bump. 0.84.0 is merged to master but still untagged, and PyPI still serves 0.83.0 — so this copy and this fix ride into the same 0.84.0 release. PyPI never sees the weaker version.
🤖 Generated with Claude Code