Skip to content

CR-README-02: make organisational memory lead, and stop a third confidence leak - #259

Merged
harishquantamix merged 1 commit into
masterfrom
cr-readme-02-public
Sep 15, 2026
Merged

harishquantamix merged 1 commit into
masterfrom
cr-readme-02-public

Conversation

@quantamixsol

Copy link
Copy Markdown
Owner

The problem

The 0.84.0 README asserted "organisational intelligence" in the hero and then reverted to code in every proof beneath it.

A word count hid it. The first 70 lines scored 15 code-words to 19 org-words — apparently balanced — because the org words sat in the hero while the structure underneath was code-only:

Where What it said
"Why GraQle exists" opened "Your AI coding tool…"
90-second proof scan repo → a code question
Documents a sub-section titled "the knowledge that isn't code" — the exception, not the category
"How it works" step 1 documents "auto-linked to the code they describe" — subordinated
"The compounding advantage" opened "it knows your codebase"
"What teams use it for" 6 code rows to 2 org rows

Public port of private PR #343 (merged 2026-09-15, merge commit 2ac78d2b). All four files are hash-identical to the merged private/master.


The restructure

The 90-second proof now leads with documents and runs on a folder with no code in it at all. Every number was measured, not estimated:

3 policy/ADR files  →  12 nodes (3 DOCUMENT + 9 SECTION) + 9 SECTION_OF edges

Then graq learn knowledge "vendor DPA must be signed before any data access" --domain policy extracted the entity DPA and created 2 SEMANTICALLY_RELATED edges — to the vendor-onboarding document and to its "Due diligence" section. It found where the rule belonged unprompted.

That is persistent, compounding organisational memory, demonstrated with no repository present.

  • Code is now an explicit wedge, not the boundary### The same graph, for code follows the documents proof and says so plainly.
  • "How it works" step 1 leads with documents/policies/ADRs; step 2 lists SECTION_OF and SEMANTICALLY_RELATED first.
  • "What teams use it for" reordered: policies, institutional memory and cross-source questions first.

Result: first 70 lines now score code 8 / org 34 (README) and 11 / 27 (PyPI). The first scan command on both surfaces is graq scan docs.


Third attempt at the placeholder confidence leak

With no LLM configured, graq run printed Confidence: 62% directly beneath text reading [NO LLM CONFIGURED — this is a placeholder response]. In a screenshot that is indistinguishable from a governed answer.

Fixed Why it didn't hold
Attempt 1 (shipped in 0.84.0) backends/mock.py — the answer text The CLI's summary line kept printing the number
Attempt 2 print sites in def reason() graq run dispatches to def run() (main.py:401) — a different function. The source-grep test written alongside passed anyway: a false green, asserting on strings in the wrong function
Attempt 3 (this commit) run(), safety_check_command() and both reason() sites Verified by running the commands
$ graq run "what approval is needed for a large refund?"
Answer: [NO LLM CONFIGURED — this is a placeholder response, not real AI reasoning...]
Confidence: not reported (no LLM configured) | Rounds: 2 | Nodes: 5 | ...

backend_status is not usable here: it is only ever set to "failed" on an exception (core/graph.py), so the fallback path leaves it "ok". MockBackend.is_fallback is the signal that tracks the condition.

The regression test asserts per-function and scopes its sweep to result.confidence / r.confidence. profile.confidence, env.confidence and rec.confidence are deliberately excluded — computed locally from the graph, meaningful with no LLM configured. Narrowing the sweep rather than weakening the guard is the point.


Verification on this public tree

Check Result
All 4 files vs merged private/master hash-identical
README lock gate (snapshot + eu_ai_act_docs_present) 62 passed
test_run_placeholder_no_confidence.py 8 passed
Every graq command vs graq --help 0 invalid
Forbidden-word scan 0 violations both files

Release Gate (PyPI) will fail as it does on every PR touching graqle/** — not a required check; the only required context on master is ip-protection-gate.


Not done here

No version bump. 0.84.0 is merged to master but still untagged, and PyPI still serves 0.83.0 — so this copy and this fix ride into the same 0.84.0 release. PyPI never sees the weaker version.

🤖 Generated with Claude Code

…dence leak

Public port of private PR #343 (merged 2026-09-15, merge commit 2ac78d2b).
All four files are byte-identical to the merged private/master (hash-verified).

The 0.84.0 README asserted "organisational intelligence" in the hero and then
reverted to code in every proof beneath it. A word count hid this: the first 70
lines scored 15 code-words to 19 org-words, apparently balanced, because the
org words sat in the HERO while the STRUCTURE underneath was code-only:

  - "Why GraQle exists" opened "Your AI coding tool..."
  - the 90-second proof was `scan repo` -> a code question
  - documents lived in a sub-section titled "the knowledge that isn't code",
    framing them as the exception rather than the category
  - "How it works" step 1 said documents are "auto-linked to the code they
    describe", subordinating them
  - "The compounding advantage" opened "it knows your codebase"
  - "What teams use it for" was 6 code rows to 2 org rows

RESTRUCTURE (both surfaces, kept in sync)
- The 90-second proof now leads with documents and runs on a folder with NO
  CODE IN IT AT ALL. Every number was measured, not estimated: 3 policy/ADR
  files -> 12 nodes (3 DOCUMENT + 9 SECTION) + 9 SECTION_OF edges. Then
  `graq learn knowledge "vendor DPA must be signed before any data access"
  --domain policy` extracted the entity "DPA" and created 2
  SEMANTICALLY_RELATED edges to the vendor-onboarding document AND to its
  "Due diligence" section — it found where the rule belonged unprompted.
  That is persistent, compounding organisational memory demonstrated with no
  repository present.
- Code is now an explicit wedge, not the boundary: "### The same graph, for
  code" follows the documents proof and says so plainly.
- "How it works" step 1 leads with documents/policies/ADRs; step 2 lists
  SECTION_OF and SEMANTICALLY_RELATED first.
- "What teams use it for" reordered: policies, institutional memory and
  cross-source questions first.
- After: first 70 lines score code 8 / org 34 (README) and 11 / 27 (PyPI);
  the first scan command on both surfaces is `graq scan docs`.

THIRD ATTEMPT AT THE PLACEHOLDER CONFIDENCE LEAK
`graq run` with no LLM configured printed "Confidence: 62%" directly beneath
text reading "[NO LLM CONFIGURED - this is a placeholder response]".

  Attempt 1 (shipped in 0.84.0) fixed backends/mock.py - the ANSWER TEXT only.
  Attempt 2 guarded `def reason()`. But `graq run` dispatches to `def run()`
    at main.py:401, a DIFFERENT function with its own print site. The number
    kept printing, and the source-grep test written alongside PASSED anyway -
    a false green, asserting on strings present in the wrong function.
  Attempt 3 (this commit) guards run(), safety_check_command() and both
    reason() sites, verified by RUNNING the commands.

backend_status is NOT usable here: it is only ever set to "failed" on an
exception (core/graph.py), so the fallback path leaves it "ok".
MockBackend.is_fallback is the signal that tracks the condition.

The regression test asserts per-function and scopes its sweep to
result.confidence / r.confidence. profile.confidence, env.confidence and
rec.confidence are deliberately excluded - those are computed locally from the
graph and are meaningful with no LLM configured. Narrowing the sweep rather
than weakening the guard is the point.

VERIFICATION ON THIS PUBLIC TREE
- All 4 files hash-identical to merged private/master.
- README lock gate (snapshot + eu_ai_act_docs_present): 62 passed.
- tests/test_cli/test_run_placeholder_no_confidence.py: 8 passed.
- Every graq command in both READMEs machine-diffed against `graq --help`:
  0 invalid.
- Forbidden-word scan: 0 violations in both files.

NOT DONE HERE
No version bump. 0.84.0 is merged to master but still UNTAGGED and PyPI still
serves 0.83.0, so this copy and this fix ride into the same 0.84.0 release -
PyPI never sees the weaker version.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🛡️ GraQle PR Guardian

💥 Blast Radius: 4 modules affected

Module Files Changed Risk Level Impact Radius
README.md 1 🟡 T2 1
README_PYPI.md 1 🟡 T2 1
graqle 1 🟡 T2 1
tests 1 🟢 T1 1

Total blast radius: 4


🏛️ Governance Verdict

⚠️ WARN

  • T2: Advisory warnings detected. Review recommended.

🔍 SHACL Violations

No SHACL violations detected.


🔐 Approval Requirements

This PR requires approval from: T2

  • A Senior Engineer or above must approve.

Approval requirement NOT yet satisfied.


Metric Value
Blast Radius 4
Files Analyzed 4
Blocked 0
SHACL Violations 0
Verdict WARN

🔬 Powered by GraQle PR Guardian v0.1.0 · Scan completed 2026-09-15T06:29:43.755458+00:00

@harishquantamix
harishquantamix merged commit 2ad9553 into master Sep 15, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants