Skip to content

W3 reasoning-quota wall — public port (ADR-245 Decision 8) - #245

Merged
harishquantamix merged 5 commits into
masterfrom
cr-mon-w3-reasoning-quota-public
Jul 30, 2026
Merged

harishquantamix merged 5 commits into
masterfrom
cr-mon-w3-reasoning-quota-public

Conversation

@quantamixsol

@quantamixsol quantamixsol commented Jul 29, 2026

Copy link
Copy Markdown
Owner

W3 reasoning-quota wall — public port (ADR-245 Decision 8)

Ports the W3 reasoning-quota monetisation wall to the public SDK. This is a
cherry-pick of already-merged private work, not new design:

Private PR Commit Public commit here
#316 — reasoning-quota wall 39d4bf48 271d6ae5
#321 — move the wall to the SDK primitive (Decision 8) 88567ae3 bb75efe3

⚠️ Two commits, not one. The public tree had neither, so reasoning_quota.py
arrived as a modify/delete conflict when #321 was picked alone. Both are required in order.

What it does

A free monthly reasoning allowance. Over the cap, a FREE-tier user gets
ReasoningQuotaExceeded and an upgrade CTA; reads and the graph keep working.

Where the wall sits — the whole point of Decision 8

graqle/core/graph.py, three sites:

Site Why it needs its own gate
areason() the SDK primitive — CLI, MCP, chat, server/app.py all funnel through it
areason_stream() builds its own StreamingOrchestrator and never reaches areason()
reason() (sync) delegates to areason(); deliberately NOT internal=True

Plus cli/commands/debate.pyDebateOrchestrator also reasons outside areason(),
so it carries an explicit check_reasoning_quota().

Rule 1: a wall at one surface is not a wall. The original spec named one bypass;
the audit found three. areason_stream and graq debate are those bypasses.

Rule 2: internal=True means NEVER METERED, not ALREADY METERED. Marking the sync
reason() wrapper internal would have made the entire synchronous API free — the
PR #316 bypass in mirror image. A delegating wrapper must not be internal; only its
delegate charges.

Rule 3: no self-attested env exemptions. internal=True appears only at
benchmark/graq bench callsites in code. There is deliberately no CI=true
exemption — that would be a one-line unlimited-reasoning bypass.

Verification

  • Byte-identical to private/master for all 5 implementation files
    (reasoning_gate.py, reasoning_quota.py, debate.py, and both test files).
  • 269 licensing tests pass, 1 skipped.
  • areason_batchself.areason(...) with no internal arg ⇒ N-query batch = N units.
  • Gate imports are stdlib only + lazy graqle.licensing.*; zero references to
    graqle/cloud/* or graqle/server/*, so there is no ImportError→fail-open path.
  • import graqle.cli.main succeeds — no circular import.
  • Complete Orchestrator enumeration: no ungated reasoning path exists.

Pre-existing red — not from this diff

tests/test_cli/test_gate_install.py fails 9 of 44 identically on the untouched base
c51d1f26 and on this branch. Verified by checking out the base and re-running.
Unrelated to W3.

Governance — ADR-209

Phase Status
1 BOOT ✅ 75,871 nodes on bolt, graph_health.degraded: false
2 INVESTIGATE ✅ wheel built + inspected; merge state verified via gh and base advanced
3 BLAST RADIUS ✅ MEDIUM — no escalation
4 PLAN plan_3859cba5
5–6 GENERATE/APPLY ✅ cherry-pick; graq_edit(strategy="literal") for docs
7 SENTINEL graq_reason ×2 on the full diff (graq_review is DOWN). Pass 1: 5 BLOCKERs, all self-labelled unverified. Each reproduced against code and refuted with evidence. Pass 2: APPROVE, 0 blockers, 87%
8 PR ✅ this PR
9 LEARN ⏳ after merge

Rule #0git remote -v run before push. Public cherry-pick authorised only
because private #321 is merged (mergedAt 2026-07-29T10:53:02Z, base advanced to
65be4f75). Sequential gate satisfied. No version files in this diff; 0.83.0 release
is a separate PR.
Stray (inferred from graph) file excluded.

🤖 Generated with Claude Code


Update — rebased onto 21c91915 + a real regression fixed

Master gained CR-010.R6 (#244, scheduler contract), which also edits
graqle/core/graph.py. Rebased. GitHub reported MERGEABLE, zero conflicts — R6's
hunks are lines 33–1185, W3's are 1774–2020, no textual overlap.

But MERGEABLE is not correct. Running the suites together after the rebase
surfaced 11–12 real failures that git could not have detected:

Command Plain master 21c91915 With W3
pytest tests/test_licensing/ tests/test_cli/test_headless_contract.py 249 passed 11 failed

Each file passed alone on both. Ordering-dependent pollution, introduced by W3's tests.

Root cause (traced to a real traceback, not inferred)

test_reasoning_gate.py imported graqle.server.app in-process, leaving
graqle.server in sys.modules. graqle/governance/tamper_evidence/verifier.py::_assert_isolated
(moat-M2 / WS-A3) raises ImportError when any server/studio module is already
loaded. R6's suite is the first to load the verifier after the licensing suite — so the
collision only appears once both features coexist.

Second, worse defect — found while fixing the first

That test imported ReasonRequest from graqle.server.app, which does not define
it
(the models live in graqle.server.models), under a bare
except Exception: pytest.skip(...). The ImportError was swallowed, so this security
assertion — no HTTP request field may inject internal=True and buy unmetered
reasoning
never executed on any machine, on private or public.

A skip-on-bare-exception around an import is indistinguishable from a pass.

The fix (one file)

  • Probe moved to a subprocess — the server import cannot touch the parent's sys.modules.
  • Import corrected to graqle.server.models; widened to cover BatchReasonRequest too.
  • Skip narrowed to except ImportError only; anything else now fails.
  • Interpreter guard: CI runs the bare pytest shim (ci.yml:42), not
    python -m pytest, so sys.executable may not be the venv holding graqle. Without a
    guard the probe would exit 3 and skip — silently recreating the very defect this
    rewrite fixes.
    It now asserts import graqle succeeds first and fails loudly.

Evidence

  • Security test now runs (was skipping): licensing 270 passed, up 1.
  • Mutation-tested twice: probing a field that does exist → rc=1 (the assertion can
    fail); an interpreter that cannot import graqle → rc=1 (guard fails loud, never skips).
  • Previously-failing combo: 346 passed, 0 failed.
  • graqle.server in sys.modules after the licensing suite → False.
  • W3 gates intact at graph.py:1899/1901 (areason) and :2117/2119 (areason_stream);
    sync reason() still ungated by design. R6 intact.
  • test_gate_install.py fails 10/608 identically on base and branch — pre-existing,
    proven by checking out the base and re-running.

Sentinel (graq_review DOWN → graq_reason ×2, full graph, non-degraded)

Pass 1 BLOCKED with 2 blockers — both correct this time, and blocker 2's premise
was verified against the real CI config before fixing. Both fixed. Pass 2:
APPROVE, 0 blockers, 92%, unanimous.

Known red — infra, not this diff

Release Gate (PyPI): release_gate prediction provider failed: FileNotFoundError → exit 2.
Documented repo-wide flake (empty GRAQLE_LICENSE, no graqle.json/Neo4j in CI); fails on
8/8 branches including ones that merged fine. Needs an owner-approved infra fix.


Update 2 — independent pre-merge review found a paywall bypass

The earlier sentinel passes were process gates on a diff-in-progress. This is a fresh
line-by-line read of the whole change. It found 3 defects, each proven by execution.

🔴 F1 (BLOCKER) — GRAQLE_QUOTA_DIR was a one-line unlimited-reasoning bypass

resolve_quota_dir() honoured the env var unconditionally, so the meter could be
pointed anywhere. Proven: a FREE user at 30/30 gets ReasoningQuotaExceeded; set
GRAQLE_QUOTA_DIR to a fresh temp dir and the same call is allowed.

export GRAQLE_QUOTA_DIR=$(mktemp -d)   # in ~/.bashrc → unlimited free reasoning

Worse than the local-bypass class ADR-245 already accepts (deleting the quota file)
on three counts: it persists via a shell profile (set once, never repeated), it is
non-destructive (nothing looks tampered with), and it needs no repeat action.

It also contradicted this module's own rule — quota_exempt() refuses to honour
CI=true precisely because self-attested env exemptions are banned by Decision 8
rule 3
. An env var that relocates the counter is that same bypass wearing a different hat.

Fix: the override is honoured only when "pytest" in sys.modules.

⚠️ My first attempt used PYTEST_CURRENT_TESTthe sentinel correctly called that
theatre
: it is itself a self-attested env var, and I reproduced the forgery
(PYTEST_CURRENT_TEST=forged::call → override applied). It was also wrong on
timing
— that var is unset during import/collection/session fixtures (measured:
False at import while "pytest" in sys.modules is already True), so a test
resolving the dir there would have silently written to the developer's real
./.graqle. sys.modules cannot be forged from the environment and is true from
process start. Both blockers fixed.

F2 (follow-up) — unbounded file growth

data[month] = used + 1 never pruned. Seeded 36 months → all 36 retained. Now trimmed to
13 months on write; schema_version preserved; unparseable keys left alone.

F3 (follow-up) — peek() disagreed with check_and_record()

peek() checked only the tier. With GRAQLE_ENFORCE_CAPS=0 at the cap it reported
allowed=False while calls actually succeeded. Not user-visible today — its sole
caller is check_and_record's own exempt-path return (grep: 1 hit) — but the two must
not drift.

Evidence

  • Mutation-tested ×4 — reverting each fix individually makes its test fail;
    restoring it passes. Verified by editing the source back and re-running.
  • Attack now closed end-to-end: forging PYTEST_CURRENT_TEST resolves to .graqle.
  • Suite 350 passed / 0 failed (was 346 + 4 new regression tests).
  • Sentinel ×2 on these fixes: pass 1 BLOCKED (both blockers correct, both fixed),
    pass 2 APPROVE, 0 blockers, 93%.

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

🛡️ GraQle PR Guardian

💥 Blast Radius: 10 modules affected

Module Files Changed Risk Level Impact Radius
graqle 8 🟡 T2 8
tests 2 🟢 T1 2

Total blast radius: 10


🏛️ Governance Verdict

⚠️ WARN

  • T2: Advisory warnings detected. Review recommended.

🔍 SHACL Violations

No SHACL violations detected.


🔐 Approval Requirements

This PR requires approval from: T2

  • A Senior Engineer or above must approve.

Approval requirement NOT yet satisfied.


Metric Value
Blast Radius 10
Files Analyzed 10
Blocked 0
SHACL Violations 0
Verdict WARN

🔬 Powered by GraQle PR Guardian v0.1.0 · Scan completed 2026-07-30T15:42:02.225456+00:00

quantamixsol and others added 3 commits July 30, 2026 08:11
The reasoning-FREQUENCY wall in the multi-wall lattice (independent of the node-cap
SIZE wall). Free tier = 30 graq run/reason invocations per calendar month; paid
unlimited. Turns "reason a lot for free forever" into a real upgrade trigger.

- NEW graqle/licensing/reasoning_quota.py: ReasoningQuota + ReasoningQuotaExceeded.
  Tier from the VERIFIED manager.current_tier ONLY (a raw env/key never grants
  unlimited — CR-LIC-03b rule). Enforced ON by default (GRAQLE_ENFORCE_CAPS opt-OUT,
  shared with node-cap). internal=True EXEMPT (reasoning invoked by another metered
  action e.g. PR-Guardian never double-charges). FAIL-OPEN on any file/meter error —
  only ReasoningQuotaExceeded escapes (re-raised before the broad except).
- main.py: _enforce_reasoning_quota() gates the 4 USER-value reasoning paths BEFORE
  areason runs: run(), bench() query loop, reason() single + batch. The 2 internal
  probes (bench smoke-test, safety_check) are correctly exempt.

Sentinel graq_reason pass1 BLOCK (BLOCKER-1: 3 of 6 areason sites ungated) -> gated the
user paths, classified the internal probes -> pass2 APPROVE 92% 0-BLOCKER (MAJOR-2
exception-escapes-fail-open + MAJOR-3 paid-tier fail-safe both verified). 17 tests
(block-after-cap, paid-unlimited, internal-exempt, verified-tier-not-raw-env, fail-open,
malformed-json, quota-escapes-fail-open). Zero regressions (pre-existing gate_install
failures confirmed on clean master). Ships in the public wheel. Own version lane — NOT 0.81.0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PR #316 put the quota wall in the CLI only, so the MCP graq_reason tool, the
chat agent and api.py reached graph.areason() around it — free unlimited
reasoning on a primary surface. The rule from that review: a wall at one
surface is not a wall. This enforces at the primitive instead.

- NEW graqle/licensing/reasoning_gate.py: one middleware, check_reasoning_quota().
  internal=True is the ONLY exemption; ReasoningQuotaExceeded is re-raised before
  the broad except (fail-open on genuine meter faults, now logged at WARNING).
- graph.areason() and graph.areason_stream() gate on entry. reason() delegates to
  areason (charged once); areason_batch() fans out per query (charged N).
- graq debate gated: DebateOrchestrator.run() never reached areason.
- reasoning_quota.py: cross-process file lock (msvcrt/fcntl) around the
  read-modify-write so racing runs cannot both spend the last unit.
- CLI keeps only the upgrade-CTA wrapper; it no longer meters (no double-charge).
- Benchmark runners + graq bench pass internal=True so evals never block a
  free-tier contributor.

Sentinel: 3 passes on the full 75,773-node graph. Pass 1 BLOCK (3 blockers) ->
pass 2 BLOCK (2 new) -> pass 3 APPROVE 91%, 0 blockers. BLOCKER-1 (unwalled
areason_stream + debate) was real and is fixed. BLOCKER-2 (CI auto-exempt) was
real: `export CI=true` was a one-line bypass — the exemption is removed entirely.
BLOCKER-3/4/5 were refuted with code evidence and pinned as regression tests.

Tests: 269 licensing pass, 100% statement coverage on both quota modules, 354
core pass, 521 CLI pass (10 pre-existing test_gate_install failures, identical
set with and without this change). Five mutation experiments prove detection:
removing either wall, disabling the lock (8/8 threads took the last unit),
inverting the re-raise, and injecting internal=True at a server endpoint each
fail the corresponding guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…urity assertion

Two defects in test_server_request_model_cannot_set_internal, both surfaced by
rebasing W3 onto master's CR-010.R6:

1. sys.modules pollution. Importing a graqle.server module in-process leaves it
   loaded for the rest of the session, and tamper_evidence.verifier refuses to
   import when a server/studio module is present (moat-M2 _assert_isolated,
   WS-A3). One import in this file caused 11 unrelated failures in
   tests/test_cli/test_headless_contract.py when the suites ran together, while
   each file passed alone. Now probed in a subprocess.

2. The assertion never ran. It imported ReasonRequest from graqle.server.app,
   which does not define it (the models live in graqle.server.models), under a
   bare `except Exception: pytest.skip(...)`. The ImportError was swallowed, so
   this security check silently skipped on every machine. A skip-on-any-exception
   guard around an import is indistinguishable from a pass.

Sentinel pass 1 raised two further blockers, both real and both fixed here:

- Skip narrowed to ImportError (not ModuleNotFoundError). A genuinely-absent
  optional extra can surface as a plain ImportError — a transitive C-extension
  that fails to load, or a broken re-export — which would have hard-failed CI on
  a machine where the extra is simply not installed.
- Interpreter guard added. CI runs the bare `pytest` shim (ci.yml:42), not
  `python -m pytest`, so sys.executable is not guaranteed to be the venv holding
  graqle. Without the guard the probe would exit 3 and SKIP, silently losing the
  assertion again — the very defect this rewrite fixes. It now fails loudly:
  a wrong interpreter is an environment bug, never a reason to drop coverage.

Also widened to cover BatchReasonRequest, not just ReasonRequest.

Mutation-tested twice: probing a field that DOES exist returns rc=1 (the
assertion can fail), and an interpreter that cannot import graqle returns rc=1
(the guard fails loudly rather than skipping). Combined run 346 passed / 0
failed (was 12 failed); licensing 270 (up 1 — the un-skipped test); no server
leak. W3 gates intact at graph.py:1899/2117 and R6 intact post-rebase.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
quantamixsol and others added 2 commits July 30, 2026 17:35
…ne old months

Pre-merge line-by-line review of this PR found three defects the process gates
missed. Each is fixed and pinned with a mutation-tested regression test.

F1 (BLOCKER) — GRAQLE_QUOTA_DIR was a self-attested one-line paywall bypass.
resolve_quota_dir() honoured the env var unconditionally, so a FREE user at the
cap who ran `export GRAQLE_QUOTA_DIR=$(mktemp -d)` got a fresh empty counter on
every invocation: unlimited free reasoning. Proven by execution — the same
check_reasoning_quota() call raised ReasoningQuotaExceeded before repointing the
var and was ALLOWED after.

This is categorically worse than the local-bypass class ADR-245 already accepts
(deleting .graqle/reasoning_quota.json): it persists via a shell profile so it is
set once and never repeated, it is non-destructive so nothing looks tampered
with, and it needs no repeat action. It also contradicted this module's own rule
— quota_exempt() refuses to honour CI=true precisely because self-attested env
exemptions are banned (Decision 8 rule 3). An env var that relocates the counter
is that same bypass wearing a different hat.

Fix: the override is honoured only while PYTEST_CURRENT_TEST is set, i.e. inside
a real pytest process. Tests can still redirect the meter; a production run
cannot. Forging the marker is no longer a quiet, documented override.

F2 (follow-up) — the quota file grew a month key forever; nothing pruned it.
Now trimmed to 13 months on write. Non-month keys (schema_version) are preserved,
and unparseable keys are left alone rather than silently discarded.

F3 (follow-up) — peek() checked only the tier while check_and_record() also
short-circuits on quota_enforcement_enabled(), so with GRAQLE_ENFORCE_CAPS=0 at
the cap peek() said allowed=False while calls actually succeeded. Nothing
user-facing consumes peek() today (its only caller is the exempt-path
short-circuit), but the two must not drift.

Mutation-tested: reverting each fix individually makes its test fail (verified
all three). Bypass confirmed closed end-to-end: resolve_quota_dir() returns
.graqle in a production run regardless of the env var. Suite 350 passed / 0
failed (was 346 + 4 new).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sentinel pass 1 blocked the previous commit and was right on both counts.

BLOCKER 1 — the fix was theatre. _under_pytest() keyed off PYTEST_CURRENT_TEST,
which is itself a self-attested env var: it replaced one bypass with another.
Proven forgeable:
    PYTEST_CURRENT_TEST=forged::call GRAQLE_QUOTA_DIR=/tmp/x  ->  resolved /tmp/x
Now `"pytest" in sys.modules or "_pytest" in sys.modules`. An attacker cannot
import pytest into a process that never imported it, so sys.modules closes what
an env check cannot. Same attack now resolves to .graqle.

BLOCKER 2 — wrong timing, silent real-store pollution. PYTEST_CURRENT_TEST is
set per-test (setup/call/teardown) and is ABSENT during module import,
collection and session-scoped fixtures. Measured at import time:
PYTEST_CURRENT_TEST -> False, "pytest" in sys.modules -> True. So a test that
resolved the quota dir in any of those phases would have silently fallen through
to the developer's REAL ./.graqle and mutated their actual quota count.
sys.modules is true from process start, covering every phase uniformly.

The regression test now runs the production case in a SUBPROCESS (in-process,
pytest is necessarily loaded) and FORGES PYTEST_CURRENT_TEST inside it, pinning
the refutation so the weak guard cannot come back. The companion test asserts
the override still works with PYTEST_CURRENT_TEST deleted — proving the
collection-time path is covered.

Mutation-tested: restoring the env-var guard makes the forged-env test fail.
Suite 350 passed / 0 failed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@harishquantamix
harishquantamix merged commit b79c63d into master Jul 30, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants