Skip to content

docs: record why the ruff layer stays off, and stop it failing silently - #805

Merged
quadseven merged 1 commit into
mainfrom
docs/record-why-ruff-layer-stays-off
Jul 31, 2026
Merged

docs: record why the ruff layer stays off, and stop it failing silently#805
quadseven merged 1 commit into
mainfrom
docs/record-why-ruff-layer-stays-off

Conversation

@quadseven

Copy link
Copy Markdown
Owner

Size: XS

Why

#787 shipped a deterministic lint source gated behind GRUG_LINT_EVIDENCE=ruff.
It has never run, and I could not have known why from the code. Two reasons,
both measured today and now recorded in the module so nobody repeats the
experiment.

1. ruff is not in the runtime image

Verified against the live consumer:

exec: "ruff": executable file not found in $PATH

So setting the env var alone changes nothing - every call takes the
FileNotFoundError path and returns (). The comment said "inert unless
GRUG_LINT_EVIDENCE=ruff", which was not the truth: it is inert, and would
stay inert if enabled.

2. The selection found nothing real on this codebase

Run against services/ with the exact _SELECT/_IGNORE: 127 findings, 34
outside tests, and not one confirmable true positive.

rule n what it actually caught
S105/S106 72 NAMES - SECRET_HEADER is a header name, EXPOSED_SECRET is one of grug's own SAST rule names. 67 of 72 in tests
S608 13 "SQL injection" on f-strings interpolating the module constant TTL_LIVE, every user value bound via %s - correct, safe code
B023 2 real pattern, but both closures run synchronously inside the same loop iteration
ASYNC 0 clean

What changed

Documentation and one log level. The default is unchanged - it stays OFF,
which the measurement vindicates.

Promoted the missing-binary log from info to warning: reaching it means an
operator deliberately enabled a finding source and is getting silent nothing.
That deserves a line someone will actually see.

Acceptance criteria

  • The reason the layer cannot work is stated where the flag is defined
  • The measurement is recorded with per-rule numbers, not a verdict
  • Default behaviour unchanged
  • Enabling it without the binary now logs a warning, not an info line

Out of scope

Verification

  • 1840 passed, 2 skipped
  • Every number above produced by running ruff with the module's own constants
  • Each high-volume rule inspected by hand at the source line, not counted and
    assumed

Part of #707.

#787 shipped a deterministic lint source gated behind
GRUG_LINT_EVIDENCE=ruff and it has never run. Two reasons, both
measured 2026-07-31 and now recorded in the module so nobody repeats
the experiment:

1. ruff IS NOT IN THE RUNTIME IMAGE. Verified against the live
   consumer: exec: "ruff": executable file not found in $PATH. Setting
   the env var alone changes nothing - every call takes the
   FileNotFoundError path and returns (). The comment said 'inert
   unless enabled', which was not the truth: it is inert, and would
   stay inert if enabled.

2. THE SELECTION FOUND NOTHING REAL HERE. Against services/ with the
   exact _SELECT/_IGNORE: 127 findings, 34 outside tests, not one
   confirmable true positive.
     S105/S106 (72) fired on NAMES - SECRET_HEADER is a header name,
                    EXPOSED_SECRET is one of grug's own SAST rule
                    names; 67 of 72 were in tests
     S608 (13)      'SQL injection' on f-strings interpolating the
                    module constant TTL_LIVE, every user value bound
                    via %s - correct, safe code
     B023 (2)       real pattern, but both closures run SYNCHRONOUSLY
                    inside the same iteration
     ASYNC (0)      clean

Default unchanged - it stays OFF, which the measurement vindicates.
The B family is the only part worth revisiting; S measured 0/29 and
should be dropped from _SELECT before anyone enables this. Not re-tuned
here: changing the selection without a fresh measurement is how the
last speculative source reached 71% of all output (#767).

Also promoted the missing-binary log from info to warning. Reaching it
means an operator deliberately enabled a finding source and is getting
silent nothing; that deserves a line they will actually see.

Claude-Session: https://claude.ai/code/session_01Weq76G5TQhR9ogo4zCK1Gd
@quadseven
quadseven merged commit 43debd8 into main Jul 31, 2026
18 checks passed
@quadseven
quadseven deleted the docs/record-why-ruff-layer-stays-off branch July 31, 2026 03:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant