Skip to content

feat(afk): log every rule check so stats.py can report AFK sessions - #33

Merged
0x7067 merged 3 commits into
0x7067:mainfrom
griffinwork40:afk-check-log
Oct 2, 2026
Merged

0x7067 merged 3 commits into
0x7067:mainfrom
griffinwork40:afk-check-log

Conversation

@griffinwork40

@griffinwork40 griffinwork40 commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Why

In AFK the rule hooks were the one part of claude-jev you couldn't see. jev-calls.jsonl shows that Jev was called, but not which session or file, which rules, what Jev scored, or what the hook did. A check that ended before Jev was asked (no rules, none in scope, none relevant) left no record at all, so "Jev found nothing" and "Jev never ran" looked the same.

Since the adapter was installed on 2026-09-27, Griffin's machine has logged about 2,870 rules calls and 268 stop-sweep calls, and none of it shows up in stats.py.

What

adapters/afk/src/shared/check-log.ts (new) writes one row per check to jev-router-log.jsonl, tagged "host": "afk":

kind when fields
rules Jev answered same shape as src/rules.ts logDecision: phase, session_id, file, n_rules, n_scoped_out, n_irrelevant, probs by rule id, violations with band, blocked, ms
rules-skip ended before asking Jev reason: no-rules, rules-unreadable, none-in-scope, or none-relevant
rules-error the Jev call threw ms, error (300 chars)
  • rules.ts and stop-sweep.ts write the row inside judge / main, before writeOutput, so a hook killed after answering still leaves its record.
  • added_head and input_hash are null: no edited code goes into the log. The cost is that Rule outcomes cannot score AFK blocks, but it couldn't anyway, because it reads Claude Code transcripts.
  • stats.py gains AFK rule checks, printed only when AFK rows exist: per-phase checks, clean, flagged, blocked, skipped, and errors; skip reasons; the most-raised rules; and, for each warned or blocked rule, what the next check of the same file scored it (cleared, still flagged, not rechecked, no later check). That last line is labelled observational.
  • Existing sections are unchanged. AFK rules rows join Rule calibration the way Claude Code rows do; skip and error rows use their own kinds, so they don't.
  • No version bump. Left to you.

Verified

CI gates run locally in the worktree: tsc --noEmit (root and adapters/afk), oxlint (0 errors), ruff format --check and ruff check (0.16.8), compileall, check_no_comments, check_no_stubs, and the not json smoke pipes for src/rules.ts, adapters/afk/src/rules.ts, and adapters/afk/src/stop-sweep.ts.

I also fed live hook events through the adapter with CLAUDE_CONFIG_DIR set to a temp home and a real key, using an AFK.md with two rules:

event result row
edit adding console.log to src/a.ts blocked (0.98) rules, blocked: ["never-call-console-log-in"], 135 ms
README edit clean rules, prob 0.02
edit in a repo with no instruction files nothing rules-skip, no-rules
same edit with TYPESAFE_API_KEY=ts-fake-key nothing rules-error, HTTP 401
Stop with no turn rules nothing rules-skip, phase: turn, none-in-scope
clean follow-up edit to src/a.ts clean follow-through line reads cleared 1

python3 scripts/stats.py against that home exits 0 and prints the new section.

Worth knowing (AFK side, not this PR)

AFK still drops a non-blocking PreToolUse additionalContext. runPreDispatchGates applies only updatedInput from the decision, and injectContext is used only on the block path. So the README's "uncertain matches are not delivered" is still true. Griffin's AFK traces count 285 flag-band notes since 2026-09-27, and none of them reached the agent. This log is now the only place they show up. I'm filing it on agent-afk separately.


Summary by cubic

Logs every AFK rule check to jev-router-log.jsonl so stats.py can report AFK sessions. Previously the AFK adapter's rule hooks left no trace in reports, and checks that ended before Jev was asked recorded nothing at all.

What gets logged

  • One row per check, tagged "host": "afk", written before writeOutput so a hook killed after answering still leaves its record.
  • rules rows match the Claude Code hook's shape with added_head and input_hash set to null, so no edited code enters the log.
  • rules-skip rows record why the check ended (no-rules, rules-unreadable, none-in-scope, none-relevant); rules-error rows record the failure message. A Jev failure while classifying instruction files now logs as rules-error too; only a plain read failure stays rules-unreadable. Two rules sharing an id keep separate probabilities, violation ids, and block ids in the row.

stats.py report

  • New AFK rule checks section prints per-phase counts, skip reasons, the most-raised rules, and what the next check of the same file said about each warned rule, labeled as observational since there's no unwarned comparison group.
  • In that follow-through, a later rules-skip of the file counts as not rechecked, and the score comes from the next check that actually asked Jev.
  • AFK rules rows join the existing Rule calibration section; skip and error rows carry their own kinds and don't. Rule outcomes and its flagged-only list skip AFK rows; the rest of the report is unchanged.

Written for commit 170768f. Summary will update on new commits.

Review in cubic

The AFK adapter's rules.ts and stop-sweep.ts made Jev calls but wrote no
decision row, so the only AFK record was jev-calls.jsonl: no session, file,
rule ids, probabilities, or outcome, and nothing at all when a check ended
before Jev was asked.

Each check now appends one row to jev-router-log.jsonl, tagged
"host": "afk":

- kind "rules": the Claude Code hook's row shape (probs, violations,
  blocked, ms), with added_head and input_hash left null so no edited code
  is logged
- kind "rules-skip": no-rules, rules-unreadable, none-in-scope,
  none-relevant
- kind "rules-error": the failed call's message

stats.py gains an "AFK rule checks" section: per-phase counts, skip
reasons, most-raised rules, and what the next check of the same file said
about each warned rule. Existing sections are unchanged; skip and error
rows carry their own kinds so they do not enter rule calibration.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread adapters/afk/src/rules.ts Outdated
Comment thread adapters/afk/src/stop-sweep.ts
Comment thread scripts/stats.py
Comment thread scripts/stats.py Outdated
cursoragent and others added 2 commits October 2, 2026 22:53
A Jev failure while classifying instruction files was logged as
rules-skip, so stats hid it among checks that never called Jev. That
throw is now rules-error. A read failure that never calls Jev stays
rules-unreadable.

Rules that share an id kept only the last probability. Logged keys use
the id, id-2 scheme, including violation and blocked ids.

A later rules-skip of a warned file counts as not rechecked. The score
still comes from the next check that asked Jev. AFK rows stay in Rule
calibration and out of Rule outcomes.

Co-authored-by: Pedro Guimarães <0x7067@users.noreply.github.com>
CHANGELOG.md conflicted: keep the unreleased AFK log fixes and the
0.29.2 integrity-hash release that landed on main.

Co-authored-by: Pedro Guimarães <0x7067@users.noreply.github.com>
@0x7067
0x7067 merged commit 5941c75 into 0x7067:main Oct 2, 2026
2 checks passed
@griffinwork40
griffinwork40 deleted the afk-check-log branch October 3, 2026 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants