Repository navigation
feat(afk): log every rule check so stats.py can report AFK sessions - #33
Merged
Merged
Conversation
The AFK adapter's rules.ts and stop-sweep.ts made Jev calls but wrote no decision row, so the only AFK record was jev-calls.jsonl: no session, file, rule ids, probabilities, or outcome, and nothing at all when a check ended before Jev was asked. Each check now appends one row to jev-router-log.jsonl, tagged "host": "afk": - kind "rules": the Claude Code hook's row shape (probs, violations, blocked, ms), with added_head and input_hash left null so no edited code is logged - kind "rules-skip": no-rules, rules-unreadable, none-in-scope, none-relevant - kind "rules-error": the failed call's message stats.py gains an "AFK rule checks" section: per-phase counts, skip reasons, most-raised rules, and what the next check of the same file said about each warned rule. Existing sections are unchanged; skip and error rows carry their own kinds so they do not enter rule calibration.
There was a problem hiding this comment.
All reported issues were addressed across 6 files
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
A Jev failure while classifying instruction files was logged as rules-skip, so stats hid it among checks that never called Jev. That throw is now rules-error. A read failure that never calls Jev stays rules-unreadable. Rules that share an id kept only the last probability. Logged keys use the id, id-2 scheme, including violation and blocked ids. A later rules-skip of a warned file counts as not rechecked. The score still comes from the next check that asked Jev. AFK rows stay in Rule calibration and out of Rule outcomes. Co-authored-by: Pedro Guimarães <0x7067@users.noreply.github.com>
CHANGELOG.md conflicted: keep the unreleased AFK log fixes and the 0.29.2 integrity-hash release that landed on main. Co-authored-by: Pedro Guimarães <0x7067@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
In AFK the rule hooks were the one part of claude-jev you couldn't see.
jev-calls.jsonlshows that Jev was called, but not which session or file, which rules, what Jev scored, or what the hook did. A check that ended before Jev was asked (no rules, none in scope, none relevant) left no record at all, so "Jev found nothing" and "Jev never ran" looked the same.Since the adapter was installed on 2026-09-27, Griffin's machine has logged about 2,870
rulescalls and 268stop-sweepcalls, and none of it shows up instats.py.What
adapters/afk/src/shared/check-log.ts(new) writes one row per check tojev-router-log.jsonl, tagged"host": "afk":rulessrc/rules.tslogDecision:phase,session_id,file,n_rules,n_scoped_out,n_irrelevant,probsby rule id,violationswith band,blocked,msrules-skipreason:no-rules,rules-unreadable,none-in-scope, ornone-relevantrules-errorms,error(300 chars)rules.tsandstop-sweep.tswrite the row insidejudge/main, beforewriteOutput, so a hook killed after answering still leaves its record.added_headandinput_hasharenull: no edited code goes into the log. The cost is thatRule outcomescannot score AFK blocks, but it couldn't anyway, because it reads Claude Code transcripts.stats.pygainsAFK rule checks, printed only when AFK rows exist: per-phase checks, clean, flagged, blocked, skipped, and errors; skip reasons; the most-raised rules; and, for each warned or blocked rule, what the next check of the same file scored it (cleared,still flagged,not rechecked,no later check). That last line is labelled observational.rulesrows joinRule calibrationthe way Claude Code rows do; skip and error rows use their own kinds, so they don't.Verified
CI gates run locally in the worktree:
tsc --noEmit(root andadapters/afk),oxlint(0 errors),ruff format --checkandruff check(0.16.8),compileall,check_no_comments,check_no_stubs, and thenot jsonsmoke pipes forsrc/rules.ts,adapters/afk/src/rules.ts, andadapters/afk/src/stop-sweep.ts.I also fed live hook events through the adapter with
CLAUDE_CONFIG_DIRset to a temp home and a real key, using anAFK.mdwith two rules:console.logtosrc/a.tsrules,blocked: ["never-call-console-log-in"], 135 msrules, prob 0.02rules-skip,no-rulesTYPESAFE_API_KEY=ts-fake-keyrules-error,HTTP 401rules-skip,phase: turn,none-in-scopesrc/a.tscleared 1python3 scripts/stats.pyagainst that home exits 0 and prints the new section.Worth knowing (AFK side, not this PR)
AFK still drops a non-blocking
PreToolUseadditionalContext.runPreDispatchGatesapplies onlyupdatedInputfrom the decision, andinjectContextis used only on the block path. So the README's "uncertain matches are not delivered" is still true. Griffin's AFK traces count 285 flag-band notes since 2026-09-27, and none of them reached the agent. This log is now the only place they show up. I'm filing it on agent-afk separately.Summary by cubic
Logs every AFK rule check to
jev-router-log.jsonlsostats.pycan report AFK sessions. Previously the AFK adapter's rule hooks left no trace in reports, and checks that ended before Jev was asked recorded nothing at all.What gets logged
"host": "afk", written beforewriteOutputso a hook killed after answering still leaves its record.rulesrows match the Claude Code hook's shape withadded_headandinput_hashset to null, so no edited code enters the log.rules-skiprows record why the check ended (no-rules,rules-unreadable,none-in-scope,none-relevant);rules-errorrows record the failure message. A Jev failure while classifying instruction files now logs asrules-errortoo; only a plain read failure staysrules-unreadable. Two rules sharing an id keep separate probabilities, violation ids, and block ids in the row.stats.py report
AFK rule checkssection prints per-phase counts, skip reasons, the most-raised rules, and what the next check of the same file said about each warned rule, labeled as observational since there's no unwarned comparison group.rules-skipof the file counts asnot rechecked, and the score comes from the next check that actually asked Jev.rulesrows join the existingRule calibrationsection; skip and error rows carry their own kinds and don't.Rule outcomesand its flagged-only list skip AFK rows; the rest of the report is unchanged.Written for commit 170768f. Summary will update on new commits.