[feat] upvote: collect adoption without relying on AI self-declaration (follow-up to #369) #723

Problem

Follow-up to #369 (closed by #371 "make upvote collection reliable for learning promotion", plus the #614 nudge-loop fix). #371 made the referenced-doc-ids nudge fire on every Stop instead of once per session and loosened the parser. Re-measured on current main: upvoted_count is still structurally zero in practice. #371 did not move the needle.

Evidence from a real team repo's votes/*.yaml (git provider):

• 1221 vote records total; 1083 have recalled_count > 0 (max 68 for a single doc).
• Only 2 have upvoted_count > 0, and both are testuser test fixtures (api-retry-pattern, k8s-rolling-upgrade). Zero real upvotes across the whole team.
• recalled_count is healthy; upvoted_count is the only counter that is dead.

Why the persistent nudge from #371 didn't fix it:

1. Upvote still depends on the main agent voluntarily writing <!-- teamai:referenced-doc-ids: [...] --> (src/builtin-rules.ts:165-176, collected at src/hook-handlers.ts:360). This is a weak contract — the model has to remember and choose to emit a specific HTML comment at the end of its reply.
2. The nudge does not actually force the model to declare. The nudge returns additionalContext on the Stop event (src/utils/hook-output.ts:13). Claude Code only blocks a stop and forces another turn on decision: "block" (the contract is called out in src/omp-hooks.ts:24); additionalContext on Stop does not continue the session. So the session ends and the declaration is never produced. Empirically, scanning 2387 local transcripts, 54 carried a recall signal but only ~2 carried a genuine (non-placeholder, non-subagent) referenced-doc-ids declaration — the rest were subagent noise or rule-template placeholders (doc-id-1, 你实际用到的-doc-id).
3. Even a correct declaration can be silently dropped. One session declared 5 doc-ids that exactly matched its recalled set (re-running the parser confirms verified = 5), yet that user's local and team votes files show upvoted_count: 0, no upvoted_delta, no last_upvoted_at for all 5. The whole handler is wrapped in try { … } catch {} with no logging (src/hook-handlers.ts:449), so any failure — 4.5s Stop timeout (src/hook-handlers.ts:58), path/username mismatch, a parse throw — vanishes without a trace.

Downstream effect: the upvoted term and the upvoted/recalled ratio in confidence collapse to 0 (src/maintenance/confidence.ts), and the promote gate upvoted >= 5 is unreachable. This issue is only about the collection mechanism, not the thresholds (see Alternatives).

Proposed Solution

Soliciting design input for a new collection mechanism that does not depend on the AI voluntarily declaring references. The design space, for discussion (not prescribing one):

1. Background LLM-judge (detached, after session). When a session recalled docs, a background: true Stop handler (mirroring the updateHandler detached pattern) asks the local signed-in CLI whether the final reply substantively used each recalled doc, then writes upvotes. Pros: no AI cooperation needed, no extra in-session turn, no loop risk. Cons: one extra local-CLI LLM call per recall session; only helps upgraded users; ceiling limited by how often recall is genuinely adopted (adoption is structurally sparse).
2. Tool-use evidence. Count a recall as adopted when the agent opens the recalled doc's file via Read/Grep/Glob in the same session. High precision, zero cooperation needed. But near-zero recall for learnings, because the recommended recall path injects the subagent's summary as text and the model rarely opens the file itself — so this alone barely feeds promotion.
3. Subagent-side attribution. The teamai-recall subagent already reliably emits recalled-doc-ids (.claude/agents/teamai-recall.md). Explore attributing adoption from the subagent interaction rather than the main agent self-declaring.

Whatever mechanism is chosen, it should: (a) not require the AI to self-declare; (b) not force extra turns or risk loops (cf. #614); (c) log failures instead of swallowing them — the silent catch {} at src/hook-handlers.ts:449 must go regardless; (d) keep HTTP teams out of upvote collection (see Alternatives).

Alternatives Considered

• Switch the nudge to decision: "block" with a per-session cap. Would force the model to declare, but is intrusive (an extra turn + cost on every recall session), the model can still declare empty [] → 0 upvotes, and it is loop-prone (the exact failure fix: prevent infinite nudge loop when model declares empty referenced-doc-ids #614 patched). Rejected as the primary path; at most a cap-bounded supplement.
• Lower the promote upvoted >= 5 threshold. Explicitly out of scope. This papers over the loss instead of fixing collection; fix(votes): make upvote collection reliable for learning promotion #371 already chose not to touch thresholds, and this issue keeps that line. The goal is for real upvote signal to flow.
• Relax gitOnly: true on votesSyncHandler (src/hook-handlers.ts:565) so HTTP-provider teams collect upvotes. Out of scope — HTTP teams do not need upvote collection by design; gitOnly stays.

Additional Context

Key code locations:

Related: #369 (original diagnosis), #371 (persistent-nudge fix that did not resolve it), #614 (nudge-loop fix), #368 (teamai viz surfaces these counts).

Scope guards (per discussion before filing): do not change promote thresholds; HTTP teams do not need upvote collection (gitOnly is intentional).