Skip to content

docs(skills): task-email-audit — round-2 learnings (conditional no-sends, 401-rejected rows, retry resets)#73

Merged
sweetmantech merged 2 commits into
mainfrom
feat/audit-skill-round2-learnings
Jul 2, 2026
Merged

docs(skills): task-email-audit — round-2 learnings (conditional no-sends, 401-rejected rows, retry resets)#73
sweetmantech merged 2 commits into
mainfrom
feat/audit-skill-round2-learnings

Conversation

@sweetmantech

@sweetmantech sweetmantech commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Prep so tomorrow's post-deploy audit runs without operator feedback — net-negative diff (+25/−51): the new learnings are funded by removing what the shipped fixes made obsolete.

Added (as tightened caveats)

  1. Classify runs sent / correct no-send / failed — conditional prompts (e.g. the rewritten roster-wide new-video sweep) make no-send a success; read the run's final narration before judging.
  2. rejected ≠ always guard-blocked — expired ephemeral keys (runs >~20 min) log 401 retries as rejected in bursts; triage via the trace before reporting guard stats.
  3. Part counts can shrink — workflow-step retries reset the persisted assistant message.
  4. Vercel-login-expired fallback on the prod TRIGGER_SECRET_KEY step.

Removed (obsolete after chat#1829 + chat#1833 shipped)

  • §4 "Blocked empties" listing section — the §1 headline already counts rejected rows; interpretation now lives in the caveat.
  • §5 "Helper adoption" — the helper shipped and is a registered default skill; the adoption metric served its rollout purpose.
  • The dated 2026-07-01 "reference point" anecdote, the pre-fix Resend-scraping fallback method (no one audits pre-06-30 windows), and baseline percentages (the linked issues carry the history).
  • Sections renumbered (§6/§7 → §4/§5), all cross-refs updated.

CI green: check_resolvable, run_resolver_eval (structural), validate_manifests.

🤖 Generated with Claude Code

…eds no feedback

Three findings from the 2026-07-02 benchmark rounds that would misclassify
tomorrow's results:
- Conditional task prompts (roster-wide new-video sweep) make no-send a SUCCESS;
  classify runs sent / correct-no-send / failed by reading the run's conclusion.
- rejected rows can be expired-ephemeral-key 401s (burst of ~35 from one long run),
  not guard blocks — distinguish via the trace before reporting guard stats.
- Workflow-step retries reset the persisted assistant message (part counts shrink);
  don't read that as data loss.
Plus: vercel-login-expired fallback note on the prod-key step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR updates the "Caveats" section of the recoup-internal-task-email-audit SKILL.md documentation with expanded guidance on interpreting audit data: conditional no-send outcomes, rejected email log causes, chat_messages part-count fluctuations, and vercel login token error handling.

Changes

SKILL.md Caveats Update

Layer / File(s) Summary
Expanded audit interpretation caveats
skills/recoup-internal-task-email-audit/SKILL.md
Adds guidance for classifying conditional "no send" outcomes, distinguishing sandbox/API 401 rejections from guard-block rejections, noting chat_messages part-count shrinkage from durable workflow retries, and handling expired vercel login for TRIGGER_SECRET_KEY.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related PRs

  • recoupable/skills#69: Introduced the original Caveats section in the same SKILL.md file that this PR expands.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main docs update and the key learnings added to the task-email-audit skill.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/audit-skill-round2-learnings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

…nings, pre-fix era removed)

Per review: additions should be paid for with removals. The round-2 learnings
(conditional no-sends are successes; rejected can be 401 key-expiry; retry resets
part counts; vercel-login fallback) are now folded into tightened caveats, funded
by cutting what the shipped fixes made obsolete: the blocked-empties listing
section (§1 headline already counts them), the helper-adoption section (helper
shipped + registered), the dated 07-01 reference-point anecdote, the pre-fix
Resend-scraping fallback method, and baseline percentages (linked issues carry
the history). Sections renumbered (old §6/§7 → §4/§5), all refs updated.

Net vs main: +25/−51.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sweetmantech sweetmantech merged commit d86e68a into main Jul 2, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant