Skip to content

feat: switch to Qwen3-0.6B for summarization - #7

Merged
joemckenney merged 2 commits into
mainfrom
joe/qwen3
Jan 30, 2026
Merged

joemckenney merged 2 commits into
mainfrom
joe/qwen3

Conversation

@joemckenney

Copy link
Copy Markdown
Owner

Summary

  • Upgrade LLM from Qwen2.5-0.5B to Qwen3-0.6B (397MB vs 491MB - 19% smaller)
  • Simplify summarization prompt to just "Summarize this terminal output in 1-2 sentences."
  • Add /no_think directive to disable Qwen3's reasoning mode for faster, more concise output
  • Add strip_think_blocks() fallback to filter any <think> tags from output

Why Qwen3?

  • Newer model with improved instruction-following
  • Smaller download size despite more parameters
  • Better summarization quality with simpler prompts
  • Official GGUF support via unsloth/Qwen3-0.6B-GGUF

Test plan

  • cargo test passes
  • Manual testing: summaries are concise and accurate
  • Model downloads successfully from HuggingFace
  • /no_think prevents verbose reasoning output

@joemckenney
joemckenney merged commit 7a6605d into main Jan 30, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant