Skip to content

Add SIQ-1-35B MoE — TrueScore 87.3 (#7 leaderboard) - #1

Open
AlexWortega wants to merge 1 commit into
Weschera:mainfrom
AlexWortega:add-siq1-35b
Open

Add SIQ-1-35B MoE — TrueScore 87.3 (#7 leaderboard)#1
AlexWortega wants to merge 1 commit into
Weschera:mainfrom
AlexWortega:add-siq1-35b

Conversation

@AlexWortega

Copy link
Copy Markdown

Summary

  • SIQ-1-35B scores 87.3 TrueScore (rank #7), placing above DeepSeek V4 Flash and below Gemma 4 E4B on the leaderboard
  • Canonical run: think-OFF, repeats=2, temp=0.3, Q4_K_M on single RTX A6000 46 GB
  • Adds deployment recipe recipes/siq1-35b-moe.md with full serving command and results

Results

TrueScore Quality Cal Rel Eff Resp Pass@1 Pass@K RelGap Latency
87.3 79.2 88.6 93.0 100.0 96.8 91.2% 86.0% 5.3% 0.65s

Architecture note

SIQ-1-35B uses Qwen3_5MoeForCausalLM — a hybrid linear-attention MoE (3 linear-attn : 1 full-attn per block, 256 experts, 3B active params). vLLM and SGLang do not support this text-only arch — only llama.cpp serves it correctly via GGUF arch detection. Serving command:

llama-server -m SIQ-1-35B.Q4_K_M.gguf -ngl 99 -c 65536 \
  --host 0.0.0.0 --port 8080 --jinja --flash-attn on -np 1

Decode: 141 tok/s · Prefill: 4722 tok/s (linear-attn layers make long-context prefill very fast)

Test plan

  • Ran canonical eval (repeats=2, thinking=off, temp=0.3, timeout=300)
  • Run MD saved to results/runs/siq1-35b-20260626-134736.md
  • Leaderboard table renumbered (rows 8–17)
  • Serving method row added for SIQ-1-35B
  • Recipe file added with full results table

🤖 Generated with Claude Code

Canonical run: think-OFF, repeats=2, temp=0.3, Q4_K_M on RTX A6000.
Scores: Quality 79.2 · Cal 88.6 · Rel 93.0 · Eff 100 · Resp 96.8
Pass@1 91.2% · Pass@K 86.0% · RelGap 5.3% · median latency 0.65s

Architecture note: Qwen3_5MoeForCausalLM (hybrid linear-attn MoE,
256 experts, 3B active params). vLLM/SGLang do not support this
text-only arch — llama.cpp only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant