MentallyChallenged Qwen

Controlled experiments for editing local LLM weights with disorder-inspired trait profiles, then measuring where each variant improves or fails on official benchmarks.

Important: this project does not claim that a model has ADHD, autism, OCD, mania, dementia, depression, or any human condition. The profile names are engineering shorthand for reproducible weight perturbations inspired by cognitive traits such as distractibility, detail focus, associative fluency, memory decay, over-stabilized checking, or reduced initiative.

Headline Results

Main base model:

Qwen/Qwen2.5-3B-Instruct

The strongest results come from a two-stage pipeline:

1. Apply a disorder-inspired weight perturbation profile to Qwen2.5-3B.
2. Train a small LoRA adapter on an official benchmark training split and evaluate with EleutherAI lm-evaluation-harness.

Current best Qwen trait seeds by official skill:

Skill	Official task	Best trait seed	Score	Baseline	Relative gain
Science / pattern reasoning	ARC-Challenge acc	ADHD-like	0.5700	0.4275	+33.3%
Science / pattern confidence	ARC-Challenge acc_norm	Mania-like	0.5717	0.4369	+30.9%
Physical commonsense	PIQA acc	OCD-like	0.7862	0.7497	+4.9%
Physical commonsense confidence	PIQA acc_norm	Autism-like	0.7933	0.7356	+7.8%
Open-book science QA	OpenBookQA acc	Autism-like / OCD-like	0.4140	0.3320	+24.7%
Open-book science QA confidence	OpenBookQA acc_norm	OCD-like	0.5180	0.4460	+16.1%

That split is the core finding: different trait profiles specialize in different benchmark categories.

Fair Base-LoRA Controls

The latest check trained the unmodified Qwen base model with the same LoRA recipes. That control explains most of the large gains, but not all of the category ordering:

Category	Best trait seed	Trait score	Base-LoRA control	Trait edge
ARC raw accuracy	ADHD-like	0.5700	0.5631	+0.0068
ARC normalized accuracy	Mania-like	0.5717	0.5717	+0.0000
PIQA raw accuracy	OCD-like	0.7862	0.7840	+0.0022
PIQA normalized accuracy	Autism-like	0.7933	0.7911	+0.0022
OpenBookQA raw accuracy	Autism-like / OCD-like	0.4140	0.4080	+0.0060
OpenBookQA normalized accuracy	OCD-like	0.5180	0.5080	+0.0100

Four ARC improvement attempts did not beat the current ADHD-like winner: LoRA rank 8 reduced raw ARC accuracy to 0.5563, lowering the learning rate to 1e-4 reduced it to 0.5410, expanding the adapter to all attention/MLP projections fell to 0.5324, and a second ADHD-like seed landed at 0.5529.

The latest refinement tried to push the strongest ARC confidence profile harder: mania-like s0.25, s0.5, and s0.75 were all trained with the same ARC-Challenge LoRA recipe. The result was a real sweet spot rather than a monotonic gain: s0.25 improved mania raw accuracy to 0.5657, but s0.5 remained best on normalized accuracy at 0.5717, and s0.75 degraded.

How The Profiles Work

Profiles are YAML files in profiles/ that specify targeted edits to model tensors:

• Gaussian noise scaled by each tensor's standard deviation.
• Attention pathway edits split across query/key focus and value/output context paths.
• MLP pathway edits split across gate/up associative pathways and down-projection consolidation paths.
• Optional embedding, language-head, and normalization scaling.
• Layer-banded edits so early, middle, and late layers can be affected differently.

Examples:

Profile	Engineering hypothesis	Current strongest result
adhd_like	Attention jitter plus associative MLP boost	Best ARC-Challenge raw accuracy
mania_like	Amplified associative fluency with more output energy	Best ARC-Challenge normalized accuracy
ocd_like	Over-stabilized attention/norms and checking behavior	Best PIQA raw accuracy and OpenBookQA normalized accuracy
autism_like	Sharpened attention and rule/detail stability	Best PIQA normalized accuracy and tied OpenBookQA raw accuracy
dyslexia_like	Noisier token/output language pathways with preserved semantic paths	Solid PIQA and ARC transfer
anxiety_like	Monitoring/error-avoidance bias with damped throughput	Strong ARC normalized accuracy tier

See profiles/TRAIT_MANIFEST.md for the full profile map and source grounding.

Reproduce The Main Qwen Matrix

Set up the environment:

python -m venv .benchvenv
.\.benchvenv\Scripts\Activate.ps1
python -m pip install -U pip
python -m pip install -e .[train]

Install the CUDA-enabled PyTorch build that matches your GPU/driver from the official PyTorch install page before running GPU training.

Run the current official trait-refinement matrix:

$env:CWL_QWEN_TRAIT_PROFILES = "adhd_like,autism_like,ocd_like,anxiety_like,dyslexia_like,mania_like"
$env:CWL_QWEN_TRAIT_TASKS = "arc_challenge,piqa,openbookqa"
$env:CWL_QWEN_TRAIT_MAX_STEPS = "4000"
$env:CWL_QWEN_TRAIT_LORA_R = "4"
.\scripts\run_qwen_trait_refinement_matrix.ps1

This creates local model checkpoints under models/, raw evaluation outputs under results/, and uses official benchmark tasks through lm-evaluation-harness.

Run A Single Perturbation

python -m cognitive_weight_lab.perturb `
  --model Qwen/Qwen2.5-3B-Instruct `
  --profile profiles/adhd_like.yaml `
  --severity 0.5 `
  --output models/qwen2.5-3b-adhd-like-s0_5 `
  --dtype float16

Run Official Evaluation

$env:CWL_EVAL_MODEL = "models/official-tuned/qwen2.5-3b-mania-like-s0_5-arc-challenge-lora-r4"
$env:CWL_EVAL_TASKS = "arc_challenge"
$env:CWL_EVAL_OUT = "results/official-lm-eval-mania-arc"
.\scripts\run_official_lm_eval.ps1

Default official tasks are hellaswag,arc_challenge,winogrande; longer suites like GSM8K or MMLU should be run as dedicated jobs.

Repository Layout

profiles/                      Disorder-inspired perturbation profiles
src/cognitive_weight_lab/       Perturbation and benchmark code
scripts/                        Training, evaluation, and matrix runners
docs/official-results/          Committed official benchmark summaries and charts
docs/benchmark-graphs/          Custom-suite matrix charts
BENCHMARKS.md                   Detailed experiment log

Generated checkpoints and raw outputs are intentionally ignored by git:

models/
results/
cache/
*.safetensors

Official Result Artifacts

• docs/official-results/qwen_trait_skill_winners.csv
• docs/official-results/qwen_trait_skill_winners.svg
• docs/official-results/qwen_base_lora_controls.csv
• docs/official-results/qwen_base_lora_controls.svg
• docs/official-results/qwen_arc_optimization_attempts.csv
• docs/official-results/qwen_arc_optimization_attempts.svg
• docs/official-results/qwen_arc_trait_refinement.svg
• docs/official-results/qwen_mania_severity_sweep.csv
• docs/official-results/qwen_mania_severity_sweep.svg
• docs/official-results/arc_challenge_lora_results.csv
• docs/official-results/piqa_lora_results.csv
• docs/official-results/openbookqa_lora_results.csv

Research Writeup Prep

• docs/research/RESEARCH_JOURNAL.md is a journal-style pilot manuscript draft based on the current data.
• docs/research/RESEARCH_PROTOCOL.md defines the minimum replication matrix needed for a journal-style claim.
• docs/research/MANUSCRIPT_OUTLINE.md sketches the paper structure and safe result language.
• docs/research/official_results_all.csv aggregates local official benchmark JSON outputs.
• docs/research/qwen_trait_claims.csv compares Qwen variants against the matching Qwen baseline.

Safety And Interpretation

This is an interpretability and robustness project, not a clinical simulator. The results should be described as:

trait-seeded supervised optimization on official benchmark splits

not as proof that a human disorder improves AI performance. The useful scientific question is narrower and testable: whether different structured weight perturbations create different benchmark strengths, weaknesses, and optimization paths.

Roadmap

• Add more official tasks for memory, long-context retrieval, math, coding, and instruction following.
• Run severity sweeps for the current winners instead of only severity 0.5.
• Compare trait-seeded LoRA against LoRA from the unmodified base model.
• Add confidence intervals and repeated seed runs for the strongest claims.
• Package result summaries into a static report/dashboard.