Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ MatHud pairs an interactive drawing canvas with an AI assistant to help visualiz
```
4. Provide at least one AI provider API key by setting environment variables or creating `.env` in the project root:
```env
OPENAI_API_KEY=sk-... # OpenAI models (GPT-4o, GPT-5, o3, etc.)
OPENAI_API_KEY=sk-... # OpenAI models (GPT-5.6, GPT-5.5, GPT-4.1, etc.)
ANTHROPIC_API_KEY=sk-ant-... # Anthropic models (Claude Opus/Sonnet/Haiku 4.5)
OPENROUTER_API_KEY=sk-or-... # OpenRouter models (Gemini, DeepSeek, Llama, etc.)
```
Expand Down Expand Up @@ -190,9 +190,9 @@ MatHud supports three AI providers. The model dropdown dynamically shows only mo

| Provider | Environment Variable | Models |
|----------|---------------------|--------|
| **OpenAI** | `OPENAI_API_KEY` | GPT-5.2, GPT-5, GPT-4.1, GPT-4o, o3, o4-mini, GPT-3.5 Turbo, etc. |
| **Anthropic** | `ANTHROPIC_API_KEY` | Claude Opus 4.5, Claude Sonnet 4.5, Claude Haiku 4.5 |
| **OpenRouter** | `OPENROUTER_API_KEY` | Gemini 3 Pro/Flash, Gemini 2.5 Pro, DeepSeek V3.2, Grok, Llama, Gemma, and more (paid and free tiers) |
| **OpenAI** | `OPENAI_API_KEY` | GPT-5.6 (Sol/Terra/Luna), GPT-5.5, GPT-5.2, GPT-4.1 family, GPT-4o mini |
| **Anthropic** | `ANTHROPIC_API_KEY` | Claude Fable 5, Claude Opus 4.8, Claude Sonnet 5, Claude Haiku 4.5 |
| **OpenRouter** | `OPENROUTER_API_KEY` | Gemini 3.1 Pro/3.5 Flash, DeepSeek V4 Pro, Qwen, GLM, Grok, MiniMax, Llama, Gemma, and more (paid and free tiers) |

Models without vision support are labeled "(text only)" in the dropdown. When no API keys are configured, the dropdown shows "No API keys configured".

Expand Down
2 changes: 1 addition & 1 deletion cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ python -m cli.main chat new [--port PORT]
```

**Options:**
- `--model, -m`: AI model (e.g., gpt-4o, claude-3-5-sonnet)
- `--model, -m`: AI model (e.g., gpt-5.5, claude-sonnet-5)
- `--vision, -v`: Include canvas snapshot
- `--no-stream`: Wait for complete response
- `--json`: Output as JSON
Expand Down
4 changes: 2 additions & 2 deletions cli/chat.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ def chat() -> None:
@click.option(
"--model",
"-m",
help="AI model to use (e.g., gpt-4o, claude-3-5-sonnet)",
help="AI model to use (e.g., gpt-5.5, claude-sonnet-5)",
)
@click.option(
"--vision",
Expand Down Expand Up @@ -224,7 +224,7 @@ def send(

mathud chat send "Draw a circle with center A and radius 50" --vision

mathud chat send "What is the derivative of x^2?" --model gpt-4o
mathud chat send "What is the derivative of x^2?" --model gpt-5.5
"""
if not check_server(DEFAULT_HOST, port):
click.echo(click.style(f"Server is not running on port {port}", fg="red"), err=True)
Expand Down
10 changes: 5 additions & 5 deletions documentation/Reference Manual.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3561,7 +3561,7 @@ Dependencies:

**Key Features:**
- **MathJax Configuration**: LaTeX rendering with inline `\(...\)` and display `$$...$$` math
- **AI Model Selection**: Dropdown with GPT-4 family and GPT-3.5-turbo options
- **AI Model Selection**: Provider-grouped dropdown populated from `/api/available_models`
- **Vision Toggle**: Conditional enabling based on model capabilities
- **Responsive Design**: Flexible dual-pane layout for different screen sizes

Expand Down Expand Up @@ -3766,7 +3766,7 @@ Dependencies:
Provides tool discovery via fast local keyword/category matching (default) or AI-powered semantic matching. The search mode is selected by the `TOOL_SEARCH_MODE` environment variable:

- `local` — keyword + category index, no API call (~0.01ms per query)
- `api` — AI-powered semantic search via gpt-5-nano
- `api` — AI-powered semantic search via gpt-4.1-nano
- `hybrid` (default) — local first, falls back to API when confidence is low

**Architecture:**
Expand Down Expand Up @@ -3958,9 +3958,9 @@ Manages model-specific capabilities (vision support, reasoning mode, provider as
and provides factory methods for creating model instances from identifiers.

Supported Providers:
- PROVIDER_OPENAI ("openai"): GPT-5.2, GPT-5, GPT-4.1, GPT-4o, o3, o4-mini, GPT-3.5 Turbo, etc.
- PROVIDER_ANTHROPIC ("anthropic"): Claude Opus 4.5, Claude Sonnet 4.5, Claude Haiku 4.5
- PROVIDER_OPENROUTER ("openrouter"): Gemini, DeepSeek, Grok, Llama, Gemma, and more (paid and free tiers)
- PROVIDER_OPENAI ("openai"): GPT-5.6 (Sol/Terra/Luna), GPT-5.5, GPT-5.2, GPT-4.1 family, GPT-4o mini
- PROVIDER_ANTHROPIC ("anthropic"): Claude Fable 5, Claude Opus 4.8, Claude Sonnet 5, Claude Haiku 4.5
- PROVIDER_OPENROUTER ("openrouter"): Gemini, DeepSeek, Qwen, GLM, Grok, MiniMax, Llama, Gemma, and more (paid and free tiers)
```

**Key Methods:**
Expand Down
102 changes: 55 additions & 47 deletions server_tests/test_ai_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,20 @@
Tests for the AIModel configuration module.

Tests model configurations, factory methods, and capability flags including
the new is_reasoning_model flag for Responses API routing.
the is_reasoning_model flag used for Responses API routing (OpenAI) and for
omitting the temperature parameter on adaptive-thinking Claude models.
"""

from __future__ import annotations

import unittest

from static.ai_model import AIModel
from static.ai_model import (
PROVIDER_ANTHROPIC,
PROVIDER_OPENAI,
PROVIDER_OPENROUTER,
AIModel,
)


class TestAIModel(unittest.TestCase):
Expand All @@ -27,46 +33,40 @@ def test_model_initialization_defaults(self) -> None:
model = AIModel(identifier="test-model", has_vision=False)
self.assertFalse(model.is_reasoning_model)

def test_from_identifier_gpt5_chat_latest(self) -> None:
"""Test GPT-5-chat-latest is a reasoning model with vision."""
model = AIModel.from_identifier("gpt-5-chat-latest")
self.assertEqual(model.id, "gpt-5-chat-latest")
def test_from_identifier_gpt56_sol(self) -> None:
"""Test GPT-5.6 Sol is a reasoning model with vision."""
model = AIModel.from_identifier("gpt-5.6-sol")
self.assertEqual(model.id, "gpt-5.6-sol")
self.assertTrue(model.has_vision)
self.assertTrue(model.is_reasoning_model)

def test_from_identifier_gpt52_chat_latest(self) -> None:
"""Test GPT-5.2-chat-latest is a reasoning model with vision."""
model = AIModel.from_identifier("gpt-5.2-chat-latest")
self.assertEqual(model.id, "gpt-5.2-chat-latest")
def test_from_identifier_gpt55_medium_reasoning(self) -> None:
"""Test GPT-5.5 (the default) has medium reasoning effort configured."""
model = AIModel.from_identifier("gpt-5.5")
self.assertEqual(model.id, "gpt-5.5")
self.assertTrue(model.has_vision)
self.assertTrue(model.is_reasoning_model)
self.assertEqual(model.reasoning_effort, "medium")

def test_from_identifier_gpt52_medium_reasoning(self) -> None:
"""Test GPT-5.2 has medium reasoning effort configured."""
"""Test GPT-5.2 remains a reasoning model with medium reasoning effort."""
model = AIModel.from_identifier("gpt-5.2")
self.assertEqual(model.id, "gpt-5.2")
self.assertTrue(model.has_vision)
self.assertTrue(model.is_reasoning_model)
self.assertEqual(model.reasoning_effort, "medium")

def test_from_identifier_o3(self) -> None:
"""Test o3 is a reasoning model with vision."""
model = AIModel.from_identifier("o3")
self.assertEqual(model.id, "o3")
def test_from_identifier_gpt56_luna(self) -> None:
"""Test GPT-5.6 Luna is a reasoning model with vision."""
model = AIModel.from_identifier("gpt-5.6-luna")
self.assertEqual(model.id, "gpt-5.6-luna")
self.assertTrue(model.has_vision)
self.assertTrue(model.is_reasoning_model)

def test_from_identifier_o4_mini(self) -> None:
"""Test o4-mini is a reasoning model with vision."""
model = AIModel.from_identifier("o4-mini")
self.assertEqual(model.id, "o4-mini")
self.assertTrue(model.has_vision)
self.assertTrue(model.is_reasoning_model)

def test_from_identifier_gpt4o(self) -> None:
"""Test GPT-4o is a standard model (not reasoning) with vision."""
model = AIModel.from_identifier("gpt-4o")
self.assertEqual(model.id, "gpt-4o")
def test_from_identifier_gpt41(self) -> None:
"""Test GPT-4.1 is a standard model with vision."""
model = AIModel.from_identifier("gpt-4.1")
self.assertEqual(model.id, "gpt-4.1")
self.assertTrue(model.has_vision)
self.assertFalse(model.is_reasoning_model)

Expand All @@ -77,19 +77,13 @@ def test_from_identifier_gpt4o_mini(self) -> None:
self.assertTrue(model.has_vision)
self.assertFalse(model.is_reasoning_model)

def test_from_identifier_gpt41(self) -> None:
"""Test GPT-4.1 is a standard model with vision."""
model = AIModel.from_identifier("gpt-4.1")
self.assertEqual(model.id, "gpt-4.1")
self.assertTrue(model.has_vision)
self.assertFalse(model.is_reasoning_model)

def test_from_identifier_gpt35_turbo(self) -> None:
"""Test GPT-3.5-turbo is a standard model without vision."""
model = AIModel.from_identifier("gpt-3.5-turbo")
self.assertEqual(model.id, "gpt-3.5-turbo")
def test_from_identifier_deepseek_no_vision(self) -> None:
"""Test a text-only OpenRouter model reports no vision support."""
model = AIModel.from_identifier("deepseek/deepseek-v4-pro")
self.assertEqual(model.id, "deepseek/deepseek-v4-pro")
self.assertFalse(model.has_vision)
self.assertFalse(model.is_reasoning_model)
self.assertEqual(model.provider, PROVIDER_OPENROUTER)

def test_from_identifier_unknown_model(self) -> None:
"""Test unknown model gets default config (no vision, not reasoning)."""
Expand All @@ -99,39 +93,53 @@ def test_from_identifier_unknown_model(self) -> None:
self.assertFalse(model.is_reasoning_model)

def test_get_default_model(self) -> None:
"""Test default model is GPT-5.2 with medium reasoning."""
"""Test default model is GPT-5.5 with medium reasoning."""
model = AIModel.get_default_model()
self.assertEqual(model.id, AIModel.DEFAULT_MODEL)
self.assertEqual(model.id, "gpt-5.2")
self.assertEqual(model.id, "gpt-5.5")
self.assertTrue(model.is_reasoning_model)
self.assertEqual(model.reasoning_effort, "medium")

def test_str_representation(self) -> None:
"""Test string representation returns model identifier."""
model = AIModel.from_identifier("gpt-4o")
self.assertEqual(str(model), "gpt-4o")
model = AIModel.from_identifier("gpt-4.1")
self.assertEqual(str(model), "gpt-4.1")

def test_all_reasoning_models_identified(self) -> None:
"""Test that all reasoning models are correctly identified."""
reasoning_models = ["gpt-5-chat-latest", "gpt-5.2-chat-latest", "gpt-5.2", "o3", "o4-mini"]
"""Test that all OpenAI Responses-API models are flagged as reasoning models."""
reasoning_models = ["gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna", "gpt-5.5", "gpt-5.2"]
for model_id in reasoning_models:
model = AIModel.from_identifier(model_id)
self.assertTrue(model.is_reasoning_model, f"{model_id} should be a reasoning model")
self.assertEqual(model.provider, PROVIDER_OPENAI)

def test_all_standard_models_identified(self) -> None:
"""Test that all standard models are correctly identified as non-reasoning."""
"""Test that all OpenAI Chat Completions models are non-reasoning."""
standard_models = [
"gpt-4.1",
"gpt-4.1-mini",
"gpt-4.1-nano",
"gpt-4o",
"gpt-4o-mini",
"gpt-5-nano",
"gpt-3.5-turbo",
]
for model_id in standard_models:
model = AIModel.from_identifier(model_id)
self.assertFalse(model.is_reasoning_model, f"{model_id} should NOT be a reasoning model")
self.assertEqual(model.provider, PROVIDER_OPENAI)

def test_anthropic_adaptive_thinking_models_flagged_reasoning(self) -> None:
"""Adaptive-thinking Claude models are flagged is_reasoning_model so the provider
omits the temperature parameter (which they reject with a 400)."""
for model_id in ["claude-fable-5", "claude-opus-4-8", "claude-sonnet-5"]:
model = AIModel.from_identifier(model_id)
self.assertEqual(model.provider, PROVIDER_ANTHROPIC)
self.assertTrue(model.is_reasoning_model, f"{model_id} should be flagged is_reasoning_model")
self.assertTrue(model.has_vision)

def test_anthropic_haiku_accepts_temperature(self) -> None:
"""Claude Haiku 4.5 still accepts temperature, so it stays non-reasoning."""
model = AIModel.from_identifier("claude-haiku-4-5")
self.assertEqual(model.provider, PROVIDER_ANTHROPIC)
self.assertFalse(model.is_reasoning_model)

def test_model_configs_completeness(self) -> None:
"""Test that MODEL_CONFIGS has both required keys for all models."""
Expand Down
12 changes: 6 additions & 6 deletions server_tests/test_openai_api_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ def test_initialization_default_model(self, mock_openai: Mock) -> None:
@patch("static.openai_api_base.OpenAI")
def test_initialization_custom_model(self, mock_openai: Mock) -> None:
"""Test API initializes with custom model."""
custom_model = AIModel.from_identifier("gpt-4o")
custom_model = AIModel.from_identifier("gpt-4.1")
api = OpenAIAPIBase(model=custom_model)
self.assertEqual(api.model.id, "gpt-4o")
self.assertEqual(api.model.id, "gpt-4.1")

@patch("static.openai_api_base.OpenAI")
def test_initialization_messages(self, mock_openai: Mock) -> None:
Expand Down Expand Up @@ -97,16 +97,16 @@ def test_reset_conversation(self, mock_openai: Mock) -> None:
def test_set_model(self, mock_openai: Mock) -> None:
"""Test set_model changes the model."""
api = OpenAIAPIBase()
api.set_model("gpt-4o")
self.assertEqual(api.model.id, "gpt-4o")
api.set_model("gpt-4.1")
self.assertEqual(api.model.id, "gpt-4.1")
self.assertFalse(api.model.is_reasoning_model)

@patch("static.openai_api_base.OpenAI")
def test_set_model_to_reasoning(self, mock_openai: Mock) -> None:
"""Test set_model to a reasoning model."""
api = OpenAIAPIBase()
api.set_model("o3")
self.assertEqual(api.model.id, "o3")
api.set_model("gpt-5.5")
self.assertEqual(api.model.id, "gpt-5.5")
self.assertTrue(api.model.is_reasoning_model)

@patch("static.openai_api_base.OpenAI")
Expand Down
38 changes: 19 additions & 19 deletions server_tests/test_openai_responses_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -654,7 +654,7 @@ class TestOpenAIResponsesAPIIntegration(unittest.TestCase):
"""Integration tests that actually call the OpenAI Responses API.

These tests require a valid OPENAI_API_KEY environment variable
and use reasoning models (GPT-5, o3, o4-mini).
and use reasoning models (GPT-5.6, GPT-5.5).
They are skipped if the key is not available.
"""

Expand All @@ -675,8 +675,8 @@ def setUp(self) -> None:
def test_integration_response_stream_format(self) -> None:
"""Test actual Responses API call returns correct format (minimal tokens)."""
api = OpenAIResponsesAPI()
# Use o4-mini as it's a reasoning model, limit tokens
api.set_model("o4-mini")
# Use gpt-5.5 as it's a reasoning model, limit tokens
api.set_model("gpt-5.5")
api.max_tokens = 20

prompt = json.dumps({"user_message": "Say: OK", "use_vision": False})
Expand All @@ -699,7 +699,7 @@ def test_integration_response_stream_format(self) -> None:
def test_integration_reasoning_tokens(self) -> None:
"""Test that reasoning models stream reasoning tokens (minimal tokens)."""
api = OpenAIResponsesAPI()
api.set_model("o4-mini")
api.set_model("gpt-5.5")
api.max_tokens = 30

prompt = json.dumps({"user_message": "2+2=?", "use_vision": False})
Expand All @@ -725,7 +725,7 @@ def test_integration_reasoning_tokens(self) -> None:
def test_integration_event_types_are_valid(self) -> None:
"""Test that all streamed events have valid types (minimal tokens)."""
api = OpenAIResponsesAPI()
api.set_model("o4-mini")
api.set_model("gpt-5.5")
api.max_tokens = 10

prompt = json.dumps({"user_message": "1", "use_vision": False})
Expand Down Expand Up @@ -757,38 +757,38 @@ def tearDown(self) -> None:
def test_set_model_to_reasoning(self, mock_openai: Mock) -> None:
"""Test setting model to a reasoning model."""
api = OpenAIResponsesAPI()
api.set_model("o3")
api.set_model("gpt-5.6-sol")

self.assertEqual(api.model.id, "o3")
self.assertEqual(api.model.id, "gpt-5.6-sol")
self.assertTrue(api.model.is_reasoning_model)

@patch("static.openai_api_base.OpenAI")
def test_set_model_to_o4_mini(self, mock_openai: Mock) -> None:
"""Test setting model to o4-mini."""
def test_set_model_to_gpt56_luna(self, mock_openai: Mock) -> None:
"""Test setting model to gpt-5.6-luna."""
api = OpenAIResponsesAPI()
api.set_model("o4-mini")
api.set_model("gpt-5.6-luna")

self.assertEqual(api.model.id, "o4-mini")
self.assertEqual(api.model.id, "gpt-5.6-luna")
self.assertTrue(api.model.is_reasoning_model)
self.assertTrue(api.model.has_vision)

@patch("static.openai_api_base.OpenAI")
def test_set_model_to_gpt5(self, mock_openai: Mock) -> None:
"""Test setting model to GPT-5-chat-latest."""
def test_set_model_to_gpt56_terra(self, mock_openai: Mock) -> None:
"""Test setting model to gpt-5.6-terra."""
api = OpenAIResponsesAPI()
api.set_model("gpt-5-chat-latest")
api.set_model("gpt-5.6-terra")

self.assertEqual(api.model.id, "gpt-5-chat-latest")
self.assertEqual(api.model.id, "gpt-5.6-terra")
self.assertTrue(api.model.is_reasoning_model)
self.assertTrue(api.model.has_vision)

@patch("static.openai_api_base.OpenAI")
def test_set_model_to_gpt52_chat_latest(self, mock_openai: Mock) -> None:
"""Test setting model to GPT-5.2-chat-latest."""
def test_set_model_to_gpt55(self, mock_openai: Mock) -> None:
"""Test setting model to GPT-5.5."""
api = OpenAIResponsesAPI()
api.set_model("gpt-5.2-chat-latest")
api.set_model("gpt-5.5")

self.assertEqual(api.model.id, "gpt-5.2-chat-latest")
self.assertEqual(api.model.id, "gpt-5.5")
self.assertTrue(api.model.is_reasoning_model)
self.assertTrue(api.model.has_vision)

Expand Down
4 changes: 2 additions & 2 deletions server_tests/test_prompt_pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,7 @@ def tearDown(self) -> None:
os.environ.pop(key, None)
clear_search_cache()

# -- reasoning model (o3): uses create_response_stream, consumed via /send_message --
# -- reasoning model (gpt-5.5): uses create_response_stream, consumed via /send_message --

@patch.object(OpenAIResponsesAPI, "create_response_stream")
def test_reasoning_model_derivative(self, mock_stream: Mock) -> None:
Expand All @@ -330,7 +330,7 @@ def test_reasoning_model_derivative(self, mock_stream: Mock) -> None:
_tool_call("derive", {"expression": "x^3", "variable": "x"}),
],
)
resp = self.client.post("/send_message", json=_make_payload("derivative of x^3", "o3"))
resp = self.client.post("/send_message", json=_make_payload("derivative of x^3", "gpt-5.5"))
data = json.loads(resp.data)

self.assertEqual(resp.status_code, 200)
Expand Down
Loading
Loading