Add typed VideoURL multimodal support

.claude/.codebase-info/.map-state.json

@@ -1,31 +1,32 @@
 {
+  "schemaVersion": 1,
   "tool": "codebase-mapper",
-  "version": "2.8.0",
-  "mappedAt": "2026-07-18",
-  "gitCommit": "73dbc8be54f4ca32e71d28f5d9a7648dd9628c9b",
+  "version": "2.14.4",
+  "mappedAt": "2026-08-11",
+  "gitCommit": "c80c7520a703018d36abc97f2199dcaf3d9c29a3",
   "documents": [
     "architecture.md",
-    "tech-landscape.md",
+    "coding-style.md",
+    "communication.md",
+    "dependencies.md",
     "directory-structure.md",
     "entry-points.md",
     "modules.md",
-    "communication.md",
-    "dependencies.md",
+    "onboarding.md",
     "patterns.md",
-    "coding-style.md",
-    "onboarding.md"
+    "tech-landscape.md"
   ],
   "hashes": {
-    "INDEX.md": "17eaa0ba82f159465e126597aac349c6e4429330887b58742636734e15854313",
-    "architecture.md": "7c908f047e6b0f08824ccd7c3f674ad47bd547dde234d0bd424a6d2784c0cbb6",
-    "tech-landscape.md": "cad1a11e5e63013a10cf07657d248196d0e3e4e14203eba7b777c1c123628198",
-    "directory-structure.md": "7e3fc4c3c8b89b6627beabcb6ddebd74b0ac7094e52718926fc87a94c932df84",
-    "entry-points.md": "6bd4ccae3d975196e03fc2c4e86007eba89f1f6427232fd8adce78ec97d6b432",
-    "modules.md": "1238c76664edb1c1667fda23f39507bcab5b760b57a3a3728d7104b2cf8032e7",
-    "communication.md": "49741a7455b0f162881e1cfb2d797dc8e3832cc8782796bbcb8a10529f2c5a0b",
+    "INDEX.md": "50e7beb7784d3d01f8ce0e85b842889f5c033ca98cbb30981fd0b532c1495c8f",
+    "architecture.md": "8e6be8d2795231ea5919d7cdc23a201595f9fed9724b635e6cc686b2bf24581f",
+    "coding-style.md": "b63e8dff06d16a2d882f02624c62f490291317aec8b3032f6d79b3ee035e8567",
+    "communication.md": "6c73cad50151b73a6be63c8f0745d3f7a6653a861d37afce660c512ac30b8819",
     "dependencies.md": "b445a920c121e27006fec824add6865d01cf71212a517e5ff8b2058d3999037a",
-    "patterns.md": "12b96c2d4f8bc7507bcae39b2b385b9db4d0dec95ef6a2049958663aff021705",
-    "coding-style.md": "2f939544333b927cf8731f5bb906ce03cb3a46afa50c9735011ff8f7e4caca9e",
-    "onboarding.md": "3d28c671dc095a2aadb297bd7ab200c677779a99c7534e03e55a3d64c935f07e"
+    "directory-structure.md": "ca200817789c4d1fdc41d1683c37740c9837bf323b18881cb2351ef8a56c4926",
+    "entry-points.md": "62fd8ff478fc27c3658776626059bdd4101cab12a74a48cdd8787f9ca7b83f97",
+    "modules.md": "7a8772e5f31be0070003c9a7ce78eca5df95dfb6a1c9dd471a6be97df81084aa",
+    "onboarding.md": "3d28c671dc095a2aadb297bd7ab200c677779a99c7534e03e55a3d64c935f07e",
+    "patterns.md": "f3a339a7bc711c4586062139d135e1d79129653cf82fc430cd256028d25e2dcc",
+    "tech-landscape.md": "700b3ba7bb5e47ec14086e2b94ad65ad956c0bfbe1c463f96c39493143e0c912"
   }
 }

.claude/.codebase-info/INDEX.md

@@ -1,12 +1,12 @@
 # Codebase Map — Atomic Agents
 
-*Last Updated: 2026-07-18*
+*Last Updated: 2026-08-11*
 
 Atomic Agents is a lightweight, modular Python framework for building agentic AI applications as
 composable, schema-driven building blocks (built on Instructor + Pydantic). This repository is a
 `uv`-workspace **monorepo**: the core framework, a TUI tool installer, a tool library, and examples.
 
-**Stack:** Python ≥3.12 · Instructor · Pydantic v2 · LiteLLM · MCP · Textual · uv + Hatchling
+**Stack:** Python ≥3.12 · Instructor · Pydantic v2 · LiteLLM · MCP · Textual · uv + Hatchling · Pyright · `VideoURL` multimodal content
 **Shape:** Monorepo — `atomic-agents/` (core lib) · `atomic-assembler/` (CLI) · `atomic-forge/` (tools) · `atomic-examples/` (examples)
 **Package:** `atomic-agents` v2.9.1 on PyPI · core import package is `atomic_agents`
 

.claude/.codebase-info/architecture.md

@@ -1,6 +1,6 @@
 # Architecture
 
-*Last Updated: 2026-07-05*
+*Last Updated: 2026-08-11*
 
 ## Summary
 
@@ -46,8 +46,8 @@ Tool install (atomic-assembler TUI):
 | Component | Path | Responsibility |
 |-----------|------|----------------|
 | Core agent | `atomic-agents/atomic_agents/agents/atomic_agent.py` | `AtomicAgent`, `AgentConfig`, run / stream / async methods |
-| Base contracts | `atomic-agents/atomic_agents/base/` | `BaseIOSchema`, `BaseTool`, `BaseToolConfig`, `BaseResource`, `BasePrompt` |
-| Context | `atomic-agents/atomic_agents/context/` | `SystemPromptGenerator`, `BaseDynamicContextProvider`, `BaseChatHistory` (pluggable memory contract) + `ChatHistory` |
+| Base contracts | `atomic-agents/atomic_agents/base/` | `BaseIOSchema`, `BaseTool`, `BaseToolConfig`, `BaseResource`, `BasePrompt`, `VideoURL` |
+| Context | `atomic-agents/atomic_agents/context/` | `SystemPromptGenerator`, `BaseDynamicContextProvider`, `BaseChatHistory` (pluggable memory contract) + `ChatHistory` with Instructor media and `VideoURL` |
 | Connectors | `atomic-agents/atomic_agents/connectors/mcp/` | Model Context Protocol tools / resources / prompts |
 | Utils | `atomic-agents/atomic_agents/utils/` | Token counting (LiteLLM), tool-message formatting |
 | Assembler (CLI) | `atomic-assembler/atomic_assembler/` | Textual TUI to fetch/install forge tools |
@@ -60,8 +60,7 @@ Tool install (atomic-assembler TUI):
 2. On `run(input)`, the `SystemPromptGenerator` assembles the system message from
    background / steps / output-instructions plus any registered **context providers** (evaluated live
    at call time).
-3. The `ChatHistory` (typed `Message`s, multimodal-aware) is serialized into the provider message
-   list; oldest *turns* are trimmed to respect `max_context_tokens`.
+3. The `ChatHistory` (typed `Message`s, multimodal-aware) is serialized into provider messages; `VideoURL` becomes an OpenAI-compatible `video_url` content part, while token counting uses a `[video content]` placeholder because LiteLLM cannot count video parts. Oldest *turns* are trimmed to respect `max_context_tokens`.
 4. `client.chat.completions.create(response_model=output_schema)` performs the structured LLM call via
    Instructor. Streaming and async variants exist: `run_stream`, `run_async`, `run_async_stream`.
 5. The validated output schema is appended to history and returned. Instructor **hooks**

.claude/.codebase-info/coding-style.md

@@ -1,6 +1,6 @@
 # Coding Style
 
-*Last Updated: 2026-06-13*
+*Last Updated: 2026-08-11*
 
 ## Tooling
 - **Formatter:** Black, line length **127** (`[tool.black]` in `pyproject.toml`; also a pre-commit hook).
@@ -10,6 +10,7 @@
 - **Pre-commit** (`.pre-commit-config.yaml`): trailing-whitespace, end-of-file-fixer, check-yaml,
   check-added-large-files, Black, Flake8.
 - **CI** runs `black --check` and `flake8` across all four subprojects (`code-quality.yml`).
+- **Type checker:** Pyright, configured by `pyrightconfig.json` to use the uv-managed `.venv`.
 
 ## Conventions
 | Kind | Convention | Example |

.claude/.codebase-info/communication.md

@@ -1,6 +1,6 @@
 # Communication & Integrations
 
-*Last Updated: 2026-06-13*
+*Last Updated: 2026-08-11*
 
 The framework exposes no HTTP API of its own; "communication" here means how it talks to LLM
 providers and external tools.
@@ -11,13 +11,17 @@ providers and external tools.
   `client.chat.completions.create(response_model=OutputSchema)` (and `create_partial` for streaming).
 - Provider-agnostic: OpenAI, Anthropic, Google Gemini, MiniMax, and 100+ models via LiteLLM. Mode is
   configurable (`Mode.TOOLS` is the default).
+- Multimodal content uses Instructor's Image/Audio/PDF types plus the framework's `VideoURL`;
+  `ChatHistory.get_history()` converts `VideoURL` to an OpenAI-compatible `video_url` dict, which
+  Instructor forwards unchanged to providers that support video inputs.
 - Provider quirks live in `AgentConfig`: `system_role`, `assistant_role` (use `"model"` for Gemini),
   `tool_result_role` (auto-detected).
 - Code: `atomic-agents/atomic_agents/agents/atomic_agent.py`.
 
 ## Token accounting (via LiteLLM)
 - `atomic-agents/atomic_agents/utils/token_counter.py` uses LiteLLM's `token_counter` for
-  provider-agnostic counts, which drive the context-window trimming in `AtomicAgent`.
+  provider-agnostic counts, which drive the context-window trimming in `AtomicAgent`. Video content
+  parts are represented by a `[video content]` text placeholder because LiteLLM cannot count them.
 
 ## MCP — Model Context Protocol
 - `atomic-agents/atomic_agents/connectors/mcp/` turns MCP server capabilities into agent

.claude/.codebase-info/directory-structure.md

@@ -1,6 +1,6 @@
 # Directory Structure
 
-*Last Updated: 2026-07-18*
+*Last Updated: 2026-08-11*
 
 ## Root Layout
 
@@ -9,11 +9,11 @@ atomic-agents/                  # repo root (uv workspace)
 ├── atomic-agents/              # CORE framework project (PyPI: atomic-agents)
 │   └── atomic_agents/          #   import package
 │       ├── agents/             #     AtomicAgent, AgentConfig
-│       ├── base/               #     BaseIOSchema, BaseTool, BaseResource, BasePrompt
+│       ├── base/               #     BaseIOSchema, BaseTool, BaseResource, BasePrompt, VideoURL
 │       ├── context/            #     SystemPromptGenerator, BaseChatHistory/ChatHistory, context providers
 │       ├── connectors/mcp/     #     Model Context Protocol integration
 │       └── utils/              #     token counter, tool-message formatting
-│   └── tests/                  #   pytest suite (agents/, base/, context/, connectors/, utils/)
+│   └── tests/                  #   pytest suite (agents/, base/, context/, connectors/, utils/; VideoURL tests in base/)
 ├── atomic-assembler/           # Textual TUI (`atomic` command) to install forge tools
 │   └── atomic_assembler/       #   main.py, app.py, screens/, widgets/, utils.py, constants.py
 ├── atomic-forge/               # library of standalone tools (NOT a package)
@@ -27,6 +27,7 @@ atomic-agents/                  # repo root (uv workspace)
 ├── guides/                     # DEV_GUIDE.md and contributor guides
 ├── scripts/                    # generate_llms_files.py (llms.txt index + llms-*.txt bundles)
 ├── pyproject.toml              # package metadata, deps, [tool.black], uv workspace
+├── pyrightconfig.json          # Pyright configuration for the uv-managed `.venv`
 ├── context7.json               # Context7 indexing config + v2 API rules for AI assistants
 ├── build_and_deploy.ps1        # version bump + uv build/publish
 ├── AGENTS.md                   # the project's own design philosophy (imported by CLAUDE.md)

.claude/.codebase-info/entry-points.md

@@ -1,6 +1,6 @@
 # Entry Points
 
-*Last Updated: 2026-07-05*
+*Last Updated: 2026-08-11*
 
 ## 1. Library API (the primary entry point)
 
@@ -18,6 +18,8 @@ result = agent.run(BasicChatInputSchema(chat_message="Hello"))
 - **Module:** `atomic-agents/atomic_agents/agents/atomic_agent.py`
 - **Run methods:** `run`, `run_stream`, `run_async`, `run_async_stream`.
 - Custom agents define their own `BaseIOSchema` subclasses and pass them as `AtomicAgent[In, Out]`.
+- **Public multimodal type:** import `VideoURL` from `atomic_agents` for video inputs; `ChatHistory`
+  serializes it as an OpenAI-compatible `video_url` content part before the provider call.
 
 ## 2. CLI — `atomic`
 

.claude/.codebase-info/modules.md

@@ -1,6 +1,6 @@
 # Key Modules
 
-*Last Updated: 2026-07-05*
+*Last Updated: 2026-08-11*
 
 ## Core framework — `atomic-agents/atomic_agents/`
 
@@ -17,8 +17,9 @@
 - **Purpose:** The typed contracts everything else implements.
 - **Key files:** `base_io_schema.py` (`BaseIOSchema` — Pydantic base; non-empty docstring enforced and
   used as the schema description), `base_tool.py` (`BaseTool[In, Out]`, `BaseToolConfig`),
-  `base_resource.py` (`BaseResource`), `base_prompt.py` (`BasePrompt`).
-- **Exposes via package root:** `BaseIOSchema`, `BaseTool`, `BaseToolConfig`.
+  `base_resource.py` (`BaseResource`), `base_prompt.py` (`BasePrompt`), and `multimodal.py`
+  (`VideoURL` — OpenAI-compatible `video_url` content-part model for video inputs).
+- **Exposes via package root:** `BaseIOSchema`, `BaseTool`, `BaseToolConfig`, `VideoURL`.
 
 ### context
 - **Location:** `atomic-agents/atomic_agents/context/`
@@ -27,7 +28,8 @@
   `base_chat_history.py` (`BaseChatHistory` — interface-only ABC declaring the memory contract
   `AtomicAgent` depends on; the pluggable seam for custom/persistent backends),
   `chat_history.py` (`ChatHistory`, `Message` — the built-in in-memory implementation of
-  `BaseChatHistory`: multimodal Image/Audio/PDF, turn grouping, `dump()`/`load()` serialization).
+  `BaseChatHistory`: multimodal Image/Audio/PDF plus `VideoURL`, turn grouping, `dump()`/`load()`
+  serialization).
 - **Note:** `AgentConfig.history` is typed to `BaseChatHistory`, so any conforming backend drops in.
 
 ### connectors/mcp

.claude/.codebase-info/patterns.md

@@ -1,6 +1,6 @@
 # Patterns & Conventions
 
-*Last Updated: 2026-07-05*
+*Last Updated: 2026-08-11*
 
 ## Atomicity
 Build with small, single-purpose, composable parts ("LEGO blocks"): each agent, tool, and context
@@ -30,8 +30,10 @@ the next's input. See `atomic-examples/deep-research` and `orchestration-agent`.
   `copy()`, plus the `history`/`current_turn_id` attributes). It is the documented, dependency-free
   seam for plugging in custom/persistent backends; `AgentConfig.history` is typed to it.
 - `ChatHistory` is the built-in implementation: stores typed `Message`s grouped into turns (a
-  user+assistant pair shares a `turn_id`), is multimodal-aware (Image/Audio/PDF), and supports
-  `dump()`/`load()`. `AtomicAgent` trims the oldest whole turns to honor `max_context_tokens`.
+  user+assistant pair shares a `turn_id`), is multimodal-aware (Image/Audio/PDF plus `VideoURL`),
+  and supports `dump()`/`load()`. `VideoURL` is converted to an OpenAI-compatible `video_url`
+  content part for provider messages. `AtomicAgent` trims the oldest whole turns to honor
+  `max_context_tokens`, using a text placeholder when LiteLLM cannot count video parts.
 - Custom backend pattern: subclass `ChatHistory` and override `add_message`/`load` to persist (see
   the `persistent-memory` example and the "Writing a Custom Memory Backend" guide section).
 
@@ -42,8 +44,8 @@ the next's input. See `atomic-examples/deep-research` and `orchestration-agent`.
 ## Testing
 - `pytest` (+ `pytest-asyncio` for async, `pytest-cov` for coverage), with `unittest.mock` for LLM
   clients. Core tests in `atomic-agents/tests/` mirror the package layout (`agents/`, `base/`,
-  `context/`, `connectors/mcp/`, `utils/`). Discovery (`pytest.ini`): files `test_*.py`, classes
-  `Test*`, functions `test_*`.
+  `context/`, `connectors/mcp/`, `utils/`), including `tests/base/test_multimodal.py` for `VideoURL`.
+  Discovery (`pytest.ini`): files `test_*.py`, classes `Test*`, functions `test_*`.
 
 ## Configuration
 - Runtime config is explicit via `AgentConfig` (client, model, history, roles, mode,

.claude/.codebase-info/tech-landscape.md

@@ -1,6 +1,6 @@
 # Technology Landscape
 
-*Last Updated: 2026-07-18*
+*Last Updated: 2026-08-11*
 
 ## Source-of-Truth Files
 
@@ -11,6 +11,7 @@
 | Legacy install shims | `setup.py`, `requirements.txt` |
 | Lint | `.flake8` |
 | Format + git hooks | `.pre-commit-config.yaml`, `[tool.black]` in `pyproject.toml` |
+| Type checking | `pyrightconfig.json` |
 | Tests | `pytest.ini`, `.coveragerc` |
 | CI/CD | `.github/workflows/` |
 | Release | `build_and_deploy.ps1` |
@@ -30,6 +31,7 @@
 | HTTP / Git | requests, GitPython | assembler fetches forge from GitHub |
 | Build backend | Hatchling | `[build-system]` |
 | Workspace / lock / publish | uv | `uv sync` / `uv build` / `uv publish` |
+| Static type checking | Pyright | configured for the workspace `.venv` via `pyrightconfig.json` |
 | Docs | Sphinx + MyST + RTD theme | + `sphinxcontrib-mermaid`, deployed to GitHub Pages |
 
 ## Infrastructure

atomic-agents/atomic_agents/__init__.py

@@ -4,7 +4,7 @@
 
 # Core exports - base classes only
 from .agents.atomic_agent import AtomicAgent, AgentConfig, BasicChatInputSchema, BasicChatOutputSchema
-from .base import BaseIOSchema, BaseTool, BaseToolConfig
+from .base import BaseIOSchema, BaseTool, BaseToolConfig, VideoURL
 
 # Version info - read from pyproject.toml via package metadata
 from importlib.metadata import version as _version
@@ -19,4 +19,5 @@
     "BaseIOSchema",
     "BaseTool",
     "BaseToolConfig",
+    "VideoURL",
 ]

atomic-agents/atomic_agents/agents/atomic_agent.py

@@ -431,7 +431,8 @@ def _serialize_history_for_token_count(self) -> List[Dict[str, Any]]:
 
         This method converts instructor multimodal objects (Image, Audio, PDF) to the
         OpenAI format that LiteLLM's token counter expects. Text content is also
-        converted to the proper multimodal text format when mixed with media.
+        converted to the proper multimodal text format when mixed with media. Content-part
+        dicts (e.g. video) become text placeholders because LiteLLM cannot count them.
 
         Returns:
             List[Dict[str, Any]]: History messages in LiteLLM-compatible format.
@@ -462,6 +463,11 @@ def _serialize_history_for_token_count(self) -> List[Dict[str, Any]]:
                                 f"Using placeholder for estimation."
                             )
                             serialized_content.append({"type": "text", "text": f"[{media_type.lower()} content]"})
+                    elif isinstance(item, dict):
+                        # get_history() emits pre-lowered content-part dicts (e.g. video).
+                        # LiteLLM's token counter only accepts text and image_url parts,
+                        # so estimate them with a placeholder.
+                        serialized_content.append({"type": "text", "text": f"[{item.get('type', 'unknown')} content]"})
                     else:
                         # Unknown type - convert to string
                         serialized_content.append({"type": "text", "text": str(item)})

atomic-agents/atomic_agents/base/__init__.py

@@ -4,6 +4,7 @@
 from .base_tool import BaseTool, BaseToolConfig
 from .base_resource import BaseResource, BaseResourceConfig
 from .base_prompt import BasePrompt, BasePromptConfig
+from .multimodal import VideoURL
 
 __all__ = [
     "BaseIOSchema",
@@ -13,4 +14,5 @@
     "BaseResourceConfig",
     "BasePrompt",
     "BasePromptConfig",
+    "VideoURL",
 ]

atomic-agents/atomic_agents/base/multimodal.py

@@ -0,0 +1,35 @@
+"""Multimodal content types that Instructor does not provide."""
+
+from typing import Any, Dict, Optional
+
+from pydantic import BaseModel, Field
+
+
+class VideoURL(BaseModel):
+    """
+    Video reference sent to the LLM as an OpenAI-compatible ``video_url`` content part.
+
+    Instructor ships Image, Audio, and PDF types but no video type
+    (see https://github.com/567-labs/instructor/discussions/2520), so this class fills
+    the gap for providers that accept ``video_url`` content parts, such as MiniMax
+    and Qwen-VL.
+
+    Attributes:
+        url (str): HTTP(S) or data: URL of the video.
+        fps (Optional[float]): Frame sampling rate, for providers that accept it.
+        detail (Optional[str]): Detail level, for providers that accept it.
+    """
+
+    url: str = Field(..., description="HTTP(S) or data: URL of the video.")
+    fps: Optional[float] = Field(default=None, description="Frame sampling rate, for providers that accept it.")
+    detail: Optional[str] = Field(default=None, description="Detail level, for providers that accept it.")
+
+    def to_openai(self) -> Dict[str, Any]:
+        """
+        Build the OpenAI-compatible content part for this video.
+
+        Returns:
+            Dict[str, Any]: A ``{"type": "video_url", "video_url": {...}}`` content part,
+            omitting optional parameters that were not set.
+        """
+        return {"type": "video_url", "video_url": self.model_dump(exclude_none=True)}