Deploy-ready: fully static app, client-side debate engine, zero backend needed

.gitignore

@@ -0,0 +1,9 @@
+node_modules/
+.env
+.next/
+__pycache__/
+*.pyc
+benchmark/results/
+.DS_Store
+.vercel
+out/

README.md

@@ -1 +1,139 @@
-# ai-intelligence-squared
+# AI² — Artificial Intelligence Squared
+
+> LLM Debate Benchmark: The top 10 frontier AI models debate head-to-head in Intelligence Squared format, judged by AI jury panels.
+
+<img alt="AI² Hero Banner" src="web/public/images/hero-banner.png" width="100%" />
+
+## 🏗 Architecture
+
+### Models vs Agents
+
+A **model** is a type of brain (e.g., Claude Opus 4.6). An **agent** is a brain instance with its own isolated context. The same model can power multiple agents simultaneously — what matters is that each agent has completely separate context.
+
+In each debate:
+- **2 debater-agents**: argue FOR and AGAINST the motion
+- **10 judge-agents**: one per model (including the debaters), each with isolated evaluator context and a unique persona
+
+### Debate Format (Intelligence Squared)
+
+1. **Opening Statements** — Each debater frames their position (600 tokens)
+2. **Rebuttals** — Respond to opponent's opening, attack assumptions (500 tokens)
+3. **Cross-Examination** — 3 rounds of Q&A between debaters (150/300 tokens)
+4. **Audience Questions** — Judges generate questions, debaters answer (300 tokens)
+5. **Closing Statements** — Compress strongest points, no new arguments (300 tokens)
+
+### Scoring
+
+- 10 judges vote before and after the debate
+- Winner = side with the highest **vote conversion** (Δ = final% − initial%)
+- Confidence-weighted persuasion tracked for soft signal
+- ELO ratings updated after each matchup (K=32)
+
+## 🏆 Top 10 Models
+
+| # | Model | Provider | Arena Score |
+|---|-------|----------|-------------|
+| 1 | Claude Opus 4.6 (Thinking) | Anthropic | 1503 |
+| 2 | Claude Opus 4.6 | Anthropic | 1497 |
+| 3 | Gemini 3.1 Pro Preview | Google | 1493 |
+| 4 | Grok 4.20 | xAI | 1490 |
+| 5 | Gemini 3 Pro | Google | 1486 |
+| 6 | GPT-5.4 (High) | OpenAI | 1484 |
+| 7 | Grok 4.20 (Reasoning) | xAI | 1480 |
+| 8 | GPT-5.2 Chat | OpenAI | 1477 |
+| 9 | Grok 4.20 Multi-Agent | xAI | 1475 |
+| 10 | Gemini 3 Flash | Google | 1474 |
+
+## 📊 Judge Persona Map
+
+Each model gets a fixed persona when acting as a judge:
+
+| Model | Judge Persona |
+|-------|--------------|
+| Claude Opus 4.6 (Thinking) | Risk-averse economist |
+| Claude Opus 4.6 | Philosophy professor |
+| Gemini 3.1 Pro Preview | Neutral academic |
+| Grok 4.20 | Contrarian thinker |
+| Gemini 3 Pro | Environmental activist |
+| GPT-5.4 (High) | Corporate executive |
+| Grok 4.20 (Reasoning) | Skeptical policymaker |
+| GPT-5.2 Chat | Optimistic technologist |
+| Grok 4.20 Multi-Agent | Union labor representative |
+| Gemini 3 Flash | Data scientist & AI researcher |
+
+## 🚀 Getting Started
+
+### Prerequisites
+
+- Node.js 22+
+- Python 3.12+
+- OpenRouter API key
+
+### Run the Web App
+
+```bash
+cd web
+npm install
+npm run dev
+```
+
+Open http://localhost:3000
+
+### Run the Benchmark
+
+```bash
+# Set your API key
+export OPENROUTER_API_KEY=sk-or-v1-...
+
+# Install dependencies
+pip install aiohttp python-dotenv
+
+# Run tests first
+python3 -m benchmark.test_debate
+
+# Run full benchmark (45 matchups)
+python3 -m benchmark.runner
+
+# Sync results to web app
+./sync_results.sh
+```
+
+### Live Arena
+
+Visit the `/arena` page to:
+1. Enter your OpenRouter API key
+2. Select two models
+3. Choose a topic
+4. Watch the debate unfold in real-time with SSE streaming
+
+## 📁 Project Structure
+
+```
+├── benchmark/               # Python benchmark engine
+│   ├── models.py           # Model definitions & configs
+│   ├── prompts.py          # All prompt templates
+│   ├── api_client.py       # OpenRouter API wrapper
+│   ├── audience.py         # Judge agent module
+│   ├── debate.py           # Debate orchestration
+│   ├── scoring.py          # ELO & scoring
+│   ├── runner.py           # Full benchmark runner
+│   └── test_debate.py      # Step-by-step tests
+├── web/                    # Next.js web application
+│   ├── src/app/            # App Router pages
+│   ├── src/components/     # React components
+│   ├── src/lib/            # Types, utils, data
+│   └── public/images/      # Generated model icons
+└── sync_results.sh         # Sync benchmark → web app
+```
+
+## 🔬 Unique Analytics
+
+- **Self-judging bias**: Does a model favor itself when it's both debater and judge?
+- **Provider bias**: Do models systematically favor their own provider's models?
+- **Cross-model matrix**: 10×10 grid showing every judge×debater relationship
+- **Persuasion resistance**: Which models are hardest to convince as judges?
+- **Stance flip rate**: How often each judge changes position after a debate
+
+## 📄 License
+
+MIT

benchmark/__init__.py

@@ -0,0 +1 @@
+# AI² Benchmark Engine

benchmark/api_client.py

@@ -0,0 +1,200 @@
+"""
+AI² Benchmark — OpenRouter API Client
+
+Async wrapper around the OpenRouter chat completions API with retry logic,
+rate-limit handling, and cost tracking.
+"""
+
+import asyncio
+import json
+import os
+import time
+from dataclasses import dataclass, field
+
+import aiohttp
+
+
+@dataclass
+class APIStats:
+    """Track cumulative API usage across the benchmark run."""
+    total_requests: int = 0
+    total_input_tokens: int = 0
+    total_output_tokens: int = 0
+    total_cost: float = 0.0
+    errors: int = 0
+    retries: int = 0
+
+
+# Global stats tracker
+stats = APIStats()
+
+# Rate-limit: max concurrent requests per model to avoid 429s
+_semaphores: dict[str, asyncio.Semaphore] = {}
+_global_semaphore: asyncio.Semaphore | None = None
+
+API_BASE = "https://openrouter.ai/api/v1/chat/completions"
+
+
+def _get_api_key() -> str:
+    key = os.environ.get("OPENROUTER_API_KEY", "")
+    if not key:
+        raise RuntimeError("OPENROUTER_API_KEY environment variable not set")
+    return key
+
+
+def _get_semaphore(model_id: str) -> asyncio.Semaphore:
+    """Per-model semaphore to limit concurrency (avoid rate limits)."""
+    if model_id not in _semaphores:
+        _semaphores[model_id] = asyncio.Semaphore(2)  # max 2 concurrent per model
+    return _semaphores[model_id]
+
+
+def get_global_semaphore() -> asyncio.Semaphore:
+    """Global semaphore to limit total concurrent requests."""
+    global _global_semaphore
+    if _global_semaphore is None:
+        _global_semaphore = asyncio.Semaphore(10)  # max 10 total concurrent (3 parallel debates)
+    return _global_semaphore
+
+
+async def chat_completion(
+    model_id: str,
+    messages: list[dict],
+    max_tokens: int = 800,
+    config: dict | None = None,
+    temperature: float = 0.7,
+    response_format: dict | None = None,
+    timeout: float = 120.0,
+    api_key: str | None = None,
+) -> dict:
+    """
+    Make a chat completion request to OpenRouter.
+
+    Args:
+        model_id: OpenRouter model ID (e.g., "anthropic/claude-opus-4.6")
+        messages: List of message dicts with "role" and "content"
+        max_tokens: Maximum tokens in the response
+        config: Extra model config (e.g., {"reasoning": {"effort": "high"}})
+        temperature: Sampling temperature
+        response_format: Optional response format (e.g., {"type": "json_object"})
+        timeout: Request timeout in seconds
+        api_key: Optional API key override (for user-provided keys)
+
+    Returns:
+        dict with keys: content, usage, model, raw_response
+    """
+    key = api_key or _get_api_key()
+
+    payload = {
+        "model": model_id,
+        "messages": messages,
+        "max_tokens": max_tokens,
+        "temperature": temperature,
+    }
+
+    # Merge extra config (e.g., reasoning settings)
+    if config:
+        payload.update(config)
+
+    if response_format:
+        payload["response_format"] = response_format
+
+    headers = {
+        "Content-Type": "application/json",
+        "Authorization": f"Bearer {key}",
+        "HTTP-Referer": "https://ai-squared-benchmark.vercel.app",
+        "X-Title": "AI² Intelligence Squared Benchmark",
+    }
+
+    model_sem = _get_semaphore(model_id)
+    global_sem = get_global_semaphore()
+
+    max_retries = 4
+    base_delay = 2.0
+
+    for attempt in range(max_retries + 1):
+        async with global_sem:
+            async with model_sem:
+                try:
+                    async with aiohttp.ClientSession() as session:
+                        async with session.post(
+                            API_BASE,
+                            json=payload,
+                            headers=headers,
+                            timeout=aiohttp.ClientTimeout(total=timeout),
+                        ) as resp:
+                            body = await resp.json()
+
+                            # Handle rate limits
+                            if resp.status == 429:
+                                delay = base_delay * (2 ** attempt)
+                                stats.retries += 1
+                                print(f"  ⚠ Rate limited on {model_id}, retrying in {delay}s...")
+                                await asyncio.sleep(delay)
+                                continue
+
+                            # Handle other errors
+                            if resp.status != 200:
+                                error_msg = body.get("error", {}).get("message", str(body))
+                                if attempt < max_retries:
+                                    delay = base_delay * (2 ** attempt)
+                                    stats.retries += 1
+                                    print(f"  ⚠ Error {resp.status} on {model_id}: {error_msg}, retrying in {delay}s...")
+                                    await asyncio.sleep(delay)
+                                    continue
+                                stats.errors += 1
+                                raise RuntimeError(f"API error {resp.status} for {model_id}: {error_msg}")
+
+                            # Extract response
+                            choice = body.get("choices", [{}])[0]
+                            message = choice.get("message", {})
+                            content = message.get("content", "")
+                            usage = body.get("usage", {})
+
+                            # Update stats
+                            stats.total_requests += 1
+                            stats.total_input_tokens += usage.get("prompt_tokens", 0)
+                            stats.total_output_tokens += usage.get("completion_tokens", 0)
+
+                            return {
+                                "content": content,
+                                "usage": usage,
+                                "model": body.get("model", model_id),
+                                "finish_reason": choice.get("finish_reason", ""),
+                            }
+
+                except asyncio.TimeoutError:
+                    if attempt < max_retries:
+                        delay = base_delay * (2 ** attempt)
+                        stats.retries += 1
+                        print(f"  ⚠ Timeout on {model_id} (attempt {attempt+1}), retrying in {delay}s...")
+                        await asyncio.sleep(delay)
+                        continue
+                    stats.errors += 1
+                    raise RuntimeError(f"Timeout after {max_retries+1} attempts for {model_id}")
+
+                except aiohttp.ClientError as e:
+                    if attempt < max_retries:
+                        delay = base_delay * (2 ** attempt)
+                        stats.retries += 1
+                        print(f"  ⚠ Connection error on {model_id}: {e}, retrying in {delay}s...")
+                        await asyncio.sleep(delay)
+                        continue
+                    stats.errors += 1
+                    raise
+
+    # Should not reach here
+    raise RuntimeError(f"Exhausted retries for {model_id}")
+
+
+def print_stats():
+    """Print cumulative API usage statistics."""
+    print(f"\n{'='*50}")
+    print(f"API Usage Statistics")
+    print(f"{'='*50}")
+    print(f"Total requests:      {stats.total_requests}")
+    print(f"Total input tokens:  {stats.total_input_tokens:,}")
+    print(f"Total output tokens: {stats.total_output_tokens:,}")
+    print(f"Retries:             {stats.retries}")
+    print(f"Errors:              {stats.errors}")
+    print(f"{'='*50}")