[codex] add OpenRouter sweeps, pricing, and status panels

[TheComplianceAide]

Summary

This PR adds infrastructure for broader model-regression sweeps, a new computer-use planning category, daily OpenRouter pricing context, and a small dashboard trust/status surface.

• Adds an OpenRouter chat-completions adapter for opt-in OpenRouter/open-weight model runs.
• Adds a pinned OpenRouter manifest workflow so broad sweeps are reproducible and config import stays offline.
• Persists the exact model manifest selected for each benchmark run, so exported historical data does not drift when the configured model list changes.
• Adds safety controls for OpenRouter runs: model-count cap, model-level concurrency cap, and lower default per-model OpenRouter test concurrency.
• Adds a text-evaluated Computer-Use Planning category with Windows, macOS, and verification-discipline tests.
• Updates static JSON export and frontend data loading so dynamic model sets can render in the site.
• Adds a lightweight OpenRouter pricing snapshot/export path and a dashboard cost-per-1M-token panel.
• Adds a small trust/status panel for community-facing credibility context.
• Swaps the site logo/favicon to a simple stick-figure graph-gun SVG.
• Adds docs/research-basis.md with 2026 references for benchmark design, computer-use agents, and cybersecurity-agent evaluation.

Rationale

The goal is to make ModelRegression useful beyond a small fixed frontier-model set. OpenRouter support lets the same suite cover dozens of API-hosted and open-weight candidates, while pinned manifests keep runs comparable over time.

The computer-use category is deliberately labeled as a planning proxy. It tests identity checks, GUI-state grounding, recovery behavior, and verified-completion discipline without claiming to be a live Windows/macOS automation harness yet.

The pricing panel gives the dashboard a simple daily cost signal without coupling OpenRouter pricing into every model record. OpenRouter negative sentinel prices are ignored in the generated snapshot.

The cybersecurity/GRC motivation is generalized from operational failure modes: wrong account or tenant context, secret leakage in artifacts, provider-specific evidence drift, stale health/build markers, and unsupported success claims.

Validation

• py -3 -m compileall benchmark
• Default config import: 4 models, 11 categories, 33 tests, no OpenRouter catalog fetch
• Explicit OpenRouter IDs import: adds 2 opt-in OpenRouter models
• OpenRouter manifest generator smoke: generated and loaded a 2-model manifest from the live catalog
• DB/run manifest smoke: seeded models/categories/tests and persisted 4 run-model records
• Empty export smoke: initialized DB with no completed run exports valid static JSON stubs
• Missing-schema export smoke: fresh SQLite file exports valid static JSON stubs
• git diff --check
• py -3 -m py_compile benchmark\openrouter_pricing.py benchmark\export_json.py
• npm run build
• In-app browser smoke: dashboard, /categories/computer-use, and /methodology render without runtime errors
• Verified http://localhost:3004/ returns 200 and loads /images/stick-figure-shooting-graph.svg for the visible logo and favicon.

[TheComplianceAide]

@HackingDave I opened this because your regression framing lines up with a recurring frustration I have seen in real agent/cybersecurity workflows: models can regress in verification discipline, account/context handling, and completion claims even when the prompt style stays stable.

This contribution tries to make that measurable in two practical ways:

• OpenRouter/open-weight sweeps are opt-in, manifest-pinned, and cost/concurrency bounded, so the suite can compare many more models without live-catalog drift.
• Computer-Use Planning adds Windows/macOS planning and verification-discipline tests, explicitly labeled as a text proxy until there is a live GUI harness.

I also added a short research note with current 2026 benchmark context for static benchmark limits, computer-use agents, and cyber-agent evaluation. Happy to adjust the shape if you want this narrower or split into smaller PRs.

[TheComplianceAide]

Small note from the regression trenches: this PR is basically a seatbelt for the classic "same prompt, new model personality, surprise cliff" problem.

If it catches even one model confidently saying a desktop task is done while the Send button is still missing, it has earned its keep.

[TheComplianceAide]

Army-to-Marine translation: I tried to make this a practical range card, not a white paper. Same prompt goes downrange, every model gets scored, and we verify whether it actually completed the mission instead of saluting a missing Send button.

Respectfully submitted from the Army side of the aisle.