fix(evaluator): simplify provider concurrency handling

[laceyp99]

Summary

• replace the RPM-derived semaphore calculation with a configurable per-provider cloud concurrency cap that defaults to four
• preserve typed provider throttling as a distinct rate_limited evaluation outcome in persisted results, summaries, and analysis views
• centralize legacy outcome fallback logic in a shared compatibility helper
• document the limits of concurrency control and the operational guidance for lower provider quotas

Decision and reasoning

The previous implementation described its semaphore as RPM-based rate limiting, but a semaphore only bounds simultaneous requests; it cannot enforce requests per minute.

#26 explored a comprehensive Eval-owned scheduler with per-model RPM pacing, separate model and global concurrency caps, a provider-dispatch acknowledgement boundary, typed throttling behavior, and durable task manifests and transition journals. That work demonstrated a viable direction, but it also introduced a large scheduling and recovery subsystem into Eval. We closed that PR as an implementation alternative because its complexity is disproportionate to the current personal-evaluator use case, not because strict quota handling is no longer valuable.

This PR takes the smaller, explicit step: Eval owns provider-scoped concurrency and makes persistent throttling observable. It does not present that control as complete rate limiting.

Current behavior

• Cloud evaluations allow at most four concurrent requests to each provider by default.
• Callers can override the cap with max_cloud_concurrency.
• The cap does not guarantee compliance with provider RPM, TPM, or RPD quotas.
• Provider SDK retry behavior varies and is not guaranteed by Eval.
• Persistent typed provider throttling is saved as rate_limited and remains distinct in summaries and analysis views. The overview card names a single exception type, aggregates mixed run exceptions, and reports N/A when no pass-rate-eligible results exist.
• Callers using lower account limits or expensive workloads should reduce concurrency or split work into smaller evaluations.

Future rate-limiting work

We are still considering the correct long-term design for rate limiting. A robust solution needs to account for RPM and TPM together rather than pacing only request starts, and may also need to consider RPD, account-tier overrides, token estimation and reconciliation, Retry-After behavior, SDK retry ownership, and coordination across concurrent runs.

That likely requires coordinated work in both conductor-core and conductor-eval, with quota ownership placed near Core's provider dispatch boundary while Eval remains a consumer of the public engine. Issue #10 therefore remains open as an enhancement while that design is worked through.

Validation

• uv sync --locked --all-extras
• uv run --locked --all-extras ruff format --check .
• uv run --locked --all-extras ruff check .
• uv run --locked --all-extras pytest -q — 129 passed
• uv build

[laceyp99]

Reviewed! Everything looks good after the commit I made, as it cleaned up the visual display of the generation failures on the dashboard.