Conversation
- Wake throttled model queues immediately instead of waiting for pacing delays - Add regression coverage for cancelling interrupted pacing waits
|
Closing this in favor of #27 after comparing the two approaches holistically. This PR implements genuine per-model RPM pacing, but doing so also introduces a substantial scheduling and durability subsystem into Eval: dispatch acknowledgements, multiple concurrency layers, task manifests, an append-only transition journal, queue cancellation, and recovery-oriented state transitions. That machinery is defensible for production-grade quota scheduling, but it is disproportionate for this personal evaluator and materially increases the surface area we would need to maintain and trust. #27 makes the narrower architectural choice explicit: Eval owns a configurable per-provider concurrency cap and records typed provider throttling as a distinct result, but it does not claim to enforce RPM, TPM, or RPD quotas. For the expected supervised workloads, that is the preferred tradeoff. If strict quota scheduling becomes necessary later, it should be implemented once in conductor-core at the provider dispatch boundary rather than as Eval-specific orchestration. This closure is therefore a scope and maintainability decision, not a judgment that the implementation here lacks rigor. |
Summary
provider_callprogress event as the dispatch boundary and durably journaldispatchedbefore allowing a provider request to continue.pre_dispatch_failed; these tasks do not advance the model RPM clock or stop the remaining model queue.close #10
Testing
uv sync --locked --all-extras- passeduv run --locked --all-extras ruff format --check .- 16 files already formatteduv run --locked --all-extras ruff check .- passeduv run --locked --all-extras pytest -q- 162 passeduv build- passed