While #47 was in flight, main landed 71 commits including three modules that overlap it. The merge resolved the one hard conflict (auc) toward main, but three duplicates are still present and should be consolidated before the eval CLI (#48) is built on top of them.
Already resolved in #47
scripts/eval-vec.ts dropped its own auc/cohensD and re-exports src/metrics.ts. Main had independently made the same average-rank tie correction, and additionally returns NaN for an empty group. There is exactly one AUC again, which is main's stated rule.
Still duplicated
| in #47 |
main's equivalent |
notes |
restAiRunner (src/ai-runner.ts) |
cloudflareRunner (scripts/runner-lib.ts) |
same Workers AI REST call |
localAiRunner (src/ai-runner.ts) |
createOpenAiRunner (src/local-runners.ts) |
main's also ships createOllamaRunner for the native API |
resolveEvalRunner (scripts/eval-runner.ts) |
resolveBackend + parseArgs (scripts/runner-lib.ts) |
eval-runner.ts is imported by nothing |
Recommended shape
- Delete
scripts/eval-runner.ts and its tests. It is dead code duplicating runner-lib.ts. Its 64 tests encode real edge cases (empty flag values, space-separated flags, prototype-named flags, last-wins) — port the valuable ones onto runner-lib.ts's parser rather than discarding them, since resolveBackend has not been probed for those cases.
- Make
localAiRunner delegate to createOpenAiRunner, keeping selectAiRunner and the LOCAL_AI_* env contract. The Worker-side wiring is genuinely additive — main's DO is still Workers-AI-only — but it should not carry a second OpenAI client.
- Drop
restAiRunner once nothing imports it. calibrate.ts already moved to runner-lib in the merge.
Affects #48
#48 tells N6/N7 to use resolveEvalRunner. If this consolidation lands first, they should use resolveBackend from runner-lib.ts instead. Sequence this before #48, or update #48's briefs.
Acceptance criteria
- Exactly one OpenAI-compatible runner, one Workers AI REST runner, one backend resolver, one AUC.
- No behaviour change to the Worker:
/api/debug/ai still reports which backend answered, and the DO still honours LOCAL_AI_BASE_URL.
- Full suite green, typecheck clean over both tsc projects.
While #47 was in flight, main landed 71 commits including three modules that overlap it. The merge resolved the one hard conflict (
auc) toward main, but three duplicates are still present and should be consolidated before the eval CLI (#48) is built on top of them.Already resolved in #47
scripts/eval-vec.tsdropped its ownauc/cohensDand re-exportssrc/metrics.ts. Main had independently made the same average-rank tie correction, and additionally returnsNaNfor an empty group. There is exactly one AUC again, which is main's stated rule.Still duplicated
restAiRunner(src/ai-runner.ts)cloudflareRunner(scripts/runner-lib.ts)localAiRunner(src/ai-runner.ts)createOpenAiRunner(src/local-runners.ts)createOllamaRunnerfor the native APIresolveEvalRunner(scripts/eval-runner.ts)resolveBackend+parseArgs(scripts/runner-lib.ts)eval-runner.tsis imported by nothingRecommended shape
scripts/eval-runner.tsand its tests. It is dead code duplicatingrunner-lib.ts. Its 64 tests encode real edge cases (empty flag values, space-separated flags, prototype-named flags, last-wins) — port the valuable ones ontorunner-lib.ts's parser rather than discarding them, sinceresolveBackendhas not been probed for those cases.localAiRunnerdelegate tocreateOpenAiRunner, keepingselectAiRunnerand theLOCAL_AI_*env contract. The Worker-side wiring is genuinely additive — main's DO is still Workers-AI-only — but it should not carry a second OpenAI client.restAiRunneronce nothing imports it.calibrate.tsalready moved torunner-libin the merge.Affects #48
#48 tells N6/N7 to use
resolveEvalRunner. If this consolidation lands first, they should useresolveBackendfromrunner-lib.tsinstead. Sequence this before #48, or update #48's briefs.Acceptance criteria
/api/debug/aistill reports which backend answered, and the DO still honoursLOCAL_AI_BASE_URL.