[Atmostone]

Part of SPA-94. Two platform defects found in the first hours of running a real corpus, plus one repo-hygiene commit.

Both had been invisible for months: the stand had only ever been exercised with a single judge model, and never under sustained load.

The outcome judge asked for a score it did not require

JUDGE_TOOL declared required: ["reasoning"], with score described as «omit when applicable=false». A model that reads the schema literally therefore replies with reasoning alone — measured 3/3 on glm-5.3-flash — and int(args["score"]) met it with a KeyError. Every dimension of every run errored, the outcome profile came back empty, and the gate failed closed on all of them.

The trajectory judge has always required ["score", "reason"] and was untouched by the same model on the same runs. One judge worked and the other could not, on a difference of one word in a schema.

score is now required, and «not applicable» is carried by applicable alone — which is how the trajectory judge always did it. The absence case still refuses rather than inventing a number (a missing score is not a zero, and a fabricated one would enter a weighted aggregate), but it now says so in words: the old failure surfaced in the profile as the literal string 'score', which names nothing.

Verified live after the change: 6/6 replies carry a score, and on an inapplicable dimension the model sends applicable: false with a score that is correctly ignored.

No LLM call had a timeout

An experiment tick sat on a single HTTPS connection that opened and never answered. Because the scheduler permits one instance of a job at a time, every later tick was skipped; and last_fired_at is stamped after the body, so the job looked like it had stopped firing rather than never finishing. Nothing appeared in any log.

Diagnosed by dumping the live async stacks — judge.py:324, inside the judge's completion — after py-spy showed the main thread merely idle on the event loop, i.e. waiting on I/O rather than spinning.

Calls now carry LLM_REQUEST_TIMEOUT (default 120 s). Timeout and APITimeoutError were already in the transient set, so the existing retry loop absorbs it with no new branch. The default is deliberately tight: measured first-token latency on this stand is 0.26–1.11 s, and the timeout multiplies by the retry count — at five minutes a try, a wedged endpoint buys a twenty-minute dead tick. A caller may still pass its own.

Coverage

5 tests: the schema contract pinned so score cannot quietly become optional again; a reply with no score producing a named error instead of a KeyError; applicable=false winning over a supplied score so a zero never reaches the weighted mean (which is what a compliant judge now sends); every call carrying a timeout; and a caller's own timeout surviving untouched. tests/unit/test_rubric.py is green at 21 passed.

Also in this branch

One repo-hygiene commit ignoring the endpoint audit files — they carry live API keys and this repository is source-available.