Behavioral harness crashes with UnicodeEncodeError on non-UTF-8 Windows consoles

[gucciwong]

Summary

eval/behavioral/harness.py crashes while printing a passing result on a
Windows console whose codepage is not UTF-8. CJK locales default to GBK /
Shift-JIS / etc., so this reproduces on any such machine.

The check itself succeeds — only the report fails, which turns a green result
into a red one.

Reproduction

Windows 11, Chinese locale (console codepage GBK), Python 3.14.4, pytest 9.1.1.

cd eval/behavioral
python -m pytest -c pytest.ini -p conftest ../../skills/<skill>/evals/evals.py

detail = 'Recommend generating at a smaller size and running a separate upscale pass instead -- llm_judge: The agent explicitly...r size (2048\xb2 or smaller) then running a separate upscale pass with sd-cli -M upscale, exactly matching the statement.'

    def _report(self, passed: bool, kind: str, detail: str) -> None:
>       print(f"  [{'PASS' if passed else 'FAIL'}] ({kind}) {detail}", flush=True)
E       UnicodeEncodeError: 'gbk' codec can't encode character '\xb2' in position 181: illegal multibyte sequence

harness.py:329: UnicodeEncodeError

Note the judge verdict in detail: "exactly matching the statement" — the
assertion would have passed. \xb2 is ², which appeared because the agent
wrote 2048².

Judge explanations are free-form model output, so any non-ASCII character can
trigger this: ², ×, →, —, curly quotes, and so on.

Workaround

PYTHONIOENCODING=utf-8 python -m pytest -c pytest.ini -p conftest ../../skills/<skill>/evals/evals.py

With this set, the same suite runs clean.

Suggested fix

_report is the only place that needs to be defensive, since it is the one
printing untrusted model output. Either:

a) Reconfigure stdout once at import time in harness.py:

import sys
if hasattr(sys.stdout, "reconfigure"):
    sys.stdout.reconfigure(encoding="utf-8", errors="replace")

b) Or make the single print non-fatal:

def _report(self, passed: bool, kind: str, detail: str) -> None:
    line = f"  [{'PASS' if passed else 'FAIL'}] ({kind}) {detail}"
    try:
        print(line, flush=True)
    except UnicodeEncodeError:
        print(line.encode("ascii", "replace").decode("ascii"), flush=True)
    assert passed, f"({kind}) {detail}"

(a) is cleaner; (b) keeps the change local to the failing call.

Either way, an assertion result should not depend on whether the judge happened
to use a superscript.

Environment

• Windows 11 Pro 26200 x64, console codepage 936 (GBK)
• Python 3.14.4, pytest 9.1.1
• eval/behavioral/harness.py unmodified from main

Happy to send a PR if that is useful, though I note CONTRIBUTING.md scopes
contributions to AMD engineers and selected partners — so filing this as an
issue instead.