simready-benchmark: --output-dir is honoured by the run stage but ignored by the plan stage, so the run always aborts

[NV-JBapst]

Summary

Passing --output-dir to simready-benchmark makes the run fail every time. The plan stage ignores the flag and writes plan.json to the path derived from project_config.toml, but the run stage honours it and looks for plan.json under the directory that was passed. The run then aborts because the plan is not where it expects.

simready-benchmark 2026.6.5.

Reproduction

$ simready-benchmark --assets <asset>.usd --output-dir D:\some\other\dir

{"type": "plan_complete",
 "output": "D:\...\simready-foundation\_testing\plan.json",     <-- flag ignored
 "assets": 1, "profiles": 1, "tests": 9, "runs": 8, "warnings": 0}

{"type": "cli_error", "error": "plan_load_failed",
 "message": "Failed to load plan: [Errno 2] No such file or directory:
             'D:\some\other\dir\plan.json'"}                     <-- flag honoured

The plan is generated successfully (8 runs) and written to {project_root}/_testing/plan.json, while the run stage looks in <output-dir>/plan.json.

--help documents the flag as:

--output-dir DIR    Output directory for run/report (default: {project_root}/_testing).

so "run" output is explicitly in scope for the flag.

Also visible in --show-config

--show-config reports output_dir as resolved from project_config, and setting [paths].output_dir in engines.toml has no effect either — project_config wins:

output_dir
----------
  resolved:    D:\...\simready-foundation\_testing
  contributed by: project_config -- {project_root}/_testing

So there appears to be no working way to relocate benchmark output away from the project root: the engines.toml key is overridden, and the CLI flag is only half-applied.

Impact

Minor but confusing — the failure surfaces as a missing-file error naming a path the user never expected a plan to be written to, several layers from the actual cause. The workaround is simply to not pass --output-dir and accept {project_root}/_testing.

Suggested fix

Apply the resolved output directory consistently across plan, run, stamp and report — or, if the plan location is deliberately fixed to the project root, have the run stage load the plan from that same resolved location rather than from --output-dir.

Environment

simready-benchmark 2026.6.5, simready-benchmark-engine-kit 2026.6.5, Isaac Sim 6.0.1, Windows 11, Python 3.12.