Skip to content

mirai bench compare cannot measure against a base on some machines, and measures the wrong build when it can #8

Closed

Description

@5hif7y

Summary

mirai bench compare --base main failed or measured the wrong thing in three ways, all found while measuring #12:

  1. Git Bash outside %ProgramFiles%\Git. The base could not be built: mirai: could not run C:\Program Files\Git\bin\bash.exe: Not_Exist. shell_executable looks only there, although mirai.cmd also looks in ProgramFiles(x86), LocalAppData\Programs, GIT_INSTALL_ROOT and the git.exe of PATH. This machine has Git for Windows in C:\Workbench\PKGMGR\gitforwindows.
  2. Different optimization for base and head. The head was built --release and the base --dev, because bench calls render_base_prepare without release=true. The report would compare two different optimizations.
  3. A base that predates a new parameter fails. When the head sends a parameter the base lacks, the base refuses it. render_call tries to drop the unknown field and measure without it. But it looks for the text unknown field ", while the API now writes "memory" is not a field: the fields are …, so that fallback never matches. bench has no fallback at all.

Acceptance criteria

  • Git Bash is found wherever mirai.cmd finds it.
  • The base of bench compare is built --release, like the head.
  • render and bench drop a field the base does not take, say so, and measure without it.

Out of scope

Activity

  1. self-assigned this
    on Nov 9, 2026
  2. 5hif7y commented on Nov 15, 2026

    @5hif7y
    MemberAuthor

    Two of the three fixes this issue asked for are on main now: b6db65c builds the base and the head alike, and ae45487 retries without the field named in error.data.unknown_field. #11 is left with the third: finding Git Bash through git --exec-path, in mirai.cmd and in the tool alike.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions