fix --apply writes unsupported-extension files with exit 0 and vacuous verification while unanalyzable .razor is refused

[openhoo-hooflow]

This report was generated by AI and independently verified before publication.

Summary

fix --apply writes unsupported-extension files with exit 0 and vacuous verification while unanalyzable .razor is refused

Classification

Bug

Version and provenance

Unchanged source commit f6ad02a57c887e54ff22e8484ce1eb69c1f92c70; hoonarqube 0.10.4. Native binary SHA-256: c2fc5396a2af0f3ac5401b5a10f0a4461e8df054ca641e96cfea75f2a7b14a87. Built with Rust 1.96.0 using cargo build --locked --offline -p hoonarqube-cli.

Reproduction

Save the following as repro.py. Set HOO_NATIVE to the binary built from the source commit above. The script creates disposable inputs and does not modify the checkout.

#!/usr/bin/env python3
"""Trigger/control probe: `hoonarqube fix --apply` writes (and reports exit 0)
on files its own analyzer declares unsupported, with zero analysis/verification.

Contract: QUICKFIX.md "Verification workflow" step 5 (lines 186-189) and README
"Automatic fixes": apply mode re-runs analysis on projected content before
writing, rejects any increased rule count including mechanical-only rewrites,
and exits nonzero (file untouched) for warnings or unverified fixes. The .razor
path already follows this: unanalyzable => exit 1, no write.

Defect: for extensions with no registered analyzer, `analyze` returns Ok(None)
and crates/hoonarqube-cli/src/main.rs:534 (`Ok(None) if targeted.is_empty() =>
return true`) makes verify_projected_rewrite vacuously pass, so a mechanical
newline append is written and reported as a clean success (exit 0, "1 mechanical
fix(es)", "unverified 0", "regressions 0") even though `hoonarqube analyze` on
the same explicit path exits 2 "language is unsupported".

Usage: repro.py trigger|control   (HOO_NATIVE env var names the binary)
  trigger: bytes 'x' in t.xyz  -> defect: analyze rejects file, fix writes it.
  control: bytes 'x' in t.py   -> clean:  analyzable file, verified repair.
"""

import json
import os
import subprocess
import sys
import tempfile

TIMEOUT = 120


def run(binary, *argv, cwd):
    proc = subprocess.run(
        [binary, *argv],
        cwd=cwd,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        text=True,
        timeout=TIMEOUT,
    )
    return proc.returncode, proc.stdout, proc.stderr


def dump_json(stdout):
    try:
        doc = json.loads(stdout)
    except json.JSONDecodeError:
        return None
    return doc


def main() -> int:
    mode = sys.argv[1] if len(sys.argv) > 1 else ""
    if mode not in ("trigger", "control"):
        print("usage: repro.py trigger|control", file=sys.stderr)
        return 2
    binary = os.environ.get("HOO_NATIVE")
    if not binary:
        print("HOO_NATIVE not set", file=sys.stderr)
        return 2

    with tempfile.TemporaryDirectory() as tmp:
        # Trigger and control differ ONLY in the file extension: same bytes 'x',
        # same missing final newline. t.py is analyzable; t.xyz is not.
        name = "t.xyz" if mode == "trigger" else "t.py"
        path = os.path.join(tmp, name)
        with open(path, "wb") as handle:
            handle.write(b"x")

        # Context: what does `analyze` say about this exact explicit path?
        a_rc, a_out, a_err = run(binary, "analyze", "--json", name, cwd=tmp)
        a_doc = dump_json(a_out)
        print(f"[analyze {name}] exit={a_rc}")
        print(f"  stdout(first 240): {a_out[:240]!r}")
        print(f"  stderr(first 240): {a_err[:240]!r}")
        if a_doc:
            files = a_doc.get("project", {}).get("files", [])
            for row in files:
                print(f"  file status={row.get('status')} reason={row.get('reason')!r}")
            print(f"  complete={a_doc.get('complete')} findings={a_doc.get('findings')}")

        before = open(path, "rb").read()
        f_rc, f_out, f_err = run(binary, "fix", "--apply", "--json", name, cwd=tmp)
        after = open(path, "rb").read()
        f_doc = dump_json(f_out)
        print(f"[fix --apply {name}] exit={f_rc}")
        print(f"  bytes: {before!r} -> {after!r} (changed={before != after})")
        print(f"  stderr(first 240): {f_err[:240]!r}")
        if f_doc:
            for row in f_doc.get("files", []):
                print(
                    "  row: written={written} mechanical={mechanical} "
                    "verified={verified} unverified={unverified} "
                    "regressions={regressions}".format(**row)
                )
            print(
                "  totals: " + ", ".join(
                    f"{key}={f_doc.get(key)}" for key in
                    ("applied", "mechanical", "verified", "unverified", "regressions")
                )
            )

        # Supporting contrast (diagnostic only, not asserted): .razor is also
        # unanalyzable and the same mechanical newline is REFUSED there.
        razor = os.path.join(tmp, "r.razor")
        with open(razor, "wb") as handle:
            handle.write(b"hello")
        r_before = open(razor, "rb").read()
        rz_rc, rz_out, rz_err = run(binary, "fix", "--apply", "r.razor", cwd=tmp)
        r_after = open(razor, "rb").read()
        print(f"[contrast fix --apply r.razor] exit={rz_rc} changed={r_before != r_after}")
        print(f"  stderr(first 200): {rz_err[:200]!r}")

        if mode == "control":
            # Clean control: analyzable file, mechanical repair verified by
            # re-analysis, exit 0, newline appended exactly once.
            assert a_rc == 0, f"control analyze should succeed, got {a_rc}: {a_err}"
            assert f_rc == 0, f"control apply should succeed, got {f_rc}: {f_err}"
            assert after == b"x\n", f"control should append newline, got {after!r}"
            print("CONTROL PASS: analyzable mechanical repair applied, exit 0")
            return 0

        # Trigger: the analyzer declares this file unsupported, so verification
        # of the mechanical rewrite is impossible. Per QUICKFIX.md step 5 the
        # apply must exit nonzero and leave the file untouched (as .razor does).
        # Defect: exit 0, file written, reported as fully clean.
        problems = []
        if a_rc == 0:
            problems.append(
                "expected analyze to reject unsupported extension (context), got 0"
            )
        if f_rc == 0:
            problems.append(
                "fix --apply exits 0 on a file analyze rejects as unsupported "
                "(README/QUICKFIX step 5: unverifiable rewrite must exit nonzero)"
            )
        if before != after:
            problems.append(
                f"file written without any analysis/verification: {before!r} -> {after!r}"
            )
        if f_doc:
            row = (f_doc.get("files") or [{}])[0]
            if row.get("written") and row.get("unverified") == 0:
                problems.append(
                    "reported written=true with unverified=0 (vacuous verification)"
                )
        if problems:
            print("TRIGGER FAIL: unsupported-extension write reported as verified")
            for item in problems:
                print(f"  - {item}")
            return 1
        print("TRIGGER unexpectedly clean: no defect observed")
        return 0


if __name__ == "__main__":
    sys.exit(main())

Run python3 repro.py trigger (observed exit 1) and python3 repro.py control (observed exit 0).

Expected and actual behavior

Expected: Apply mode must re-run analysis on projected content before any write and exit nonzero (file untouched) when verification is impossible. The unanalyzable .razor path already follows this: fix --apply r.razor -> exit 1, "cannot analyze fixes for r.razor", file unchanged. A file whose extension has no registered analyzer (analyze exits 2, status=unsupported, "language is unsupported") is equally unanalyzable, so its mechanical newline rewrite is unverifiable and must be refused the same way. README 'Automatic fixes' (lines 701-708) makes the promise unconditional, 'including after a mechanical-only rewrite'.

Actual: For e.g. t.xyz containing b'x' with no final newline: analyze t.xyz exits 2 with status=unsupported, yet fix --apply --json t.xyz exits 0 and rewrites the file to b'x\n', reporting written=true, mechanical=1, verified=0, unverified=0, regressions=0 and no warnings. Root cause: crates/hoonarqube-cli/src/main.rs:534 in verify_projected_rewrite - Ok(None) if targeted.is_empty() => return true - makes verification vacuously pass whenever the analyzer returns no report and no rule fix is targeted, so write_applied_content proceeds with zero analysis. Also asymmetric with directory mode, which silently ignores unsupported files (no write, no warning), and confined to UTF-8 text (non-UTF-8 binaries are refused by the read gate), so harm is bounded to an unverified newline append reported as a clean success.

Evidence and scope

Current contract: QUICKFIX.md.

5. Verify: apply mode tests every rule fix independently, re-runs the combined analysis,
   requires targeted rule-count reduction, rejects any increased rule count (including from
   mechanical-only rewrites), rejects symlinked or late-modified inputs, and exits nonzero for
   conflicts, warnings, or unverified fixes.

Trigger output:

[analyze t.xyz] exit=2
  stdout(first 240): '{"schema_version":1,"files":[],"project":{"metrics":{"files":0,"lines":0,"code_lines":0,"comment_lines":0},"files":[{"path":"t.xyz","classification":"source","status":"unsupported","metrics":null,"duplication":null,"reason":"language is uns'
  stderr(first 240): 'duplication analysis skipped because one or more eligible source files failed\nt.xyz: language is unsupported\n'
  file status=unsupported reason='language is unsupported'
  complete=None findings=None
[fix --apply t.xyz] exit=0
  bytes: b'x' -> b'x\n' (changed=True)
  stderr(first 240): ''
  row: written=True mechanical=1 verified=0 unverified=0 regressions=0
  totals: applied=0, mechanical=1, verified=0, unverified=0, regressions=0
[contrast fix --apply r.razor] exit=1 changed=False
  stderr(first 200): 'cannot analyze fixes for r.razor: Razor analysis requires a complete trusted C# compiler context\n'
TRIGGER FAIL: unsupported-extension write reported as verified
  - fix --apply exits 0 on a file analyze rejects as unsupported (README/QUICKFIX step 5: unverifiable rewrite must exit nonzero)
  - file written without any analysis/verification: b'x' -> b'x\n'
  - reported written=true with unverified=0 (vacuous verification)

Adjacent clean control output:

[analyze t.py] exit=0
  stdout(first 240): '{"schema_version":1,"files":[{"path":"t.py","language":"python","issues":[{"rule_key":"python:S113","message":"Add a new line at the end of this file \\"t.py\\".","range":{"start":{"line":0,"column":0},"end":{"line":0,"column":0}}},{"rule_key'
  stderr(first 240): ''
  file status=complete reason=None
  complete=None findings=None
[fix --apply t.py] exit=0
  bytes: b'x' -> b'x\n' (changed=True)
  stderr(first 240): ''
  row: written=True mechanical=1 verified=0 unverified=0 regressions=0
  totals: applied=0, mechanical=1, verified=0, unverified=0, regressions=0
[contrast fix --apply r.razor] exit=1 changed=False
  stderr(first 200): 'cannot analyze fixes for r.razor: Razor analysis requires a complete trusted C# compiler context\n'
CONTROL PASS: analyzable mechanical repair applied, exit 0

The exact probe was replayed against unchanged source in a fresh environment. One independent review checked semantics, the reproduction, scope and possible duplicates. A reference difference alone is not proof of a native defect.

Acceptance criteria

Replay discovery-probes/repro.py with HOO_NATIVE set: python3 repro.py trigger (t.xyz = b'x') must exit 1, printing analyze exit=2/status=unsupported, fix exit=0, bytes b'x' -> b'x\n', row written=True unverified=0, and the .razor contrast (exit 1, unchanged); python3 repro.py control (t.py = b'x', only the extension differs) must exit 0 with analyze exit 0 and verified newline append. Executed on HOO_NATIVE hoonarqube 0.10.4: trigger exit=1 with all diagnostics shown, control exit=0 PASS.