Skip to content

check_doc_gate.py: a broken/unparseable config exits 1 identically to a real violation #2306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jaylfc merged 1 commit into from
Aug 9, 2026
Merged

check_doc_gate.py: a broken/unparseable config exits 1 identically to a real violation #2306

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions scripts/check_doc_gate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,13 @@
DEFAULT_CONFIG = REPO_ROOT / "docs" / "doc-gate.toml"
DEFAULT_TRAILER = "Docs-Reviewed:"

# Exit codes: 0 clean, 1 a doc-gate violation, 2 a config/usage error
# (broken, missing, or unparseable config). Distinct so a misconfigured gate
# is never mistaken for a real documentation-drift violation.
EXIT_OK = 0
EXIT_VIOLATION = 1
EXIT_CONFIG_ERROR = 2

# A path-like token: one of the four known repo prefixes followed by a run of
# non-whitespace / non-quoting characters. The negative lookbehind stops us
# matching a prefix that is actually embedded inside a larger path (e.g. the
Expand Down Expand Up @@ -254,10 +261,10 @@ def get_trailer(config: dict) -> str:
def _report(failures: list[str]) -> int:
if not failures:
print("doc-gate: clean")
return 0
return EXIT_OK
for failure in failures:
print(f"DOC-GATE FAIL: {failure}")
return 1
return EXIT_VIOLATION


def main(argv: list[str] | None = None) -> int:
Expand All @@ -273,7 +280,11 @@ def main(argv: list[str] | None = None) -> int:
group.add_argument("--base", help="Compare <base>...HEAD (CI / commit-msg)")

args = parser.parse_args(argv)
config = load_config(args.config)
try:
config = load_config(args.config)
except (tomllib.TOMLDecodeError, OSError) as e:
print(f"doc-gate: config error: {args.config}: {e}", file=sys.stderr)
Comment on lines +283 to +286

@qodo-code-review qodo-code-review Bot Aug 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

1. Schema errors still crash 🐞 Bug ☼ Reliability

main() only maps TOML parse/read failures to EXIT_CONFIG_ERROR; a syntactically valid but
structurally invalid config (wrong types for keys like gate/rules) can still raise later and likely
exit 1, making some broken configs indistinguishable from real violations again.
Agent Prompt 
## Issue description
`scripts/check_doc_gate.py` now returns `EXIT_CONFIG_ERROR` for `TOMLDecodeError` and `OSError` during `load_config()`, but it still assumes the loaded TOML has the expected structure/types. A syntactically valid TOML with wrong types (e.g., `gate = "x"`) can crash later (e.g., `str` has no `.get()`), which typically exits with status 1 and undermines the “config errors are distinct from violations” contract.

## Issue Context
The PR’s goal is to ensure config problems do not look like doc-gate violations. This currently holds for unreadable/unparseable files, but not for schema/type errors.

## Fix Focus Areas
- scripts/check_doc_gate.py[255-258]
- scripts/check_doc_gate.py[270-307]

## Suggested implementation sketch
- Add a small `validate_config(config: dict) -> None` that checks:
  - `config` is a `dict`
  - `gate` is absent or a `dict`
  - `rules` is absent or a `list` of `dict`
  - any other sections used in `main()` (`invariants.referenced_paths_scan`, etc.) have expected types
- Call `validate_config()` immediately after `load_config()`.
- Catch `ValueError` (raised by validation) in the same `except` block and return `EXIT_CONFIG_ERROR`.
- Add a unit test with a syntactically valid TOML that violates schema (e.g., `gate = "x"`) asserting `EXIT_CONFIG_ERROR` and a clear stderr message.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

return EXIT_CONFIG_ERROR
Comment on lines +283 to +287

@coderabbitai coderabbitai Bot Aug 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
python - <<'PY'
import io
import tomllib

try:
    tomllib.load(io.BytesIO(b"\xff"))
except UnicodeDecodeError:
    print("confirmed")
else:
    raise SystemExit("expected UnicodeDecodeError")
PY

Repository: jaylfc/taOS

Length of output: 159

🏁 Script executed:

#!/bin/bash
set -euo pipefail

python - <<'PY'
import io, tomllib

for value in (b"\xff", b"\x80", b"\x80\x80\x80"):
    try:
        tomllib.load(io.BytesIO(value))
    except Exception as exc:
        print(type(exc).__name__, bool(isinstance(exc, tomllib.TOMLDecodeError)), bool(isinstance(exc, UnicodeDecodeError)))
    else:
        print("parsed OK")
PY

printf '\n--- scripts/check_doc_gate.py context ---\n'
sed -n '250,300p' scripts/check_doc_gate.py

printf '\n--- config error tests/usages ---\n'
rg -n "EXIT_CONFIG_ERROR|load_config|TOMLDecodeError|UnicodeDecodeError|bad\.write_bytes|config error" scripts -S

Repository: jaylfc/taOS

Length of output: 2804

Handle invalid UTF-8 as a configuration error.

tomllib.load() decodes the file before TOML parsing, so invalid UTF-8 raises UnicodeDecodeError, not tomllib.TOMLDecodeError. Catch that exception here so a corrupt configuration returns EXIT_CONFIG_ERROR instead of escaping the config-error handler. Add a regression test that writes invalid UTF-8 bytes, such as bad.write_bytes(b"\xff").

Proposed fix 
-    except (tomllib.TOMLDecodeError, OSError) as e:
+    except (tomllib.TOMLDecodeError, UnicodeDecodeError, OSError) as e:
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
try:
config = load_config(args.config)
except (tomllib.TOMLDecodeError, OSError) as e:
print(f"doc-gate: config error: {args.config}: {e}", file=sys.stderr)
return EXIT_CONFIG_ERROR
try:
config = load_config(args.config)
except (tomllib.TOMLDecodeError, UnicodeDecodeError, OSError) as e:
print(f"doc-gate: config error: {args.config}: {e}", file=sys.stderr)
return EXIT_CONFIG_ERROR
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/check_doc_gate.py` around lines 283 - 287, Update the exception tuple
in the config-loading handler around load_config to include UnicodeDecodeError,
ensuring invalid UTF-8 returns EXIT_CONFIG_ERROR with the existing diagnostic.
Add a regression test that writes invalid bytes such as b"\xff" and verifies the
configuration-error outcome.


if args.command == "invariants":
files_to_scan = config.get("invariants", {}).get("referenced_paths_scan", [])
Expand Down
31 changes: 31 additions & 0 deletions tests/test_doc_gate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -295,3 +295,34 @@ def test_desktop_shell_test_file_does_not_trigger(self):
("A", "desktop/src/stores/__tests__/theme-store.test.ts"),
]
assert dg.evaluate_rules(changed, [], CHANGELOG_RULE_CONFIG) == []


class TestConfigErrorExitCode:
"""A broken/unparseable config must exit distinctly from a real violation.

Regression: previously an unparseable config raised an unhandled traceback
that exited 1 -- identical to a genuine doc-gate violation -- so a typo in
docs/doc-gate.toml looked just like a missing changelog."""

def test_unparseable_config_returns_config_error_code(self, tmp_path, capsys):
bad = tmp_path / "bad.toml"
bad.write_text('key = "unterminated string\n')
rc = dg.main(["--config", str(bad), "print-trailer"])
captured = capsys.readouterr()
assert rc == dg.EXIT_CONFIG_ERROR
assert rc != dg.EXIT_VIOLATION
assert "config error" in captured.err

def test_missing_config_file_returns_config_error_code(self, tmp_path):
missing = tmp_path / "does_not_exist.toml"
rc = dg.main(["--config", str(missing), "print-trailer"])
assert rc == dg.EXIT_CONFIG_ERROR
assert rc != dg.EXIT_VIOLATION

def test_real_violation_returns_violation_not_config_error(self):
"""Exit code 1 (violation) must remain distinct from exit code 2
(config error)."""
changed = [("A", "tinyagentos/routes/themes.py")]
failures = dg.evaluate_rules(changed, [], APPS_RULE_CONFIG)
assert dg._report(failures) == dg.EXIT_VIOLATION

Loading