Skip to content

Return scan-error exit status for partial URL timeouts #35

Description

@tayfuryldz

Context

HeaderProof documents exit code 0 as "Scan completed with no verified technical findings" and exit code 2 as "Scan failed or completed with scan errors". A URL that exhausts its URL budget after baseline is recorded as partial_timeout, but _result_exit_code() currently checks only error and partial_error before returning clean/finding status.

I reproduced this through the public CLI on current main with a controlled localhost endpoint: the baseline completed immediately and later probe requests were delayed until the fixed per-URL budget expired.

Observed console summary:

urls=1 scanned=0 partial_error=0 partial_timeout=1 findings=0 suppressed=0 errors=0

Observed process result:

exit=0

The scan did not complete, so CI can currently interpret an incomplete HeaderProof run as a clean successful scan.

Expected contract

A partial_timeout is an incomplete scan state and must not be indistinguishable from a completed clean scan.

The existing documented precedence already treats scan errors before findings in _result_exit_code(). Keep that policy consistent unless the public exit-code contract is intentionally revised.

Acceptance criteria

  • A run with partial_timeout > 0 cannot return EXIT_CLEAN.
  • With the current documented contract, incomplete/error state retains scan-error precedence over verified findings.
  • Fully completed clean scans still return 0.
  • Fully completed scans with verified findings still return 1.
  • error and partial_error behavior remains unchanged.
  • Regression coverage exercises _result_exit_code() for clean, findings, error, partial_error, partial_timeout, and mixed finding + partial_timeout cases.
  • At least one integration-level regression demonstrates the incomplete URL-budget state is surfaced as a non-clean process result.
  • docs/CI.md remains accurate after the change.
  • Full test/lint/type-check suite stays green.

Keep the fix at the result/exit-code boundary; do not turn partial timeout into a finding or alter evidence-gating semantics.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggood first issueGood for newcomershelp wantedExtra attention is neededtestsTest coverage, fixtures, and validation

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions