Skip to content

tell an audit that found nothing from one that could not run #80

Description

@haribo

Why

npm audit and cargo audit both query a remote advisory service on every run. When that service is unreachable, the job fails with exit code 1 — exactly as it does when it finds a high-severity vulnerability. PR #79 hit this twice in a row: read ECONNRESET, then 503 Service Unavailable from registry.npmjs.org. Neither run said anything about the dependencies, and both blocked the PR.

Two failures that mean opposite things must not look the same. "No vulnerability" and "no answer" are not the same verdict, and today the gate reports them identically.

Build

  • Retry the advisory request before failing — a handful of attempts with a growing wait, since these outages last seconds to minutes.
  • When every attempt fails to reach the service, fail with a message that says so, distinct from the message a real finding produces. The job still fails — an audit that did not run is not a pass — but whoever reads the log knows within one line whether to look at dependencies or at a status page.
  • Both jobs, npm audit and cargo audit: cargo audit fetches the RustSec advisory database over the network and has the same failure mode, even if it has not bitten yet.

Validation

  • A run with the service unreachable fails with the unreachable message, not the vulnerability one.
  • A run against a dependency with a known high-severity advisory still fails, with the finding.
  • A transient failure followed by a success does not fail the job.

Out of scope

Changing the severity threshold, and vendoring the advisory database.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: choreCI, tooling, maintenance, cleanup

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions