Skip to content

Add pip-audit / OSV dependency vulnerability scanning to CI #414

@dgenio

Description

@dgenio

Summary

Add a CI job that runs pip-audit (or the OSV scanner) against the resolved dependency set on a schedule and on dependency-changing PRs, reporting known vulnerabilities in ChainWeaver's runtime and dev dependencies.

Why this matters

ChainWeaver has strong supply-chain hygiene in places (Dependabot for pip and Actions, trusted publishing via OIDC, detect-secrets), but there is no vulnerability scanning of the dependency tree in CI (verified: no pip-audit/OSV/CodeQL/SBOM references). Security-conscious adopters and emerging regulatory pressure (EU CRA) treat dependency auditing as baseline. This complements Dependabot (which proposes upgrades) by actively flagging known-vulnerable resolved versions.

Current evidence

  • .github/dependabot.yml covers pip + github-actions weekly — upgrade proposals, not vulnerability gating.
  • No pip-audit, OSV, CodeQL, Trivy, SBOM, or CycloneDX reference anywhere in .github/, docs/, or pyproject.toml (verified by grep).
  • .github/workflows/ already runs a rich matrix and floor/latest-deps canaries, so adding a scan job fits the established CI conventions.

External context

pip-audit (PyPA) and the OSV scanner are the standard PyPI-ecosystem vulnerability scanners; many projects run them weekly and on dependency PRs.

Proposed implementation

  1. Add .github/workflows/security-audit.yml (or a job in ci.yml) running pip-audit against the installed environment on schedule + dependency-touching PRs.
  2. Decide gating policy: start non-blocking (report to job summary) to avoid breaking PRs on transitive advisories, with a documented path to gating once noise is understood.
  3. Optionally audit both the floor-deps and latest-deps resolutions (reusing those existing jobs' setups) to catch advisories at both bounds.
  4. Document the policy in SECURITY.md.

AI-agent execution notes

Inspect .github/workflows/ci.yml (floor/latest-deps jobs to reuse), .github/dependabot.yml, SECURITY.md, pyproject.toml dependency policy comment. Keep the base package's zero-LLM-SDK / minimal-runtime-dep posture in mind — auditing must not pull heavy tooling into the runtime. Start non-blocking to respect the repo's "promote to gating after stability" discipline.

Acceptance criteria

  • A scheduled + PR-triggered job runs pip-audit/OSV and publishes findings to the job summary.
  • The policy (advisory vs gating) is documented in SECURITY.md.
  • The job covers at least the latest-deps resolution; floor-deps coverage is noted as optional.

Test plan

The workflow itself (dry-run on a branch); verify it reports a seeded known-vulnerable pin in a test run, then reverts.

Documentation plan

SECURITY.md (audit policy), docs/security.md if present, CHANGELOG (security-tooling note).

Migration and compatibility notes

Not expected to require migration.

Risks and tradeoffs

Advisory noise from transitive deps can cause alert fatigue if gating too early — start non-blocking. Scanning adds CI minutes (one scheduled job + PR job).

Suggested labels

security, ecosystem, reliability

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions