Skip to content

Step 14b/23 (Phase 3): Signatures — Implement ClamAV hash-only signature fetcher #161

Description

@Jordan231111

Summary

Implement a Python script that fetches ClamAV signature databases, extracts hash-only signatures using `sigtool`, and outputs them in the project's normalized format.

Depends On: Step 14a (Licensing matrix — ClamAV must be approved)
Parent Issue: #12 (Integrate Real-World Malware Signatures)

Context

ClamAV provides `main.cvd` and `daily.cvd` signature databases under GPL-2.0. Using `sigtool --list-sigs`, we can extract MD5 and SHA1 file hashes without redistributing the full ClamAV database.

Key Changes

  1. Create `python_scripts/fetch_clamav_hashes.py`:
    • Download `main.cvd` and `daily.cvd` from ClamAV mirror
    • Use `sigtool` (or parse .hdb/.hsb files directly) to extract hash signatures
    • Support MD5 and SHA1 hash types
    • Output normalized JSON: `[{"hash": "...", "hash_type": "md5|sha1", "name": "...", "source": "clamav"}]`
    • Deduplicate on hash_type + hash value
  2. Add error handling: mirror unavailable, corrupted download, sigtool not installed
  3. Add `--output` flag for output file path
  4. Document ClamAV setup requirements in script docstring

Acceptance Criteria

  • Script successfully downloads and extracts ClamAV hash signatures
  • Both MD5 and SHA1 hashes are extracted
  • Output format matches project's normalized signature schema
  • Deduplication removes redundant entries
  • Graceful failure if ClamAV mirrors are unreachable or sigtool unavailable
  • Script can be run standalone and as part of the aggregator pipeline

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:signaturesSignature sources / licensingdifficulty:intermediateModerate complexityphase:3-updates-signaturesUpdates & signatures (Steps 13–14)priority:p0Critical path / blockingtrack:post-mvpDeferred until after the MVP demo; still aligned with the 2026 plan end goals

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions