Griping about Grype (Grype was taking a very long time on a very small repo)

[wesley-dean]

I recently moved MegaLinter from GitHub Actions to a local Jenkins environment and noticed something odd: Grype was consistently taking about three minutes to scan a very small Bash repository. The generated artifact is only about 12.3 KB, yet Grype was taking roughly 187-193 seconds per run.

My first thought was database download latency, so I put Grype's database endpoint behind a Nexus raw proxy. That successfully cached the roughly 145 MB compressed vulnerability database, but it made essentially no difference: a cold run took about 193 seconds, and another run took about 190 seconds.

The problem turned out to be the ephemeral MegaLinter container. Because MegaLinter was being run with docker run --rm, Grype's local database cache disappeared after every run. Each invocation therefore had to recreate its local database state even though the upstream archive was already cached.

I added an optional Docker volume for Grype's cache and set GRYPE_DB_CACHE_DIR to the mounted location. The first run with an empty volume still took about 192 seconds, as expected.

The second run took 4.2 seconds.

So, in this environment:

• No persistent Grype cache: ~190 seconds
• Persistent Grype cache, cold: ~192 seconds
• Persistent Grype cache, warm: 4.2 seconds

The Nexus proxy is still useful because it keeps database downloads local when a Jenkins node needs to populate or refresh its cache, while the node-local Docker volume prevents every disposable MegaLinter container from rebuilding that cache from scratch.

I thought this might be useful to anyone running MegaLinter in short-lived containers and wondering why Grype appears disproportionately slow on otherwise tiny repositories.

[nvuillam]

Great tip, it would be nice to add it in grype linter description :)

[wesley-dean]

I wonder if including in the documentation a matrix of what scanners cover what tooling might be nice. For example, running Checkov against a repo with some documentation and an awk script may not be maximally helpful.

Use this scanner if you have...

[wesley-dean]

Also, I'm reviewing other scanners in the repository descriptor that could benefit from a more-durable cache (e.g., Checkov's external modules, Trivy's database, OSV-Scanner etc.) to continue the experiment. I wonder if it might be useful to include in their documentation an item like, "Cachable Locations" (or something like that) with a list of 0 or more directories that could benefit from local cache volumes. I don't know if that could / would / should translate to GitHub Actions and its caching, but it's definitely working well for me locally.

[wesley-dean]

I added caches for Grype, Trivy, Kingfisher, Syft, and Checkov (and OSV-Scanner, even though it wasn't enabled in this experiment) and I was able to drop total runtime to 58 seconds. On GitHub, those same runs took an average of 2:34s. That said, since my Jenkins instance polls that repo every minute, the time to notice changes (30 seconds on average) plus the runtime puts me at about a 1:28s start to finish now. It's cheaper and faster without eliminating any functionality. 🎉 🎉 🎉

It's not a perfect comparison as the GitHub Actions runtimes also included pulling the MegaLinter image

[wesley-dean]

Disabling the linters I didn't need brought total runtime down to 31s; that plus the average time to notice changes puts the entire process from pushing a commit to having results to 61s on average.