analysis_lab.md

Scanner battery — analysis

How to read this. The scanners measure different things — some look for CVEs in installed packages, others for embedded secrets, image-hardening issues, malware signatures, or live web findings. So compare scanners within a category, never across: a TruffleHog secret "missing" from Trivy isn't a coverage gap — Trivy doesn't scan for secrets. Within a category, scanners still disagree by design (different vuln databases, dependency resolution, CVE↔package mappings), so low pairwise overlap and a tall "found by 1 tool" bar are expected (see docs/papers/). A finding seen by only one scanner in a category others also cover is a triage item / coverage signal, not automatically a false positive; one seen by 3+ independent scanners is the highest-confidence one. "Most findings" ≠ "best scanner" — extra findings are often duplication or over-broad CPE matching.

• targets in corpus: 145 (scanned: 145, with container IP: 145)
• scanners that produced data: 8 · merged findings: 475937

What was measured (category → scanners that produced findings in it):

• pkg-vuln — CVEs in installed packages / dependencies — 428349 findings, by: grype, osv, trivy
• sbom-component — inventory (not findings) — 43553 findings, by: syft
• secret — embedded credentials / keys — 3210 findings, by: trivy, trufflehog
• image-config — image hardening / Dockerfile smells — 825 findings, by: dockle, trivy

severity	count
critical	15811
high	65731
medium	191006
low	62617
info	83857
unknown	56915

Throughput

• wall clock: 1.9h for 145 containers
• 76.7 containers / hour (1.28 / min)
• per container: avg 3.8m, median 2.9m
• scanner-CPU time: 10.5h total → parallel efficiency ≈ 5.5× (avg concurrent scanner containers)

Per-scanner

scanner	categories	runs	ok	err	timeout	skip	avg	p50	p95	peak mem	findings	exclusive
grype	pkg-vuln	145	145	0	0	0	1.2m	50s	3.8m	2217MB	281066	280298 (280298)
trivy	image-config,pkg-vuln,secret	145	100	45	0	0	39s	24s	2.0m	912MB	92087	92087 (92087)
osv	pkg-vuln	145	144	1	0	0	29s	14s	1.5m	3178MB	56241	55473 (55473)
syft	sbom-component	145	145	0	0	0	30s	15s	1.6m	1351MB	43553	(sole · 43553)
trufflehog	secret	145	145	0	0	0	38s	23s	1.9m	3065MB	3122	3122 (3122)
dockle	image-config	145	145	0	0	0	16s	4s	1.5m	63MB	636	636 (636)
semgrep	—	145	0	145	0	0	5s	0s	42s	0MB	0	(sole · 0)
clamav	—	12	12	0	0	0	5.6m	4.1m	11.1m	1221MB	0	(sole · 0)

exclusive = findings only this scanner reported, and another scanner covering the same category was running too (so it's a real divergence); (N) = its total solo findings. (sole · N) means it's the only scanner of its category here — its N solo findings just reflect that no comparable tool ran.

Within-category agreement

pkg-vuln — 3 scanners: grype, osv, trivy

428349 merged pkg-vuln finding(s):

• found by 1 scanner(s): 427581 (100%)
• found by 2 scanner(s): 768 (0%)

Pairwise overlap (Jaccard):

	grype	osv	trivy
grype	—	0.00	0.00
osv	0.00	—	0.00
trivy	0.00	0.00	—

secret — 2 scanners: trivy, trufflehog

3210 merged secret finding(s):

• found by 1 scanner(s): 3210 (100%)

Pairwise overlap (Jaccard):

	trivy	trufflehog
trivy	—	0.00
trufflehog	0.00	—

image-config — 2 scanners: dockle, trivy

825 merged image-config finding(s):

• found by 1 scanner(s): 825 (100%)

Pairwise overlap (Jaccard):

	dockle	trivy
dockle	—	0.00
trivy	0.00	—

Severity × category

severity	image-config	pkg-vuln	sbom-component	secret	total
critical	0	15807	0	4	15811
high	146	65508	0	77	65731
medium	146	187731	0	3129	191006
low	533	62084	0	0	62617
info	0	40304	43553	0	83857
unknown	0	56915	0	0	56915

Most exposed containers (top 20)

container	ip	findings	crit	high	med	low	severity
metasploitable3	172.30.1.3	34337	168	1125	23036	3618	▓▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░·····
metasploitable3-alt	172.30.1.4	16272	151	479	8906	2031	▓▒▒▒▒▒▒▒▒▒▒▒▒▒░░░·······
hackazon	172.30.7.2	14934	2	119	8475	2219	▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░░······
bwapp	172.30.9.2	13162	0	117	7973	1834	▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░······
bwapp-alt	172.30.9.4	13162	0	117	7973	1834	▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░······
wackopicko	172.30.39.1	12159	0	95	6737	1633	▒▒▒▒▒▒▒▒▒▒▒▒▒░░░·······
sqli	172.31.1.107	11754	0	89	6958	1368	▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░·······
mutillidae	172.30.9.3	10340	0	75	6342	1146	▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░······
dvna	172.31.1.104	10182	329	1930	3604	993	█▓▓▓▓▓▒▒▒▒▒▒▒▒░░········
node6	172.31.1.39	9891	556	2412	2976	1070	█▓▓▓▓▓▓▒▒▒▒▒▒▒░░░·······
xvwa	172.30.40.9	9631	0	63	6705	1012	▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░·····
python27	172.31.1.36	9426	354	2889	3587	880	█▓▓▓▓▓▓▓▒▒▒▒▒▒▒▒▒░░·····
railsgoat-alt	172.30.8.5	9256	287	1760	3563	943	█▓▓▓▓▓▒▒▒▒▒▒▒▒▒░░·······
python35	172.31.1.37	8123	301	2282	3001	874	█▓▓▓▓▓▓▓▒▒▒▒▒▒▒▒▒░░░·····
dns-vuln	172.30.5.1	6923	39	196	4013	2354	▓▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░░░░░░·
express-old	172.30.22.2	6664	358	1308	1599	520	█▓▓▓▓▓▒▒▒▒▒▒░░···········
node8	172.31.1.40	6369	296	1139	1424	498	█▓▓▓▓▒▒▒▒▒░░············
python36	172.31.1.38	6287	291	1371	1520	202	█▓▓▓▓▓▒▒▒▒▒▒░···········
artifactory5	172.31.1.54	6171	603	1770	1589	665	██▓▓▓▓▓▓▓▒▒▒▒▒▒░░░······
mediawiki-old	172.30.32.1	6134	326	1835	2260	462	█▓▓▓▓▓▓▓▒▒▒▒▒▒▒▒▒░░·····