should `protocols/fuzz-test` be on CI?

[lucasbalieiro]

while working on #1947, I noticed that protocols/fuzz-test isn’t included in the CI.

During the review, @Shourya742 spotted an old reference to roles_logic_sv2 there: #1947 (review)
It wasn’t caught by the CI because that target isn’t being executed anywhere.

Currently, the only fuzz-test being executed is the one from buffer:

stratum/.github/workflows/test.yaml

Lines 60 to 67 in 1223d14

	- name: fuzz tests
	run: |
	if [ ${{ matrix.os }} == "ubuntu-latest" ]; then
	./run.sh 30
	else
	echo "Skipping fuzz test on ${{ matrix.os }} - not supported"
	fi
	working-directory: utils/buffer/fuzz

[erickcestari]

Hi!

To get the most value from fuzz testing, we should restructure how we run it in CI (running for 30 seconds in the CI doesn't help much).

Fuzz tests are meant to run continuously (ideally 24/7) to discover edge cases, with discovered inputs saved to a corpus.

For CI, we should use these fuzz tests as regression tests instead:

1. Maintain a corpus repository (could be a separate GitHub repo where contributors can submit interesting test cases they've discovered)
2. In CI, run the fuzz targets against this existing corpus to catch regressions
3. This ensures we're testing against known edge cases without needing long-running fuzz sessions in CI

This way, we get fast, deterministic CI runs while still benefiting from ongoing fuzzing efforts.

[plebhash]

thanks @erickcestari

would you be willing to put together a PoC with:

• some infra to keep tests running 24/7
• a corpus repo

[erickcestari]

thanks @erickcestari

would you be willing to put together a PoC with:

• some infra to keep tests running 24/7
• a corpus repo

Yes! I can help to startup fuzzing in stratum, but I don't currently have a server to run fuzzing for stratum 24/7.

Since right now we have just some targets, I plan to first refactor/add new fuzz targets and then we can start creating a corpus repo.

I forgot to mention that some targets (the simpler ones) don't need to be run forever. If we see that the coverage gets stuck (e.g., doesn't increase anymore), it means either our fuzzing is not able to generate a valid input (in which case we'll need to create a custom mutator) or it means that the fuzzer could reach all the code paths and we are done.

[GitGab19]

Given our recent work on fuzzing, especially on #1992, we can close this issue now.