Add simavr-based functional test for UnitTest sketch (make sim-test)#139
Merged
Conversation
Run the UnitTest sketch on a simulated ATmega328P (simavr) and compare its OK/FAIL verdicts against a committed golden baseline. This gives CI a functional test of actual step timing on emulated 8-bit hardware, not just compile checks - a verdict flip like issue #93 (LINEAR_SPEED running 4x fast at low rpm) would be caught. - test/simavr-run.sh: runs firmware under simavr, waits for the new "TESTS COMPLETE" marker (kills the sim on completion, since loop() never exits), strips ANSI/simulator noise, and diffs OK/FAIL verdict lines against examples/UnitTest/uno-simavr.txt with digits stripped, so toolchain-version timing drift does not cause false failures while any verdict flip does. --update regenerates the baseline. - Makefile: sim-test and sim-test-update targets (pio build + script). - .github/workflows/platformio.yml: SimTest job (apt install simavr, make sim-test). - .devcontainer/Dockerfile: install simavr. - examples/UnitTest/uno-simavr.txt: golden baseline; FAILs at high rpm are the 16 MHz hardware speed limit and are part of the baseline. - examples/UnitTest/UnitTest.ino: print TESTS COMPLETE at end of setup. - AGENTS.md, test/README: document the procedure. Validation: make sim-test passes twice (deterministic, 14 verdict lines, ~8.4s sim run); flipping one verdict in the golden makes it fail with a diff; TIMEOUT=2 correctly reports the missing completion marker; all 5 pio envs still build with the sketch change. Agent: claude-fable-5 (Claude Code 2.1.153); implementation by claude-opus-4-8 subagent, test runs by claude-sonnet-5 subagent, verified by claude-fable-5; max context and thinking level not exposed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Retains and automates the simavr test procedure used to verify #137 and #138, so CI (and anyone locally) gets a functional test of real step timing on an emulated ATmega328P — not just compile checks. A verdict flip like #93 (LINEAR_SPEED running 4× fast at low rpm) would fail this test.
What's included
test/simavr-run.sh— runs the UnitTest firmware undersimavr -m atmega328p -f 16000000, waits for a newTESTS COMPLETEmarker (the sketch'sloop()never exits, so the script kills the simulator on completion or timeout), strips ANSI codes and simulator noise, then compares OK/FAIL verdict lines against the golden baseline with digits stripped — so µs timings that drift with toolchain versions don't cause false failures, while any OK↔FAIL flip does.--updateregenerates the baseline.make sim-test/make sim-test-update— build (pio run -e uno) + run/regenerate.SimTestjob in the PlatformIO workflow (installs simavr via apt, runsmake sim-test).examples/UnitTest/uno-simavr.txt— golden baseline (57 lines, 14 verdicts), joining the existing per-board reference outputs. Note: some FAILs at high rpm are the 16 MHz hardware speed limit, not bugs, and are part of the baseline.examples/UnitTest/UnitTest.ino— printsTESTS COMPLETEat the end ofsetup()(completion/crash detection; harmless on real hardware).Test results
make sim-testin the devcontainer: PASS, 14 verdict lines match, run twice — output is byte-identical between runs (simavr is deterministic); full sketch run completes in ~8.4 s wall clock.TIMEOUT=2correctly exits 1 with "completion marker never appeared" (hang/crash detection).uno,adafruit_feather_m0,nodemcuv2,esp32dev,teensylc) still build with the sketch change.Performance impact
None on the library itself (no
src/changes). CI gains one ~2-minute job.Agent: claude-fable-5 (Claude Code 2.1.153) — implementation by claude-opus-4-8 subagent, test runs by claude-sonnet-5 subagent, verified by claude-fable-5; max context and thinking level not exposed to the agents
🤖 Generated with Claude Code