Include compRefTests - #11134
Conversation
|
According to claude, this change will not take effect until the CI process switching to a new reference test version (i.e. |
did it run locally? |
|
I remember talking to Mikhail and Alex about this, and one of our concerns was how long executing the tests would take. We need some sort of comparison to understand the impact this will have on our reference tests run to understand the tradeoff of adding these new tests to the current CI pipeline. If they aren't too expensive to run, it is likely fine. But if they add too much time, we might need to consider some alternative (e.g. run it nightly). |
|
Looks like the full change in previous commits made by Claude caused significant regression. I'll force push a simpler change that doesn't. Edit: Actually, it looks like someone already a fix for this issue on master, so I'll just include that instead. |
425aa46 to
b81364b
Compare
b81364b to
d041005
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Reviewed by Cursor Bugbot for commit 2aa5421. Configure here.

Integrate new compliance reference tests into teku's CI process
https://github.com/Consensys/teku-internal/issues/277
Note
Medium Risk
Production fork-choice timeliness is tied to block import outcomes; incorrect confirm/invalidate logic could affect timely/late head signaling, though changes are scoped to retry and rejected-gossip paths.
Overview
Adds consensus-specs compliance reference tests to CI by downloading
comptests.tar.gz(tagged releases only; skipped on nightly) and extracting just thefork_choice_compliancesubtree so the full archive is not expanded.Updates
ForkChoiceTestExecutorfor those vectors: attestations deferred by fork choice are no longer treated as successful, andviable_for_head_roots_and_weightscan assert optional payload status per head root.Fixes block timeliness so observations stay unconfirmed until a block is successfully imported: failed or rejected import attempts (including gossip REJECT) clear stale unconfirmed entries, later arrivals can refresh the value, and confirm on successful import.
BlockTimelinessTrackertests reflect the new refresh/invalidate/confirm behavior.Reviewed by Cursor Bugbot for commit a1128ad. Bugbot is set up for automated code reviews on this repo. Configure here.