Problem
When a deterministic step fails, the runner reports:
Stdout (last 1,024 bytes):
For any test runner that streams results and prints a summary last (node
--test TAP, cargo, vitest, pytest), the last 1,024 bytes are always the
tail of the run — the totals and the final few passing cases. The failing
case is by construction earlier in the stream, so it is exactly the part that
gets cut.
Concrete case
A failing node --test step in a relay PR reported:
ok 13 - pty-exit: child process exit is detected
ok 14 - pty-exit: long-running process stays alive until released
1..14
# tests 14
# pass 13
# fail 1
Every line visible in the failure report is ok. The one not ok was in
tests 1–12 and never appeared. The report proves a failure happened and
withholds which one — the single fact the reader needs.
Recovering it meant compiling the suite and re-running all 14 tests locally
(~2 min, and it did not reproduce), because the journal SQLite is not
uploaded as a CI artifact.
Why the existing escape hatches don't cover it
flows replay <run> --at <step> needs --data-dir, which points inside
.workflow-artifacts/ on the runner. Not uploaded, so it is gone once the
job ends.
- Re-running with more verbosity re-runs the whole matrix.
Proposals (any one would resolve this)
- Prefer the head+tail over the tail. Show first 512 + last 512 bytes
with a … N bytes elided … marker. Cheap, no config, and catches runners
that print failures inline.
- Make the byte budget configurable —
outputTailBytes on the step, or
RELAYFLOWS_OUTPUT_TAIL_BYTES. 1,024 is very small for a test step.
- Grep-forward on failure. Let a step declare
failureHighlight: ["^not ok", "^FAILED", "^\\s+✖"]; on non-zero exit the
runner also prints matching lines from anywhere in the captured stream.
This is the most targeted fix — it surfaces the failing case regardless of
where it sits in the stream.
(3) plus (1) would mean a red step names what failed without a local repro.
Impact
Any 80-to-100 campaign that runs a real test suite in a deterministic step.
The gate correctly goes red and then cannot say why, which pushes diagnosis
back to a local re-run — the exact loop the gate exists to avoid.
Problem
When a deterministic step fails, the runner reports:
For any test runner that streams results and prints a summary last (node
--testTAP, cargo, vitest, pytest), the last 1,024 bytes are always thetail of the run — the totals and the final few passing cases. The failing
case is by construction earlier in the stream, so it is exactly the part that
gets cut.
Concrete case
A failing
node --teststep in a relay PR reported:Every line visible in the failure report is
ok. The onenot okwas intests 1–12 and never appeared. The report proves a failure happened and
withholds which one — the single fact the reader needs.
Recovering it meant compiling the suite and re-running all 14 tests locally
(~2 min, and it did not reproduce), because the journal SQLite is not
uploaded as a CI artifact.
Why the existing escape hatches don't cover it
flows replay <run> --at <step>needs--data-dir, which points inside.workflow-artifacts/on the runner. Not uploaded, so it is gone once thejob ends.
Proposals (any one would resolve this)
with a
… N bytes elided …marker. Cheap, no config, and catches runnersthat print failures inline.
outputTailByteson the step, orRELAYFLOWS_OUTPUT_TAIL_BYTES. 1,024 is very small for a test step.failureHighlight: ["^not ok", "^FAILED", "^\\s+✖"]; on non-zero exit therunner also prints matching lines from anywhere in the captured stream.
This is the most targeted fix — it surfaces the failing case regardless of
where it sits in the stream.
(3) plus (1) would mean a red step names what failed without a local repro.
Impact
Any 80-to-100 campaign that runs a real test suite in a deterministic step.
The gate correctly goes red and then cannot say why, which pushes diagnosis
back to a local re-run — the exact loop the gate exists to avoid.