Skip to content

A shard dump with no PINAX_TEST_MATRIX is filtered out of a matrix report without a diagnostic, and the summary counts only the survivors #118

Description

@sotashimozono

src/testset.jl:1516:

pairs = [(r,c) for (r,c) in zip(roots, cells) if !isempty(c)]

The dispatch at :1509 enters _render_matrix as soon as two or more distinct non-empty cells exist. So in a run where eight shards set PINAX_TEST_MATRIX and one job does not, that job's entire tree -- its pages, its checks, its failures -- is dropped. There is no _diag! and no @warn.

_summary_line(rowsroot) at :1531 is then computed over pairs only, so the overview reports a green N/N that omits a whole shard.

Why this matters

This is the exact failure the feature own comments say it exists to prevent:

ext/PinaxTestShardsExt.jl:8-16
src/testset.jl:1370

A report missing a shard reads as a smaller suite that passed, not as an incomplete run.

Compare _warn_if_undumpable (src/testset.jl:1595), which warns loudly about a single dropped figure, while a dropped shard is silent.

Fix

An empty cell in a matrix render should be an error, or at minimum a diagnostic that reaches the summary line and both faces -- not a filter.

Reported by review; not independently reproduced.

Activity

  1. added
    bugSomething isn't working
    test-reportThe Test -> Pinax bridge (experimental)
    silent-failureWrong output that looks self-consistent; no signal to the reader
    on Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingsilent-failureWrong output that looks self-consistent; no signal to the readertest-reportThe Test -> Pinax bridge (experimental)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions