Skip to content

odp_rbh_to_ribbon: Snakemake >=8 runs run: blocks in spawned subprocesses, hiding the real traceback #126

@conchoecia

Description

@conchoecia

Summary

Running scripts/odp_rbh_to_ribbon fails with an opaque
snakemake.exceptions.SpawnedJobError and Error in rule make_plot_base: message: None. No traceback from the run: block reaches stderr, so a
failure inside the rule is undiagnosable from the Snakemake output alone.

Environment

  • odp at main (commit e00c977).
  • Reproduced with Snakemake 8.30.0 and 9.14.5.
  • requirements-dev.txt pins snakemake>=7,<9, but 8.x already exhibits
    the behavior below.

What happens

make_plot_base (and make_plot_loci) are run:-directive rules.
Snakemake >=8 executes run: jobs by re-spawning itself as a subprocess
(--mode subprocess). When the code inside the run: block raises, the
parent reports only:

Error in rule make_plot_base:
    message: None
WorkflowError: At least one job did not complete successfully.

Running the spawned command manually does not help — it recursively
re-spawns and swallows the traceback again.

Every odp workflow built around run: blocks (this one, and others) becomes
hard to debug: any real error like a pandas API change, a missing column, a bad
config value surface only as SpawnedJobError.

In this specific case the underlying error was a pandas incompatibility
(groupby(...).apply(...) index handling; ValueError: Length of new names must be 1, got 2 in subway_plot), consistent with the existing pandas<3
pin in requirements-dev.txt. But that root cause was invisible until the
plotting code was run outside Snakemake.

Workaround

Calling subway_plot (and its helpers) directly, bypassing the Snakemake
wrapper, surfaces the real traceback immediately and produces the figure.

Suggestions

  1. Document the supported Snakemake range in README / requirements-dev.txt
    <9 is pinned but 8.x still spawns and hides tracebacks, so the
    pin alone is not sufficient guidance.
  2. Consider migrating run:-block rules to script: directives (or thin
    importable module functions) so exceptions propagate to stderr.
  3. Alternatively, add a short "debugging" note showing how to invoke the
    plotting functions directly when a run: rule fails.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions