Skip to content

Fix rules that combine sample outputs failing when dataset has a large number of samples#73

Merged
zjnolen merged 5 commits intomasterfrom
fix/combine-many-samples
Mar 3, 2026
Merged

Fix rules that combine sample outputs failing when dataset has a large number of samples#73
zjnolen merged 5 commits intomasterfrom
fix/combine-many-samples

Conversation

@zjnolen
Copy link
Copy Markdown
Owner

@zjnolen zjnolen commented Feb 16, 2026

Fixes #72. I think most of these errors are from cat receiving more arguments than it can handle. Trying to fix it by using echo and passing the output to xargs, which seems to work in the current shell container.

@zjnolen
Copy link
Copy Markdown
Owner Author

zjnolen commented Feb 18, 2026

Updated the following rules to use python scripts instead of shell commands:

angsd_makeBamlist now uses make_bamlist.py

compile_endo_cont, merge_ind_depth, merge_ibs_ref_bias, compile_kinship_stats_sfs, aggregate_fst_global, and aggregate_fst_window now all use concat_files.py

In all cases, outputs should be identical to those of previous pipeline versions, so reruns can be safely ignored if previous versions ran successfully.

@zjnolen zjnolen merged commit 3b6d22e into master Mar 3, 2026
3 of 4 checks passed
@zjnolen zjnolen deleted the fix/combine-many-samples branch March 20, 2026 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Workflow fails on local combine rules with many inputs (likely path or command length limit)

1 participant