Step-wise intro to Viash components

src/c1/README.md

@@ -0,0 +1,26 @@
+## Install or use container?
+
+Installation requires compilation: <https://www.htslib.org/download/>
+
+Or, use a container:
+
+```
+quay.io/biocontainers/samtools:1.19.2--h50ea8bc_1
+```
+
+This is one way to do it:
+
+``` bash
+docker run -it \
+  -v `pwd`:`pwd` \
+  -w `pwd` \
+  quay.io/biocontainers/samtools:1.19.2--h50ea8bc_1 \
+  stat test.paired_end.sorted.bam
+```
+
+But:
+
+- what if the container does not exist?
+- need to remember the (technical) arguments for Docker
+- need to remember the container pointer
+

src/c2/README.md

@@ -0,0 +1,20 @@
+## Create a script?
+
+Making it easier to run the tool, let's create a script:
+
+``` bash
+#!/bin/bash
+
+docker run -it \
+  -v $(pwd):$(pwd) \
+  -w $(pwd) \
+  quay.io/biocontainers/samtools:1.19.2--h50ea8bc_1 \
+  stat "$1"
+```
+
+And yes, that works!
+
+But:
+
+- what if I need to add additional arguments?
+- this is a tool that is relatively well supported, but what if this is a Python script?

src/c2/samtools.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+docker run -it \
+  -v $(pwd):$(pwd) \
+  -w $(pwd) \
+  quay.io/biocontainers/samtools:1.19.2--h50ea8bc_1 \
+  stat "$1"

src/c3/README.md

@@ -0,0 +1,20 @@
+## Add output argument
+I want to run this in batch or use this as part of a workflow, output should be in a file. Let's add an output argument to the script:
+
+``` bash
+#!/bin/bash
+
+docker run -it \
+  -v $(pwd):$(pwd) \
+  -w $(pwd) \
+  quay.io/biocontainers/samtools:1.19.2--h50ea8bc_1 \
+  stat "$1" >"$2"
+```
+
+But this is not very flexible, does not include validation or parameter checking. It also does not allow for additional arguments?!
+
+```bash
+❯ src/c3/samtools.sh test.paired_end.sorted.bam
+src/c3/samtools.sh: line 3: : No such file or directory
+```
+

src/c3/samtools.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+docker run -it \
+  -v $(pwd):$(pwd) \
+  -w $(pwd) \
+  quay.io/biocontainers/samtools:1.19.2--h50ea8bc_1 \
+  stat "$1" >"$2"

src/c4/README.md

@@ -0,0 +1,43 @@
+## Add argument parsing
+
+The script is error-prone now, input is not checked at all. At a minimum, we would need to have something like:
+``` bash
+#!/bin/bash
+
+usage() {
+  echo "Usage: $0 -i <input file> -o <output file>" 1>&2
+  exit 1
+}
+
+while getopts ":i:o:" arg; do
+  case "${arg}" in
+  i)
+    i=${OPTARG}
+    ;;
+  o)
+    o=${OPTARG}
+    ;;
+  *)
+    usage
+    ;;
+  esac
+done
+shift $((OPTIND - 1))
+
+if [ -z "${i}" ] || [ -z "${o}" ]; then
+  usage
+fi
+
+docker run -it \
+  -v $(pwd):$(pwd) \
+  -w $(pwd) \
+  quay.io/biocontainers/samtools:1.19.2--h50ea8bc_1 \
+  stat "$i" >"$o"
+```
+
+This is not fun anymore. With the help of some coding AI, it's not necessarily hard but not fun.
+
+But:
+
+- What about additional arguments?!
+- What if you want to check if file exists, argument is integer, ... ?

src/c4/samtools.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+
+usage() {
+  echo "Usage: $0 -i <input file> -o <output file>" 1>&2
+  exit 1
+}
+
+while getopts ":i:o:" arg; do
+  case "${arg}" in
+  i)
+    i=${OPTARG}
+    ;;
+  o)
+    o=${OPTARG}
+    ;;
+  *)
+    usage
+    ;;
+  esac
+done
+shift $((OPTIND - 1))
+
+if [ -z "${i}" ] || [ -z "${o}" ]; then
+  usage
+fi
+
+docker run -it \
+  -v $(pwd):$(pwd) \
+  -w $(pwd) \
+  quay.io/biocontainers/samtools:1.19.2--h50ea8bc_1 \
+  stat "$i" >"$o"

src/c5/README.md

@@ -0,0 +1,3 @@
+## Use Viash
+
+What if we use Viash to create a shareable component?

src/c5/config.vsh.yaml

@@ -0,0 +1,21 @@
+name: samtools_5
+
+arguments:
+  - name: --input
+    type: file
+    required: true
+  - name: --output
+    type: file
+    required: true
+    direction: output
+
+resources:
+  - type: bash_script
+    path: script.sh
+
+engines:
+  - type: docker
+    image: quay.io/biocontainers/samtools:1.19.2--h50ea8bc_1
+
+runners:
+- type: executable

src/c5/script.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+set -e
+
+samtools stats \
+  "$par_input" \
+  >"$par_output"
+
+exit 0

src/c6/README.md

@@ -0,0 +1,3 @@
+## More advanced options with Viash
+
+Add unit tests, documentation, custom setup, argument parsing options, etc.

src/c6/config.vsh.yaml

@@ -0,0 +1,38 @@
+name: samtools_6
+
+arguments:
+  - name: --input
+    type: file
+    description: |
+      Input file.
+    required: true
+    must_exist: true
+  - name: --output
+    alternatives: -o
+    type: file
+    description: |
+      Output file.
+    example: "out.txt"
+    required: true
+    direction: output
+
+resources:
+  - type: bash_script
+    path: script.sh
+
+test_resources:
+  - type: bash_script
+    path: test.sh
+  - type: file
+    path: test_data
+
+engines:
+  - type: docker
+    image: quay.io/biocontainers/samtools:1.19.2--h50ea8bc_1
+    setup:
+      - type: docker
+        run: |
+          samtools --version 2>&1 | grep -E '^(samtools|Using htslib)' | \
+          sed 's#Using ##;s# \([0-9\.]*\)$#: \1#' > /var/software_versions.txt
+runners:
+- type: executable

src/c6/script.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+set -e
+
+samtools stats \
+  "$par_input" \
+  >"$par_output"
+
+exit 0

src/c6/test.sh

@@ -0,0 +1,78 @@
+#!/bin/bash
+
+test_dir="${meta_resources_dir}/test_data"
+
+############################################################################################
+
+echo ">>> Test 1: $meta_functionality_name"
+"$meta_executable" \
+  --input "$test_dir/test.paired_end.sorted.bam" \
+  --bai "$test_dir/test.paired_end.sorted.bam.bai" \
+  --output "$test_dir/test.paired_end.sorted.txt"
+
+echo ">>> Checking whether output exists"
+[ ! -f "$test_dir/test.paired_end.sorted.txt" ] && echo "File 'test.paired_end.sorted.txt' does not exist!" && exit 1
+
+echo ">>> Checking whether output is non-empty"
+[ ! -s "$test_dir/test.paired_end.sorted.txt" ] && echo "File 'test.paired_end.sorted.txt' is empty!" && exit 1
+
+echo ">>> Checking whether output is correct"
+# compare using diff, ignoring the line stating the command that was passed.
+diff <(grep -v "^# The command" "$test_dir/test.paired_end.sorted.txt") \
+    <(grep -v "^# The command" "$test_dir/ref.paired_end.sorted.txt") || \
+    (echo "Output file ref.paired_end.sorted.txt does not match expected output" && exit 1)
+
+rm "$test_dir/test.paired_end.sorted.txt"
+
+############################################################################################
+
+echo ">>> Test 2: $meta_functionality_name with --remove_dups"
+"$meta_executable" \
+  --remove_dups \
+  --input "$test_dir/test.paired_end.sorted.bam" \
+  --bai "$test_dir/test.paired_end.sorted.bam.bai" \
+  --output "$test_dir/test.d.paired_end.sorted.txt"
+
+echo ">>> Checking whether output exists"
+[ ! -f "$test_dir/ref.d.paired_end.sorted.txt" ] && echo "File 'ref.d.paired_end.sorted.txt' does not exist!" && exit 1
+
+echo ">>> Checking whether output is non-empty"
+[ ! -s "$test_dir/ref.d.paired_end.sorted.txt" ] && echo "File 'ref.d.paired_end.sorted.txt' is empty!" && exit 1
+
+echo ">>> Checking whether output is correct"
+# compare using diff, ignoring the line stating the command that was passed.
+diff <(grep -v "^# The command" "$test_dir/test.d.paired_end.sorted.txt") \
+    <(grep -v "^# The command" "$test_dir/ref.d.paired_end.sorted.txt") || \
+    (echo "Output file ref.d.paired_end.sorted.txt does not match expected output" && exit 1)
+
+rm "$test_dir/test.d.paired_end.sorted.txt"
+
+############################################################################################
+
+echo ">>> Test 3: $meta_functionality_name with --remove_overlaps"
+"$meta_executable" \
+  --remove_overlaps \
+  --input "$test_dir/test.paired_end.sorted.bam" \
+  --bai "$test_dir/test.paired_end.sorted.bam.bai" \
+  --output "$test_dir/test.p.paired_end.sorted.txt" 
+
+echo ">>> Checking whether output exists"
+[ ! -f "$test_dir/ref.p.paired_end.sorted.txt" ] && echo "File 'ref.p.paired_end.sorted.txt' does not exist!" && exit 1
+
+echo ">>> Checking whether output is non-empty"
+[ ! -s "$test_dir/ref.p.paired_end.sorted.txt" ] && echo "File 'ref.p.paired_end.sorted.txt' is empty!" && exit 1
+
+
+echo ">>> Checking whether output is correct"
+# compare using diff, ignoring the line stating the command that was passed.
+diff <(grep -v "^# The command" "$test_dir/test.p.paired_end.sorted.txt") \
+    <(grep -v "^# The command" "$test_dir/ref.p.paired_end.sorted.txt") || \
+    (echo "Output file ref.p.paired_end.sorted.txt does not match expected output" && exit 1)
+
+rm "$test_dir/test.p.paired_end.sorted.txt"
+
+############################################################################################
+
+echo ">>> All tests passed successfully."
+
+exit 0