Installation & module error

[OZTaekOppa]

Hi,

I am trying to install it on HPC (PBSpro) but II got this error.

Followed "Developing the Snakemake pipeline"

1. Conda activated: tox and snakemake
2. git clone pore-c.git
3. cd pore-c
4. tox -e snakemake --notest

(PipCmakeToxSnakeM) XXXX@flashlite2:~/miniconda3/envs/pore-c> tox -e snakemake --notest
.tox recreate: /home/XXXXX/miniconda3/envs/pore-c/.tox/.tox
.tox installdeps: tox-conda, tox >= 3.25.0
ERROR: unknown environment 'snakemake'

Alternatively, I have tried as below.

1. conda install pore-c
2. git clone https://github.com/nanoporetech/Pore-C-Snakemake.git
3. cd pore-c-snakemake
4. conda env create
5. conda activate pore_c_snakemake

After this, I have activated "pore-c-snakemake"

(pore-c-snakemake) XXXXX@flashlite2:~/miniconda3/envs> conda -h
usage: conda [-h] [-V] command ...

conda is a tool for managing and deploying applications, environments and packages.

Options:

positional arguments:
command
clean Remove unused packages and caches.
compare Compare packages between conda environments.
config Modify configuration values in .condarc. This is modeled after the git config command. Writes to the user .condarc file (/home/XXXXX/.condarc) by default.
create Create a new conda environment from a list of specified packages.
help Displays a list of available conda commands and their help strings.
info Display information about current conda install.
init Initialize conda for shell interaction. [Experimental]
install Installs a list of packages into a specified conda environment.
list List linked packages in a conda environment.
package Low-level conda package utility. (EXPERIMENTAL)
remove Remove a list of packages from a specified conda environment.
uninstall Alias for conda remove.
run Run an executable in a conda environment.
search Search for packages and display associated information. The input is a MatchSpec, a query language for conda packages. See examples below.
update Updates conda packages to the latest compatible version.
upgrade Alias for conda update.

optional arguments:
-h, --help Show this help message and exit.
-V, --version Show the conda version number and exit.

conda commands available from other packages:
env
(pore-c-snakemake) XXXXX@flashlite2:~/miniconda3/envs> snakemake --use-conda test -j 4 --config=output_dir=results.test
Error: no Snakefile found, tried Snakefile, snakefile, workflow/Snakefile, workflow/snakefile.

Both attempts are not successful. Do you have any idea on this matter?

Looking forward to your reply!

Kind regards,

[OZTaekOppa]

Hi again,

Adding to the previous post, I might have managed to install it.
However, there is an error. Is it normal?

(PipCmakeToxSnakeM) XXXXX@flashlite2:/miniconda3/envs/Pore-C-Snakemake> conda activate pore-c-snakemake
(pore-c-snakemake) XXXXX@flashlite2:/miniconda3/envs/Pore-C-Snakemake> snakemake -j 8 test --use-conda --config pore_c_version=dev
Building DAG of jobs...
Creating conda environment envs/juicer_tools.yml...
Downloading and installing remote packages.
Environment for envs/juicer_tools.yml created (location: .snakemake/conda/1c2800ef)
Creating conda environment envs/pairtools.yml...
Downloading and installing remote packages.
Environment for envs/pairtools.yml created (location: .snakemake/conda/905247b4)
Creating conda environment envs/pairix.yml...
Downloading and installing remote packages.
Environment for envs/pairix.yml created (location: .snakemake/conda/6cc10e4b)
Creating conda environment envs/cooler.yml...
Downloading and installing remote packages.
Environment for envs/cooler.yml created (location: .snakemake/conda/afdeade6)
Using shell: /usr/bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Job counts:
count jobs
2 add_refgenome
2 create_contact_fofn
2 create_hic
2 create_hicRef
2 create_mcool_file
1 download_juicer_tools
2 import_basecalls
2 index_pairs
2 merge_contact_files
2 summarise_contacts
1 test
2 to_cooler
2 to_mnd
2 to_salsa_bed
2 to_sorted_pairs
2 to_unsorted_pairs
2 virtual_digest
32

[Thu Jun 2 13:07:45 2022]
rule download_juicer_tools:
output: results/juicebox/juicer_tools.jar
log: results/logs/results/juicebox/juicer_tools.jar.log
jobid: 23

[Thu Jun 2 13:07:45 2022]
checkpoint import_basecalls:
output: results/basecall/NlaIII_run02.rd.catalog.yaml, results/basecall/NlaIII_run02.rd.summary.csv, results/basecall/NlaIII_run02.rd.read_metadata.parquet
log: results/logs/results/basecall/NlaIII_run02.rd.catalog.yaml.log
jobid: 16
benchmark: results/benchmarks/results/basecall/NlaIII_run02.rd.catalog.yaml.bench.txt
wildcards: enzyme=NlaIII, run_id=run02
Downstream jobs will be updated after completion.

[Thu Jun 2 13:07:45 2022]
rule add_refgenome:
output: results/refgenome/GRCh38.rg.catalog.yaml, results/refgenome/GRCh38.rg.metadata.csv, results/refgenome/GRCh38.rg.chromsizes, results/refgenome/GRCh38.rg.fa, results/refgenome/GRCh38.rg.fa.fai
log: results/logs/results/refgenome/GRCh38.rg.catalog.yaml.log
jobid: 22
benchmark: results/benchmarks/results/refgenome/GRCh38.rg.catalog.yaml.bench.txt
wildcards: refgenome_id=GRCh38
threads: 5

[Thu Jun 2 13:07:45 2022]
checkpoint import_basecalls:
output: results/basecall/NlaIII_run01.rd.catalog.yaml, results/basecall/NlaIII_run01.rd.summary.csv, results/basecall/NlaIII_run01.rd.read_metadata.parquet
log: results/logs/results/basecall/NlaIII_run01.rd.catalog.yaml.log
jobid: 14
benchmark: results/benchmarks/results/basecall/NlaIII_run01.rd.catalog.yaml.bench.txt
wildcards: enzyme=NlaIII, run_id=run01
Downstream jobs will be updated after completion.

[Thu Jun 2 13:07:45 2022]
[Thu Jun 2 13:07:45 2022]
[Thu Jun 2 13:07:45 2022]
Error in rule import_basecalls:
Error in rule import_basecalls:
Error in rule add_refgenome:
jobid: 14
jobid: 16
jobid: 22
output: results/basecall/NlaIII_run01.rd.catalog.yaml, results/basecall/NlaIII_run01.rd.summary.csv, results/basecall/NlaIII_run01.rd.read_metadata.parquet
output: results/basecall/NlaIII_run02.rd.catalog.yaml, results/basecall/NlaIII_run02.rd.summary.csv, results/basecall/NlaIII_run02.rd.read_metadata.parquet
output: results/refgenome/GRCh38.rg.catalog.yaml, results/refgenome/GRCh38.rg.metadata.csv, results/refgenome/GRCh38.rg.chromsizes, results/refgenome/GRCh38.rg.fa, results/refgenome/GRCh38.rg.fa.fai
log: results/logs/results/basecall/NlaIII_run01.rd.catalog.yaml.log (check log file(s) for error message)
log: results/logs/results/basecall/NlaIII_run02.rd.catalog.yaml.log (check log file(s) for error message)
log: results/logs/results/refgenome/GRCh38.rg.catalog.yaml.log (check log file(s) for error message)
shell:
pore_c --dask-scheduler-port 0 --dask-num-workers 1 reads prepare .test/resources/GM12878_NlaIII_reads.fq.gz results/basecall/NlaIII_run01.rd --max-read-length 100000 --batch-size 50000 2> results/logs/results/basecall/NlaIII_run01.rd.catalog.yaml.log
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
shell:
pore_c --dask-scheduler-port 0 --dask-num-workers 1 reads prepare .test/resources/GM12878_NlaIII_reads.fq.gz results/basecall/NlaIII_run02.rd --max-read-length 100000 --batch-size 50000 2> results/logs/results/basecall/NlaIII_run02.rd.catalog.yaml.log
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
shell:
pore_c --dask-scheduler-port 0 --dask-num-workers 5 refgenome prepare .test/resources/GRCh38.fasta.gz results/refgenome/GRCh38.rg --genome-id GRCh38 2> results/logs/results/refgenome/GRCh38.rg.catalog.yaml.log
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Activating conda environment: /home/XXXXX/miniconda3/envs/Pore-C-Snakemake/.snakemake/conda/1c2800ef
[Thu Jun 2 13:07:59 2022]
Finished job 23.
1 of 32 steps (3%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/uqhjung3/miniconda3/envs/Pore-C-Snakemake/.snakemake/log/2022-06-02T130134.885090.snakemake.log

However, it looks ok with this.

(pore-c-snakemake) XXXXX@flashlite2:~/miniconda3/envs/Pore-C-Snakemake> snakemake --use-conda test -j 4 --config=output_dir=results.test

There was no complaint at all.
Hope this is all good now.

Thanks.

[OZTaekOppa]

I have tried again on HPC (PBSpro).

1. Work directory attempt:

#!/bin/bash -l
#PBS -A qris-uq
#PBS -N BrahM_PrC
#PBS -l walltime=22:00:00
#PBS -l select=1:ncpus=8:mem=100G

cd $PBS_O_WORKDIR

conda activate pore-c-snakemake
conda activate PipCmTxSmGcc

Pore-C command

snakemake --use-conda brahm -j 8 --config=output_dir=/QRISdata/Q4840/Brahman/ASM_Mom/Pore-Casm/Results.brahm

The error message was

Error: no Snakefile found, tried Snakefile, snakefile, workflow/Snakefile, workflow/snakefile.

2. Conda Pore-C-Snakemake directory attempt:

The same executed script.

The error message was

###############################################################################
AssertionError in line 93 of /home/uqhjung3/miniconda3/envs/Pore-C-Snakemake/rules/common.smk:
fast5_directory
File "/home/XXXXX/miniconda3/envs/Pore-C-Snakemake/Snakefile", line 33, in
File "/home/XXXXX/miniconda3/envs/Pore-C-Snakemake/rules/reads.smk", line 28, in
File "/home/XXXXX/miniconda3/envs/Pore-C-Snakemake/rules/common.smk", line 93, in lookup_value
########################### Job Execution History #############################
JobId:242405.flashmgr2
UserName:XXXXX
GroupName:qris-uq
JobName:BrahM_PrC
SessionId:30341
ResourcesRequested:mem=100gb,ncpus=8,place=free,walltime=22:00:00
ResourcesUsed:cpupercent=75,cput=00:00:17,mem=10440kb,ncpus=8,vmem=389144kb,walltime=00:00:18
QueueUsed:General
AccountString:qris-uq
ExitStatus:1
###############################################################################

Did I miss something?

[OZTaekOppa]

I have managed to make it work. However, I have another question. Is it required to put "sequencing_summary_path" in basecalls.tsv? If yes, where can I retrieve this info? From a guppy /basecalling run (https://nf-co.re/nanoseq/1.0.0/output)?