fix(contract): make the 1.0 output contract match where each file is written, and declare --pedigree

docs/output-contract.md

@@ -10,7 +10,7 @@ The full list is one file, [`assets/output_contract.tsv`](https://github.com/Gei
 - **`--tools` names:** every name `main.nf` accepts (`pharmcat`, `clinvar`, `vcfanno`, ...).
 - **`run-all.sh`:** its two arguments (`<sample> <male|female>`) and every switch in its `SWITCHES` list (`TOOLS`, `THREADS`, `SKIP_TRIM`, `INTERVALS`, `REF_FASTA`, ...).
 - **Result files:** every file the Nextflow pipeline publishes, which is also what `run-all.sh` writes, at its path under `GENOME_DIR/<sample>/` (or `--outdir`). For each one the contract gives its step, its format and the fields a reader can count on: the columns of a table, the INFO and FORMAT fields of a VCF header, the top-level keys of a JSON file (`summary.json`'s keys are the ones its schema, `tests/schema/summary.schema.json`, requires).
-- **Single-step scripts:** the files listed with leg `bash` or `both`. Some single steps write to other folders than the pipeline does (step 7 writes PharmCAT's reports to `vcf/`, the pipeline to `pharmcat/`; step 16b writes to `mosdepth/`, the pipeline to `coverage/`). Both names are kept. A single-step output that is not in the file is not part of the promise yet.
+- **Single-step scripts:** the files listed with leg `bash` or `both`. A row is `both` when the step's script (`scripts/<step>-*.sh`) writes the same path under `GENOME_DIR`. Some single steps write to other folders than the pipeline does (step 7 writes PharmCAT's reports to `vcf/`, the pipeline to `pharmcat/`; step 16b writes to `mosdepth/`, the pipeline to `coverage/`). Both names are kept. A single-step output that is not in the file is not part of the promise yet.
 
 ## What counts as a break
 
@@ -40,7 +40,7 @@ The engines that run by default, and every step that existed before 1.0, are `st
 
 Two checks read the contract, both with [`scripts/ci/check-output-contract.py`](https://github.com/GeiserX/OwnGenome/blob/main/scripts/ci/check-output-contract.py):
 
-- **On every pull request, with no run** (`tests/test_output_contract.py`, in the `unit-tests` job of the Guard workflow): every param of `nextflow_schema.json`, every `--tools` name of `main.nf` and every switch of `run-all.sh` has a row, and every row still names one. Every file row of a Nextflow process still matches a file that process writes and publishes, and every file a module publishes has a row. A removed or renamed stable item fails as `BREAKING`. A new one with no row fails too, with the message to add it, so the contract is always the full list. The test also plants each kind of break on a copy of the code (a renamed output, a removed param, a removed switch, a new tool and a new output with no row) and requires each to fail.
+- **On every pull request, with no run** (`tests/test_output_contract.py`, in the `unit-tests` job of the Guard workflow): every param of `nextflow_schema.json`, every `--tools` name of `main.nf` and every switch of `run-all.sh` has a row, and every row still names one. Every file row of a Nextflow process still matches a file that process writes and publishes, in the folder of a `publishDir` that keeps it (a `saveAs` that returns `null` for the file does not count), and every file a module publishes has a row. A published row with leg `nextflow` fails when its step's script writes the same path too, since its leg is then `both`; the check reads the script with its shell variables filled in, so a path it cannot follow is not flagged, and a `both` row it does not find is not refused. Every default in `nextflow_schema.json` must equal the value in `nextflow.config`. A removed or renamed stable item fails as `BREAKING`. A new one with no row fails too, with the message to add it, so the contract is always the full list. The test also plants each kind of break on a copy of the code (a renamed output, a removed param, a removed switch, a new tool and a new output with no row, a schema default that differs from `nextflow.config`, a row in the folder a `saveAs` drops the file from, and a `nextflow` row a step script also writes) and requires each to fail.
 - **On the HG002 end-to-end run** (`tests/e2e/output-contract-1-hg002.sh`): every stable file whose `e2e` column names a leg must be in the results that leg wrote, under its contracted name, with its required fields. The bash leg is the single-step scripts the base cases run; the Nextflow leg is case `60-nextflow`. Rows whose step the fixture run does not reach have `e2e` set to `-` and are held by the static check only. The two files `run-all.sh` writes itself, `run_manifest.tsv` and `logs/run_status.tsv`, are listed but no check holds them yet: neither the fixture run nor a module writes them. The same case copies the Nextflow results, drops the `clnsig` column of the ClinVar hits table and renames the ROH summary, and requires the check to fail on both. It also prints every file each leg wrote with its fields, which is what to read when the contract is written again.
 
 ## Changing the contract

main.nf

@@ -61,13 +61,12 @@ workflow {
     // ─── Validate inputs ────────────────────────────────────────────────
     // Every parameter must be one nextflow_schema.json declares: Nextflow
     // takes any --name as a new parameter, so a misspelt option (--outdirr,
-    // --max-cpus) would be ignored with no word. --pedigree is read with
-    // containsKey and has no default; config_profile_* are the test
-    // profiles' labels.
+    // --max-cpus) would be ignored with no word. config_profile_* are the
+    // test profiles' labels.
     def schema_params = new groovy.json.JsonSlurper()
         .parseText(file("${projectDir}/nextflow_schema.json").text)['definitions']
         .values().collectMany { group -> (group['properties'] ?: [:]).keySet() as List }
-    def known_params = schema_params + ['pedigree', 'config_profile_name', 'config_profile_description']
+    def known_params = schema_params + ['config_profile_name', 'config_profile_description']
     def unknown_params = params.keySet().findAll { k -> !known_params.contains(k) }.sort()
     if (unknown_params) {
         error "unknown parameter${unknown_params.size() > 1 ? 's' : ''} " +
@@ -332,9 +331,8 @@ workflow {
     // leading family column: one family runs, the one --family_name names
     // (needed when the file holds more than one). Here only the family and
     // sample columns are read; FAMILY_PREPARE (bin/family_report.py prepare)
-    // checks the rest before the joint call starts. Read with containsKey,
-    // so a run without the option is not told about an undefined parameter.
-    def pedigree   = params.containsKey('pedigree') ? params.pedigree : null
+    // checks the rest before the joint call starts.
+    def pedigree   = params.pedigree
     // The family's folder under --outdir: --family_name, else 'family'
     def family_dir = params.family_name ?: 'family'
     def family_ids = []

nextflow.config

@@ -121,6 +121,7 @@ params {
     viral_ref          = null       // viral: the viral genomes of step 47 (setup.sh --viral-data: reference/viral-<version>/viral_genomes.fa)
     hpo                = null       // exomiser: the symptoms as HPO ids, comma separated (HP:0001250,HP:0001263); setting it runs exomiser (step 48)
     exomiser_data      = null       // exomiser: Exomiser's data folder (setup.sh --exomiser-data: reference/exomiser-<release>)
+    pedigree           = null       // opt-in: a pedigree or families file (assets/pedigree.example.tsv, assets/families.example.tsv); runs the family workflow (steps 38, 39)
     family_name        = null       // --pedigree: the family to run of a families file (assets/families.example.tsv) that holds more than one; also its folder under --outdir (default family)
     exomiser_proband   = null       // exomiser: the sample the --hpo terms describe; needed when the samplesheet has more than one
     exomiser_assembly  = 'hg38'     // exomiser: the assembly of its data; hg19 only for Exomiser's cut-down test release (CI)

nextflow_schema.json

@@ -105,7 +105,7 @@
                 "tools": {
                     "type": "string",
                     "description": "Comma-separated list of tools to run; an unknown name stops the run. DB-heavy tools (vep, slivar, clinical_filter, cpsr, clinvar, expansion_hunter, sample_qc) and extras (stranger, ancestry (needs --ancestry_ref and prs), ancestry_fine (needs --ancestry_fine_panel), ancient (needs --ancient_panel; samples with a gVCF), pypgx, SV callers, cram_archive, cyrius, parascopy, paraphase (PacBio HiFi BAMs only), y_haplogroup (male samples), y_str (Y-STR repeat counts, male samples), consumer_export (samples with a gVCF), bloodtype (blood group antigens; RhD needs the BAM), lpa (the LPA KIV-2 repeat count; needs the BAM), vntyper (MUC1 VNTR frameshifts behind ADTKD-MUC1; needs the BAM), viral (viral DNA in the unmapped reads; needs --viral_ref and a BAM), exomiser (runs when --hpo is set), chip (clonal haematopoiesis from the BAM; needs --chip_gff), archaic (needs --archaic_data), mei (Alu, L1 and SVA insertions with MEGAnE; needs --megane_kmers and a BAM), mocha (mosaic chromosomal alterations; needs --mocha_data), locityper (a second opinion at HLA class II, KIR, CYP2D6 and FCGR; samples with a BAM; needs --locityper_data)) are opt-in. prs and vcfanno are skipped with a warning until --pgs_scoring or a score file is set. With pharmcat, PharmCAT also gets the HLA types of hla_typing and a CYP2D6 call pypgx and cyrius agree on (PGX_CONSENSUS).",
-                    "default": "pharmcat,cpic,vcfanno,roh,prs,mito_haplogroup,telomere_hunter,mosdepth,mito_variants,html_report,multiqc",
+                    "default": "pharmcat,cpic,vcfanno,roh,prs,mito_haplogroup,telomere_hunter,mosdepth,mito_variants,traits,html_report,multiqc",
                     "fa_icon": "fas fa-toolbox"
                 }
             }
@@ -344,6 +344,12 @@
                     "description": "Exomiser's data folder, which `scripts/setup.sh --exomiser-data` installs as reference/exomiser-<release> (its <release>_hg38 and <release>_phenotype folders). Required with --hpo.",
                     "fa_icon": "fas fa-folder"
                 },
+                "pedigree": {
+                    "type": "string",
+                    "format": "file-path",
+                    "description": "Opt-in: run the family workflow (steps 38 and 39) on the samples of the samplesheet this file names: a pedigree (assets/pedigree.example.tsv), or a families file with a leading family column (assets/families.example.tsv). A families file that holds more than one family needs --family_name, and only that family runs. The family that runs needs two or more of those samples, each with a gVCF, and --somalier_sites.",
+                    "fa_icon": "fas fa-sitemap"
+                },
                 "family_name": {
                     "type": "string",
                     "pattern": "^[a-zA-Z0-9._][a-zA-Z0-9._-]*$",