diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f99374..3db09b6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,45 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [Unreleased] + +### Fixed + +- Regional plots select rows before axis scaling, labels and LD lookup, preserve + lead identity and honor shared/per-panel LD settings. Invalid stacked lead + positions are rejected; canonical and legacy frames can share a stack. +- Label eligibility is resolved once from the selected lead row. Fresh reference + LD replaces an existing `R2` column without losing coloring or changing row order. +- Colocalization projects fields from their declared source before merging, so + unrelated metadata cannot change effect colors or satisfy a missing column. + Disabling effect coloring also disables its effect-column requirements. +- GTEx preserves chromosome and absolute position; relative TSS distance is no + longer accepted as an absolute coordinate. GTF/GFF3 display names are preserved + independently of attribute order. +- Genome-wide column resolution reaches QQ compositions and Miami hover. Missing + categorical groups remain visible, including groups omitted from a custom order. + Pandas categorical columns render correctly in categorical Manhattan plots. +- PLINK resolves paths before changing directory and recognizes registered Ensembl + species aliases. Unknown chromosome-set support is rejected explicitly. +- Recombination plotting reads caller directories without replacing files or + applying an implicit liftover. Managed maps reject unavailable target assemblies. +- Archive ingestion rejects links and unsafe or duplicate members, streaming map + text into generated canonical paths. Concurrent downloads use private staging. +- Gene and exon caches publish together and retain the previous entry after an + interrupted write. Legacy separate CSV pairs are treated as cache misses. +- Matplotlib and Bokeh heatmap cells share their boundaries with SNP highlights. +- Example comparison generates outside the checkout, preserves manual edits and + returns a failing status for unexpected differences. + +### Changed + +- FINEMAP/CAVIAR loaders preserve PIPs and supplied membership without inventing + credible sets. Figures with no membership use the existing PIP-only display. +- `calculate_colocalization_overlap` matches chromosome plus absolute position. + Position-only inputs require `common_chrom`; output columns are canonical. +- `Species.plink_flags=None` means unknown support. An explicit empty tuple keeps + PLINK's human chromosome defaults. + ## [3.1.1] - 2026-09-08 ### Added diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 36850aa..024558d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -65,7 +65,7 @@ markdown, mermaid, link-check, or test failures will not merge. - **Base branch:** open PRs against `main`. - **Tests first:** follow test-driven development — add or update tests in `tests/` before (or alongside) the implementation. Mock PLINK calls rather - than requiring a local install; see `tests/test_ld.py` for the pattern. + than requiring a local install; see `tests/test_ld_process.py` for the pattern. - **Changelog:** add an entry to `CHANGELOG.md` under the `## [Unreleased]` section, using the `Added` / `Changed` / `Fixed` / `Removed` categories. - **Docs:** update `README.md`, `docs/USER_GUIDE.md`, `docs/ARCHITECTURE.md`, @@ -73,8 +73,11 @@ markdown, mermaid, link-check, or test failures will not merge. features, or changes behavior that users rely on. - **Example plots:** if your change touches a backend or a panel, run `scripts/example_diff.sh`. It regenerates the examples and lists the exports - whose content changed after normalising generated ids; the suite does not - see serialised output. Commit only the files it reports, using `--keep`. + whose content changed after normalising generated ids. Generation runs in a + temporary directory and leaves the checkout untouched. Exit 1 reports differences; + exit 2 reports a failure. Review the changes, then use `--keep` to accept only + changed exports. Acceptance refuses to overwrite manually modified exports. + Commit the accepted outputs with the source change. - **Commits:** keep messages focused on *what* changed and *why*. Do not include AI or tool attribution. - **Scope:** one logical change per PR. Refactors and feature work belong in diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 7a0bb3e..4e9801f 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -14,6 +14,19 @@ of three interchangeable backends: matplotlib (static PNG/PDF), plotly backend-pluggable pipeline: validation → data preparation → backend-agnostic plot assembly → backend-specific rendering. +Regional preparation resolves each panel's columns and LD options, selects its +rows once and carries the selected lead onward. Genome-wide preparation projects +configured roles to canonical columns before sharing layout with QQ and Miami. +Colocalization projects each source before merging, so caller metadata cannot +rename internal fields. The backends use the shared `cell_edges` geometry for +heatmap cells and highlights. + +Reference-data ownership is explicit. Caller map directories are read-only; +managed caches alone may download and replace generations. Download writers have +private staging files, map archives stream regular members into canonical names, +and gene/exon pairs publish as one atomically replaced ZIP. See +[ADR 0009](adr/0009-resolved-inputs-and-owned-publication.md). + ## Component Diagram ```mermaid @@ -32,6 +45,8 @@ graph TD subgraph Prepare["Data Preparation"] DATA[_data.py: shared p-value intake] + REGIONAL["plotter.py: selected regional inputs and resolved leads"] + CACHE["_gene_cache.py: atomic gene and exon archive"] LD[ld.py: PLINK wrapper] RECOMB[recombination.py: maps + CanFam4 liftover] ENSEMBL["reference_genes.py:
gene fetch by build
(ensembl.py, ucsc.py)"] @@ -72,15 +87,18 @@ graph TD SCHEMA --> EQTLV EQTLV --> COLORS LD --> COLORS - COLORS --> LZ - DATA --> LZ + COLORS --> PANELS + DATA --> REGIONAL + LZ --> REGIONAL + REGIONAL --> LD + REGIONAL --> PANELS + ENSEMBL --> CACHE COLORS --> MP COLORS --> SP COLORS --> MIAMI COLORS --> LDH COLORS --> CP RECOMB --> LZ - LZ --> PROTO LZ --> PANELS MP --> PANELS SP --> PANELS @@ -229,7 +247,7 @@ stages: | Regional panels | Internal modules | `src/pylocuszoom/panels/{association,finemapping,eqtl,genes,heatmap}.py` | One module per panel type, each holding its value type, the constructor it builds itself through, and the `draw` method that draws it. A panel carries its resolved mode, region, hover contract and layout, so drawing inspects no columns | | `MiamiRequest`, `MiamiPanel`, `miami_plan` | Internal module | `src/pylocuszoom/panels/miami.py` | The Miami figure: a request the plotter resolves, a panel that draws one mirrored Manhattan half with its SNP annotations, and the builder that lays two of them on a `FigurePlan` with the cross-panel highlights | | `PhewasPanel`, `ForestPanel` | Internal module | `src/pylocuszoom/panels/stats.py` | The PheWAS and forest panels, each built through `from_frame` and drawing itself. Every family is a panel value with `draw` on a `FigurePlan`; no family holds a renderer class | -| `ColocPanel` | Internal module | `src/pylocuszoom/panels/coloc.py` | The colocalization scatter: the merged frame, its resolved column names and lead index, drawing itself with both threshold lines through `add_significance_line` | +| `ColocPanel` | Internal module | `src/pylocuszoom/panels/coloc.py` | The colocalization scatter: the projected frame with fixed source-owned column roles and lead index, drawing itself with both threshold lines through `add_significance_line` | | `LDHeatmapPanel` | Internal module | `src/pylocuszoom/panels/ld_heatmap.py` | The standalone heatmap: the matrix, its ids, and the lead and highlight indices, drawing itself | | `ManhattanPlotter` | Class | `src/pylocuszoom/manhattan_plotter.py` | Genome-wide Manhattan and QQ plots | | `StatsPlotter` | Class | `src/pylocuszoom/stats_plotter.py` | PheWAS and forest plots | diff --git a/docs/CODEMAP.md b/docs/CODEMAP.md index 5d931cd..8241ef0 100644 --- a/docs/CODEMAP.md +++ b/docs/CODEMAP.md @@ -199,7 +199,8 @@ Data transformation between validated input and backend-ready primitives. | 3d | get_recombination_rate_for_region | Region-filtered recomb rate | [recombination.py](../src/pylocuszoom/recombination.py) | | 3d | download_canine_recombination_maps | Lazy-download bundled maps | [recombination.py](../src/pylocuszoom/recombination.py) | | 3d | recomb_for_region, RecombResult | The one place the skip-the-overlay decision is made, reported as a value | [recombination.py](../src/pylocuszoom/recombination.py) | -| 3d | download_recombination_maps, RecombSource | Species-generic download, extract and publish; the record carries everything that varies | [recombination.py](../src/pylocuszoom/recombination.py) | +| 3d | download_recombination_maps, RecombSource | Species-generic download, canonical member streaming and publication; the record carries everything that varies | [recombination.py](../src/pylocuszoom/recombination.py) | +| 3e | prepare_genomewide_frames | Per-input column projection before genome-wide layout and composition | [manhattan.py](../src/pylocuszoom/manhattan.py) | | 3e | prepare_manhattan_frames | Cumulative-position Manhattan prep against one shared `GenomeLayout` | [manhattan.py](../src/pylocuszoom/manhattan.py) | | 3e | GenomeLayout | Chromosome order, offsets, colours, ticks, and x limits for every panel of a figure | [manhattan.py](../src/pylocuszoom/manhattan.py) | | 3f | prepare_qq_data | Observed vs expected QQ data | [qq.py](../src/pylocuszoom/qq.py) | @@ -208,12 +209,14 @@ Data transformation between validated input and backend-ready primitives. | 3h | source_for, get_genes_for_build | The build-to-source routing and the one fetch-and-cache orchestration | [reference_genes.py](../src/pylocuszoom/reference_genes.py) | | 3h | ensembl_source, fetch_overlap_frames | Ensembl REST client | [ensembl.py](../src/pylocuszoom/ensembl.py) | | 3h | ucsc_source, fetch_track_frames | UCSC track client, used for CanFam3.1, CanFam4 and FelCat9 | [ucsc.py](../src/pylocuszoom/ucsc.py) | -| 3h | gene cache | On-disk cache shared by both gene sources | [_gene_cache.py](../src/pylocuszoom/_gene_cache.py) | -| 3j | enrich_with_ld | Calls PLINK for lead-SNP R² and merges it into the GWAS frame under one recovery policy | [_ld_plotting.py](../src/pylocuszoom/_ld_plotting.py) | +| 3h | gene cache | Atomic gene/exon archive cache shared by both gene sources | [_gene_cache.py](../src/pylocuszoom/_gene_cache.py) | +| 3j | _AssociationInput | Region-selected data and resolved per-panel options | [plotter.py](../src/pylocuszoom/plotter.py) | +| 3j | enrich_with_ld | Calls PLINK for lead-SNP R² and assigns values by SNP ID while preserving selected rows | [_ld_plotting.py](../src/pylocuszoom/_ld_plotting.py) | | 3j | prepare_pvalue_data | Shared p-value intake: filtering, zero-value mode, finite `-log10` | [_data.py](../src/pylocuszoom/_data.py) | | 3j | prepare_eqtl_for_plotting | eQTL panel prep | [eqtl.py](../src/pylocuszoom/eqtl.py) | -| 3j | calculate_colocalization_overlap | Colocalisation overlap between two association frames | [eqtl.py](../src/pylocuszoom/eqtl.py) | -| 3j | add_snp_labels | SNP label placement and lead-proximity filtering | [labels.py](../src/pylocuszoom/labels.py) | +| 3j | calculate_colocalization_overlap | Significant coordinate overlap on chromosome and absolute position | [eqtl.py](../src/pylocuszoom/eqtl.py) | +| 3j | select_label_candidates | Shared lead-proximity eligibility for regional and standalone SNP labels | [_label_data.py](../src/pylocuszoom/_label_data.py) | +| 3j | add_snp_labels | SNP label ranking and placement | [labels.py](../src/pylocuszoom/labels.py) | | 3j | liftover | CanFam3.1 to CanFam4 coordinate lift for recombination maps | [_liftover.py](../src/pylocuszoom/_liftover.py) | | 3j | UNSET, resolve_threshold | The significance-threshold sentinel every threshold-bearing plotter uses, which keeps `None` meaning "draw no line" | [_plotter_utils.py](../src/pylocuszoom/_plotter_utils.py) | | 3i | Regional panels | The five regional panel value types, each with the `draw` method that draws it, one per module | [panels/](../src/pylocuszoom/panels/) | diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index a49fa24..de1d6f7 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -43,18 +43,20 @@ Implementation: `liftover` leaf holding downloaded chain files - [`_gene_cache.cache_root()`](../src/pylocuszoom/_gene_cache.py) -You can also override the cache location explicitly by passing `output_dir` -to `download_canine_recombination_maps()` / `ensure_recomb_maps()` — this -bypasses the environment variables entirely. +To pre-download maps into a chosen directory, call +`download_canine_recombination_maps(output_dir="/path/to/maps")`. +Passing `recomb_data_dir` to the plotter, or `data_dir` to the map helpers, +selects a read-only caller directory. It never downloads or replaces files there, +and its coordinates must already use the requested build. With no directory, +`ensure_recomb_maps()` manages the platform cache and may download built-in maps. ## Programmatic Configuration (Pydantic Models) -The user-facing API uses plain keyword arguments (`plot()`, -`plot_stacked()`). Internally these kwargs are validated by frozen -Pydantic models defined in -[`src/pylocuszoom/config.py`](../src/pylocuszoom/config.py). You normally -do not construct these directly, but they define the canonical set of -options and their defaults. +The plotting methods take frozen Pydantic values such as `ColumnConfig`, +`DisplayConfig`, `LDConfig` and `PanelInputs`, defined in +[`src/pylocuszoom/config.py`](../src/pylocuszoom/config.py). Region coordinates +and per-panel overrides remain keyword arguments. The public values can be +reused across calls; each call resolves its effective options before rendering. ### `RegionConfig` — genomic region (required) @@ -176,8 +178,9 @@ If you need per-environment behaviour, do it at the caller level, e.g.: - Set `XDG_CACHE_HOME` / `LOCALAPPDATA` per machine to control where reference data is cached. -- Pre-download reference data in CI with `ensure_recomb_maps()` pointing at - a shared directory, then set `output_dir=` accordingly at runtime. +- Pre-download canine maps with `download_canine_recombination_maps(output_dir=...)`, + then pass that directory as `recomb_data_dir` to the plotter. Caller maps must + already use the requested genome build. - On Databricks, the `/dbfs/FileStore/reference_data/recombination_maps` path is selected automatically. diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index 64d5322..618e87e 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -64,7 +64,7 @@ The common development commands are: | `uv run pre-commit run --all-files` | Run the full pre-commit suite against every file in the repo. | | `uv build` | Build the wheel and sdist via hatchling into `dist/`. | | `uv run python examples/generate_example_plots.py` | Regenerate example plots shown in the README. | -| `scripts/example_diff.sh [--keep]` | Regenerate the examples and list the exports whose content changed after id normalisation; the equivalence check for backend and panel changes. | +| `scripts/example_diff.sh [--keep]` | Generate outside the checkout and compare exports with HEAD. Exit 1 means differences; exit 2 means failure. `--keep` accepts generated changes only when affected exports have no manual edits. | | `uv lock` | Refresh `uv.lock` after changing dependencies in `pyproject.toml`. | See [CONTRIBUTING.md](../CONTRIBUTING.md) for the full pre-commit and pre-PR checklists. diff --git a/docs/TESTING.md b/docs/TESTING.md index 37380f1..ca1d354 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -65,7 +65,9 @@ Tests live under `tests/`. Files follow the `test_*.py` naming convention and ma - `tests/test_data_intake.py` — the shared p-value intake policy - `tests/test_manhattan_plotter.py`, `tests/test_qq.py`, `tests/test_manhattan.py` — Manhattan/QQ coverage - `tests/test_stats_plotter.py`, `tests/test_phewas.py`, `tests/test_forest.py` — statistical plots -- `tests/test_ld.py` — PLINK wrapper (driven through the `fake_plink` fixture; no real PLINK binary required) +- `tests/test_ld.py` — PLINK command construction and species flags +- `tests/test_ld_parsing.py` — single-lead and pairwise PLINK output parsing +- `tests/test_ld_process.py` — PLINK file validation, executable discovery and process execution, using `fake_plink` without a real PLINK binary - `tests/test_backends.py` — the shared `PlotBackend` surface and the matplotlib backend - `tests/test_plotly_backend.py`, `tests/test_bokeh_backend.py` — the interactive backends - `tests/test_notebook_backends.py` — Plotly/Bokeh notebook compatibility, parametrised over both backends through `tests/figure_probes.py` @@ -105,7 +107,7 @@ Hypothesis strategies shared across tests live in `tests/strategies.py`. ### Guidelines - **Assert on observable outputs, not mock call counts.** Check returned figures, DataFrame columns/shapes, written files, and raised exceptions. Reserve `assert_called_once_with` for true system boundaries (PLINK subprocess, HTTP, filesystem dispatch). -- **Drive PLINK through `fake_plink`** — tests must not require a real PLINK installation. The `fake_plink` fixture in `conftest.py` patches `subprocess.run` and writes a real `.ld` file at the path the command asked for, so command construction, output parsing and the R2 merge all stay inside the test. Assert on the frame `calculate_ld` returns, not on what the mock received: a command flag is already pinned by `TestBuildLdCommand` and `TestBuildPairwiseLdCommand`, which call the pure builders and assert on the list they return. +- **Drive PLINK through `fake_plink`** — tests must not require a real PLINK installation. The `fake_plink` fixture in `conftest.py` patches `subprocess.run` and writes a real `.ld` file at the path the command asked for, so command construction, output parsing and the R2 assignment all stay inside the test. Assert on the frame `calculate_ld` returns, not on what the mock received: a command flag is already pinned by `TestBuildLdCommand` and `TestBuildPairwiseLdCommand`, which call the pure builders and assert on the list they return. - **State a rendering behaviour once, not once per backend.** A fact about the figure (which marker, whether it hovers, whether it exports) belongs in one `@pytest.mark.parametrize("backend_name", INTERACTIVE_BACKENDS)` test reading a probe from `tests/figure_probes.py`. Hand-written per-backend twins drift: the bokeh eQTL marker test used to pass with the negative-effect glyph never drawn. A genuine library-specific regression still belongs in `test_plotly_backend.py` or `test_bokeh_backend.py`. - **Cover edge cases**: empty DataFrames, missing required columns, mismatched list lengths, single-SNP regions, and cross-chromosome filtering. - **Respect the 30s timeout.** If a test is legitimately slow, override with `@pytest.mark.timeout(60)` rather than raising the global default. diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md index fdb51e8..d7b7a58 100644 --- a/docs/USER_GUIDE.md +++ b/docs/USER_GUIDE.md @@ -1022,7 +1022,17 @@ eqtl_df = load_gtex_eqtl( eqtl_df = load_eqtl_catalogue("eqtl_results.tsv", gene="TP53") ``` -**Output columns:** `pos`, `p_value`, `gene`, `effect` +**Output columns:** `pos`, `p_value`, `gene`, `effect_size`, and `chr` when supplied. +GTEx variant IDs provide both chromosome and absolute position. A file carrying +only relative `tss_distance` is rejected because it does not locate a variant. + +`calculate_colocalization_overlap(gwas_df, eqtl_df)` matches canonical chromosome +and absolute position, without allele harmonization. Custom names use +`gwas_chrom_col`, `eqtl_chrom_col` and the existing position/p-value arguments. +If both inputs are already scoped to one chromosome and omit `chr`, pass +`common_chrom=1` explicitly. Any supplied chromosome values must agree with it. +The result has `chr`, `pos`, `p_value_gwas` and `p_value_eqtl` columns. This helper +finds significant coordinate matches; it does not perform statistical colocalization. ### Fine-mapping Loaders @@ -1040,7 +1050,7 @@ from pylocuszoom import load_susie, load_finemap fm_df = load_susie("susie_results.tsv") # Output: pos, pip, cs (credible set, 0 = not in CS) -# FINEMAP results (assigns CS based on 95% PIP threshold) +# FINEMAP results (preserves PIPs and any supplied credible-set membership) fm_df = load_finemap("finemap_output.snp") # Use in plot @@ -1051,7 +1061,12 @@ fig = plotter.plot_stacked( ) ``` -**Output columns:** `pos`, `pip`, `cs` (credible set assignment) +**Output columns:** `pip`, plus `pos` and `cs` where the source supplies them. +FINEMAP and CAVIAR loaders no longer infer credible sets from cumulative PIPs. +Supply membership from the inference method that produced your results, or plot +PIPs without set assignments. CAVIAR requires a SNP annotation merge to add +absolute positions before plotting. `cs_col` chooses the output name for supplied +membership; it does not request inference. ### Gene Annotation Loaders @@ -1099,6 +1114,27 @@ frame still carrying the pre-4.0 `ps` and `p_wald` names is accepted with a `DeprecationWarning` until 5.0.0. Other names are supported through `ColumnConfig` and `GenomeWideConfig`. +Regional plots select chromosome and inclusive position bounds before choosing a +lead, scaling axes, labeling points or calculating LD. A frame without `chr` is +assumed to contain only the requested chromosome. In stacks, shared `LDConfig` +values apply to every panel unless a per-panel list overrides them. A lead +position shared by multiple variants selects the strongest p-value at that +position, with input order breaking ties. That selected row also defines label +eligibility; nearby non-lead variants are excluded before ranking labels. +Requesting reference LD replaces an existing `R2` column in the prepared plot +data. The caller's frame is unchanged. Regional heatmaps sort SNPs and both +matrix axes together, and require distinct retained genomic positions. + +Genome-wide stacks resolve supported legacy names independently for each frame. +QQ compositions and Miami hover read those same resolved columns. Unselected +metadata never replaces a configured role. Requested colocalization LD columns +must exist in their declared source frame. Effect columns are required in their +declared sources only when `color_by_effect=True`. + +Categorical Manhattan plots render missing categories as `Uncategorised`. +An explicit category order sets priority; other observed categories append in +alphabetical order so retained observations remain visible. + ```python gwas_df = pd.DataFrame({ "chr": [1, 1, 1], @@ -1213,7 +1249,13 @@ plotter = LocusZoomPlotter(species="canine", genome_build="canfam4") Recombination maps are automatically downloaded on first use (~50MB), into `recombination_maps` under the platform cache. The CanFam3.1 to CanFam4 liftover chain downloads into a `liftover` directory beside it, so replacing a -map set never touches the chain. +map set never touches the chain. Managed maps requested in another assembly +without a registered conversion are skipped with a build-unavailable warning. + +An explicit `recomb_data_dir` is caller-owned, read-only and already in the +requested build. This works for every species, including `species=None`, and +requires only the chromosomes used by the plot. No automatic liftover runs on +caller maps. ### Feline @@ -1266,7 +1308,16 @@ overrides the fetched one. | canine, dog | canis_lupus_familiaris | | feline, cat | felis_catus | -Any valid Ensembl species name also works (e.g., `sus_scrofa` for pig). +Any valid Ensembl species name also works for annotation (e.g., `sus_scrofa` +for pig). Registered Ensembl names are aliases of their species records, so +`canis_lupus_familiaris` receives the same PLINK flags as `canine`. + +LD calculation requires known chromosome-set flags. Unknown PLINK support raises +an error; supply a `Species` record with explicit `plink_flags` when adding a +species. An empty tuple explicitly selects PLINK's human defaults, while `None` +means support is unknown. Relative reference, working-directory and executable +paths resolve against the caller's directory before PLINK starts. Bare executable +names search PATH. **Region Limit:** Maximum 5Mb per request (Ensembl API limitation). For larger regions, provide `genes_df` directly. @@ -1295,6 +1346,9 @@ UCSC's `ncbiRefSeq` is a transcript-level track, so transcripts sharing a symbol - Windows: `%LOCALAPPDATA%/pylocuszoom/ensembl/{ensembl_species}/` and `%LOCALAPPDATA%/pylocuszoom/ucsc/{ucsc_genome}/` A CanFam3.1 or FelCat9 plot caches under `ucsc/canFam3/` or `ucsc/felCat9/`, not under `ensembl/`. +Each entry atomically publishes one ZIP containing both gene and exon CSVs. +Older separate CSV pairs become cache misses and are fetched again; clearing the +cache removes both formats. The returned count is files removed, one per new entry. ```python # Clear cache when needed diff --git a/docs/adr/0009-resolved-inputs-and-owned-publication.md b/docs/adr/0009-resolved-inputs-and-owned-publication.md new file mode 100644 index 0000000..3e964b4 --- /dev/null +++ b/docs/adr/0009-resolved-inputs-and-owned-publication.md @@ -0,0 +1,54 @@ +# ADR 0009: Resolve data roles before rendering and publish only owned state + +Status: Accepted + +## Context + +The figure and panel model already gives every plot one rendering path. Its +inputs still allowed several interpretations downstream: a numeric lead position +could identify a different row, merged column suffixes could shadow configured +fields, and a temporary alias resolution could be lost by a composed plot. +Filesystem paths similarly failed to distinguish caller data from managed caches +or an incomplete write from a published entry. + +## Decision + +Resolve each regional frame's selected rows, column names, LD options and lead +identity before panel construction. Genome-wide preparation projects configured +roles per frame before building shared layout. Colocalization projects each +source's requested roles before merging; internal column names have one meaning. +Chromosome and absolute position define coordinate overlap. Position-only callers +must explicitly identify their shared chromosome. Fine-mapping loaders preserve +reported PIPs and membership without making an implicit statistical inference. + +Use the existing canonical heatmap edges in drawing adapters and SNP outlines. +Order regional heatmap coordinates with both matrix axes before drawing them. +Keep FigurePlan, panel-owned drawing, the shared layout and public config values. + +An explicit map directory is caller-owned and read-only. Only the default managed +cache may install a built-in map set or apply its known assembly conversion. +Map archives supply regular text members, streamed under generated canonical +names. General filesystem extraction is unnecessary. + +Each HTTP download owns a private temporary file. Gene and exon annotations form +one cache entry, an archive replaced only after both members have been written. +A failed update leaves the previous entry available; unreadable entries are +cache misses. Example verification likewise generates outside the checkout and +only copies outputs back through an explicit acceptance operation. + +## Consequences + +Extra input metadata cannot redefine selected column roles. One prepared regional +input supplies all association consumers. Writers cannot share an in-progress +filename or publish half an annotation entry. Custom map files survive plotting. + +Position-only overlap calls must supply `common_chrom`. FINEMAP/CAVIAR calls that +relied on inferred membership now receive PIPs without a credible-set column. +Legacy annotation CSV pairs are cold misses. These behavior changes are deliberate +corrections to ambiguous or scientifically incorrect contracts; migration details +are in the user guide and changelog. + +Regression tests observe native plotted points and cell bounds, loaded data, +subprocess paths resolved from its working directory, concurrent publication and +preservation of caller files. The example comparison checks serialized exports +and returns nonzero on a difference. diff --git a/examples/bokeh/ld_heatmap_bokeh.html b/examples/bokeh/ld_heatmap_bokeh.html index 0c421af..beac939 100644 --- a/examples/bokeh/ld_heatmap_bokeh.html +++ b/examples/bokeh/ld_heatmap_bokeh.html @@ -18,10 +18,10 @@ -
+
- -
+
-