pgscatalog_meta

Code for systematic evaluation of polygenic score (PGS) portability across ancestries using data from the PGS Catalog.

Overview

This repository contains the analysis pipeline used to:

• Download and process PGS Catalog performance data.
• Build evaluation-level datasets linking PGS, performance metrics, sample sets, traits, and ancestry.
• Harmonize AUC estimates and confidence intervals.
• Perform inverse-variance weighted meta-analysis of AUC.
• Quantify portability as ΔAUC, defined as target ancestry AUC minus European ancestry AUC.
• Generate publication figures, including the faceted ΔAUC forest plot.

Data source

All data are derived from the PGS Catalog. The pipeline uses two complementary PGS Catalog resources:

1. the PGS Catalog REST API, accessed through the quincunx R package; and
2. the PGS Catalog bulk metadata files, downloaded from the official PGS Catalog FTP metadata release.

Raw downloaded and normalized files are cached locally under data/pgs_cache/.

Repository structure

pgscatalog_meta/
├── analysis/
│   ├── 01_Installation_API_and_perfomance_pull.R
│   ├── 02_loading_PGS_matrix.R
│   ├── 03_bulk_metadata_pull.R
│   ├── 04_PGS_systemic_portability_unique_pss.R
│   ├── 05_pgs_auc_ci_audit.R
│   ├── 06_meta_ivw.R
│   ├── 07_i_square_filter.R
│   └── 08_generate_publication_figures.R
├── data/
│   └── pgs_cache/              # generated; not tracked if large
├── results/                    # generated analysis outputs
├── meta_roadmap_two_stages/     # generated meta-analysis tables and figures
├── README.md
└── LICENSE

Only the scripts in analysis/ are required to reproduce the main publication figure. Output folders are generated by the scripts.

Main analysis pipeline

To reproduce the publication figure from raw data, run the scripts in the following order.

1. 01_Installation_API_and_perfomance_pull.R

Downloads performance metrics from the PGS Catalog API using quincunx and saves normalized tables to data/pgs_cache/.

Main outputs include:

• pm_<STAMP>.rds
• ppm_<STAMP>.csv
• pss_links_<STAMP>.csv
• samples_<STAMP>.csv
• classm_<STAMP>.csv
• optional .parquet versions if arrow is available

2. 02_loading_PGS_matrix.R

Loads the normalized API-derived tables and checks the relationships between performance metrics, sample sets, samples, and ancestry labels.

This is mainly a validation and sanity-check step before combining API and bulk metadata resources.

3. 03_bulk_metada_pull.R

Downloads PGS Catalog bulk metadata files and derives the training ancestry categories used downstream.

Main output:

• train_bucket_<STAMP>.csv

The train_bucket variable groups each PGS according to the ancestry composition of its development or training data.

4. 04_PGS_systemic_portability_unique_pss.R

Builds the evaluation-level dataset using unique evaluation sample sets and combines performance metrics, evaluation ancestry, sample size information, trait labels, and training ancestry buckets.

This step creates the systemic portability input tables used by later AUC and ΔAUC analyses.

5. 05_pgs_auc_ci_audit.R

Harmonizes AUC estimates and confidence intervals across performance records.

This step audits reported and inferred confidence intervals and produces the final evaluation table used for meta-analysis.

Key output:

• eval_df_final_auc_ci.csv

6. 06_meta_ivw.R

Performs Stage-1 inverse-variance weighted meta-analysis.

For each trait, PGS, and evaluation ancestry, this script pools AUC values across available evaluation cohorts using a logit-scale inverse-variance weighted model.

Main output:

• meta_roadmap_two_stages/<STAMP>/tables/stage1_pooled_cells_<STAMP>.csv

This file is the main Stage-1 input for heterogeneity filtering and final figure generation.

7. 07_i_square_filter.R

Applies the heterogeneity filter to the Stage-1 pooled estimates.

Cells with high heterogeneity are removed when:

• k_eval >= 2, and
• I² > 80%.

Main outputs:

• stage1_I2_summary_<STAMP>.csv
• stage1_pooled_cells_I2filtered_<STAMP>.csv
• I² diagnostic plots under plots/heterogeneity/

8. 08_generate_publication_figures.R

Generates the final publication figures from the I²-filtered Stage-1 table.

This script rebuilds the paired ΔAUC analysis in a self-contained way and applies the final reproduction filters used for the faceted forest plot.

Main outputs include:

• paired_weightedtest_by_bucket_rebuilt_<STAMP>.csv
• reproduction_gate_heatmap_df_<STAMP>.csv
• reproduction_gate_keep_traits_<STAMP>.csv
• deltaAUC_forest_faceted_data_<STAMP>.csv
• deltaAUC_forest_faceted_<STAMP>.pdf

Main publication figure

The main publication figure is the faceted ΔAUC forest plot generated by 08_generate_publication_figures.R.

It shows:

• ΔAUC for each target ancestry relative to European evaluation cohorts.
• Results grouped by trait and training ancestry bucket.
• Inverse-variance weighted ΔAUC estimates across PGS.
• 95% confidence intervals.
• Visual distinction between estimates supported by pooled cells and those based only on single-evaluation cells.

The final ΔAUC definition is:

ΔAUC = AUC_target ancestry − AUC_European ancestry

Negative values indicate lower predictive performance in the target ancestry compared with European evaluation cohorts.

Reproducibility notes

• Run scripts from the repository root, not from inside analysis/, unless paths are manually adjusted.
• Scripts are designed to be run sequentially from step 1 to step 8.
• Each script writes intermediate results to disk and later scripts reuse those files.
• Most output filenames include a date stamp in the format YYYYMMDD.
• To reproduce an identical analysis, use the same input cache and the same date-stamped intermediate files.
• A fresh run may differ from older runs if the PGS Catalog has been updated since the previous download.

Requirements

R version 4.0 or newer is recommended.

Main R packages used by the pipeline include:

• quincunx
• dplyr
• tidyr
• readr
• stringr
• purrr
• janitor
• forcats
• ggplot2
• scales
• glue
• tibble
• jsonlite

Some scripts install missing packages automatically, but a clean reproducible run should ideally start from a documented R environment.

Suggested run command

From the repository root:

Rscript analysis/01_Installation_API_and_perfomance_pull.R
Rscript analysis/02_loading_PGS_matrix.R
Rscript analysis/03_bulk_metada_pull.R
Rscript analysis/04_PGS_systemic_portability_unique_pss.R
Rscript analysis/05_pgs_auc_ci_audit.R
Rscript analysis/06_meta_ivw.R
Rscript analysis/07_i_square_filter.R
Rscript analysis/08_generate_publication_figures.R

License

See the LICENSE file.