liftover blocks when lifting joint VCFs from ROSMAP (bg37 -> hg19)

[milcs40]

Hello,

I have successfully used bcftools +liftover in the past for large joint VCFs (ADNI cohort, with ~800 samples).
I was trying to perform the same liftover, this time on joint VCFs from the ROSMAP cohort. These are ~1000 samples, bg37, split per chromosomes.

I started by trying to lift the whole chromosomes. At the speed things were going, I thought one chromosome would take more than one month.
I then decided to subset the VCFs for just a few particular regions of interest. This reduced the VCFs to around 5000 positions per VCF.

Unfortunately, this didn't solve the issue. It seems bcftools +liftover is just stuck in there. And I have no way to know what is going on.

Any clue?

[freeseek]

Maybe some annoying indels are causing a bug in the realignment step. Can you send me a sites-only VCF chunk where you are experiencing BCFtools/liftover getting stuck?

[jjfarrell]

I was have a similar slowness issue #25 with a warning.

[freeseek]

Latest release of BCFtools/liftover had a fix for #25

[milcs40]

Maybe some annoying indels are causing a bug in the realignment step. Can you send me a sites-only VCF chunk where you are experiencing BCFtools/liftover getting stuck?

Hello!
I will be happy to help.
But I am unsure how... I have no single error message. The liftover just gets stuck here:

Warning: source contig chrM has length 16569 in the VCF and length 16571 in the chain file INFO/AC is handled by AC rule INFO/AF is handled by AF rule INFO/MLEAC is handled by AGR rule INFO/MLEAF is handled by AGR rule FORMAT/PL is handled by AGR rule WARNING: bcftools version mismatch .. bcftools at 1.21, the plugin "liftover" at 1.20

If you give me some pointers, I can try to help finding what is making it get stuck.
With GATK liftover, this is done in a few hours (all chromosomes). I got one single chromosome stuck for one week!

[freeseek]

Keep splitting the VCF into smaller chunks until you find the variant that is causing BCFtools/liftover to get stuck. Then report the variant

[milcs40]

I split the VCFs in SNPs and everything else. And surely, SNPs lifts like a breeze, while the rest just sits there for ages.
I further split the culprit set of variants into 100 chunks. As expected, some chunks take 0 seconds to lift, whereas some others take more than 4 hours.

Would it help if I shared a small VCF with 100 variants that are slowing down the liftover?

[freeseek]

Keep splitting until you find the indel that is causing the issue and report it here

[milcs40]

I found a variant that takes 3.43 hours to complete.
I am trying to find a few others.

How do you want me to report the variants?
Do you want me to give you the single variant VCF?
This is a joint VCF with ~1000 samples.

[freeseek]

I don't need the genotypes of the samples. I only need CHROM, POS, REF, and ALT

[milcs40]

I split non-SNP VCFs in chunks of 100 variants.
For those chunks that were taking too long time, I further split them into 10 variants' VCFs. I then tested the time for each variant.

Here are two variants that were really long to lift.

Variant  : 1:55448789  ID=.  REF=T  ALT=*
Time_sec : 12362 sec

Variant  : 1:55485163  ID=.  REF=C  ALT=*
Time_sec : 14552 sec

The only thing I see in common is a * in ALT.

I can try to look for more, if you think it can help.

[freeseek]

I can replicate this issue with version 2024-09-27 of BCFtools/liftover but not with version 2025-08-19. What version are you using? You can check by running bcftools +liftover -h

[jjfarrell]

@milcs40 I had some similar slowness and odd warnings/errors when I had a missmatch between the liftover plugin version and the bcftools, and htslib versions. Since your run has this warning, WARNING: bcftools version mismatch .. bcftools at 1.21, the plugin "liftover" at 1.20, I would next try using consistent versions of 1.22 versions of bcftools, htslib and score plug-ins and see if that fixes things.

[freeseek]

The version inconsistency is unlikely to be relevant. What version of liftover are you using? What variant is causing the slowdown?