Hi!
I am currently using a large dataset (N ~80M SNPs). I have two questions.
Q1)
While the coord step ran ,with errors such as opening chrom 23,24,25 and 26, still gave the out put file. Is this issue ignorable?
88.46%"Unable to open object (object 'chrom_23' doesn't exist)"
Did not find chromosome 23 in SS dataset.
Continuing.
92.31%"Unable to open object (object 'chrom_24' doesn't exist)"
Did not find chromosome 24 in SS dataset.
Continuing.
96.15%"Unable to open object (object 'chrom_26' doesn't exist)"
Did not find chromosome 26 in SS dataset.
Continuing.
100.00%
/python/3.6.8/lib/python3.6/site-packages/numpy/lib/scimath.py:274: RuntimeWarning: divide by zero encountered in log
return nx.log(x)
Q2)
But in the gibbs phase, the first two f values (01 and 0.3) ran successfully, but threw an assertion error as described below.
=============================== �[96mLDpred v. 1.0.10�[0m ===============================
Applying LDpred with LD radius: 200
6188957 SNP effects were found
Calculating LDpred-inf weights
Starting LDpred gibbs with f=1.0000
4.55%
9.09%
13.64%
18.18%
22.73%
27.27%
31.82%
36.36%
40.91%
45.45%
50.00%
54.55%
59.09%
63.64%
68.18%
72.73%
77.27%
81.82%
86.36%
90.91%
95.45%
100.00%
100.00%
Starting LDpred gibbs with f=0.3000
4.55%
9.09%
13.64%
18.18%
22.73%
27.27%
31.82%
36.36%
40.91%
45.45%
50.00%
54.55%
59.09%
63.64%
68.18%
72.73%
77.27%
81.82%
86.36%
90.91%
95.45%
100.00%
100.00%
Starting LDpred gibbs with f=0.1000
4.55%
9.09%
13.64%
18.18%
22.73%
27.27%
31.82%
36.36%
40.91%
45.45%
50.00%
54.55%
59.09%
63.64%
68.18%
72.73%
77.27%
81.82%/.local/lib/python3.6/site-packages/ldpred/ld.py:365: ComplexWarning: Casting complex values to real discards the imaginary part
summary_dict[1.11]={'name':'Genome-wide (LDscore) estimated heritability:','value':'%0.4f'%gw_h2_ld_score_est}
.local/lib/python3.6/site-packages/ldpred/ld.py:366: ComplexWarning: Casting complex values to real discards the imaginary part
summary_dict[1.12]={'name':'Chi-square lambda (inflation statistic).','value':'%0.4f'%chi_square_lambda}
local/lib/python3.6/site-packages/ldpred/LDpred_inf.py:48: ComplexWarning: Casting complex values to real discards the imaginary part
updated_betas[start_i: stop_i] = sp.dot(A_inv * n , beta_hats[start_i: stop_i]) # Adjust the beta_hats
local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:170: ComplexWarning: Casting complex values to real discards the imaginary part
curr_post_means[snp_i] = cd['hdmp_hdmpn'] * postp * res_beta_hat_i
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:180: ComplexWarning: Casting complex values to real discards the imaginary part
curr_betas[snp_i] = proposed_beta # UPDATE BETA
local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:55: ComplexWarning: Casting complex values to real discards the imaginary part
rv_scalars[:]=sampl_var_shrink_factor* sp.sqrt((hdmp_hdmpn) * (1.0 / ldpred_n))
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:390: ComplexWarning: Casting complex values to real discards the imaginary part
f.write('%s %d %s %s %s %0.4e %0.4e\n' % (chrom, pos, sid, nt1, nt2, raw_beta, ldpred_beta))
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:164: RuntimeWarning: overflow encountered in cdouble_scalars
postp = numerator / (numerator + d_const_b2_exp)
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:164: RuntimeWarning: invalid value encountered in cdouble_scalars
postp = numerator / (numerator + d_const_b2_exp)
Traceback (most recent call last):
File "/python/3.6.8/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/python/3.6.8/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/.local/lib/python3.6/site-packages/ldpred/main.py", line 8, in
run.main()
File "/.local/lib/python3.6/site-packages/ldpred/run.py", line 337, in main
main_with_args(sys.argv[1:])
File "/.local/lib/python3.6/site-packages/ldpred/run.py", line 323, in main_with_args
LDpred_gibbs.main(p_dict)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 422, in main
sampl_var_shrink_factor=1, verbose=p_dict['debug'], summary_dict=summary_dict)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 328, in ldpred_genomewide
snp_lrld=snp_lrld, print_progress=False)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 165, in ldpred_gibbs
assert sp.isreal(postp), 'The posterior mean is not a real number? Possibly due to problems with summary stats, LD estimates, or parameter settings.'
AssertionError: The posterior mean is not a real number? Possibly due to problems with summary stats, LD estimates, or parameter settings.
Is this an error rising due to memory issue or LD radius?
Thanks in advance!!
Hi!
I am currently using a large dataset (N ~80M SNPs). I have two questions.
Q1)
While the coord step ran ,with errors such as opening chrom 23,24,25 and 26, still gave the out put file. Is this issue ignorable?
88.46%"Unable to open object (object 'chrom_23' doesn't exist)"
Did not find chromosome 23 in SS dataset.
Continuing.
92.31%"Unable to open object (object 'chrom_24' doesn't exist)"
Did not find chromosome 24 in SS dataset.
Continuing.
96.15%"Unable to open object (object 'chrom_26' doesn't exist)"
Did not find chromosome 26 in SS dataset.
Continuing.
100.00%
/python/3.6.8/lib/python3.6/site-packages/numpy/lib/scimath.py:274: RuntimeWarning: divide by zero encountered in log
return nx.log(x)
Q2)
But in the gibbs phase, the first two f values (01 and 0.3) ran successfully, but threw an assertion error as described below.
=============================== �[96mLDpred v. 1.0.10�[0m ===============================
Applying LDpred with LD radius: 200
6188957 SNP effects were found
Calculating LDpred-inf weights
Starting LDpred gibbs with f=1.0000
4.55%
9.09%
13.64%
18.18%
22.73%
27.27%
31.82%
36.36%
40.91%
45.45%
50.00%
54.55%
59.09%
63.64%
68.18%
72.73%
77.27%
81.82%
86.36%
90.91%
95.45%
100.00%
100.00%
Starting LDpred gibbs with f=0.3000
4.55%
9.09%
13.64%
18.18%
22.73%
27.27%
31.82%
36.36%
40.91%
45.45%
50.00%
54.55%
59.09%
63.64%
68.18%
72.73%
77.27%
81.82%
86.36%
90.91%
95.45%
100.00%
100.00%
Starting LDpred gibbs with f=0.1000
4.55%
9.09%
13.64%
18.18%
22.73%
27.27%
31.82%
36.36%
40.91%
45.45%
50.00%
54.55%
59.09%
63.64%
68.18%
72.73%
77.27%
81.82%/.local/lib/python3.6/site-packages/ldpred/ld.py:365: ComplexWarning: Casting complex values to real discards the imaginary part
summary_dict[1.11]={'name':'Genome-wide (LDscore) estimated heritability:','value':'%0.4f'%gw_h2_ld_score_est}
.local/lib/python3.6/site-packages/ldpred/ld.py:366: ComplexWarning: Casting complex values to real discards the imaginary part
summary_dict[1.12]={'name':'Chi-square lambda (inflation statistic).','value':'%0.4f'%chi_square_lambda}
local/lib/python3.6/site-packages/ldpred/LDpred_inf.py:48: ComplexWarning: Casting complex values to real discards the imaginary part
updated_betas[start_i: stop_i] = sp.dot(A_inv * n , beta_hats[start_i: stop_i]) # Adjust the beta_hats
local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:170: ComplexWarning: Casting complex values to real discards the imaginary part
curr_post_means[snp_i] = cd['hdmp_hdmpn'] * postp * res_beta_hat_i
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:180: ComplexWarning: Casting complex values to real discards the imaginary part
curr_betas[snp_i] = proposed_beta # UPDATE BETA
local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:55: ComplexWarning: Casting complex values to real discards the imaginary part
rv_scalars[:]=sampl_var_shrink_factor* sp.sqrt((hdmp_hdmpn) * (1.0 / ldpred_n))
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:390: ComplexWarning: Casting complex values to real discards the imaginary part
f.write('%s %d %s %s %s %0.4e %0.4e\n' % (chrom, pos, sid, nt1, nt2, raw_beta, ldpred_beta))
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:164: RuntimeWarning: overflow encountered in cdouble_scalars
postp = numerator / (numerator + d_const_b2_exp)
.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py:164: RuntimeWarning: invalid value encountered in cdouble_scalars
postp = numerator / (numerator + d_const_b2_exp)
Traceback (most recent call last):
File "/python/3.6.8/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/python/3.6.8/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/.local/lib/python3.6/site-packages/ldpred/main.py", line 8, in
run.main()
File "/.local/lib/python3.6/site-packages/ldpred/run.py", line 337, in main
main_with_args(sys.argv[1:])
File "/.local/lib/python3.6/site-packages/ldpred/run.py", line 323, in main_with_args
LDpred_gibbs.main(p_dict)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 422, in main
sampl_var_shrink_factor=1, verbose=p_dict['debug'], summary_dict=summary_dict)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 328, in ldpred_genomewide
snp_lrld=snp_lrld, print_progress=False)
File "/.local/lib/python3.6/site-packages/ldpred/LDpred_gibbs.py", line 165, in ldpred_gibbs
assert sp.isreal(postp), 'The posterior mean is not a real number? Possibly due to problems with summary stats, LD estimates, or parameter settings.'
AssertionError: The posterior mean is not a real number? Possibly due to problems with summary stats, LD estimates, or parameter settings.
Is this an error rising due to memory issue or LD radius?
Thanks in advance!!