KeyError: 'umap_cell_embeddings'

[kenneditodd]

I am trying to do singlecellVR starting from a seurat object. I have tried with my own data and the pbmc example dataset shown. I can convert to .loom just fine. I get a KeyError: 'umap_cell_embeddings' when i run scvr from my conda environment.

command:
scvr -f seurat3d_10xpbmc.loom -t seurat -a annotations.txt -g genes.txt -o seurat_report

Standard output is shown below. How do i fix this problem?

- Single cell VR preprocessing -
Version 1.1

Converting 'seurat' analysis result ...
reading in loom file ...
OMP: Info #271: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
Traceback (most recent call last):
  File "/Users/m214960/miniconda3/envs/scvr/bin/scvr", line 8, in <module>
    sys.exit(main())
  File "/Users/m214960/miniconda3/envs/scvr/lib/python3.9/site-packages/scvr/command_line.py", line 93, in main
    scvr.output_seurat_cells(adata,ann_list,gene_list=gene_list,reportdir=output)
  File "/Users/m214960/miniconda3/envs/scvr/lib/python3.9/site-packages/scvr/converters.py", line 194, in output_seurat_cells
    assert (adata.obsm['umap_cell_embeddings'].shape[1]>=3),\
  File "/Users/m214960/miniconda3/envs/scvr/lib/python3.9/site-packages/anndata/_core/aligned_mapping.py", line 148, in __getitem__
    return self._data[key]
KeyError: 'umap_cell_embeddings'

[lucapinello]

Thanks for reporting this bug! I am CCing the main developers of the tool, I am sure they can help you with this matter.

…

On Mon, Oct 18, 2021 at 2:01 PM kenneditodd ***@***.***> wrote: I am trying to do singlecellVR starting from a seurat object. I have tried with my own data and the pbmc example dataset shown. I can convert to .loom just fine. I get a KeyError: 'umap_cell_embeddings' when i run scvr from my conda environment. command: scvr -f seurat3d_10xpbmc.loom -t seurat -a annotations.txt -g genes.txt -o seurat_report Standard output is shown below. How do i fix this problem? - Single cell VR preprocessing - Version 1.1 Converting 'seurat' analysis result ... reading in loom file ... OMP: Info #271: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead. Traceback (most recent call last): File "/Users/m214960/miniconda3/envs/scvr/bin/scvr", line 8, in sys.exit(main()) File "/Users/m214960/miniconda3/envs/scvr/lib/python3.9/site-packages/scvr/command_line.py", line 93, in main scvr.output_seurat_cells(adata,ann_list,gene_list=gene_list,reportdir=output) File "/Users/m214960/miniconda3/envs/scvr/lib/python3.9/site-packages/scvr/converters.py", line 194, in output_seurat_cells assert (adata.obsm['umap_cell_embeddings'].shape[1]>=3), File "/Users/m214960/miniconda3/envs/scvr/lib/python3.9/site-packages/anndata/_core/aligned_mapping.py", line 148, in *getitem* return self._data[key] KeyError: 'umap_cell_embeddings' — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#16>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIH72W5BKPQ4HDRXRNFUB3UHROIRANCNFSM5GHHSBMA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[davidfstein]

Hi @kenneditodd, sorry you're having trouble with the converter. Currently, I'm not able to reproduce the error you're encountering, as running scvr -f seurat3d_10xpbmc.loom -t seurat -a annotations.txt -g genes.txt -o seurat_report successfully creates the output needed for singlecellVR for me. Could you please post the output from conda list? Could you also try creating a new conda environment and running as follows:

conda create -n scvr python=3.7
conda activate scvr
pip install scvr scvelo
scvr -f seurat3d_10xpbmc.loom -t seurat -a annotations.txt -g genes.txt -o seurat_report

[kenneditodd]

Hello @davidfstein thanks for the quick reply!

Okay I created a new conda environment exactly like you mentioned. Here is the output from conda list.

# packages in environment at /Users/m214960/miniconda3/envs/scvr:
#
# Name                    Version                   Build  Channel
anndata                   0.7.6                    pypi_0    pypi
ca-certificates           2021.10.8            h033912b_0    conda-forge
cached-property           1.5.2                    pypi_0    pypi
click                     8.0.3                    pypi_0    pypi
cycler                    0.10.0                   pypi_0    pypi
h5py                      3.4.0                    pypi_0    pypi
importlib-metadata        4.8.1                    pypi_0    pypi
joblib                    1.1.0                    pypi_0    pypi
kiwisolver                1.3.2                    pypi_0    pypi
libcxx                    12.0.1               habf9029_0    conda-forge
libffi                    3.4.2                he49afe7_4    conda-forge
libzlib                   1.2.11            h9173be1_1013    conda-forge
llvmlite                  0.37.0                   pypi_0    pypi
loompy                    3.0.6                    pypi_0    pypi
matplotlib                3.4.3                    pypi_0    pypi
natsort                   7.1.1                    pypi_0    pypi
ncurses                   6.2                  h2e338ed_4    conda-forge
networkx                  2.6.3                    pypi_0    pypi
numba                     0.54.1                   pypi_0    pypi
numexpr                   2.7.3                    pypi_0    pypi
numpy                     1.20.3                   pypi_0    pypi
numpy-groupies            0.9.14                   pypi_0    pypi
openssl                   3.0.0                h0d85af4_1    conda-forge
packaging                 21.0                     pypi_0    pypi
pandas                    1.3.4                    pypi_0    pypi
patsy                     0.5.2                    pypi_0    pypi
pillow                    8.4.0                    pypi_0    pypi
pip                       21.3               pyhd8ed1ab_0    conda-forge
pynndescent               0.5.5                    pypi_0    pypi
pyparsing                 2.4.7                    pypi_0    pypi
python                    3.7.10          hf3644f1_104_cpython    conda-forge
python-dateutil           2.8.2                    pypi_0    pypi
python_abi                3.7                     2_cp37m    conda-forge
pytz                      2021.3                   pypi_0    pypi
readline                  8.1                  h05e3726_0    conda-forge
scanpy                    1.8.1                    pypi_0    pypi
scikit-learn              1.0                      pypi_0    pypi
scipy                     1.7.1                    pypi_0    pypi
scvelo                    0.2.4                    pypi_0    pypi
scvr                      1.1                      pypi_0    pypi
seaborn                   0.11.2                   pypi_0    pypi
setuptools                58.2.0           py37hf985489_0    conda-forge
sinfo                     0.3.4                    pypi_0    pypi
six                       1.16.0                   pypi_0    pypi
sqlite                    3.36.0               h23a322b_2    conda-forge
statsmodels               0.13.0                   pypi_0    pypi
stdlib-list               0.8.0                    pypi_0    pypi
tables                    3.6.1                    pypi_0    pypi
threadpoolctl             3.0.0                    pypi_0    pypi
tk                        8.6.11               h5dbffcc_1    conda-forge
tqdm                      4.62.3                   pypi_0    pypi
typing-extensions         3.10.0.2                 pypi_0    pypi
umap-learn                0.5.1                    pypi_0    pypi
wheel                     0.37.0             pyhd8ed1ab_1    conda-forge
xlrd                      1.2.0                    pypi_0    pypi
xz                        5.2.5                haf1e3a3_1    conda-forge
zipp                      3.6.0                    pypi_0    pypi
zlib                      1.2.11            h9173be1_1013    conda-forge

I have ran this two ways. First, I used all the provided input from the dropbox and I can now replicate that portion.

Second, I converted the pbmc seurat object to a loom file myself using the methodology show here.
https://nbviewer.org/github/pinellolab/singlecellvr/blob/master/examples/seurat_10xpbmc.ipynb?flush_cache=true
This is my output when I convert the object.

Saving data from RNA as /matrix
  |=======================================================| 100%
Adding slot counts for assay RNA
Adding layer counts
  |=======================================================| 100%
Adding slot scale.data for assay RNA
Adding layer scale.data
Adding col attribute CellID
Adding col attribute orig.ident
Adding col attribute nCount_RNA
Adding col attribute nFeature_RNA
Adding col attribute percent.mt
Adding col attribute RNA_snn_res.0.5
Adding col attribute seurat_clusters
Adding row attribute Gene
Class: loom
Filename: /Users/m214960/Documents/VR/seurat3d_10xpbmc.loom
Access type: H5F_ACC_RDWR
Listing:
       name    obj_type dataset.dims dataset.type_class
      attrs   H5I_GROUP         <NA>               <NA>
  col_attrs   H5I_GROUP         <NA>               <NA>
 col_graphs   H5I_GROUP         <NA>               <NA>
     layers   H5I_GROUP         <NA>               <NA>
     matrix H5I_DATASET 2638 x 13714          H5T_FLOAT
  row_attrs   H5I_GROUP         <NA>               <NA>
 row_graphs   H5I_GROUP         <NA>               <NA>

Here is my R session info

R version 4.0.3 (2020-10-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] Matrix_1.3-4          loomR_0.2.0           itertools_0.1-3       iterators_1.0.13      hdf5r_1.3.4           R6_2.5.1             
 [7] SeuratDisk_0.0.0.9019 reticulate_1.22       tibble_3.1.5          patchwork_1.1.1       stringr_1.4.0         SeuratObject_4.0.2   
[13] Seurat_4.0.4          purrr_0.3.4           ggplot2_3.3.5         dplyr_1.0.7           cowplot_1.1.1         BiocParallel_1.24.1  
[19] edgeR_3.32.1          limma_3.46.0         

loaded via a namespace (and not attached):
  [1] sn_2.0.0              plyr_1.8.6            igraph_1.2.6          lazyeval_0.2.2        splines_4.0.3         listenv_0.8.0        
  [7] scattermore_0.7       TH.data_1.1-0         digest_0.6.28         htmltools_0.5.2       fansi_0.5.0           magrittr_2.0.1       
 [13] tensor_1.5            cluster_2.1.2         ROCR_1.0-11           remotes_2.4.1         globals_0.14.0        matrixStats_0.61.0   
 [19] sandwich_3.0-1        spatstat.sparse_2.0-0 prettyunits_1.1.1     colorspace_2.0-2      ggrepel_0.9.1         rbibutils_2.2.4      
 [25] xfun_0.26             callr_3.7.0           crayon_1.4.1          jsonlite_1.7.2        spatstat.data_2.1-0   survival_3.2-13      
 [31] zoo_1.8-9             glue_1.4.2            polyclip_1.10-0       gtable_0.3.0          leiden_0.3.9          pkgbuild_1.2.0       
 [37] future.apply_1.8.1    BiocGenerics_0.36.1   abind_1.4-5           scales_1.1.1          mvtnorm_1.1-3         DBI_1.1.1            
 [43] miniUI_0.1.1.1        Rcpp_1.0.7            metap_1.5             plotrix_3.8-2         viridisLite_0.4.0     xtable_1.8-4         
 [49] tmvnsim_1.0-2         spatstat.core_2.3-0   bit_4.0.4             stats4_4.0.3          htmlwidgets_1.5.4     httr_1.4.2           
 [55] RColorBrewer_1.1-2    TFisher_0.2.0         ellipsis_0.3.2        ica_1.0-2             pkgconfig_2.0.3       farver_2.1.0         
 [61] uwot_0.1.10           deldir_1.0-5          locfit_1.5-9.4        utf8_1.2.2            tidyselect_1.1.1      labeling_0.4.2       
 [67] rlang_0.4.11          reshape2_1.4.4        later_1.3.0           munsell_0.5.0         tools_4.0.3           cli_3.0.1            
 [73] generics_0.1.0        mathjaxr_1.4-0        ggridges_0.5.3        evaluate_0.14         fastmap_1.1.0         yaml_2.2.1           
 [79] goftest_1.2-3         bit64_4.0.5           processx_3.5.2        knitr_1.36            fitdistrplus_1.1-6    RANN_2.6.1           
 [85] pbapply_1.5-0         future_1.22.1         nlme_3.1-153          mime_0.12             compiler_4.0.3        rstudioapi_0.13      
 [91] curl_4.3.2            plotly_4.10.0         png_0.1-7             spatstat.utils_2.2-0  stringi_1.7.5         ps_1.6.0             
 [97] RSpectra_0.16-0       lattice_0.20-45       multtest_2.46.0       vctrs_0.3.8           mutoss_0.1-12         pillar_1.6.3         
[103] lifecycle_1.0.1       BiocManager_1.30.16   spatstat.geom_2.3-0   Rdpack_2.1.2          lmtest_0.9-38         RcppAnnoy_0.0.19     
[109] data.table_1.14.2     irlba_2.3.3           httpuv_1.6.3          promises_1.2.0.1      KernSmooth_2.23-20    gridExtra_2.3        
[115] parallelly_1.28.1     codetools_0.2-18      MASS_7.3-54           assertthat_0.2.1      rprojroot_2.0.2       withr_2.4.2          
[121] sctransform_0.3.2     mnormt_2.0.2          multcomp_1.4-17       mgcv_1.8-38           grid_4.0.3            rpart_4.1-15         
[127] tidyr_1.1.4           rmarkdown_2.11        Rtsne_0.15            numDeriv_2016.8-1.1   Biobase_2.50.0        shiny_1.7.1   

Again when I try to run the scvr command I get this output.

- Single cell VR preprocessing -
Version 1.1

Converting 'seurat' analysis result ...
reading in loom file ...
Traceback (most recent call last):
  File "/Users/m214960/miniconda3/envs/scvr/bin/scvr", line 8, in <module>
    sys.exit(main())
  File "/Users/m214960/miniconda3/envs/scvr/lib/python3.7/site-packages/scvr/command_line.py", line 93, in main
    scvr.output_seurat_cells(adata,ann_list,gene_list=gene_list,reportdir=output)
  File "/Users/m214960/miniconda3/envs/scvr/lib/python3.7/site-packages/scvr/converters.py", line 194, in output_seurat_cells
    assert (adata.obsm['umap_cell_embeddings'].shape[1]>=3),\
  File "/Users/m214960/miniconda3/envs/scvr/lib/python3.7/site-packages/anndata/_core/aligned_mapping.py", line 148, in __getitem__
    return self._data[key]
KeyError: 'umap_cell_embeddings'

[davidfstein]

Thanks for that info @kenneditodd ! I believe this issue is being caused by changes in the as.loom function between Seurat 3.x and 4.x. We will update the scvr package so that it supports loom files produced by Seurat 4.x. In the meantime, if you downgrade to Seurat 3.x, you should be able to successfully create the singlecellVR zip file.

[qinqian]

Hi @kenneditodd,

As mentioned by @davidfstein, this issue is caused by the incompatibility problem when upgrading Seurat 3.x to Seurat 4.0.x. We have resolved the problem now, please take a look at the new Jupiter notebook (https://github.com/qinqian/singlecellvr/blob/master/examples/seurat4_pbmc3k.ipynb), the idea is to use SeuratDisk to export the data into h5ad file. The corresponding scvr API has been updated as well by scvr -f demo.h5ad -t seurat -a annotations.txt -g genes.txt -o seurat_report, please replace previous annotations.txt with https://github.com/qinqian/singlecellvr/blob/master/examples/annotations.txt, and replace previous loom file with the h5ad file generated by the notebook.

Best,
Qian

[kenneditodd]

@qinqian @davidfstein I can't get it to work with Seurat 4.x. This was my R chunk.

SaveH5Seurat(pbmc, filename = "pbmc3k.h5Seurat")
Convert("pbmc3k.h5Seurat", dest = "h5ad")

R output

Creating h5Seurat file for version 3.1.5.9900
Adding counts for RNA
Adding data for RNA
Adding scale.data for RNA
Adding variable features for RNA
Adding feature-level metadata for RNA
Adding cell embeddings for pca
Adding loadings for pca
No projected loadings for pca
Adding standard deviations for pca
No JackStraw data for pca
Adding cell embeddings for umap
No loadings for umap
No projected loadings for umap
No standard deviations for umap
No JackStraw data for umap
Validating h5Seurat file
Adding scale.data from RNA as X
Transfering meta.features to var
Adding data from RNA as raw
Transfering meta.features to raw/var
Transfering meta.data to obs
Adding dimensional reduction information for pca
Adding feature loadings for pca
Adding dimensional reduction information for umap

I created my conda environment like this

conda create -n scvr python=3.7
conda activate scvr
pip install scvr scvelo

Ran this command

scvr -f pbmc3k.h5ad -t seurat -a annotations.txt -g genes.txt -o seurat_report

This was my error

- Single cell VR preprocessing -
Version 1.1

Converting 'seurat' analysis result ...
Traceback (most recent call last):
  File "/Users/m214960/miniconda3/envs/scvr/bin/scvr", line 8, in <module>
    sys.exit(main())
  File "/Users/m214960/miniconda3/envs/scvr/lib/python3.7/site-packages/scvr/command_line.py", line 90, in main
    assert (filename.lower().endswith(('.loom'))), "For Seurat only .loom file is supported."
AssertionError: For Seurat only .loom file is supported.

Are you able to send a yaml file so i can make sure I have the same conda environment?

[qinqian]

We didn't update the scvr on the pypi, could you please cd scvr && python setup.py install?

Best,
Qian

[kenneditodd]

@qinqian
Okay I cloned the repository, did cd scvr and then python setup.py install . I am still getting the following error.

scvr -f pbmc3k.h5ad -t seurat -a annotations.txt -g genes.txt -o seurat_report
- Single cell VR preprocessing -
Version 1.1

Converting 'seurat' analysis result ...
Traceback (most recent call last):
  File "/Users/m214960/miniconda3/envs/scvr/bin/scvr", line 33, in <module>
    sys.exit(load_entry_point('scvr==1.1', 'console_scripts', 'scvr')())
  File "/Users/m214960/miniconda3/envs/scvr/lib/python3.7/site-packages/scvr/command_line.py", line 90, in main
    assert (filename.lower().endswith(('.loom'))), "For Seurat only .loom file is supported."
AssertionError: For Seurat only .loom file is supported.

Let me know if I did anything wrong or misunderstood.

[qinqian]

Please delete scvr in your directory first and then setup the new script. As the assert in the latest code is 93th line (https://github.com/pinellolab/singlecellvr/blob/master/scvr/scvr/command_line.py#L93).