cell typing using negative binomial and f-test pvalue for error detection

[lidanwu]

• fixed issue Investigate fastLM to speed up module #15: use fastLMPure() for linear regression fitting, use corresponding f-test and p-value calculation for error detection instead of lrtest() for alternative and null models.
• fixed issue likelihood calcualtion tLLRv2 #16: use negative binomial model to calculate the log-likelihood of each cell/transcript group lldist() belong to reference cell clusters in all cell typing steps; but still use original transcript log-likelihood ratio for segmentation error detection.

Tested on TAP data, and found the new pipeline has slower processed speed despite fewer cells going through the pipeline.

• fewer number of cells being flagged for segmentation error (~90% of earlier resegmentation results) and also fewer low-score transcript groups identified, fewer merging or trimming events;
• processed 122 FOVs of 12.87 GB file size within 18.56hr, slower than earlier 17.35hr for same dataset;
  It seems lldist() calculation is rate-limiting and requires optimization in speed.

[davidpross]

@lidanwu Since you are finding it to be slow, why don't you bring over the optimization to lldist from InSituType? Or make InSituType a dependency and use the same function that is implemented in Rcpp.

[lidanwu]

Good suggestion. I’m not familiar with Rcpp implementation. Could you please advise on how to put that in the FastReseg? I didn't want to use import from InSituType since the handling for cells of zero counts are different and InSituType package is not public available.

…

On Oct 24, 2022, at 9:58 AM, Dave Ross ***@***.***> wrote:  @lidanwu Since you are finding it to be slow, why don't you bring over the optimization to lldist from InSituType? Or make InSituType a dependency and use the same function that is implemented in Rcpp. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[davidpross]

The InSituType package is now public. Will the lls function that is available there work? It has the following signature.

lls {InSituType}	R Documentation

Value

rowSums for matrix of densities

---

[Package InSituType version 0.99.2 Index]

lls {InSituType} R Documentation sum from negative binomial density function Description Probability density function of the negative binomial distribution (written in C++)

Usage
lls(mat, s, x, bg, size_dnb)
Arguments
mat
dgCMatrix expression counts

s
numeric scaling factor

x
numeric expression for reference profile

bg
numeric background level

size_dnb
int Dispersion parameter

Value
rowSums for matrix of densities

[Package InSituType version 0.99.2 Index]

[lidanwu]

I didn't realize InSituType is now public. Yes, importing lls function would work.

[lidanwu]

Encountered a memory issue when using parallel::mclapply() in logliks calculation . The logliks returned has value = "Error : cannot allocate vector of size 1.9 Gb\n" and fewer elements than the total number of reference cell types. Changing the number of cores would result in different number of elements returned and different cell types with error suggesting it's not a data issue. Running the same dataset with same script on AWS instead on hades resolved the issue.
It seems large RAM is required for lldist based cell typing in the pipeline.