Question about the position encoding of 'pe = 0.' code

[lishensuo]

https://github.com/OmicsML/CellPLM/blob/b59ee7688b1bd2a745856a610248b46f15019a22/CellPLM/embedder/omics.py#L88C1-L101C17

In the last forwrd part of CellPLM/CellPLM/embedder/omics.py, I notice that the pe variable is always initialized as 0 and it seems that pe not related the coordinate encoding. Does it mean that x only includes the expression embed and batch embed? Could you explain the reasons? Thanks.

    def forward(self, x_dict, input_gene_list=None):
        x = self.feat_enc(x_dict, input_gene_list)#self.act(self.feat_enc(x_dict, input_gene_list))
        if self.pe_enc is not None:
            pe_input = x_dict[self.pe_enc.pe_key]
            pe = 0.#self.pe_enc(pe_input)
            if self.inject_covariate:
                pe = pe + self.cov_enc(x_dict['batch'])
            if self.cat_pe:
                x = torch.cat([x, pe], 1)
            else:
                x = x + pe
        x = self.extra_linear(x)
        # x = self.norm0(self.dropout(x))
        return x

[wehos]

Sorry for the late response. This is a bug for sure and I'm very sad that I didn't see this issue until today. This comment should definitely be removed in order to facilitate spatial transcriptomics. I'll fix it in an update soon.

Thank you very much for this good catch!

[wehos]

Here is a quick update on the current status for this issue:

(2) All the codes in this github repo were uploaded at a certain point after we finished our manuscript. I have checked on my private github repo and noticed that the problematic codes were modified on Sep 15, 2023 (our manuscript was first submitted to NeurIPS in May 2023). I believe this change was made when I fine-tuned the model for scRNA-seq data. For now, feel free to remove the comment on spatial transcriptomics data.

Changelog from my private repo (the link is for reference purpose but is not public available):

- pe = self.pe_enc(pe_input)
+ pe = 0.#self.pe_enc(pe_input)
https://github.com/wehos/CellBert/commit/9fc5b1a7bf7b20aa9eac95beba6de067f2684619#diff-286a4c753e046ce11d844e262d909e0ae39ee1fa09b63e02d3be2371cbe78efbL78

Originally posted by @wehos in #23