Question about _digitize randomization in binning preprocessing

[zhangtianmingxp]

---

Hi, thank you for the great work and for making the code available!

I have a question about the binning preprocessing step in your code.

I noticed that in the binning step, the code does not directly use np.digitize, but instead calls a custom _digitize function, which introduces randomization when the bin edges contain repeated values:

rands = np.random.rand(len(x))  # uniform random numbers
digits = rands * (right_difits - left_digits) + left_digits

This means that if a value falls into a range of bins that all have the same edge value (due to duplicated quantiles), it will be assigned to a bin index uniformly at random between left_digits and right_difits.

---

Questions

1. What is the motivation for re-implementing _digitize with randomization, instead of using the standard deterministic np.digitize?
2. Have you tried an ablation study where this randomization is disabled (e.g., always choose the smallest possible bin index)?
   I am concerned that for very sparse data (such as single-cell gene expression), this randomness could introduce large variance in the preprocessed features.
3. Even if this randomization helps during training, how is inference handled?
   • If the same preprocessing is applied during inference, the random assignment could make predictions unstable.
   • If randomization is disabled at inference, then the input distribution would differ from training.

---

Example

For instance, with very sparse and low-cardinality rows:

Raw:       [2, 2, 1, 1, 1, 1]
Norm:      [2500, 2500, 1250, 1250, 1250, 1250]
Log1p:     [7.82, 7.82, 7.13, 7.13, 7.13, 7.13]
Binning:   [43, 45, 26, 26, 29, 5]   # 43 and 45 come from the same original value (7.82)

Running preprocessing again could produce different bin assignments for the same value, e.g.:

[50, 50, 1, 1, 1, 1]  # in extreme cases

In sparse settings, even a single random misassignment can disproportionately affect the resulting feature vector and significantly impact downstream results.

---

Summary

• Could you clarify the intended benefit of introducing randomness in _digitize?
• Could you share whether deterministic binning was tested?
• How should this be handled for inference to ensure consistent results?

Thanks again for your time and for considering my question!

[KalinNonchev]

I am also interested in ablation on this

[LeeWang21]

I am also concerned the predictions involved cell embedding due to the randomization

[zhangtianmingxp]

Following up on this issue, I ran some experiments to further evaluate
the effect of random binning during embedding extraction via get_tokenizer().

In _digitize, random binning occurs when side="both" and bins contains duplicate values.
To make the embedding extraction deterministic, I modified the function as follows:

# Original random binning:
# rands = np.random.rand(len(x))
# digits = rands * (right_digits - left_digits) + left_digits
# digits = np.ceil(digits).astype(np.int64)

# Deterministic midpoint binning
digits = (left_digits + right_digits) // 2
return digits.astype(np.int64)

This ensures that tokenized sequences are deterministic for the same input,
removing stochasticity during embedding extraction / inference.

Experimental evaluation

I tested both strategies on the 10X PBMC RNA dataset (h5ad),
using scGPT embeddings for downstream clustering.

Two settings were tested:

• No fixed random seed
• Fixed random seed (seed = 42)

Results (no seed)

Embedding	ARI	NMI	Silhouette	kNN purity
Random binning	0.3385	0.6483	0.1622	0.7580
Deterministic midpoint binning	0.3730	0.6611	0.1816	0.7805

Results (seed = 42)

Embedding	ARI	NMI	Silhouette	kNN purity
Random binning	0.3969	0.6451	0.1610	0.7577
Deterministic midpoint binning	0.4041	0.6808	0.1808	0.7767

UMAP visualizations also show more compact and better-separated clusters
with deterministic binning .

This suggests deterministic binning improves embedding stability and reproducibility during inference.

[KalinNonchev]

@zhangtianmingxp thanks for the analysis!

This suggests deterministic binning improves embedding stability and reproducibility during inference.

Well, if you select the right seed :-)

[zhangtianmingxp]

Well, if you select the right seed :-)

Looks like the right seed beats deterministic binning. 🙂

[LeeWang21]

Well, if you select the right seed :-)

Looks like the right seed beats deterministic binning. 🙂

Thank you for running the test. I am a bit confused about the use of the random seed in your experiment. After your modification, it appears that the function itself does not introduce any randomness. In this case, where should the random seed be specified?

[zhangtianmingxp]

Thank you for running the test. I am a bit confused about the use of the random seed in your experiment. After your modification, it appears that the function itself does not introduce any randomness. In this case, where should the random seed be specified?

Thanks for the question.

You’re absolutely right that after the modification, the _digitize function itself no longer introduces any randomness.
The fixed random seed is included only as an additional comparison, since it affects other stochastic parts of the pipeline.

[YvyuanWang]

For example, consider a cell with ~2000 expressed genes (non-zero), where ~1000 of them have a raw count of 1. When B = 50, a large fraction of genes share exactly the same expression value. After log1p transformation, these identical raw counts may still exhibit slight floating-point variations depending on preprocessing or batch-specific effects. As a result, these genes could be distributed across multiple bins rather than being assigned consistently to the same bin. In extreme cases, the same raw count (e.g., 1) might be mapped to a relatively low (1)bin in one batch and a much higher bin(25) in another.