[Feature]: implement public tokenize method to SparseEmbedding

[dancixx]

What feature would you like to request?

Expose the tokenize method as a public method in SparseEmbedding models.

Is there any additional information you would like to provide?

No response

[joein]

#564

[Dylancouzon]

Sorry for the delay here.

#564 was closed in favour of #583, which shipped token_count() instead, our call was to not expose tokenize itself on the public facade. token_count() is public on every text and sparse embedding class if that covers what you need.

Closing this as not planned.