"Życie" (life) is lemmatized into "żyto" (rye)

[DamianBisewski]

I'm working on a project and experimenting with spacy. I have discovered that in many contexts the word "życie" (life) is transformed into "żyto" (rye) instead of "życie" (life). I'm using Python 3.9.9 and Spacy 0.1.0. Sample sentences are "Jakie życie, taka śmierć", "Stała się dla mnie przewodnikiem na całe życie". There were more sentences analysed, "życie" was always mapped to "żyto".

[DamianBisewski]

I forgot to mention that I use the dictionary pl_core_news_sm v3.2.0.

[DamianBisewski]

I tested the code also for pl_core_news_lg v.3.2.0.

[ryszardtuora]

What do you mean by "using dictionary pl_core_news_sm v3.2.0". How do you load the model? In any case, this repo does not concern the official spaCy models, downloadable via their API, but rather the custom pl_spacy_model_morfeusz model. I am not entirely sure how these official models work as of now, as they've strayed from the initial implementation that I was involved in. I've tested the sentences your provide, and "życie" is lemmatized properly in our model (pl_spacy_model_morfeusz), so you might want to try using that, but the problem might appear elsewhere, as disambiguating between different lemmas is done based on morphological properties of the tokens, and so if two interpretations are possible, and they share the same morphological profile, the model will always choose the first one of them.

[DamianBisewski]

I installed the models via python -m spacy download pl_core_news_lg. I load the model in Python via spacy.load(). I will try model pl_spacy_model_morfeusz and I also understand that the disambiguation will not always be perfectly accurate.