[en] Include lemma on inflected forms

[cacfd3a]

Consider the Russian word ёж.

Consider the generated information from the declension table for ежа́ (singular accusative):

    {
      "form": "ежа́",
      "roman": "ježá",
      "source": "declension",
      "tags": [
        "accusative",
        "animate",
        "singular"
      ]
    }

ru-noun-table helpfully adds a stress marker on the а, but the Wiktionary lemma does not have this marker: ежа. On the HTML on Wiktionary, you can see that 'ежа́' is displayed, but it links to (and has as title) 'ежа'.

<a href="/wiki/ежа#Russian" title="ежа">ежа́</a>

Since Wiktionary itself provides the lemma, I request that you do not throw away the data on the lemma, but put it in a field of its own. For example, a dedicated lemma field to these generated forms:

    {
      "form": "ежа́",
      "lemma": "ежа", <- add this field
      "roman": "ježá",
      "source": "declension",
      "tags": [
        "accusative",
        "animate",
        "singular"
      ]
    }

This is similar to the word field and canonical tag on headwords.

[kristian-clausal]

I'm not sure what you're asking for here.

[cacfd3a]

Sorry, I was a bit too expansive. It is comparable to what you call the "canonical" form of a word: the section title (the canonical form) could be different from the page title (the lemma).

The Wiktionary declension template adds a little something to Russian words: a marker to let the reader know where stress is, since stress is non-obvious in Russian words. So word 'ежа' is displayed as 'ежа́'. But the lemma of this word is not 'ежа́'. It is 'ежа', as you can see from the URL to the declined word. The real lemma is visible in Wiktionary, but not in Kaikki. Kaikki only shows the 'canonical' form.

I believe both strings should be included: the canonical form (ежа́) and the lemma (ежа).

In Russian, it is perhaps straightforward to derive the lemma from the canonical form (just remove the accent), but I can imagine that other languages have less straightforward transformations from canonical form to lemma.

[kristian-clausal]

We are not going to add a lemma field for this, because it's too specific and relies on behavior only apparently present in Russian. What we could do is add a links field to the form data when the links differ from the contents of the form; this should catch everything 'weird', including when a form is a multiword construction with links on each separate word into different articles. I'll see what I can do.

[cacfd3a]

only apparently present in Russian

Is it? There are salient examples in other languages:

The Afar word narig has 2 canonical forms: nárig and naríg, with separate declension charts.

Both predicate forms náriigi and naríigi link to the lemma nariigi (which, ok, is not defined yet on Wiktionary).

on Kaikki, we just get:

      "form": "náriigi",
      "source": "declension",
      "tags": [
        "predicative"
      ]
    }, 
...
   {
      "form": "naríigi",
      "source": "declension",
      "tags": [
        "predicative"
      ]
    },

---

add a links field

That will probably be the cleaner solution, thanks!

[kristian-clausal]

I have been slowly working on this, and it is extremely difficult to do in the English edition extractor without rewriting a lot of it (and there would be A LOT to rewrite), at least in the case of word heads (the first line starting with the lemma). That's because we parse the head line as a string, which has lost the link data, and we do it that was for many, many reasons (other extractors do it differently and probably better, but the English extractor has to deal with a lot of bullshit). There's some code that has to do with link texts in the original text, but that is a kludge that generates text to be used in a regex that detects words when the article title contains punctuation and special signs that would otherwise mess up the parsing.

Doing this for individual cells in tables should be much simpler.

I'll continue working on it.