Deprecate or rename <foreign>

[raffazizzi]

The use of the term "foreign" can be problematic in multilingual contexts where no language (or the people that speak it) is arguably "foreign" to the one mainly used in a given doument. This has been raised as a concern by a few projects including:

• https://scholarlyediting.org/issues/41/kinship-and-longing/ (reported by @raffazizzi)
• https://ogham.celt.dias.ie/ (reported by @trishaoconnor)
• https://lyoninmourning.dhil.lib.sfu.ca/ (reported by @joeytakeda)
• https://adaptiveteinetwork.arts.ubc.ca/ (reported by @joeytakeda)

We think that @xml:lang is sufficient to indicate that a word or phrase belongs to some language other than that of the surrounding text, so it could be simply be used on <seg> or <w> to perform the same encoding action, or if a distinct element is necessary, we may want to reconsider its name.

[lb42]

There is also the existing <distinct> element which some may prefer

[raffazizzi]

distinct was brought up in conversation but we know at least of one projects was uncomfortable with its current gloss (we could adjust it), particularly the examples provided in the second clause. Quoted here for reference: "identifies any word or phrase which is regarded as linguistically distinct, for example as archaic, technical, dialectal, non-preferred, etc., or as forming part of a sublanguage."

[joeytakeda]

My opinion is that <foreign> and <distinct> are elements with specific semantics, which are useful only if you want those semantics -- i.e. to say that your interpretation of this text is that this span of stuff is foreign or distinct, and not just in a different language. The Guidelines should you want to say that this is in another language, then just use <seg>.

[trishaoconnor]

Thank you for opening this ticket @raffazizzi. Recently several linguistic researchers highlighted that the term "foreign" was problematic within the context of encoding national languages such as Irish and Welsh. The explanation that the language is foreign to the language of the encoded file was understood but not satisfactory.

Thank you @lb42 for recommending the use of <distinct>. @tuurma also recommended that this might be a feasible alternative encoding approach but I found the use of the term "sublanguage" in the gloss was also problematic within this context. I would welcome updating the definition of <distinct> to "...or as forming part of another language or sublanguage" but I agree with @joeytakeda that the connotations associated with the term "foreign" seem to be a recurring issue and that a more suitable term for designating "a word or phrase as belonging to some language other than that of the surrounding text" would be most welcome.

[lb42]

The Guidelines do actually say that foreign and distinct are provided for the case where you want to mark a codeshift/change of language but there is no specific element to attach an @xml:lang to. They don't say or imply anything else about the semantics of foreign-ness, which is what I assume people are taking exception to.

[trishaoconnor]

Thank you @lb42. Speaking from the context of only one project, ordinarily we absolutely would have attached @xml:lang to an element. In this particular context we were using the <placeName> element to encode the place name in both English as well as the original Irish or Welsh. Within the <placeName> element, we were also using the <ref> element to point to the Logainm or Historic Placenames of Wales databases. However @xml:lang was not allowed on the <ref> element so we used <foreign> instead in this instance.

<placeName type="townland">Burnfort (<ref target="https://www.logainm.ie/ga/11662"><foreign xml:lang="ga">Ráth an Tóiteáin</foreign></ref>)</placeName>

I entirely accept if this was an incorrect encoding approach to have taken in this instance, mea culpa. Perhaps I should have recommended to encode the English language placename and the Irish or Welsh language placenames with individual <placeName> elements or found a different workaround.

Still, this error has inadvertently led to a general discussion of the connotations associated with the term "foreign" for these change of language code shifts. Although the Guidelines don't state or imply anything else about the semantics of "foreign" it seems to be a concern shared by other members of the community (who undoubtedly have much better scenarios/examples than the one that I've supplied above).

[lb42]

I would definitely not have embedded the two placenames like that, irrespective of the language issue. It implies that the full name has both components, which i dont think you mean.

[joeytakeda]

I think the problem is that <foreign> is not a neutral term, even if the remarks say the opposite. And, if that were the case, then <foreign> should likely require @xml:lang (given that is its only function). So I think on balance, it makes much more sense to say "if you have nothing to hook an @xml:lang on, use <seg> or <hi>, which explicitly have no semantics. If you want to denote that a segment of text is interpretively understood as foreign, use <foreign>."

[lb42]

I am not keen at all on the idea of making xml:lang mandatory on foreign. To start with, what's the BCP46 code for "I don't know what this language is but it's definitely not French" (or whatever the parent language is)?

I am also not keen on using <hi> since that does "have semantics" -- its contents should be visually distinct from the rest of the text in some way. And those tricky little foreign words do sometimes just get used in media res with no formatting distinction. So we are stuck with seg, I suppose. However, your last sentence does allow poor old foreign to be used for text the encoder or maybe the original source wants to brand as "interpretively understood as foreign" so I am reassured. Adopting this ticket would involve some more explanatory prose no doubt, but it would not require deprecation or renaming of ,which would me iudice render invalid who knows how many thousands of existing megabytes.

[martindholmes]

I think this ticket might be a bit of an over-reaction. "Foreign" as a term in some contexts might be pejorative, and in others actually the opposite; as an English word it many meanings and usages, but as a TEI element it simply means "belonging to some language other than that of the surrounding text", which is completely inoffensive. If we start replacing element and attribute names based on their containing words which can be used pejoratively in some contexts, we'll be continually rotating the vocabulary of the schema based on rapidly-changing political and cultural mores, and we'll never keep up, and by the time the required deprecation period has elapsed the culture may well have moved on and the grounds for alteration largely evaporated. If <foreign> is beyond the pale, how about <xenoData>, or <explicit>, or @defective, or @real? We can easily pick out element and attribute names which might transient discomfort in some ephemeral political or cultural climate (<climate> -- there's another one), but we're attempting to provide a stable archival standard format for the long-term, and we have more urgent work to do than this.

Anyone who has a particular sensitivity to an element or attribute name is free to avoid it and use another with <altIdent> in their schema, or select an alternative method of encoding the phenomena with existing TEI components, surely?

[joeytakeda]

But this isn't that the word "foreign" is bad: it's that the <foreign> tag masquerades as being completely neutral -- as a convenient place on which to hang @xml:lang. As it says in the remarks, "This element is intended for use only where no other element is available to mark the phrase or words concerned. The global xml:lang attribute should be used in preference to this element where it is intended to mark the language of the whole of some text element."

But there's always another tag (<seg>) onto which you can put @xml:lang. So, my read --and many others-- of <foreign> is that it is semantic sugar for a particular type of segment -- <seg type="foreign">-- which denotes something that the encoder wants to say about their text.

We have many people who, all independently, express a similar discomfort with this semantic ambiguity. So I think my recommendation is slightly different from the what the title of the issue suggests: I think we should keep <foreign>, but clarify that it does what it says on the tin: that it describes text understood as belonging to a foreign language, not just a different one.

[raffazizzi]

@martindholmes can you explain what makes you think this request is a reflection of rapidly-changing political and cultural mores?

<foreign> makes too big of an assumption about its "surrounding text" as a monolingual and mono-cultural context; this may be truly adequate in a context where the textual object being encoded (a book in French) includes a term specifically borrowed from another language in a useful and meaningful way (e.g. latin).

It is, however, a poor instrument for texts that juggle multiple languages, but even more so in texts originating in multicultural contexts, such as the one in the project I linked: French then Spanish Louisiana with creole linguistic and cultural realities. In encoding a text that originates from this context, encoding "a word or phrase as belonging to some language other than that of the surrounding text" is not the same as encoding a foreign word. At the very least we should have a remark and a suggestion on how to handle this to avoid excessive divergence between projects (speaking of stable archival standards).

Naming textual phenomena as accurately as possible is a large part of what we do, so I'm troubled that this is received as a waste of time.

[hcayless]

I have definitely seen (and am probably responsible for) a number of clear misuses of <foreign>. It's not itself offensive in any way, but it can definitely be accidentally used in ways that aren't sensitive to their context. It represents a definite usage, e.g. a snippet of Latin or German, etc. italicized in English text. I think we just need to make it clear in the Guidelines that if you're working with multilingual texts, this is not the thing you're looking for.

[gabrielbodard]

In general, I agree that <foreign> is often misused as described, and that therefore some people (quite reasonably) don't like it. I would be happy if it had been better named when it was coined (perhaps <languageShift> would be neutral enough? Maybe not), but I'm less sanguine about the disruptive impact of changing the name of an element that is already is wide usage and works for a lot of what it is used for.

I think I tend to agree with @joeytakeda that in many cases we should encourage people to hang @xml:lang on a more neutral element like <seg> (although beware their content models are different), and in any case many texts with large chunks of multiple languages already use something like <ab> or similar to divide them up. The description of <foreign> should then be improved and clarified, to make it actually mean "text in a different language (specified or not)".

Even if this meant that 90% of our current uses of <foreign xml:lang=""> disappeared, I don't think I would be convinced by the argument to deprecate <foreign> altogether. There is still a clear, not uncommon, and not otherwise supported use case of a transcribed text in which some words are italicised/underlined/rubricated/w.ev. because they are "foreign"—whether Latin or French or transliterated Hebrew is not specified, and so the transcriber doesn't need to either; they just tag it as <foreign>.