Skip to content

Populate shortDescription field on works #6488

Description

@StepanBrychta

Archive browse designs include a new 'short description' field, which should be populated by taking the first sentence of the description field. See designs here.

Determining when a sentence ends is hard. Splitting on a sentence terminator (.!?) is an easy heuristic, but it doesn't always work. In addition, the description field contains HTML, which complicates things further.

Libraries like pysbd are available, but not always accurate. We could also write the logic ourselves to cover some of the edge cases but would likely not be able to achieve 100% accuracy. Here are some edge cases to be aware of:

Description Short description to avoid
Letter from Dr. Smith to the Institute. Second… Letter from Dr.
Dated 1.5.1943. Letter concerning research. Dated 1.
<p>Item no. 5 of the series. Another sentence.</p> <p>Item no.
<p>One sentence. Another sentence.</p> <p>One sentence.
Report on the 1918-19 epidemic (see p. 4). Written by J. S. Haldane. Report on the 1918-19 epidemic (see p.
Vol. 2 of the series covers 1900-1910. Vol. 3… Vol.
He said "stop." Then he left. He said "stop.
<p>See <a href="http://wellcome.org/thing">something. here</a>. More text.</p> <p>See <a href="http://wellcome.org/thing">something.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    Next

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions