Proposal: Evolution of the Locator model

[mickael-menu]

Motivation

The current Locator model has several limitations that have become apparent in practice.

type is not used

The type property was intended to describe the type of the resource pointed to by href. In practice it provides no value: the reading system already knows the media type of every resource from the publication manifest, and nothing in the toolkit uses it to make decisions. It is noise that every locator creator must supply and every locator consumer must ignore.

href is too constraining

href is required, which makes it awkward to express locations in publications where there is no meaningful sub-resource to point at:

• A standalone PDF is a single file: the meaningful location is a page number, not an HREF.
• A single audio file uses a time offset, not an HREF.

In both cases callers today work around the constraint by supplying a dummy or self-referential HREF, which is misleading and fragile.

No support for multiple resources

A single href cannot express a location that is simultaneously anchored to multiple resources. Two concrete use cases make this necessary:

• Multiple renditions: A publication may offer the same content as both an audio and image rendition. A reading position shared across renditions needs to reference a resource in each.
• Guided Navigation Documents: A GND could synchronize a separate EPUB package and a separate audiobook package. A locator produced by the navigator driven with the GND must simultaneously anchor to an HTML resource and an audio track so that both the visual and audio renditions can seek to the correct position.

Prior art: OPDS Progression 1.0

The OPDS Progression 1.0 addresses several of these issues with a references array that can hold multiple resource URIs with their fragments:

{
  "progression": 0.0174920,
  "references": [
    "#t=849.250",
    "chapter2.mp3#t=423.52",
    "chapter2.html#:~:text=It%20was%20expected"
  ]
}

Bare fragment references (no href, as in the audio example above) handle resource-less publications. The references array naturally extends to multiple resources.

Proposed Evolution of Locator

This is directly inspired by the OPDS Progression.

The Locator Object

Key	Definition	Format	Required
title	Contains text that can be relevant to identify or contextualize the progression, such as a chapter title.	String	No
progression	Total progression in the publication expressed as a percentage (%).	Float (0.0-1.0)	Yes
position	Synthetic 1-based virtual page in a reflowable publication.	Integer (1-)	No
page	Pre-paginated or print page number.	Integer (1-)	No
time	Location in seconds in an audio publication.	Float (0.0-)	No
text	Textual context of the locator, for example a text selection.	String	No
references	References inside the publication.	Array of Reference Objects	No

The top-level properties are used for contextualization publication-wide, for example in the user interface, but may be used as a fallback for positioning when no suitable Reference Object is found.

For example, the text property contains normalized text, ready for presentation in the user interface. But a reference using an optimized scroll-to-text fragment should be used for positioning.

Note that text does not have before and after like before. Do we need them, in the context of contextualization? They are a bit ambiguous because the reading system that consume the locator don't control the amount of prefix and suffix around the quote.

The Reference Object

A Reference object can be represented as a literal String (e.g. "chapter2.html#:~:text=It%20was%20expected") or a full object.

Key	Definition	Format	Required
href	The URI of the resource that the Reference Object points to.	String	Only if progression is set
progression	Progression in the resource expressed as a percentage.	Float (0.0-1.0)	No
fragments	Contains one or more fragment in the resource referenced by the Reference Object.	Array of strings	No

This is naturally open for extensions, for example CSS selectors which don't have an official fragment identifier.

Alternative 1: Only URI References

We could simplify and have only URIs in the references array (like in the OPDS Progression document). This is more compact and compatible with OPDS Progression, but has a couple of issues:

• progression cannot be represented with a fragment. We need to roll out a non-standard one, like #scroll=0.5284.
• Same for CSS selectors, but at least there's an unofficial spec, that we might use anyway in the Guided Navigation Objects.
• This is not easy to extend.

Alternative 2: Publication-wide fragments

The OPDS Progression allows for publication-wide fragments, such as #page=45 or #t=23.6. We could do the same and remove the top-level properties time and page. However:

• This is not as readily usable as direct and typed top-level context properties.
• Some top-level properties cannot be represented as fragment identifier, such as position.

Compatibility with the Legacy Locator Object

The new and legacy representations are asymmetrically compatible.

• Legacy to New is lossless and requires no publication context. This means stored locators can be migrated lazily, on first read, or with a bulk database migration.
• New to Legacy requires the publication's reading order to recover href and mediaType. We should expose an API that performs this reconstruction, so callers that need to write back to a legacy store or synchronize with older clients have a solution.

[chocolatkey]

From my perspective as someone who is generating WebPubs with position list documents for thousands of publications, I appreciate the backwards compatibility so they continue to function in updated versions of the toolkits

[qnga]

href is too constraining

How concretely would you benefit from the change? Locators are usually associated with a publication more or less represented as an RWPM and RWPM enforces the need for a href by resource.

The Locator Object

I'm a bit concerned about properties not referring to a specific manifestation of a publication. An agnostic progression sounds great, but in practice it's a lie. Given a location from the user point of view, total progression and position are different for each manifestation. I feel like you're trying to design the best format to persist locations, is this correct?

Guided Navigation Documents: A GND could synchronize a separate EPUB package and a separate audiobook package.

I'm not aware of that. How would it present itself?

[mickael-menu]

How concretely would you benefit from the change? Locators are usually associated with a publication more or less represented as an RWPM and RWPM enforces the need for a href by resource.

Standalone PDFs and audio clips are examples that we had to hack around to fit RWPM. Right now we set the href as publication.pdf. This was not always the case and we encountered interoperability issues between the toolkits in the past. This is a convention in the Readium toolkits, but not super interoperable with other reading systems. In essence, an HREF doesn't make sense for publications that are not packages.

The OPDS Progression doesn't require an HREF, so that's another example that is not compatible with the current Locator: https://github.com/opds-community/drafts/blob/main/opds-progression-1.0.md

Finally, we would be able to represent as locators requests such as "scroll to X% in the publication", "go to position X", "go to print-page number X". To use them directly in the API, or to store and share them. Right now to skip to a given total progression or position, an app must do that:

if let locator = await publication.locate(progression: 0.4) {
    await navigator.go(to: locator)
}

I'm a bit concerned about properties not referring to a specific manifestation of a publication. An agnostic progression sounds great, but in practice it's a lie. Given a location from the user point of view, total progression and position are different for each manifestation.

I'm not sure what you mean by a "specific manifestation of a publication"? Is this different from what we had with the existing Locator objects, are you suggesting an alternative?

I feel like you're trying to design the best format to persist locations, is this correct?

Yes, the JSON serialization that works for persistence or as an interchange format. But I want to design it so that it can work with publication-specific native types too. Do you see an issue with this proposal in that regard?

I'm not aware of that. How would it present itself?

This is to be able to download separately existing standalone EPUB and audiobooks. A bunch of GNDs could be packaged together with a RWPM referencing publications from both the EPUB and the audiobook to synchronize them together. In this specific case, there's not one "main" publication. This is still exploratory but stakeholders are interested in this.

[qnga]

I'm not sure what you mean by a "specific manifestation of a publication"? Is this different from what we had with the existing Locator objects, are you suggesting an alternative?

As I understand it, you aim to support multiple renditions in a single locator, either from a publication with guided navigation, or from two different publications. In such cases, what isposition or totalProgression referring to? Though a universal definition would be great, it's not feasible, you'll get different results if you compute them from audio or from text. And if you try to go to them, you'd better know which rendition it refers to or you'll end up at a random location.

[mickael-menu]

From what I gather working on this:

Even when playing Guided Navigation or Media Overlays, the locator should remain within the "main" rendition's referential. In the code, I referred to this as the "publication's reading order" as opposed to the "guided navigation document order" (derived from w3c/epub-specs#2943 (comment)).

So the references in a locator generated by a Guided Audio experience would still point to XHTML resources and HTML ID fragments. I'm not sure we can derived a position from just the Guided Audio navigator, and the progression would be relative to the audio playback.

I agree that this progression wouldn't be useful to restore a position in the EPUB navigator, but it's purpose is more for UI contextualization (showing the user where they are in the progression of the publication). It can be used for positioning as a fallback if references are missing.

Although IMHO we must always have references or position in an EPUB locator. Maybe this should be enforced.

[mickael-menu]

Maybe a Locator should have a progression and any of (references, position, page, time).

[qnga]

After further explanation I understand a bit better how you've got to this. I have two extra points about the scope though:

• You didn't recall the use cases, probably because you assumed they're still exactly the same as before. That's preventing us from asking ourselves if we still think that a single model is best to fit them all.
• Let's assume we're talking about the location restoration use case. Is this supposed to be an attempt at a standard location exchange format reaching apps not using Readium, somehow like the new annotation model? If it's not, a user can't import the locator in a "random" app supporting it and I feel like it's a bit the responsibility of the apps to adopt a suitable exchange format inside their ecosystem. We can provide a go-to format, but it's far less important than in the other case.

[mickael-menu]

The use cases are the same as before, do you have others in mind?

• current location saving and restoration in a given app or Readium ecosystem
• bookmarks, highlights
• contextualizing a location in the UI (displaying the % progression of the publication, ordering bookmarks, displaying the text of a highlight), etc.

It's not for a standard exchange format outside the Readium ecosystem. OPDS Progression could be used for that.

[mickael-menu]

I'm adding:

• synchronizing two renditions of the same publication (e.g. EPUB and media overlays)

[qnga]

• I think it's also used in the content iterator at the moment. Its main use case will be, I think, search once the TTS is no longer using it.