Layering highlights

Multiple simultaneous highlights are possible when cues overlap. Here is an example using mulitple WebVTT files to provide highlight layers for words, lines, and stanzas of a poem. The track order gives the highlight priority order. In this example, stanzas are highlighted, followed by lines appearing on top of that, followed by words.

<audio controls autoplay src="chapter.mp3">
 <track id="stanzas" kind="metadata" src="stanza.vtt" label="Stanzas" default/>
 <track id="lines" kind="metadata" src="line.vtt" label="Lines" default/>
 <track id="words" kind="metadata" src="word.vtt" label="Words" default/>
</audio>

WEBVTT

00:00:07.800 --> 00:00:08.200
{"selector": {"type": "CssSelector", "value": ":nth-child("1" of .stanza) > :nth-child(1 of .line)", refinedBy: {"type": "TextPositionSelector", "start": 0, "end": 3}}}

00:00:08.200 --> 00:00:08.800
{"selector": {"type": "CssSelector", "value": ":nth-child("1" of .stanza) > :nth-child(1 of .line)", refinedBy: {"type": "TextPositionSelector", "start": 5, "end": 8}}}

00:00:08.800 --> 00:00:09.200
{"selector": {"type": "CssSelector", "value": ":nth-child("1" of .stanza) > :nth-child(1 of .line)", refinedBy: {"type": "TextPositionSelector", "start": 10 "end": ,0"}}}

WEBVTT

00:00:07.800 --> 00:00:12.600
{"selector":{"type":"CssSelector","value":":nth-child(1 of .stanza) > :nth-child(1 of .line)"}}

00:00:12.600 --> 00:00:16.200
{"selector":{"type":"CssSelector","value":":nth-child(1 of .stanza) > :nth-child(2 of .line)"}}

00:00:17.200 --> 00:00:21.000
{"selector":{"type":"CssSelector","value":":nth-child(1 of .stanza) > :nth-child(3 of .line)"}}

WEBVTT

00:00:07.800 --> 00:00:31.100
{"selector":{"type":"CssSelector","value":":nth-child(1 of .stanza)"}}

00:00:32.300 --> 00:00:55.800
{"selector":{"type":"CssSelector","value":":nth-child(2 of .stanza)"}}

In that example, a cue from each WebVTT file starts at the same time. Their associated highlights can all be different, for example the stanza can have a light background, the line can have a stronger background, and the word can be underlined. All the highlights can co-exist simultaneously because each one has its own key in the HighlightRegistry. The link between the highlight's key in the HighlightRegistry and the track is given by the track's ID.

let range = createRange(cuePayload.selector)
let highlight = new Highlight(range);
CSS.highlights.set(cue.track.id, highlight); 

::highlight(stanzas) {
    background-color: lightyellow;
}

::highlight(lines) {
    text-decoration: none;
    background-color: yellow;
}
::highlight(words) {
    text-decoration: underline;
    text-decoration-style:dotted;
    text-decoration-thickness: 2px;
}

Previous/next navigation

This user interface feature lets a user move between cues. When paired with multiple simultaneous tracks, it enables variation on the previous/next navigation feature, for example allowing a way to move to the next word, sentence, or paragraph.

Cues may appear in a WebVTT file in any order, not necessarily in document flow order. They can be sorted by timestamp when it is necessary to calculate previous and next cues.

Skip/escape

A user listening to a narrated document has certain elements "set to skip", e.g. page number announcements might be turned off; or wants to "escape" from a complex structure, e.g. a table or nested list, and return to the main content flow.

HTML document semantics provide all necessary information, requiring only to reposition the audio timeline past any cues that reference a construct being escaped or skipped. Therefore including skip/escape semantics and nesting in the WebVTT file becomes unnecessary.