Pronunciation Gap Analysis & Use Cases

Precise spoken presentation of numeric values may not always be correctly determined by text to speech engines from context.  Examples include speaking a multidigit number as individual digits (100 spoken as "one, zer, zero" instead of "one hundred"), correct reading of year values, and the correct speaking of ordinal and cardinal numbers. Furthermore, some educators may have specific requirements for the spoken presentation of a numeric value which may differ from a TTS engine's default rendering. For example, the Smarter Balanced Assessment Consortium has developed Read Aloud Guidelines to be followed by human readers used by students who may require a spoken presentation of an educational test, which includes specific examples of how numeric values should be read aloud.

Precise presentation of string values may not be determined correctly by text to speech synthesizers. Examples include speaking acroyms as individual letters rather than words and providing a substitute pronunciation for a word based on context (the word "read" may be pronounced as either "reed" or "red"), and providing a phonetic pronunciation of a word to ensure the correct spoken presentation.

A new aria attribute could be used to include pronunciation content. This option may have the best chance of being added to the accessibility tree by browsers.

Embed SSML in an HTML document.

• Assistive Technology
• Browser Extensions
• Search Engines

aria-ssml as embedded JSON

When AT encounters an element with aria-ssml, the AT should enhance the UI by processing the pronunciation content and passing it to the Web Speech API or an external API (e.g., Google's Text to Speech API).

<p>
  I say <span aria-ssml='{"phoneme":{"ph":"pɪˈkɑːn","alphabet":"ipa"}}'>pecan</span>.
  You say <span aria-ssml='{"phoneme":{"ph":"ˈpi.kæn","alphabet":"ipa"}}'>pecan</span>.
</p>
<script>
  // Client will convert JSON to SSML and pass the XML string a speech API.
  var msg = new SpeechSynthesisUtterance();
  msg.text = convertJSONtoSSML(element.getAttribute('aria-ssml'));
  speechSynthesis.speak(msg);
</script>

aria-ssml referencing XML by template ID

<!-- ssml must appear inside a template to be valid -->
<template id="pecan">
<?xml version="1.0"?>
<speak version="1.1"
       xmlns="http://www.w3.org/2001/10/synthesis"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.w3.org/2001/10/synthesis
                   http://www.w3.org/TR/speech-synthesis11/synthesis.xsd"
       xml:lang="en-US">
    You say, <phoneme alphabet="ipa" ph="pɪˈkɑːn">pecan</phoneme>.
    I say, <phoneme alphabet="ipa" ph="ˈpi.kæn">pecan</phoneme>.
</speak>
</template>

<p aria-ssml="#pecan">You say, pecan. I say, pecan.</p>

<script>
  // Client will parse XML and serialize it before passing to a speech API:
  var msg = new SpeechSynthesisUtterance();
  var xml = document.getElementById('pecan').content.firstElementChild;
  msg.text = serialize(xml);
  speechSynthesis.speak(msg);
</script>

aria-ssml referencing an XML string as script tag

<script id="pecan" type="application/ssml+xml">
<speak version="1.1"
       xmlns="http://www.w3.org/2001/10/synthesis"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.w3.org/2001/10/synthesis
                   http://www.w3.org/TR/speech-synthesis11/synthesis.xsd"
       xml:lang="en-US">
    You say, <phoneme alphabet="ipa" ph="pɪˈkɑːn">pecan</phoneme>.
    I say, <phoneme alphabet="ipa" ph="ˈpi.kæn">pecan</phoneme>.
</speak>
</script>

<p aria-ssml="#pecan">You say, pecan. I say, pecan.</p>

<script>
  // Client will pass the XML string raw to a speech API.
  var msg = new SpeechSynthesisUtterance();
  msg.text = document.getElementById('pecan').textContent;
  speechSynthesis.speak(msg);
</script>

aria-ssml referencing an external XML document by URL

<p aria-ssml="http://example.com/pronounce.ssml#pecan">You say, pecan. I say, pecan.</p>

<script>
  // Client will pass the string payload to a speech API.
  var msg = new SpeechSynthesisUtterance();
  var response = await fetch(element.getAttribute('aria-ssml'));
  msg.txt = await response.text();
  speechSynthesis.speak(msg);
</script>

• aria-ssml proposal
• SSML
• Web Speech API

• aria-ssml is not a valid aria-* attribute.
• OS/Browsers combinations that do not support the serialized XML usage of the Web Speech API.

As an existing attribute, data-* could be used, with some conventions, to include pronunciation content.

• Support repeated use within the page context
• Support external file references
• Reuse existing techniques without expanding specifications

Hearing users

data-ssml as embedded JSON

When an element with data-ssml is encountered by an SSML-aware AT, the AT should enhance the user interface by processing the referenced SSML content and passing it to the Web Speech API or an external API (e.g., Google's Text to Speech API).

<h2>The Pronunciation of Pecan</h2>
<p><speak>
I say <span data-ssml='{"phoneme":{"ph":"pɪˈkɑːn","alphabet":"ipa"}}'>pecan</span>.
You say <span data-ssml='{"phoneme":{"ph":"ˈpi.kæn","alphabet":"ipa"}}'>pecan</span>.
</p>

<script>
  // Client will convert JSON to SSML and pass the XML string a speech API.
  var msg = new SpeechSynthesisUtterance();
  msg.text = convertJSONtoSSML(element.dataset.ssml);
  speechSynthesis.speak(msg);
</script>

data-ssml referencing XML by template ID

<!-- ssml must appear inside a template to be valid -->
<template id="pecan">
<?xml version="1.0"?>
<speak version="1.1"
       xmlns="http://www.w3.org/2001/10/synthesis"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.w3.org/2001/10/synthesis
                   http://www.w3.org/TR/speech-synthesis11/synthesis.xsd"
       xml:lang="en-US">
    You say, <phoneme alphabet="ipa" ph="pɪˈkɑːn">pecan</phoneme>.
    I say, <phoneme alphabet="ipa" ph="ˈpi.kæn">pecan</phoneme>.
</speak>
</template>

<p data-ssml="#pecan">You say, pecan. I say, pecan.</p>

<script>
  // Client will parse XML and serialize it before passing to a speech API:
  var msg = new SpeechSynthesisUtterance();
  var xml = document.getElementById('pecan').content.firstElementChild;
  msg.text = serialize(xml);
  speechSynthesis.speak(msg);
</script>

data-ssml referencing an XML string as script tag

<script id="pecan" type="application/ssml+xml">
<speak version="1.1"
       xmlns="http://www.w3.org/2001/10/synthesis"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.w3.org/2001/10/synthesis
                   http://www.w3.org/TR/speech-synthesis11/synthesis.xsd"
       xml:lang="en-US">
    You say, <phoneme alphabet="ipa" ph="pɪˈkɑːn">pecan</phoneme>.
    I say, <phoneme alphabet="ipa" ph="ˈpi.kæn">pecan</phoneme>.
</speak>
</script>

<script>
  // Client will pass the XML string raw to a speech API.
  var msg = new SpeechSynthesisUtterance();
  msg.text = document.getElementById('pecan').textContent;
  speechSynthesis.speak(msg);
</script>
<p data-ssml="#pecan">You say, pecan. I say, pecan.</p>

data-ssml referencing an external XML document by URL

<p data-ssml="http://example.com/pronounce.ssml#pecan">You say, pecan. I say, pecan.</p>

<script>
  // Client will pass the string payload to a speech API.
  var msg = new SpeechSynthesisUtterance();
  var response = await fetch(el.dataset.ssml)
  msg.txt = await response.text();
  speechSynthesis.speak(msg);
</script>

• aria-ssml proposal
• SSML
• Web Speech API

• Does not assume or suggest visual pronunciation help for deaf or hard of hearing
• Use of data-* requires input from AT vendors
• XML data is not indexed by search engines

HTML5 includes the XML namespaces for MathML and SVG. Therefore, using either's elements in an HTML5 document is valid. Likewise, including an SSML namespace would allow the valid use of SSML in HTML5. Because SSML's implementation is non-visual in nature, browser implementation could be slow or non-existent without affecting how authors use SSML in HTML. Browsers would treat the element like any other unknown element, as HTMLUnknownElement.

• Support valid use of SSML in HTML5 documents
• Allow visual pronunciation support

• SSML-aware technologies and browser extensions
• Search indexers

SSML

When an element with data-ssml is encountered by an SSML-aware AT, the AT should enhance the user interface by processing the referenced SSML content and passing it to the Web Speech API or an external API (e.g., Google's Text to Speech API).

<h2>The Pronunciation of Pecan</h2>
  <p><speak>
  You say, <phoneme alphabet="ipa" ph="pɪˈkɑːn">pecan</phoneme>.
  I say, <phoneme alphabet="ipa" ph="ˈpi.kæn">pecan</phoneme>.
</speak></p>

• VoiceXML 2.1
• SMIL - Synchronized Multimedia Integration Language
• PLS - Pronunciation Lexicon

SSML is not valid HTML5

Embed valid SSML in HTML using custom elements registered as ssml-* where * is the actual SSML tag name (except for p which expects the same treatment as an HTML p in HTML layout).

Support use of SSML in HTML documents.

• SSML-aware technologies and browser extensions
• Search indexers

ssml-speak: see demo

Only the <ssml-speak> component requires registration. The component code lifts the SSML by getting the innerHTML and removing the ssml- prefix from the interior tags and passing it to the web speech API. The <p> tag from SSML is not given the prefix because we still want to start a semantic paragraph within the content. The other tags used in the example have no semantic meaning. Tags like <em> in HTML could be converted to <emphasis> in SSML. In that case, CSS styles will come from the browser's default styles or the page author.

<ssml-speak>
  Here are <ssml-say-as interpret-as="characters">SSML</ssml-say-as> samples.
  I can pause<ssml-break time="3s"></ssml-break>.
  I can speak in cardinals.
  Your number is <ssml-say-as interpret-as="cardinal">10</ssml-say-as>.
  Or I can speak in ordinals.
  You are <ssml-say-as interpret-as="ordinal">10</ssml-say-as> in line.
  Or I can even speak in digits.
  The digits for ten are <ssml-say-as interpret-as="characters">10</ssml-say-as>.
  I can also substitute phrases, like the <ssml-sub alias="World Wide Web Consortium">W3C</ssml-sub>.
  Finally, I can speak a paragraph with two sentences.
  <p>
    <ssml-s>You say, <ssml-phoneme alphabet="ipa" ph="pɪˈkɑːn">pecan</ssml-phoneme>.</ssml-s>
    <ssml-s>I say, <ssml-phoneme alphabet="ipa" ph="ˈpi.kæn">pecan</ssml-phoneme>.</ssml-s>
  </p>
</ssml-speak>
<template id="ssml-controls">
  <style>
    [role="switch"][aria-checked="true"] :first-child,
    [role="switch"][aria-checked="false"] :last-child {
      background: #000;
      color: #fff;
    }
  </style>
  <slot></slot>
  <p>
    <span id="play">Speak</span>
    <button role="switch" aria-checked="false" aria-labelledby="play">
      <span>on</span>
      <span>off</span>
    </button>
  </p>
</template>

<script>
class SSMLSpeak extends HTMLElement {
  constructor() {
    super();
    const template = document.getElementById('ssml-controls');
    const templateContent = template.content;
    this.attachShadow({mode: 'open'})
      .appendChild(templateContent.cloneNode(true));
  }
  connectedCallback() {
    const button = this.shadowRoot.querySelector('[role="switch"][aria-labelledby="play"]')
    const ssml = this.innerHTML.replace(/ssml-/gm, '')
    const msg = new SpeechSynthesisUtterance();
    msg.lang = document.documentElement.lang;
    msg.text = `<speak version="1.1"
      xmlns="http://www.w3.org/2001/10/synthesis"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.w3.org/2001/10/synthesis
        http://www.w3.org/TR/speech-synthesis11/synthesis.xsd"
      xml:lang="${msg.lang}">
    ${ssml}
    </speak>`;
    msg.voice = speechSynthesis.getVoices().find(voice => voice.lang.startsWith(msg.lang));
    msg.onstart = () => button.setAttribute('aria-checked', 'true');
    msg.onend = () => button.setAttribute('aria-checked', 'false');
    button.addEventListener('click', () => speechSynthesis[speechSynthesis.speaking ? 'cancel' : 'speak'](msg))
  }
}

customElements.define('ssml-speak', SSMLSpeak);
</script>

• DOM Living Standard
• Web Speech API

• OS/Browsers combinations that do not support the serialized XML usage of the Web Speech API.
• Browsers may need to map SSML tags with CSS styles for default user agent styles.
• Without an extension or AT, only user interaction can start the Web Speech API.
• Authors or parsing may need to remove HTML content with unintended SSML semantics before serialization.

JSON-LD provides an established standard for embedding data in HTML. Unlike other microdata approaches, JSON-LD helps to reuse standardized annotations through external references.

Support use of SSML in HTML documents.

• SSML-aware technologies and browser extensions
• Search indexers

JSON-LD

<script type="application/ld+json">
{
  "@context": "http://schema.org/",
  "@id": "/Pronunciation#WKRP",
  "@type": "TextPronunciation",
  "@language": "en",
  "text": "WKRP",
  "speechToTextMarkup": "SSML",
  "phoneticText": "<say-as interpret-as=\"characters\">WKRP</say-as>"
}
</script>
<p>
  Do you listen to <span itemscope
    itemtype="http://schema.org/TextPronunciation"
    itemid="/Pronunciation#WKRP">WKRP</span>?
</p>

• Web of Things Working Group
• Schema.org

JSON-LD is not an established "type"/published schema

<Ruby> annotations are short runs of text presented alongside base text, primarily used in East Asian typography as a guide for pronunciation or to include other annotations.

ruby guides pronunciation visually.

• Support use of SSML in HTML documents.
• Offer visual pronunciation support.

• AT and browser extensions
• Search indexers

ruby with microdata

Microdata can augment the ruby element and its descendants.

<p>
  You say,
  <span itemscope="" itemtype="http://example.org/Pronunciation">
    <ruby itemprop="phoneme" content="pecan">
      pecan
      <rt itemprop="ph">pɪˈkɑːn</rt>
      <meta itemprop="alphabet" content="ipa">
    </ruby>.
  </span>
  I say,
  <span itemscope="" itemtype="http://example.org/Pronunciation">
    <ruby itemprop="phoneme" content="pecan">
      pe
      <rt itemprop="ph">ˈpi</rt>
      can
      <rt itemprop="ph">kæn</rt>
      <meta itemprop="alphabet" content="ipa">
    </ruby>.
  </span>
</p>

• HTML Living Standard
• Schema.org
• Rules for Simple Placement of Japanese Ruby
• Requirements for Japanese Text Layout (JLreq)

• AT may process annotations as content
• AT "double reading" words instead of choosing either the content or the annotation
• Only offers a few SSML expressions
• Difficult to reuse by reference
• Competing interests with conventional use

lang attribute can be applied at the document level or to individual elements. (WCAG) (AT Supported: some)

Example: <speak> In Paris, they pronounce it <lang xml:lang="fr-FR">Paris</lang> </speak>code>

voice-family property can be used to specify the gender of the voice.

Example: { voice-family: male; }

Using the <voice> element, the gender of the speaker, if supported by the TTS engine, can be specified.

Example: <voice gender="female" >Mary had a little lamb,</voice>

Using PLS, all the pronunciations can be factored out into an external PLS document which is referenced by the <lexicon> element of SSML

Example: <speak> <lexicon uri="http://www.example.com/movie_lexicon.pls"/>
                The title of the movie is: "La vita è bella" (Life is beautiful),
    which is directed by Roberto Benigni.</speak>

The following is a simple example of an SSML document. It includes an Italian movie title and the name of the director to be read in US English.

Example: The title of the movie is: 
            <speak> <phoneme alphabet="ipa" ph="ˈlɑ ˈviːɾə ˈʔeɪ ˈbɛlə">
            "La vita è bella"</phoneme> (Life is beautiful),
            which is directed by 
            <phoneme alphabet="ipa" ph="ɹəˈbɛːɹɾoʊ bɛˈniːnji""> 
            Roberto Benigni </phoneme>.</speak>
            

The aria-label and aria-labelledby attribute can be used by an author to supply a text string that will become the accessible name for the element upon which it is applied.  This usage effectively provides a mechanism for performing text substation that is supported by a screen reader. However, it is problematic for one significant reason; for users who utilize screen readers and refreshable Braille, the content that is voiced will not match the content that is sent to the refreshable Braille device. This mismatch would not be acceptable for some content, particularly for assessment content.

Pronounce the specified word or phrase as a different word or phrase. Specify the pronunciation to substitute with the alias attribute.

   <speak>
       My favorite chemical element is <sub alias="aluminum">Al</sub>,
       but Al prefers <sub alias="magnesium">Mg</sub>. 
 </speak>
 

voice-rate

The ‘voice-rate’ property manipulates the rate of generated synthetic speech in terms of words per minute.

voice-pitch

The ‘voice-pitch’ property specifies the "baseline" pitch of the generated speech output, which depends on the used ‘voice-family’ instance, and varies across speech synthesis processors (it approximately corresponds to the average pitch of the output). For example, the common pitch for a male voice is around 120Hz, whereas it is around 210Hz for a female voice.

voice-range

The ‘voice-range’ property specifies the variability in the "baseline" pitch, i.e. how much the fundamental frequency may deviate from the average pitch of the speech output. The dynamic pitch range of the generated speech generally increases for a highly animated voice, for example when variations in inflection are used to convey meaning and emphasis in speech. Typically, a low range produces a flat, monotonic voice, whereas a high range produces an animated voice.

prosody modifies the volume, pitch, and rate of the tagged speech.

  <speak>
      Normal volume for the first sentence.
      <prosody volume="x-loud">Louder volume for the second sentence</prosody>.
      When I wake up, <prosody rate="x-slow">I speak quite slowly</prosody>.
      I can speak with my normal pitch, 
      <prosody pitch="x-high"> but also with a much higher pitch </prosody>, 
      and also <prosody pitch="low">with a lower pitch</prosody>.
</speak>

The HTML <em> element marks text that has stress emphasis. The <em> element can be nested, with each level of nesting indicating a greater degree of emphasis.

The <em> element is for words that have a stressed emphasis compared to surrounding text, which is often limited to a word or words of a sentence and affects the meaning of the sentence itself. Typically this element is displayed in italic type. However, it should not be used simply to apply italic styling; use the CSS font-style property for that purpose. Use the <cite> element to mark the title of a work (book, play, song, etc.). Use the <i> element to mark text that is in an alternate tone or mood, which covers many common situations for italics such as scientific names or words in other languages. Use the <strong> element to mark text that has greater importance than surrounding text.

voice-stress

The ‘voice-stress’ property manipulates the strength of emphasis, which is normally applied using a combination of pitch change, timing changes, loudness and other acoustic differences. The precise meaning of the values therefore depend on the language being spoken.

Emphasize the tagged words or phrases. Emphasis changes rate and volume of the speech. More emphasis is spoken louder and slower. Less emphasis is quieter and faster.

        <speak>
        I already told you I 
       <emphasis level="strong">really like</emphasis> that person.
       </speak> 

The ‘speak-as’ property determines in what manner text gets rendered aurally, based upon a predefined list of possibilities.

Describes how the text should be interpreted. This lets you provide additional context to the text and eliminate any ambiguity on how Alexa should render the text. Indicate how Alexa should interpret the text with the interpret-as attribute.

<speak>
    Here is a number spoken as a cardinal number: 
    <say-as interpret-as="cardinal">12345</say-as>.
    Here is the same number with each digit spoken separately:
    <say-as interpret-as="digits">12345</say-as>.
    Here is a word spelled out: <say-as interpret-as="spell-out">hello</say-as>
</speak>

The ‘pause-before’ and ‘pause-after’ properties specify a prosodic boundary (silence with a specific duration) that occurs before (or after) the speech synthesis rendition of the selected element, or if any ‘cue-before’ (or ‘cue-after’) is specified, before (or after) the cue within the aural box model.

break represents a pause in the speech. Set the length of the pause with the strength or time attributes.

<speak>
    There is a three second pause here <break time="3s"/> 
    then the speech continues.
</speak>