This document describes user requirements for text to speech of electronic documents containing ruby.
Purpose
This document addresses concerns related to the text-to-speech
functionality in HTML documents and EPUB publications that
contain ruby annotations. While typographical aspects of ruby
are covered by [[?JLREQ]], text-to-speech
issues in this context have not received widespread
recognition. The primary focus of this document is to outline
user requirements.
In Section 2, we enumerate the various roles of ruby
annotations in relation to their associated ruby
bases. Section 3 provides an overview of potential options for
using ruby bases and/or ruby annotations in text-to-speech,
along with a discussion of the advantages and disadvantages of
each option. Section 4 addresses markup issues related to the
text-to-speech of ruby annotations. Section 5 introduces
alternative mechanisms, such as SSML and PLS. Section 6 delves
into the use of ruby annotations in translating HTML or EPUB
document to braille.
Roles of ruby annotations
Furigana, background
The primary purpose of ruby annotations is to indicate how
to pronounce CJK ideographic characters, a practice known
as Furigana (see also JLReq terminology).
In contemporary usage, it is uncommon to attach ruby
annotations to all CJK ideographic characters
in a given document. Instead, it is more common to
attach ruby annotations to only some of the CJK ideographic characters.
Ruby annotations find their application in various
contexts, including trade books, newspapers, textbooks,
teaching materials, and more, but are rarely utilized in
business documents.
Even for simple CJK ideographic characters, ruby annotations may be added for some users who have particular
difficulties with CJK ideographic characters
(in electronic documents, it is easy to make ruby annotations visible or invisible based on user preferences).
Such ruby annotations are called as furigana-added-for-enhanced-accessibility.
Some simple CJK ideographic characters have more than one possible reading and thus require ruby annotations for disambiguation.
This is common for names of people and places. For example, 山崎 (a person's name) may be read as
YAMAZAKI or YAMASAKI.
If ruby annotations are attached to only some of the CJK ideographic characters in a given document, the first occurrence
of a CJK ideographic character or a word composed of such characters may have a ruby annotation, while subsequent occurrences typically do not. This practice
assumes that users will learn the correct pronunciation from the first occurrence.
Gikun, background
Especially in Japan, ruby annotations are
also used to indicate something different from the reading of a CJK ideographic character.
Such ruby annotations are referred to as Gikun. Gikun is commonly employed in light novels and comics.
Here are some examples of Gikun:
敵
(where 敵 means 'enemy' and とも means 'friend'). The combination means 'frenemy'.
生命
(where the typical reading of 生命 is SEIMEI rather than いのち (INOCHI),
both of which mean 'Life')
背景
(where the typical reading of 背景 is HAIKEI rather than バック (back),
an English translation)
牛乳
(where the typical reading of 牛乳 is GYUUNYUU rather than ミルク (milk),
an English translation)
Even when Gikun is used for a compound word, it is unlikely to be repeated for later occurrences of the same word.
Moreover, different [=GIKUN=] may be added for subsequent occurrences of the same word.
For example, the next occurrence of 生命 may well be
生命
where ライフ (life) is an English translation.
Unusual names of people and places, background
Unusual names of people in Japan are typically written
using CJK ideographic characters but are pronounced quite
differently from the standard reading of these
characters. For instance, 男 is an unusual name,
where 男 (usually read as OTOKO)
means 'man', and あだむ represents
'Adam' in Kana.
Character names in comics, animations, and light novels can
sometimes be extremely challenging to pronounce. Many of the
character names
in Demon
Slayer (Kimetsu no Yaiba) fall into this category. For
example, almost no one can read
不死川 玄弥 as SHINAZUGAWA GENNYA without assistance.
Names of places can also be difficult to read due to
historical reasons. For instance, 神居古潭,
温根沼,
音威子府
are names of places in Hokkaido (the northern
island of Japan). These names are challenging to pronounce
because they originated from the Ainu language,
which is
entirely different from the Japanese language.
In many instances, the first occurrence of an unusual name is accompanied by a ruby annotation, but subsequent occurrences are not.
Interlinear notes, background
Interlinear notes resemble ruby annotations in appearance.
A note in JLreq introduces interlinear notes as follows:
In the example shown in
a figure referenced in the quoted note ("An example of a note in inter lines"),
徳川慶喜 (Tokugawa Yoshinobu) is accompanied by an interlinear note
"1837-1913 江戸幕府最後の将軍" (1837-1913 the last shogun of the Edo shogunate).
Other examples are: a modern kana phrase as an interlinear note for a historical kana phrase,
a standard Japanese expression as an interlinear note for an expression in a dialect,
a modern CJK ideographic character as an interlinear note for a traditional CJK ideographic character,
an English text chunk as an interlinear note for a Japanese text chunk,
and an official name as an interlinear note for an abbreviated name.
One could argue that HTML ruby elements should not be used for representing interlinear notes
(see Kobayashi Sensei's mail in Japanese).
However, it is not difficult to imagine that ruby elements are actually used for representing interlinear notes.
Ruby annotations for indicating the pronunciation of foreign phrases in language textbooks, background
In language textbooks, ruby annotations are at times
employed to indicate the pronunciation of foreign phrases
written in hiragana or katakana. For example, a Chinese
phrase 我去学校 may include
ウオ チュー シュエシャオ as a ruby annotation.
Double-sided ruby, background
A sequence of characters can be accompanied by two ruby annotations,
typically consisting of [=Furigana=] and either [=GIKUN=] or an [=interlinear note=].
In an example provided in JLreq
("An example of ruby annotations attached to both sides of the base characters"),
東南 is accompanied by たつみ and とうなん.
Here 東南 means 'southeast', with とうなん (TOUNAN) serving as [=Furigana=],
and たつみ (TATSUMI) as [=GIKUN=],
as 辰巳 (read as TATSUMI) indicates the same direction as 東南.
We offer two additional illustrative examples.
東洋 features an upper-side ruby annotation オリエント and a lower-side ruby annotation とうよう
In this example, とうよう serves as [=Furigana=], while オリエント is used as [=Gikun=]
織田信長 features an upper-side ruby annotation "1534〜82" and a lower-side ruby annotation おだのぶなが
In this example, おだのぶなが serves as [=Furigana=], while "1534〜82" is presented as an [=interlinear note=].
Which should be read aloud, ruby bases or ruby annotations, or both?
There are three possible options: (1) both ruby bases and ruby annotations, (2) ruby annotations only, and (3) ruby bases only.
Reading aloud both ruby bases and ruby annoations
In this option, both ruby bases and ruby annotations are read aloud (double reading).
Many implementations (screen readers, in particular) support this option only.
For example, foo is read aloud as 'foo bar' or 'bar foo'.
Furigana, when both read aloud
The option of reading aloud both interferes with readers' understanding significantly.
Examples of harmful double reading: Japanese
彼の名前は出羽内です。
This sentence is intended to mean "His name is
Dewanai". Double reading completely changes the meaning:
it will be interpreted as "His name is NOT Dewanai".
それでは話にならない。
This sentence is intened to mean "Nonsense!". Double
reading completely changes the meaning: it will be
interpreted as "You have to deal with it".
Examples of harmful double reading: English
Consider this English sentence having a ruby annotation: "My name is Knot".
Double reading completely changes the meaning: it will be interpreted as "My name is not Knot".
Another example: "There is a road in Autin spelled
both Manchaca
and Menchaca".
Double reading makes the road name read aloud twice,
possibly differently.
Yet another example: "Oxoerythromycin
is a ketone derived from erythromycin".
Double reading makes this compound name read aloud twice,
possibly differently.
Gikun, when both read aloud
The option of reading aloud both is sensible. It is common to read aloud ruby annotations first then ruby bases next, but it is sometimes better to read aloud ruby bases first and ruby annotations next [[?Transliteration Training Course]]).
敵 is read aloud as TEKI TOMO or TOMO TEKI, which means 'enemy friend' or 'friend enemy' (equal to 'frenemy').
生命 is read aloud as SEIMEI INOCHI or INOCHI SEIMEI,
where SEIMEI is a loan word from Chinese and INOCHI is a native Japanese word. Both means life.
Unusual names of people and places, when both read aloud
The option of reading aloud both interferes with readers' understanding significantly.
不死川玄弥
is read aloud as FUSHIKAWA GENYA SHINAZUGAWA GENYA or SHINAZUGAWA GENYA FUSHIKAWA GENYA, which suggests two persons rather than one person.
Interlinear notes, when both read aloud
The option of reading aloud both is sensible. It is necessary to read aloud ruby bases first then ruby annotations next.
For example, 徳川慶喜
is read aloud as TOKUGAWA YOSHINOBU 1837-1913 EDO BAKUFU SAIGONO SHOUGUN,
which means 'Tokugawa Yoshinobu 1837-1913, the last shogun of the Edo shogunate'.
Ruby annotations for indicating the pronunciation of foreign phrases in language books, when both read aloud
The option of reading aloud both interferes with readers' understanding significantly.
In the example of 我去学校,
even if ウオ チュー シュエシャオ is read aloud using the Japanese text-to-speech engine,
the result will not be helpful to learners because of the incorrect pronunciation and four tones.
Katakana pronunciation is also useless in languages such as English.
Double-sided ruby, when both read aloud
Since there are two ruby annotations, double-sided ruby leads to reading aloud three times.
One of the ruby annotations is typically furigana, so the description in 3.1.1 applies.
If the other ruby annotation is a Gikun, the description in 3.1.2 applies;
if it is an interlinear note, the description in 3.1.4 applies.
Reading aloud ruby annotations only
In this option, ruby annotations are read aloud but ruby bases
are not. For example, foo is
read aloud as 'bar'.
Furigana, when ruby annotations read aloud
Even native Japanese speakers may easily assume, without
thorough consideration, that the option of reading only ruby
annotations aloud will provide reasonable results. However,
this is not always the case.
Incorrect pitch accent
Each hiragarana character represents a mora (a basic timing
unit in phonology), which is typically a single vowel or a
consonant followed by a single vowel. The same sequence of
moras may mean different words depending on the pitch accent.
For example, both 雨 (rain) and
飴 (candy) consists of the same moras:
あ and め.
However, if the Tokyo accent is used as a basis, the first mora in 雨 has a
high pitch, and the second has a low pitch;
飴 has the opposite pitch accent.
Reading aloud ruby annotations rather than ruby bases often
leads to incorrect pitch accent. As an example, consider
雨が好き
(I like rain) and 飴が好き
(I like candy). In both cases, reading aloud ruby annotations
rather than ruby bases implies that the TTS engine will receive
あめが好き and create the same result.
A similar example is 牡蠣を食べる
(I eat oysters) and 柿を食べる
(I eat persimmons), where 牡蠣 and
柿 have the same two moras but opposite
pitch accents.
Incorrectly pronouncing non-particle は or へ as particles
In modern Japanese, there is basically only one way to read
each hiragana character. But は and
へ are exceptions. は
is usually read aloud as /ha/ but is read aloud as /wa/ when
it is used as a particle. Likewise, へ
is usually read aloud as /he/ but is read aloud as /e/ when
it is used as a particle.
Reading aloud ruby annotations rather than ruby bases implies
that CJK ideographic characters in ruby bases will not be
passed to the TTS engine, only hiragana characters in ruby
annotations will be.
Without CJK ideographic characters, Japanese morphological
analysis is likely to fail. For example, やがてはいしになる may be
misinterepreted as やがては いしに なる ("I will eventually become a doctor") rather
than やがて はいしに なる ("It will be abolished eventually").
Occurrences of
は or へ as
non-particles in ruby annotations may well be mistakenly
interpreted as particles. Consequently, such occurrences
of は and へ
may well be mistakenly read aloud as /wa/ and /e/, respectively.
For example, consider
やがて廃止になる.
This sentence means "It will be abolished eventually". But
if やがてはいしになる is passed to the
TTS engine, は may well be mistakenly
read aloud as /wa/ rather than /ha/. The result means
"I will eventually become a doctor".
Here are some similar examples. All occurrences of
は and へ
in ruby annotations are likely to be mistakenly read aloud.
人員配置
自然破壊
社会波紋
天皇陛下
大学併願
学級閉鎖
Inconsistency between the first and subsequent occurrences
As described in 2.1,
furigana as a ruby annotation may be attached to only the
first occurrence of a CJK ideographic character or a word
composed from such characters. Thus, there is a risk that
the first occurrence and the others are read aloud differently.
For example, consider 智子 as the name
of a character in the novel. Tthere are several possible readings of this name, such as
さとこ and
ともこ. If さとこ
as a ruby annotation is attached only to the first occurrence
of the name, it will be read as さとこ
and the other occurrences may be read as ともこ.
The reader would then think that さとこ
and ともこ are different characters.
Gikun, when ruby annotations read aloud
The option of reading aloud ruby annotations only provides an understandable result but does not properly convey the author's intention.
敵 is read aloud as TOMO, which means 'friend', but 'frenemy' is intended.
生命 will be read aloud as INOCHI(いのち).
Unusual names of people and places, when ruby annotations read aloud
The option of reading aloud ruby annotations only works correctly.
However, if the first occurrence of a name is accompanied by a ruby annotation and the other occurrences are not,
the first occurrence is read aloud differently from the others thus suggesting different persons or places.
For example, 不死川玄弥
is read aloud as SHINAZUGAWA GENYA correctly.
But later occurrences of 不死川玄弥 are read aloud as FUSHIKAWA GENYA if they do not have ruby annotations.
Interlinear notes, when ruby annotations read aloud
The option of reading aloud ruby annotations only provides incomprehensible results often.
If "1837-1913 江戸幕府最後の将軍" is attached to 徳川慶喜 as a ruby annotation,
it will be read aloud as 1837-1913 EDOBAKUFU SAIGO NO SHOGUN
(1837-1913 the last shogun of the Edo shogunate), which is reasonable.
But if only "1837-1913" is attached as a ruby annotation, the result is 1837-1913, which does not make any sense.
Ruby annotations for indicating the pronunciation of foreign phrases in language books, when ruby annotations read aloud
The option of reading aloud ruby annotations only interferes with readers' understanding significantly.
In the example of 我去学校 (ウオ チュー シュエシャオ),
even if ウオ チュー シュエシャオ is read out in the Japanese style,
it will not be helpful to learners because of the inaccurate pronunciation and the four tones (tones).
Katakana pronunciation is also useless in languages such as English.
Double-sided ruby, when ruby annotations read aloud
The option of reading aloud ruby annotations only makes two ruby annotations be read aloud while ignoring their ruby base.
Since one of the two ruby annotations is typically furigana, the description in 3.2.1 applies.
If the other ruby annotation is a Gikun, the description in 3.2.2 applies;
if it is an interlinear note, the description in 3.2.4 applies.
Reading aloud ruby bases only
In this option, ruby bases are read aloud but ruby annotations are not.
For example, foo is read aloud as foo.
Furigana, when bases read aloud
The option of reading aloud ruby bases only may or may not provide good results, depending on text-to-speech engines.
The following is a quote from [[?ACCESSIBLE_E_BOOKS]].
Furthermore, compound words made up from CJK ideographic characters in JIS X 0208 are sometimes read aloud incorrectly.
As the importance of accessibility is well recognized and text-to-speech engines are improved,
more and more words will be read aloud correctly.
However, there are some words, such as the aforementioned YAMAZAKI,
that cannot be read aloud correctly by text-to-speech engines and even native Japanese speakers.
Gikun, when bases read aloud
The option of reading aloud ruby bases only results in a perfectly understandable result.
However, since gikun is ignored, the author's intent is not completely conveyed.
敵 is read aloud as TEKI, which means 'enemy', but 'frenemy' is intended.
生命 is read out as SEIMEI.
Unusual names of people and places, when bases read aloud
The option of reading ruby bases only leads to incorrect results.
However, since every occurrence of a name is read aloud in the same way, users will not be confused.
Every occurrence 不死川 玄弥
will always be incorrectly read aloud as FUSHIKAWA GENYA, regardless of the presence or absence of ruby annotations.
Interlinear notes, when bases read aloud
The option of reading ruby bases only provides a perfectly understandable result.
However, since interline notes are ignored, the author's intention is not conveyed well.
徳川慶喜
(Tokugawa Yoshinobu 1837-1913, the last shogun of the Edo shogunate),
will be read aloud as とくがわよしのぶ (Tokugawa Yoshinobu).
Ruby annotations for indicating the pronunciation of foreign phrases in language books, when bases read aloud
The option of reading ruby bases only is most appropriate when natural languages are correctly identified
and ruby bases are read aloud by a text-to-speech engine for that language.
On the other hand, if the natural language cannot be identified or the text-to-speech engine for that language is not available,
the result is not understandable.
Double-sided ruby, when bases read aloud
The option of reading ruby bases only will ignore the two ruby annotations and read their ruby base only.
When one of the two ruby annotations is furigana, the description in 3.3.1 applies.
If the other is a gikun, the description in 3.3.2 applies, and if it is an interlinear note, the description in 3.3.4 applies.
Miscellaneous issues around ruby markup
Conversion from small kana characters to full-size kana characters
Small kana characters ゃ, ゅ, ょ, and
っ are too small when they appear in ruby annotations.
For this reason, instead of these small characters, full-size kana characters や,
ゆ, よ, and つ are used in ruby annotations.
However, since full-size kana characters are pronounced differently from small kana,
ruby annotations containing full-size kana are read aloud differently.
CSS has a mechanism for overcoming this problem.
Value 'full-size-kana' of
the text-transform property as specified in CSS Text converts
small kana characters to full-size kana.
It is thus possible to use small kana in ruby annotations while rendering them using full-size kana.
Text-to-speech engines can provide correct results even when ruby annotations are read aloud.
A single ruby element or multiple ruby elements per one compound word
When attaching a ruby annotation to a compound word consisting of multiple CJK ideographic characters in an HTML or EPUB document, one way is to create a single HTML ruby element for the entire word. However, in some cases, a separate ruby element is created for each CJK ideographic character.
For example, to attach the ruby annotation せいめい to the word 生命 (meaning “life” in Japanese), the typical approach is to create a single ruby element for this word. This ruby element may have a single rt element for “せいめい” or two rt elements (one for “せい” and another for “めい”). However, it is not entirely uncommon to see two ruby elements for this word: one for “生” and another for “命”.
Some people argue that
creating a ruby element per compound word is better than creating a ruby element for each character in a compound word. They argue that
it becomes easier for the text-to-speech engine to maintain a correspondence table between ruby bases and ruby annotations so that subsequent occurrences of the compound word without ruby can be pronounced correctly.
Meanwhile, others argue that there is a good reason to attach ruby
annotations to some, but not all, characters in a compound word.
For example, consider 佳人, where
佳 is taught in junior high schools while
人 is taught in the first grade of elementary
schools. Therefore, it makse sense to attach a ruby annotation to
佳 only (one ruby element for
佳 and no ruby element for
人). Similarly, it is reasonable to attach ruby
annotations to the first and third CJK idegraphic characters in
屯田兵 only but not to the second one (thus,
two ruby elements).
Markup for indicating furigana-added-for-enhanced-accessibility
Although furigana-added-for-enhanced-accessibility is necessary for those readers who have particular
difficulties with CJK ideographic characters, it is unnecessary or slightly disturbing for others.
If furigana-added-for-enhanced-accessibility is distinguishable from normal furigana,
it can be made visible or invisible depending on user preferences.
It is thus necessary to standardize a markup mechanism for indicating furigana-added-for-enhanced-accessibility.
Markup for indicating ruby annotations used as gikun or interlinear note
In Section 3, we have seen that ruby annotations used as gikun or interline notes should be read aloud differently from the other cases.
It is thus necessary to standardize a markup mechanism for clearly indicating ruby annotations used as gikun or interlinear note.
Alternatives to ruby
[[?SSML]] and [[?PRONUNCIATION-LEXICON]] offer alternatives for
conveying phonemic and phonetic pronunciations of CJK ideographic
characters to speech synthesis engines. These methods are not intended for visual
presentations but can offer superior control over text-to-speech compared to using ruby annotations.
SSML
[[?SSML]] employs symbol collections (such as IPA and
[[?JEITA_IT-4006]]) to represent the sounds of human
languages. Phonemic and phonetic pronunciations are conveyed
through sequences of these symbols.
[[?epub-32]] allows the use of SSML attributes within
XHTML
content documents in EPUB publications. In [[?epub-33]],
these attributes are relocated to
[[?epub-tts-10]]. Meanwhile, the W3C Accessible Platform
Architectures Working Group is developing [[?spoken-html]],
which outlines two potential methods for incorporating SSML
attributes into HTML elements.
In Japan, SSML finds extensive application in digital
textbooks, adopted by the biggest textbook publisher in
Japan. However, it has been noted that attaching SSML
attributes to CJK ideographic characters significantly
raises authoring costs. In the case of DAISY textbooks in
Japan, SSML is not used, as they contain recorded voice.
Trade books in Japan do not typically employ SSML
either.
PLS
PLS ([[PRONUNCIATION-LEXICON]]) enables the use of pronunciation lexicons, which map words to
sequences of symbol collections such as those found in IPA or
[[?JEITA_IT-4006]].
While SSML attributes are embedded within XHTML content
documents in EPUB publications, PLS lexicons
in EPUB publications are stored
externally to and referenced by XHTML content documents
(see Pronunciation Lexicons
section in [[?epub-tts-10]]). As of the present,
[[spoken-html]] does not offer a mechanism for associationg
PLS lexicons with HTML documents.
PLS is a robust tool for rendering unusual names of people
and places in text-to-speech applications. In particular, PLS allows
every occurrence of a word or phrase to be consistently pronounced,
regardless of the presence of ruby annotations. At the time of this writing, PLS is
used by at least one digital textbook publisher in Japan.
Use of ruby for automatic braille translation
The conversion of HTML documents and EPUB publications to
braille is expected to become increasingly important in the
near future.
Japanese braille lacks CJK ideographic characters and does not
distinguish between hiragana and katakana. (Note: Han braille
in Japan does include CJK ideographic characters, but it is
not widely used.)
Braille exhibits some syntactical differences from the
Japanese writing system. First, space characters are inserted
as delimiters between words. Second, two Japanese particles,
は and へ, are transcribed as they are pronounced, meaning
は and へ are represented as if they were
わ and え,
respectively. Third, う pronounced as an elongated sound is
represented using the long vowel character. For example,
to tranlsate たいよう to braille,
たいよう is first converted to たいよー and then translated to braille.
Natural language processing is required to handle these
differences during the conversion to braille. However, unlike
the case of text-to-speech, intonation is not relevant.
When converting HTML or EPUB content to braille, it is
essential to select the correct reading for each CJK
ideographic character. Choosing an incorrect reading can
result in erroneous braille output. Similar to text-to-speech,
ruby annotations provide valuable hints, while [[?SSML]] and PLS
([[?PRONUNCIATION-LEXICON]]) serve as effective alternatives.
For furigana and the transcription of unusual names of people
and places, natural language processing is more effective when
using ruby bases (typically containing CJK ideographic
characters) as the foundation. In contrast, the correct
readings are chosen when using ruby annotations as the
basis. It is also possible to combine both ruby bases and ruby
annotations.
