This document describes and prioritises gaps for the support of Arabic and Persian languages on the Web and in eBooks. In particular, it is concerned with text layout. It checks that needed features are supported in W3C specifications, in particular HTML and CSS and those relating to digital publications. It also checks whether the features have been implemented in browsers and ereaders. This is a preliminary analysis.

This document describes and prioritises gaps for the support of Arabic and Persian languages on the Web and in eBooks. In particular, it is concerned with text layout. It checks that needed features are supported in W3C specifications, in particular HTML and CSS and those relating to digital publications. It also checks whether the features have been implemented in browsers and ereaders. This document complements the document Text Layout Requirements for the Arabic Script, which describes the requirements for areas where gaps appear. It is linked to from the language matrix that tracks Web support for many languages.

The editor's draft of this document is being developed by the Arabic Layout Task Force, part of the W3C Internationalization Interest Group. It is published by the Internationalization Working Group. The end target for this document is a Working Group Note.

Sending comments on this document

If you wish to make comments regarding this document, please raise them as github issues. Only send comments by email if you are unable to raise issues on github (see links below). All comments are welcome.

To make it easier to track comments, please raise separate issues or emails for each comment, and point to the section you are commenting on  using a URL.

A summary of this report and others can be found as part of the language matrix.

Introduction

The W3C needs to make sure that the text layout and typographic needs of scripts and languages around the world are built in to technologies such as HTML, CSS, SVG, etc. so that Web pages and eBooks can look and behave as people expect around the world.

This page documents issues for a given script or language in terms of support by specifications or user agents (browsers, e-readers, etc.). A summary of this report and others can be found as part of the language matrix.

A summary of this report and others can be found as part of the language matrix.

Work flow

This version of the document is a preliminary analysis

Gap analysis work usually starts with a preliminary analysis, conducted quickly by one or a small group of experts. Then a more detailed analysis is carried out, involving a wider range of experts. The detailed analysis may involve the development of tests, in order to illustrate issues and track results for browsers. The next phase is ongoing maintenance. It is expected that the resulting document will not be frozen: as gaps are fixed, this should be noted in the document. It is also possible that new gaps are noticed or arise, and they can be added to this document when that happens.

As the gap analysis develops, the requirements for features that are problematic should be described in the companion document, Text Layout Requirements for the Arabic Script. Links to the appropriate part of that document should be added to this document as the material is created. Note that the requirements document should not contain any technology-specific information: all of that belongs here.

Prioritization

This document not only describes gaps, it also attempts to prioritise them in terms of the impact on the local user. The prioritisation is indicated by colour.

Key:

It is important to note that these colours do not indicate to what extent a particular features is broken. They indicate the impact of a broken or missing feature on the content author or end user.

Basic styling is the level that would be generally accepted as sufficient for most Web pages. Advanced level support would include additional features one might expect to include in ebooks or other advanced typographic formats. There may be features of a script or language that are not supported on the Web, but that are not generally regarded as necessary (usually archaic or obscure features). In this case, the feature can be described here, but the status should be marked as OK.

The decision as to what priority level is assigned to a described gap is down to the experts doing the gap analysis. It may not always be straightforward to decide. If a given section in this document refers to more than one feature that is broken, each with different impacts on Web users, the priority for the section should be the lowest denominator.

A cell can be scored as OK if the feature in question is specified in an appropriate specification, and is supported by user agents. A specification that is in CR or later and has two implementations in 'major' browsers will count. This means that the feature may not be supported in all browsers yet. (At some point in the future we may try to distinguish, visually, whether support is available in a specification but still pending in major browsers or applications.)

Text direction

See also General page layout & progression for features such as column layout, page turning direction, etc. that are affected by text direction.

Vertical text

Are the script requirements for vertically oriented text met? What about if you mix vertical text with scripts that are normally only horizontal? Do you need a switch to use different characters in vertical vs. horizontal text? Does the browser support short runs of horizontal text in vertical lines (tate-chu-yoko in Japanese) as expected? Is the orientation of characters and the directional ordering of characters supported as needed? See available information or check for currently needed data.

We need to clarify whether there is a particular requirement for handling arabic text specially in vertical lines, such as upright glyphs. We are also waiting on implementation of sideways values of writing-modes in order to be able to effectively use arabic text in vertical arrangements (such as book spines, table headings, etc), but that is not a problem specific to arabic.

Bidirectional text

If this script runs right-to-left, are there any issues when handling that? Is bidirectional text adequately supported? What about numbers and expressions? Do the Unicode bidi controls and HTML markup provide the support needed? Is isolation of directional runs problematic? See available information or check for currently needed data.

Isolation not fully supported

Edge still doesn't support directional isolation.

Base direction needed for string data

There are questions about how to transmit information about the expected directional behaviour of certain strings.

RLI, LRI, etc are still not well supported.

Logical CSS keyword support needed

There needs to be wider adoption of logical keywords such as start and end, rather than left and right.

Characters and phrases

Characters & encoding

Are there any character repertoire issues preventing use of this script on the Web? Do variation selectors need attention? Are there any other encoding-related issues?

Fonts

Do the standard fallback fonts used in browsers (eg. serif, sans-serif, cursive, etc.) match expectations? Are special font or OpenType features needed for this script that are not available? See available information or check for currently needed data.

Font styles, weight, etc

This covers ways of modifying the glyphs, such as for italicisation, bolding, oblique, etc. Do italic fonts lean in the right direction? Is synthesised italicisation problematic? Are there other problems relating to bolding or italicisation - perhaps relating to generalised assumptions of applicability? See available information or check for currently needed data.

Controls needed for italic slant direction

There should be means available to control the direction in which 'italicised' or 'oblique' text slants, since in some schools of Arabic script typography, text in these styles need to slant to the left.

Glyph shaping and positioning

Does the script in question require additional user control features to support alterations to the position or shape of glyphs, for example adjusting the distance between the base text and diacritics, or changing the glyphs used in a systematic way? Do you need to be able to compose/decompose conjuncts, or show characters that are otherwise hidden, etc? See available information or check for currently needed data.

Controls needed to adjust diacritic positioning

For advanced typographic purposes, the fixed position of diacritical marks relative to base letter or baseline and the logical stacking behavior of them might not be sufficient. Therefore, it should be possible to adjust the positioning and combining order and changes to the individual marks when combining. Fonts and other systems may implement the logic needed to result in an optimal presentation of diacritical mark clusters by default.

Cursive text

If this script is cursive (eg. Arabic, N'Ko, Syriac, etc), are there problems or needed features related to the handling of cursive text? Do cursive links break if parts of a word are marked up or styled? Do Unicode joiner and non-joiner characters behave as expected? See available information or check for currently needed data.

The Arabic script is cursive. This means that letters appear in different forms depending on how they join with their neighbors. Majority of modern software implementations treat Arabic text accordingly. However, for few advanced typographic behaviours, additional considerations are required; for example, adding color highlighting to individual letterforms in a word context without disrupting the joining behaviour, application of text outlining, etc.

Correct and incorrect renderings of text shadow.
Correct and incorrect renderings of text shadow.

Arabic Layout Requirement document includes a section dedicated to this topic.

Text shadow works as expected

The text-shadow property in CSS doesn't disrupt the flow of cursive text in any major browser. See a test.

Text stroke cuts joining glyphs apart

The property text-stroke is not yet in CSS, but has been implemented in major browsers under the name -webkit-text-stroke. In all major browsers, the stroke around the text interrupts the cursive flow. See a test.

Text opacity shows glyph overlap

When opacity is applied to text, Firefox and Chrome produce dark patches where the cursive glyphs overlap, but Safari and Edge don't. See the test.

Inline elements break cursive shaping

When elements surround part of a cursive run of text, and apply styling, the results often break the cursive joins. (See the results of trying to colour individual letters in the illustration below – successful on the left, unsuccessful on the right.)

After some discussion, the CSS spec requires the following:

  1. Markup alone around part of a joined up sequence must not disturb the joining behaviour.
  2. Styling that doesn't affect the characters, such as text-decoration, must not break the joins.
  3. Styling that does affect the shape of the characters should not break the joins, however the result is not well defined for complex glyph arrangements such as ligatures where the markup occurs between characters that make up the ligature.
  4. Non-zero margins, padding, and borders, will break the join, as will isolation boundaries.

See the tests and results for a range of different styling effects.

Chrome and Safari break cursive joining as soon as markup appears around a character, and so obviously fail for any type of styling application, too.

Firefox and Edge keep joins for styling that doesn't affect the shape of the characters (eg. text-decoration), and keeps it for colour changes, however they fail for changes in font-weight, font-style, and font-size, as well as for markup such as `em` and `b` tags.

It would be useful to decide on the potential impact of the failures described here, so as to prioritise the issue. Is the inability to surround/style parts of a word a significant issue? It may be problematic when defining a term (using `dfn`) if the term is only part of the run of letters between spaces, eg. after the definite article.

Transforming characters

Does your script need special text transforms that are not supported? Does your script convert letters to uppercase, capitalised and lowercase alternatives according to your typographic needs? Do you need to to convert between half-width and full-width presentation forms? See available information or check for currently needed data.

Text segmentation & selection

This is about how text is divided into graphemes, words, sentences, etc., and behaviour associated with that. Do Unicode grapheme clusters appropriately segment character units for your script? When you double- or triple-click on the text, is the expected range of characters highlighted? When you move through the text with the cursor, or backspace, etc. do you see the expected behaviour? (Some of the answers to these questions may be picker up in other sections, such as line-breaking, or initial-letter styling.) See available information or check for currently needed data.

Arabic script word boundaries, similar to Latin script, generally can be distinguished by white space and a specific subset of punctuations. There are few exceptions which are listed in Arabic Layout Requirement document.

To enforce a disjoining behavior between the letters which under circumstances of their position normally join, special Unicode character U+200C ZERO WIDTH NON-JOINER (ZWNJ) is used. This usage is detailed in Arabic Layout Requirement document.

All major desktop browsers select a whole word when you double-click on it, including when the word contains ZWNJ. Single-letter prepositions attached to the word are also selected. See some tests.

Text decoration

This is about ways of marking text (see also specific sections dedicated to quotations and inline notes/annotations). Is it possible to express emphasis or highlight content as expected? Bold, italic and under-/over-lines are not always appropriate, and some scripts have their own unique ways of doing things, that are not in the Western tradition at all. Text delimiters mark certain items or sections off from the main text, such as book names in Chinese, quotations, head markers in Tibetan, etc, and often involve the use of punctuation. Is there any behaviour that isn't well supported, such as overlines for numeric digits in Syriac? Are there issues about the positioning or use of underlines? Some aspects related to the drawing of lines alongside or through text involve local typographic considerations. Do underlines need to be broken in special ways for this script? Do you need support for additional line shapes or widths? Does the distance or position of the lines relative to the text need to vary in ways that are not achievable? Are lines correctly drawn relative to vertical text? See available information or check for currently needed data.

Underlines too close to text

It must be possible to position under- and overlines further away from the baseline than for Latin text. There is not currently a way to achieve effective underlining in a way that works with the Arabic script.

Quotations

Are there any issues when dealing with quotations marks, especially when nested? Should block quotes be indented or handled specially? See available information or check for currently needed data.

Default quotation marks for q element

If the html tag sets the language of a page, the HTML specification says that the q element should by default produce quotation marks according to the information in CLDR for that language.

For Arabic, the default quote marks should be, reading right to left, “...”, and embedded quote marks ‘...’. Firefox produces these but facing in the wrong direction. Chrome & Safari are ok. See a test and results.

For Persian, the default quote marks should be, reading right to left, «...», and embedded quote marks ‹...›. Again, Chrome & Safari are ok, but Firefox produces the wrong quotation marks, since it doesn't change them according to the language set. See a test and results.

Default quotation marks in a new language section

In addition, the default quotation marks for the q element are not set to the appropriate characters by the browser when the element appears inside an Arabic/Persian section of a page that has a different overall language. This is currently per the HTML specification (both W3C and WhatWG). There is an issue raised against the WhatWG version for this to be changed.

Embedded quotations in a different language

Arabic/Persian quotes embedded in text in another language can also be problematic if the outer language uses different quotation marks. This is due to the browsers choosing default quotation marks based on the language of the quotation, rather than that of the surrounding text. See a test. This behaviour is specified in the WhatWG version of the HTML spec, but no longer in the W3C version. There is an issue raised against the WhatWG version for this to be changed.

Marking all the above as advanced, because use of the q element is optional (quote characters can be used instead), and it can be styled using CSS for the general case.

Inline notes & annotations

The ruby spec currently specifies an initial subset of requirements for fine-tuning the typography of phonetic and semantic annotations of East Asian text, including furigana, pinyin and zhuyin fuhao systems. Is is adequate for what it sets out to do? What other controls will be needed in the future? What about other types of inline annotation, such as warichu? (For referent-type notes such as footnotes, see below.) See available information or check for currently needed data.

Numbers, dates, etc.

If the script has its own set of number digits, are there any issues in how they are used? Does the script or language use special format patterns that are problematic (eg. 12,34,000 in India)? What about date/time formats and selection - and are non-Gregorian calendars needed? Do percent signs and other symbols associated with number work correctly, and do numbers need special decorations, (like in Ethiopic or Syriac)? See available information or check for currently needed data.

Internationalization software libraries identify Arabic-Indic numerals as the set of numerals which should be used with text in Arabic language excluding a number of Arabic-speaking countries of Northern and Northwestern Africa which should use Arabic (ASCII) numerals. Surveys of publications, monetary and governmental documents, and manuscripts confirm these precedences. However, there is a considerable diverging trend from these recommendations observable on the web, digital products, and in user-generated content. If not considered in design and implementation of software products dealing with Arabic text, this discrepency could be potentially disadvantageous to the quality of text layout, digital typography, and locale-specific data processing.

Other inline features

Does the script have special ways of representing inline notes (such as kumimoji in Japanese) or other inline features that need to be supported? See available information or check for currently needed data.

Lines and Paragraphs

Line breaking

Does the browser capture the rules about the way text in your script wraps when it hits the end of a line? Does line-breaking wrap whole 'words' at a time, or characters, or something else (such as syllables in Tibetan and Javanese)? What characters should not appear at the end or start of a line, and what should be done to prevent that? See available information or check for currently needed data.

Hyphenation

Is hyphenation used for your script, or something else? If hyphenation is used, does it work as expected? (Note, this is about line-end hyphenation when text is wrapped, rather than use of the hyphen and related characters as punctuation marks.) See available information or check for currently needed data.

No hyphenation is used for Arabic when used with Arabic and Persian languages.

Text alignment & justification

When text in a paragraph needs to have flush lines down both sides, does it follow the rules for your script? Does the script need assistance to conform to a grid pattern? Does your script allow punctuation to hang outside the text box at the start or end of a line? Where adjustments are need to make a line flush, how is that done? Do you shrink/stretch space between words and/or letters? Are word baselines stretched, as in Arabic? What about paragraph indents, or the need for logical alignment keywords, such as start/end, rather than left/right? See available information or check for currently needed data.

Need better control of justification mechanisms

For justification of Arabic script text, there are various common strategies available. These could be categorized in two major groups; strategies based on adjusting inter-word or inter-character whitespace and strategies based on adjusting the letterforms.

A basic implementation must provide at least one of these strategies for adequate justification results. Advanced implementations should provide users with the necessary means to control the selection of strategies, adjustment of their attributes, and the priority with which they are being applied.

Currently, CSS specifications do not provide these advanced features, but recommend that the implementations select the justification strategy appropriate to the text.

Arabic Layout Requirement document includes a section dedicated to this topic.

Word & letter spacing

Some scripts create emphasis or other effects by spacing out the words, letters or syllables in a word. Are there requirements for this script/language that are unsupported? (For justification related spacing, see below.) See available information or check for currently needed data.

Need control of baseline stretching

It is very common in Arabic script to stretch words or phrases to a particular width (eg. to match a Latin translation or transcription above or below). This is achieved by lengthening the connections between letters, and to some extent by use of wide glyph variants, etc. The rules for which part of the text to stretch and how far are complicated - this is not the even spacing that usually occurs in tracked Latin text. There are currently no mechanisms for managing this process effectively in HTML/CSS.

Counters, lists, etc.

The CSS Counter Styles specification describes a limited set of simple and complex styles for counters to be used in list numbering, chapter heading numbering, etc.The rules plus more counter styles (totalling around 120 for over 30 scripts) are listed in the document Ready-made Counter Styles. Do these cover your needs? Are the details correct? Are there other aspects related to counters and lists that need to be addressed? See available information or check for currently needed data.

Add support for custom counter-styles

Arabic script text uses local counter styles. While some of these are supported by some browsers, the set of symbols used and their order varies by language. Custom counter styles can be created using Firefox only. The CSS spec needs to become a Rec and more browsers need to support it.

Styling initials

Does the browser or ereader correctly handle special styling of the initial letter of a line or paragraph, such as for drop caps or similar? How about the size relationship between the large letter and the lines alongide? where does the large letter anchor relative to the lines alongside? is it normal to include initial quote marks in the large letter? is the large letter really a syllable? etc. Are all of these things working as expected? .See available information or check for currently needed data.

Although initial letter styling is not an innate feature of the Arabic script, there have been occurences of its usage noted. However, the specifications and guidelines for composition of these decorative elements are undefined or insubstantial; for example, which of the joining forms of letters is to be used or how the joining behaviour is treated accross the boundary between the styled initial letter and rest of the paragraph.

Baselines & inline alignment

Does the browser support requirements for baseline alignment between mixed scripts and in general? See available information or check for currently needed data.

Other paragraph features

In your script, is the first line of text typically indented at the start of a paragraph? Are there other features of paragraph design that are peculiar to your script? See available information or check for currently needed data.

Page & book layout

General page layout & progression

How are the main text area and ancilliary areas positioned and defined? Are there any special requirements here, such as dimensions in characters for the Japanese kihon hanmen? The book cover for scripts that are read right-to-left scripts is on the right of the spine, rather than the left. Is that provided for? When content can flow vertically and to the left or right, how do you specify the location of objects, text, etc. relative to the flow? For example, keywords 'left' and 'right' are likely to need to be reversed for pages written in English and page written in Arabic. Do tables and grid layouts work as expected? How do columns work in vertical text? Can you mix block of vertical and horizontal text correctly? Does text scroll in the expected direction? See available information or check for currently needed data.

Notes, footnotes, etc.

Does your script have special requirements for notes, footnotes, endnotes or other necessary annotations of this kind in the way needed for your culture? (There is a section above for purely inline annotations, such as ruby or warichu. This section is more about annotation systems that separate the reference marks and the content of the notes.) See available information or check for currently needed data.

Page numbering, running headers, etc.

Are there special conventions for page numbering, or the way that running headers and the like are handled? See available information or check for currently needed data.

Forms & user interaction

Are vertical form controls well supported? In right-to-left scripts, is it possible to set the base direction for a form field? Is the scroll bar on the correct side? etc. See available information or check for currently needed data.

Other page layout & pagination features

Some cultures define text areas and page progression direction very differently from those in the West (eg. kihon hanmen in Japanese). Is this an issue for you? Are widows and orphans relevant? If pages progress RTL, are there issues for support with paged media? What about things such as cross-references, bookmarks, columns, printer marks, tables of contents and indexes? See available information or check for currently needed data.

Other

Culture-specific features

Sometimes a script or language does things that are not common outside of its sphere of influence. This is a loose bag of additional items that weren't previously mentioned. This section may also be relevant for observations related to locale formats (such as number, date, currency, format support).

What else?

There are many other CSS modules which may need review for script-specific requirements, not to mention the SVG, HTML, Speech, MathML and other specifications. What else is likely to cause problems for worldwide deployment of the Web, and what requirements need to be addressed to make the Web function well locally?

Show summary

Acknowledgements

Special thanks to the following people who contributed to this document (contributors' names listed in in alphabetic order).

This Person, That Person, etc

Please find the latest info of the contributors at the GitHub contributors list.