How can I use language metadata in native APIs?

Sometimes a program receives data from the Web that includes important metadata about the language of a string or a range of text. Developers working with document formats and protocols that include language metadata should preserve it and pass it to the native text APIs that process or display the associated text. This article provides links to documentation for several operating systems, programming environments, and text libraries.

Language metadata in this article means the language of the text itself. It is different from metadata about the language of the intended audience, and it is different from the user's preferred language. For that distinction, see Types of language declaration. For an introduction to language and direction metadata on the Web, see Strings on the Web: Language and Direction Metadata.

Principles

Language tags are normally BCP 47 language tags, such as fr, sr-Latn, or zh-Hant-TW. Treat the tag as data that belongs to the text and keep the whole tag when passing it between layers.

Keep the scope. If only one phrase changes language, attach the metadata to that range instead of changing the language of the whole view or document.
Choose the API for the operation. Text layout APIs use language to select glyphs, fonts, or line-breaking rules. Speech APIs use it to select a voice. Formatting and collation APIs use locale data to generate or compare text.
Keep direction separate. A language tag does not replace a base-direction or string-direction value. Supply both when possible. See How can I use direction metadata in native APIs?.

APIs

This section links to APIs for applying language metadata to text ranges, controls, and speech engines.

The list is necessarily incomplete: the Internationalization Working Group welcomes contributions and corrections. Click on the link Leave a comment in the Tell us what you think box below.

Apple Core Text:
- kCTLanguageAttributeName applies a locale identifier to a range. Core Text uses it for localized glyph selection and locale-specific line breaking.
- CTFontCreateForStringWithLanguage chooses a font for a string range using a language identifier.
Android:
- LocaleSpan
- TextView.setTextLocale and setTextLocales
- TextToSpeech.setLanguage selects the closest available language for speech.
ICU4C:
- icu::Locale::forLanguageTag parses a BCP 47 language tag. Pass the resulting locale to ICU boundary-analysis, collation, and formatting APIs as appropriate for the operation.
Java AWT and Swing:
- Component.setLocale is inherited by Container and its descendants, including Swing components. Use it to associate a locale with a component or container.
- Locale.forLanguageTag creates a Locale from a BCP 47 language tag.

Example: a language range in Android

Android's LocaleSpan is useful when one TextView contains text in more than one language.

Locale language = Locale.forLanguageTag("zh-Hant-TW");
SpannableString styled = new SpannableString(text);
styled.setSpan(new LocaleSpan(language), 0, styled.length(),
        Spanned.SPAN_EXCLUSIVE_EXCLUSIVE);
textView.setText(styled);

If the whole control has one language, TextView.setTextLocale or setTextLocales is the appropriate default. For speech, pass the language to TextToSpeech.setLanguage and check its return value.

Principles

APIs

Example: a language range in Android

Further reading