This specification specifies an Hypothetical Render Model (HRM) that constrains the presentation complexity of documents that conform to any of the TTML Profiles for Internet Media Subtitles and Captions ([[IMSC]]).
The model is not intended as a specification of the processing requirements for implementations. For instance, while the model defines glyph cache for the purpose of modelling how the number of glyph drawing operations can be reduced, it neither requires the implementation of such a cache, nor models the sub-pixel glyph positioning and anti-aliased glyph rendering that can be used to produce text output.
Furthermore, the model is not intended to constrain readability complexity.
This specification specifies an Hypothetical Render Model (HRM) that constrains the presentation complexity of a IMSC Document Instance.
This specification uses the same conventions as [[!IMSC]].
character. The character code property of a Character Information Item.
The term character is for practical purposes the same as a code point, as defined by [[?i18n-glossary]].
empty ISD. An Intermediate Synchronic Document with no presented region.
non-empty ISD. An Intermediate Synchronic Document with at least one presented region.
error. A failure to conform to the constraints defined by this specification.
IMSC Document Instance. A Document Instance that conforms to any profile defined in any edition of [[!IMSC]].
Unless noted otherwise, this specification applies to an IMSC Document Instance.
A IMSC Document Instance conforms to the Hypothetical Render Model if the sequence of Intermediate Synchronic Documents generated from it using the Intermediate Synchronic Document Construction procedure specified in [[!TTML2]] is processed without error by the HRM algorithm specified at .
Applying the Hypothetical Render Model to a Document Instance that is not an IMSC Document Instance yields results that might not reflect the complexity of the Document Instance.
In applications where sequences of Document Instances can be resolved into a single sequence of Intermediate Synchronic Documents that do not overlap each other temporally, conformance can be determined based on a synthesised Document Instance that generates an equivalent sequence of Intermediate Synchronic Documents, where minimal equivalence is limited to the content and metrics that are used to identify errors.
The objective of the HRM is to allow subtitle and caption authors and providers to verify that the content they provide does not exceed defined complexity levels, so that playback systems can render the content synchronized with the author-specified display times.
Playback systems include desktop computers, mobile devices and home theatre devices.
The HRM is not a new concept: it has been included in all versions and editions of [[IMSC]] and has remained substantially unchanged. It is refactored herein to simplify document maintenance.
IMSC Document Instances are typically authored by a first party and rendered by a second party. Unless both parties agree on the maximum complexity of a IMSC Document Instance, it is likely that:
As illustrated in , by defining a method (the HRM) to compute a proxy for the complexity of an IMSC Document Instance and specifying a complexity limit based on such proxy:
The HRM supplements the syntactic and structural constraints imposed in [[IMSC]] by imposing constraints on the contents of the presentation.
Because of the temporal and spatial variability of subtitles and captions across types of content, territories and languages, it is not possible to limit the complexity of an IMSC Document Instance using only average values.
An average-based constraint of 840 characters per minute could be met in multiple ways, with different rendering complexities. Contrast two potential approaches:
In the first, 5 characters are presented for a fraction of a second, followed by 835 characters that are then presented for over 59 seconds. This generates a high rendering complexity for the 835 characters, since there is only a brief time available to paint them.
In the second, 210 characters are painted every 15 seconds, giving 15 seconds to prepare for the next presentation. This has a much lower rendering complexity.
The HRM achieves a more accurate representation of the complexity of an IMSC Document Instance at any given time by taking into account its past complexity in addition to its instantaneous complexity. The same approach is commonly used in video to limit bitstream complexity, e.g., the Hypothetical Reference Decoder (HRD) specified in [[iso14496-10]].
The HRM defines a simple model for the rendering of subtitles and captions, and uses the time it takes to render subtitles and captions according to that model as a proxy for the complexity of the subtitles and captions. Rendering includes drawing region backgrounds, rendering and copying text, and decoding and copying images. Complexity is then limited by requiring that the time to render one subtitle or caption is shorter than the time elapsed since the previous subtitle or caption.
This simple model requires only a static analysis of the IMSC Document Instance, requires no fetching of external resources and does not require the IMSC Document Instance to be actually rendered. Several simplifying assumptions are made to achieve this. For example, the model assumes that each character is drawn independently, and accounts for that assumption being, in many cases, false, by assigning different render speeds for different scripts. In general the model is not intended to capture the actual time that an implementation takes to render subtitles and captions, but rather scale with it: a document that is twice as complex according to the model would require roughly twice as many resources to actually render.
The HRM is typically used prior to distribution of the IMSC Document Instance to the end-user, as an integral part of authoring and as a quality check before distribution.
When the HRM is used, the consequences of an IMSC Document Instance exceeding the HRM limits depends on the context:
The HRM is not intended to be used when the IMSC Document Instance is presented to end-users since:
The HRM, illustrated in , operates on a sequence of Intermediate Synchronic Documents Ei:
The model specifies a (hypothetical) time required for completely painting a non-empty ISD as a proxy for complexity. Painting includes clearing the Back Buffer, drawing region backgrounds, rendering and copying glyphs, and decoding and copying images. Complexity is then limited by requiring that painting of non-empty ISD En begins no earlier than the presentation time of the previous non-empty non-empty ISD Em and completes by the presentation time of En.
In contrast, there is no complexity involved connecting and disconnecting the Front Buffer from the display, and thus no complexity associated with empty ISDs.
Whenever applicable, constraints are specified relative to Root Container Region dimensions, allowing subtitle sequences to be authored independently of Related Video Object resolution.
To enable scenarios where the same glyphs are used in multiple successive Intermediate Synchronic Documents, e.g. to convey a CEA-608/708-style roll-up (see [[CEA-608]] and [[CEA-708]]), a Glyph Cache stores rendered glyphs across Intermediate Synchronic Documents, allowing glyphs to be copied into the Presentation Buffer instead of rendered, a more costly operation.
Similarly, a Decoded Image Cache stores decoded images across Intermediate Synchronic Documents, allowing images to be copied into the Presentation Buffer instead of decoded.
Each of the terms Presentation Compositor, Glyph Renderer, Glyph Copier, Image Decoder, and Image Copier is defined by the algorithmic requirements defined for it in this specification.
The HRM algorithm processes a sequence of Intermediate Synchronic Documents En.
Each successive non-empty ISD En is rendered by the Presentation Compositor using the following steps in order:
The Presentation Compositor begins rendering En:
The Presentation Compositor never begins rendering an ISD more than IPD ahead of its presentation time.
The duration DUR(En) for painting an Intermediate Synchronic Document En in the Back Buffer is given by:
DUR(En) = S(En) / BDraw + DURT(En) + DURI(En)
The contents of the Back Buffer are transferred instantaneously to the Front Buffer at the presentation time of a non-empty ISD En, making the latter available for display.
The Front Buffer is:
It is possible for the contents of the Front Buffer to never be displayed. This can happen, for example, if the Back Buffer is copied twice to Front Buffer between two consecutive video frame boundaries of the Related Video Object.
It SHALL be an error for the Presentation Compositor to fail to complete painting pixels for non-empty ISD En before its presentation time.
The following table specifies the values of IPD and BDraw.
|Initial Painting Delay (IPD)||1 s|
|Normalized background drawing performance factor (BDraw)||12 s-1|
BDraw effectively sets a limit on fillings regions - for example, assuming that the Root Container Region is ultimately rendered at 1920×1080 resolution, a BDraw of 12 s-1 would correspond to a fill rate of 1920×1080×12/s=23.7×220pixels s-1.
IPD effectively sets a limit on the complexity of any given Intermediate Synchronic Document.
The total normalized drawing area S(En) for Intermediate Synchronic Document En is given by:
S(En) = CLEAR(En) + PAINT(En )
where CLEAR(En) = 1.
To ensure consistency of the Back Buffer, a new Intermediate Synchronic Document requires clearing of the Root Container Region.
PAINT(En) is the normalized area to be painted for all regions that are used in Intermediate Synchronic Document En according to:
PAINT(En) = ∑Ri∈Rp NSIZE(Ri) ∙ NBG(Ri)
where Rp is the set of presented regions in the Intermediate Synchronic Document En.
NSIZE(Ri) is given by:
NSIZE(Ri) = (width of Ri ∙ height of Ri ) ÷ (Root Container Region height ∙ Root Container Region width)
NBG(Ri) is the total number of elements within the tree rooted at region Ri that satisfy the following criteria:
An element and its parent that satisfy the criteria above and share identical computed values of
tts:backgroundColor are counted as two distinct elements for the purpose of computing NBG(Ri).
set element is not included in the computation of NBG(Ri). While it can affect the
computed values of
tts:backgroundColor, it is removed during Intermediate Synchronic Document
The Presentation Compositor paints into the Back Buffer all visible pixels of presented images of Intermediate Synchronic Document En.
For each presented image, the Presentation Compositor either:
Two images are identical if and only if they reference the same encoded image source.
The duration DURI(En) for painting images of an Intermediate Synchronic Document En in the Back Buffer is given by:
DURI(En) = ∑Ii ∈ Ic NRGA(Ii) / ICpy + ∑Ij ∈ Id NSIZ(Ij) / IDec
NRGA(Ii) is the Normalized Image Area of presented image Ii and is given by:
NRGA(Ii)= (width of Ii ∙ height of Ii ) ÷ ( Root Container Region height ∙ Root Container Region width )
NSIZ(Ii) is the number of pixels of presented image Ii.
At the presentation time of Intermediate Synchronic Document En, perform the following steps in order:
It SHALL be an error if the sum of the Normalized Image Area over all images flagged retain in the Decoded Image Cache is at any time greater than the Normalized Decoded Image Cache Size (NDIBS).
The abbreviation NDIBS reflects the name of the Decoded Image Cache from earlier editions of the specification.
The following table specifies the values of ICpy, IDec, and NDBIS.
|Normalized image copy performance factor (ICpy)||6|
|Image Decoding rate (IDec)||1 × 220 pixels s-1|
|Normalized Decoded Image Cache Size (NDIBS)||0.9885|
In the context of this section, a glyph is a tuple consisting of (i) one character and (ii) the computed values of the following style properties:
In the case where a property is prohibited in a profile of [[IMSC]], the computed value of the property specified in [[ttml2]] can be used.
The Hypothetical Render Model defines a one-to-one mapping between characters and glyphs (using the definition of glyph from this document). While a one-to-one mapping between code points and glyphs (using the definition of glyph from [[?i18n-glossary]]) is common in some scripts (such as the Latin script), the actual relationship is more complex. Some scripts, such as Arabic, use different glyphs for a given character, depending on its position in a word. Some scripts require combining marks or use a sequence of code points to form a glyph. Cases exist where a given sequence of code points can have different glyph representations depending on context. This complexity is accounted for by reducing the performance of the Glyph Cache for scripts where a one-to-one mapping is not the general rule (see GCpy below).
For each glyph associated with a character in a presented region of Intermediate Synchronic Document En, the Presentation Compositor:
The duration DURT(En) for rendering the text of an Intermediate Synchronic Document En in the Back Buffer is as follows:
DURT(En) = ∑gi ∈ Γr NRGA(gi) / Ren(gi) + ∑gj ∈ Γc NRGA(gj) / GCpy
The Normalized Rendered Glyph Area NRGA(gi) of a glyph gi is given by:
NRGA(gi) = (fontSize of gi as a decimal fraction of Root Container Region height)2
NRGA(Gi) does not take into account decorations (e.g. underline), effects (e.g. outline) or actual typographical glyph aspect ratio. An implementation can determine an actual cache size needs based on worst-case glyph size complexity.
At the presentation time of Intermediate Synchronic Document En, perform the following steps in order:
It SHALL be an error if the sum of NRGA(gi) over all glyphs flagged retain in the Glyph Cache is at any time larger than the Normalized Glyph Cache Size (NGBS).
The abbreviation NGBS reflects the name of the Glyph Cache from earlier editions of the specification.
Unless specified otherwise, the following table specifies values of GCpy, Ren and NGBS.
|Normalized glyph copy performance factor (GCpy)|
|Script property, as defined at [[!UAX24]], for the character of gi||GCpy|
|any other value||3|
|Text rendering performance factor Ren(Gi)|
|Script property, as defined at [[!UAX24]], for the character of gi||Ren(Gi)|
|any other value||1.2|
|Normalized Glyph Cache Size (NGBS)|
While DURT(En) is not affected, the choice of font by the presentation processor can increase actual rendering
complexity at time of presentation. For instance, a cursive font might select different glyphs for a given grapheme (in order to maintain joining or for the start/end of the
word) even in the
Latin script. Conversely the rendering of scripts that fall in the any other
value category can in practice achieve performance comparable to, say, the
In a system where IMSC Document Instances are expected to conform to the Hypothetical Render Model, an IMSC Document Instance that does not conform to the Hypothetical Render Model might negatively impact accessibility during presentation of the IMSC Document Instance and its associated content.
This specification does not attempt to model any additional complexity for presentation processors that might arise due to the user customisation of presentation, for example as described by [[media-accessibility-reqs]]; such user customisation is not defined by [[IMSC]].
Implementers of presentation processors that support user customisation of presentation should ensure that those processors are able to present IMSC Document Instances that conform to the Hypothetical Render Model, even if the customisation effectively increases the complexity of presentation.
This specification has no inherent security or privacy implications.
The algorithm defined within this specification is used for static analysis of a resource. This specification does not define any protocol or interface for obtaining such a resource, and it does not define any interface for exposing the results of the analysis. No personal or sensitive information is processed as part of the algorithm, other than any such information that might happen to be part of the IMSC Document Instance being analysed. No information is exposed by the algorithm to any origin. No scripts are loaded or processed as part of the algorithm and no links to external resources are dereferenced.
Implementers of this specification should capture and meet privacy and security requirements for their intended application. For example, an implementation could, when reporting on an error encountered during processing of an IMSC Document Instance, include a section of the content of an IMSC Document Instance to elaborate the error. If that content could include sensitive or personal information, the implementation should ensure that any such output is provided using appropriately secure protocols. No such reporting is defined or required by this specification.
This specification does not define how, or even if, errors should be reported.
For example, an implementation could stop on the first error encountered, or continue to process the IMSC Document Instance and report every error. Or an implementation could exit with an appropriate status code without reporting any details at all.
This specification does not define any runtime exceptions, or how such exceptions should be handled.
In order to allow short (less than 100 ms) gaps between subtitles, which is common practice, the complexity of presenting empty ISDs has been reduced to zero: instead of being drawn into the Back Buffer, an empty ISD merely disconnects the Front Buffer from the display while it is presented.
The first Intermediate Synchronic Document is no longer treated differently and incurs a cost for clearing the Back Buffer.