(Packaged) Web Publications (PWP)

These Slides are Available on the Web

See: https://w3c.github.io/dpub/2016/CNCC-Taiyuan/

(Slides are in HTML)

EPUB 3 is a very mature specification

• Widely used format for digital publication
• A truly portable and non-proprietary format
• The standard format for digital books…
• …but can be also be used for magazines, journal articles, documentation, etc.

Books with illustrations

Scientific presentations

Art books

Mangas

Technical books

Books with different character sets and writing directions

…conference proceedings

…journals or magazines (articles)

…official reports of all kinds

In fact, just about anything!

Well, at least one of the secrets…

• EPUB is based on Web Technologies like HTML, CSS, SVG, JS, etc.
  • collectively referred to as the “Open Web Platform” (OWP)
• I.e., no separate rendering, formatting, etc., specification had to be defined
• Instead, EPUB relies on specifications defined by W3C and deployed in browser engines

Rough structure of an EPUB file

A good example

• The image on the right is displayed through that translucent “screen”
• This is not done through some separate image processing step
• Instead, it is based on some complex CSS features (implemented in browsers but also in EPUB readers)

Are we all done?
I.e., are OWP and EPUB a perfect match?

There are two major areas that need work:

1. bring OWP and Digital Publishing closer
2. bring the Web and Digital Publishing closer

Missing OWP features

• Some features, needed by publishing, are missing from OWP; examples:
  • pagination, page controls for headings, page numbers
  • graphics transition effects (needed in, e.g., mangas, cartoons, children books)
  • structuring information of complex documents
  • taking into account the typesetting requirements of different cultures (French, Russian, Arabic, Chinese, Japanese,…)
• W3C has a Digital Publishing Interest Group to look into these issues and cooperate with the relevant W3C groups

Bring other OWP features to publishing

• For example:
  • dynamic tools and libraries, deployed on the Web, would be beneficial in publishing
  • use external services on the Web via extra protocols
  • new Web authentication or payment methods could change the way publications are accessed
  • new device API-s could be used by dedicated software on devices
  • etc.

What is, in fact, a (digital) book?

• It is a mean to transmit ideas to readers in various formats
• It is independent of format: the same “book” should be available in different format and media
  • printed book, digital books, Web page(s)…
• Today, a Digital Book is different than a Web page (conceptually and practically)

What we get today…

• One can download an EPUB (and other formats) from a Web site…

What we get today…

• …get a very traditional text to read…

What we get today…

• …but access a different (and possibly nicer) Web page online…
• This should not be the case!

“This should not be the case!”
what does this mean?

Separation between publishing as Web sites and in an offline package should be diminished to zero

• This means:
  • it should be possible to load the publication content into a browser or a specialized reader, whatever the user prefers
  • it should be possible to read the book either offline or online, whatever the circumstances dictate
  • contents could be authored regardless of where they are used

For example: book in a browser

• On a desktop I may want to read a book just like a Web page:
  • easily follow a link “out” of the book
  • create bookmarks to “within” a page in a book
  • use useful plugins and tools that my browser may have
  • create annotations
  • sometimes I may need the computing power of my desk-top for, e.g., powerful text analysis

For example: book in a browser

• But, at other times, I may also want to use a small dedicated reader device to read the book on the beach…
• All these on the same book (not conversions from one format to the other)!

For example: I may not be online…

• I may find an article on the Web that I want to review, annotate, etc., while commuting home on a train
• I want the results of the annotations to be back online, when I am back on the Internet
• note: browsers may have an “archiving” possibility, but they are not interoperable

For example: scholarly publishing

• Paper may be published on-line, but people may want to download it for offline use
• The format of the paper should be adaptable to reading environment
  • no two column, fixed layout file that cannot be handled on an my tablet…
• The “paper” may also contain video, audio, data, programs…
  • scholarly publishing is not text only any more!

Need for the concept of a “publication” of many resources

• A publication may consist of many resources (text, images, videos, etc)
• On the Web, these are linked together, but they keep their complete “independence”
• For publishing the abstract “work” should be a single, conceptual unit on the Web: a Web Publication (WP)

Why do we need the WP concept?

• The whole book should be archived in a national library
• The publication has to be checked for integrity as a whole (e.g., in a legal publication)
• Sharing a publication with a friend requires the concept of the publication as a whole
• Publishing and library business models are based on the concept of “works”

Why do we need the WP concept?

• Value counters (counters, section numbering, footnotes, endnotes, etc.) should be consistent across the publication
• Any search, text analysis, etc., facility for a book should have access to the full publication
• User preferences should apply to the “work” rather than to a single Web page

How does that translate to the Web?

• The current Web builds on the notion of a single resource:
  • HTML source, metadata, CSS style sheet, etc.
  • each has its own URL
• Rendering is based on the interoperation of many such resources

How does that translate to the Web?

• A WP is a collection of Web resources, considered as a single Web resource
  • this means there is a unique Web address for the collection
  • the same WP can be offline or online with the same address
• Web Publications are part of the Web!

An internal representation may also be needed

• At the moment, the internal representation of (HTML) resources is based on the concept of a “Document” in the “Document Object Model (DOM)”
  • this underpins all programmatic access to content on the Web
• There may be a need for a programmatic interface on top of Documents in the DOM, representing a collection of documents

Envisioned architecture:
online

Envisioned architecture:
offline

Advances in modern browsers: Web and Service Workers

• Web Worker: a truly parallel thread within the browser
• A Service Worker is a special type of Web Worker, with additional features:
  • it is a programmable network proxy: the renderer’s network calls are caught and the request/answer can be modified on-the-fly behind the scenes
  • it has an interface to handle a local cache for networked data
  • it will stay alive even if the user moves away from the main page, and can be accessed later if he/she returns to it

Service Workers are coming…

• Browser engines are heading towards Service Workers today
  • a major technology to reconcile Mobile Applications and Web Developments
• I.e, Web Publications build on an evolution that is already happening

Manifests

• A WP needs a number of extra information
  • different type of metadata (author, title, publisher, etc.)
  • list of resources within a WP
  • table of content information, i.e., default reading order
  • unique identifier (ISBN, DOI, etc.)
  • references to more complex metadata (e.g., ONIX data)
• The traditional term is “manifest” for such information
• A detailed manifest must be specified for WP-s

Work on “Web App Manifests”

• Work is happening at W3C
  • used for Web Applications
  • contains lots of display-oriented data, and some metadata
  • describes a precise way of fetching manifests and content in a secure context
  • contains data for an application based on Service Workers
• A WP manifest can build on this evolution as well!

Packaged Web Publications (PWP)

• For a large number of use cases Web Publications (essentially, online/offline reading) is all they really need
  • e.g., scholarly journals, online magazines, brochures
• But a number of use cases rely on an extra step: to “pack” the content of a WP into a single file
  • archival, library distributions,…
  • extension of the current business model of trade publishers (built on EPUB)

A layer “on top” of WP-s

• Packaging means collecting all the content of a WP into a single file
• Some additional issues to be handled:
  • special entries to the manifest
  • archival format(s)
  • etc.
• A specific “profile” for WP may have to be defined for a PWP

Structure of an EPUB3 file

A Packaging of a Web Publication

• A Packaged Web Publication may have a very similar structure
• But moving to the Web would probably require to rethink the format of the administrative files
  • e.g., manifest would be in JSON
  • structure of metadata may different
  • etc.

PWP packaging structure

Web Archival format?

• There is an interest among some W3C members for a Web friendly packaging format:
  • streamable
  • rely, as much as possible, on existing Web technologies (e.g., HTTP)
• Use cases include:
  • retrieve an HTML file with related CSS files, images
  • access Web Applications (“Widgets”) with all libraries involved
  • Packaged Web Publications are a clear use case
• This Web Packaging proposal is not OPF based, though
  • this may lead to a different packaging in future for Digital Publishing

PWP packaging structure

However…

• There isn‘t yet a full agreement whether to develop such Web Packaging format
  • a caching architecture based on Service Workers may be enough for the use cases
• The Publishing Community may not move away from OPF
  • technical advantages of a new format must be weighted against existing deployment

A PWP Processor

• PWP-s may be handled by dedicated applications
• But the concept of a WP processor can be extended to a PWP processor, unpacking content on the fly
  • security considerations must be handled, though

A PWP Processor

Book reading needs a different approach to ergonomy

• Typical example: pagination
  • for short text, pagination is not really important
  • but what about reading, say, the Story of the Stone?
  • currently not available or used on the Web

The Houdini project & pagination

• May lead to a standard access to page-related information
• Pagination isn’t only books: slides, cards, and other “boxes” use the same functionality

Personalization

• What is the level of user control of the presentation?
• The Web and eBook traditions are vastly different:
  • in a browser, the Web designer is in full control
    • CSS alternate style sheets or user style sheets are hardly in use
    • some user interface aspects can be controlled but only for the browser as a whole
  • in an eBook reader, there is more user control
    • foreground/background color
    • choice of fonts
• There is a need to reconcile these traditions

PWP vs. EPUB3.1

• PWP does not replace EPUB 3
• It is, instead, part of a future evolution, a “vision” for the years to come

Most things are the same!

• Important to note that the content of a PWP and EPUB 3 are essentially identical
  • usage of HTML, CSS, SVG, etc.
• The main change is the “administrative” aspect of the publication (locators, manifest, etc)

Development process

• EPUB was developed at the IDPF, Web Technologies were developed at the W3C
• Strong cooperation exists since 2013 (e.g., in W3C’s Digital Publishing Interest Group)
  • many of the ideas around PWP were developed at W3C, with strong presence of experts from IDPF
• The plan is that IDPF and W3C would merge starting 2017
  • maintenance of EPUB 3.1 will continue at W3C
  • future development (of PWP and others) will be part of one or more W3C Working Groups
  • A memorandum of understanding is under review be IDPF and W3C members, respectively

Some references

Latest PWP Use Cases and Requirements draft:

https://w3c.github.io/dpub-pwp-ucr/

Latest PWP Editors’ draft:

https://w3c.github.io/dpub-pwp/

PWP Issue list:

https://github.com/w3c/dpub-pwp/issues

Thank you for your attention!

This presentation:

https://w3c.github.io/dpub/2016/CNCC-Taiyuan/
(PDF is also available for download)

My contact:

ivan@w3.org