(Pack­aged) Web Pub­li­ca­tions (PWP)

These Slides are Avail­able on the Web

See: https://w3c.github.io/dpub/2016/SWIB/

(Slides are in HTML)

EPUB 3 is a very mature specification

• Widely used for­mat for dig­i­tal pub­li­ca­tion
• A truly portable and non-pro­pri­etary for­mat
• The stan­dard for­mat for dig­i­tal books…
• …but can be also be used for mag­a­zines, jour­nal ar­ti­cles, doc­u­men­ta­tion, etc.

Books with lots of il­lus­tra­tions…

Sci­en­tific pre­sen­ta­tions

Art books

Man­gas

Tech­ni­cal books

Books with different character sets and writing directions

…conference proceedings

…journals or magazines (articles)

…of­fi­cial re­ports of all kinds

In fact, just about anything!

Well, at least one of the secrets…

• EPUB is based on Web Tech­nolo­gies like HTML, CSS, SVG, JS, etc.
  • col­lec­tively re­ferred to as the “Open Web Plat­form” (OWP)
• I.e., no sep­a­rate ren­der­ing, for­mat­ting, etc., spec­i­fi­ca­tion had to be de­fined
• In­stead, EPUB re­lies on spec­i­fi­ca­tions de­fined by W3C and de­ployed in browser en­gines

Rough structure of an EPUB file

A good example

• The image on the right is dis­played through that translu­cent “screen”
• This is not done through some sep­a­rate image pro­cess­ing step
• In­stead, it is based on some com­plex CSS fea­tures (im­ple­mented in browsers but also in EPUB read­ers)

Are we all done?
I.e., are OWP and EPUB a perfect match?

There are two major areas that need work:

1. bring OWP and Dig­i­tal Pub­lish­ing closer
2. bring the Web and Dig­i­tal Pub­lish­ing closer

Missing OWP features

• Some fea­tures, needed by pub­lish­ing, are miss­ing from OWP; ex­am­ples:
  • pag­i­na­tion, page con­trols for head­ings, page num­bers
  • graph­ics tran­si­tion ef­fects (needed in, e.g., man­gas, car­toons, chil­dren books)
  • struc­tur­ing in­for­ma­tion of com­plex doc­u­ments
  • tak­ing into ac­count the type­set­ting re­quire­ments of dif­fer­ent cul­tures (French, Russ­ian, Ara­bic, Chi­nese, Japan­ese,…)
• W3C has a Dig­i­tal Pub­lish­ing In­ter­est Group to look into these is­sues and co­op­er­ate with the rel­e­vant W3C groups

Bring other OWP features to publishing

• For ex­am­ple:
  • dy­namic tools and li­braries, de­ployed on the Web, would be ben­e­fi­cial in pub­lish­ing
  • use ex­ter­nal ser­vices on the Web via extra pro­to­cols
  • new Web au­then­ti­ca­tion or pay­ment meth­ods could change the way pub­li­ca­tions are ac­cessed
  • new de­vice API-s could be used by ded­i­cated soft­ware on de­vices
  • etc.

What is, in fact, a (dig­i­tal) book?

• It is a mean to trans­mit ideas to read­ers in var­i­ous for­mats
• It is in­de­pen­dent of for­mat: the same “book” should be avail­able in dif­fer­ent for­mat and media
  • printed book, dig­i­tal books, Web page(s)…
• Today, a Dig­i­tal Book is dif­fer­ent than a Web page (con­cep­tu­ally and prac­ti­cally)

What we get today…

• One can down­load an EPUB (and other for­mats) from a Web site…

What we get today…

• …get a very tra­di­tional text to read…

What we get today…

• …but one can ac­cess a dif­fer­ent (and pos­si­bly nicer) Web page on­line…
• This should not be the case!

“This should not be the case!”
what does this mean?

Sep­a­ra­tion be­tween pub­lish­ing as Web sites and in an of­fline pack­age should be di­min­ished to zero

• This means:
  • it should be possible to load the publication content into a browser or a specialized reader, whatever the user prefers
  • it should be possible to read the book either offline or online, whatever the circumstances dictate
  • contents could be authored regardless of where they are used

For example: book in a browser

• On a desk­top I may want to read a book just like a Web page:
  • eas­ily fol­low a link “out” of the book
  • cre­ate book­marks to “within” a page in a book
  • use use­ful plu­g­ins and tools that my browser may have
  • cre­ate an­no­ta­tions
  • some­times I may need the com­put­ing power of my desk-top for, e.g., pow­er­ful text analy­sis

For example: book in a browser

• But, at other times, I may also want to use a small ded­i­cated reader de­vice to read the book on the beach…
• All these on the same book (not con­ver­sions from one for­mat to the other)!

For example: I may not be online…

• I may find an ar­ti­cle on the Web that I want to re­view, an­no­tate, etc., while com­mut­ing home on a train
• I want the re­sults of the an­no­ta­tions to be back on­line, when I am back on the In­ter­net
• note: browsers may have an “archiv­ing” pos­si­bil­ity, but they are not in­ter­op­er­a­ble

For example: scholarly publishing

• Paper may be pub­lished on-line, but peo­ple may want to down­load it for of­fline use
• The for­mat of the paper should be adapt­able to read­ing en­vi­ron­ment
  • no two col­umn, fixed lay­out file that can­not be han­dled on an my tablet…
• The “paper” may also con­tain video, audio, data, pro­grams…
  • schol­arly pub­lish­ing is not text only any more!

Need for the concept of a “publication” of many resources

• A pub­li­ca­tion may con­sist of many re­sources (text, im­ages, videos, etc)
• On the Web, these are linked to­gether, but they keep their com­plete “in­de­pen­dence”
• For pub­lish­ing the ab­stract “work” should be a sin­gle, con­cep­tual unit on the Web: a Web Pub­li­ca­tion (WP)

Why do we need the WP concept?

• The whole book should be archived in a na­tional li­brary
• The pub­li­ca­tion has to be checked for in­tegrity as a whole (e.g., in a legal pub­li­ca­tion)
• Shar­ing a pub­li­ca­tion with a friend re­quires the con­cept of the pub­li­ca­tion as a whole
• Pub­lish­ing and li­brary busi­ness mod­els are based on the con­cept of “works”

Why do we need the WP concept?

• Value coun­ters (coun­ters, sec­tion num­ber­ing, foot­notes, end­notes, etc.) should be con­sis­tent across the pub­li­ca­tion
• Any search, text analy­sis, etc., fa­cil­ity for a book should have ac­cess to the full pub­li­ca­tion
• User pref­er­ences should apply to the “work” rather than to a sin­gle Web page

How does that translate to the Web?

• The cur­rent Web builds on the no­tion of a sin­gle re­source:
  • HTML source, meta­data, CSS style sheet, etc.
  • each has its own URL
• Ren­der­ing is based on the in­ter­op­er­a­tion of many such re­sources

How does that translate to the Web?

• A WP is a col­lec­tion of Web re­sources, con­sid­ered as a sin­gle Web re­source
  • this means there is a unique Web ad­dress for the col­lec­tion
  • the same WP can be of­fline or on­line with the same ad­dress
• Web Pub­li­ca­tions are part of the Web!

An internal representation may also be needed

• At the mo­ment, the in­ter­nal rep­re­sen­ta­tion of (HTML) re­sources is based on the con­cept of a “Doc­u­ment” in the “Doc­u­ment Ob­ject Model (DOM)”
  • this un­der­pins all pro­gram­matic ac­cess to con­tent on the Web
• There may be a need for a pro­gram­matic in­ter­face on top of Doc­u­ments in the DOM, rep­re­sent­ing a col­lec­tion of doc­u­ments

Envisioned architecture:
online

Envisioned architecture:
offline

Advances in modern browsers: Web and Service Workers

• Web Worker: a truly par­al­lel thread within the browser
• A Ser­vice Worker is a spe­cial type of Web Worker, with ad­di­tional fea­tures:
  • it is a pro­gram­ma­ble net­work proxy: the ren­derer’s net­work calls are caught and the re­quest/an­swer can be mod­i­fied on-the-fly be­hind the scenes
  • it has an in­ter­face to han­dle a local cache for net­worked data
  • it will stay alive even if the user moves away from the main page, and can be ac­cessed later if he/she re­turns to it

Service Workers are coming…

• Browser en­gines are head­ing to­wards Ser­vice Work­ers today
  • a major tech­nol­ogy to rec­on­cile Mo­bile Ap­pli­ca­tions and Web De­vel­op­ments
• I.e, Web Pub­li­ca­tions may build on an evo­lu­tion that is al­ready hap­pen­ing

An ex­am­ple of on­line/of­fline book with Ser­vice Work­ers

Manifests

• A WP needs a num­ber of extra in­for­ma­tion
  • dif­fer­ent type of meta­data (au­thor, title, pub­lisher, etc.)
  • list of re­sources within a WP
  • table of con­tent in­for­ma­tion, i.e., de­fault read­ing order
  • unique iden­ti­fier (ISBN, DOI, etc.)
  • ref­er­ences to more com­plex meta­data (e.g., ONIX data)
• The tra­di­tional term is “man­i­fest” for such in­for­ma­tion
• A de­tailed man­i­fest must be spec­i­fied for WP-s

Pack­aged Web Pub­li­ca­tions (PWP)

• For a large num­ber of use cases Web Pub­li­ca­tions (es­sen­tially, on­line/of­fline read­ing) is all they re­ally need
  • e.g., schol­arly jour­nals, on­line mag­a­zines, brochures
• But a num­ber of use cases rely on an extra step: to “pack” the con­tent of a WP into a sin­gle file
  • archival, li­brary dis­tri­b­u­tions,…
  • ex­ten­sion of the cur­rent busi­ness model of trade pub­lish­ers (built on EPUB)

A layer “on top” of WP-s

• Pack­ag­ing means col­lect­ing all the con­tent of a WP into a sin­gle file
• Some ad­di­tional is­sues to be han­dled:
  • spe­cial en­tries to the man­i­fest
  • archival for­mat(s)
  • etc.
• A spe­cific “pro­file” for WP may have to be de­fined for a PWP

Structure of an EPUB3 file

A Packaging of a Web Publication

• A Packaged Web Publication may have a very similar structure
• But moving to the Web would probably require to rethink the format of the administrative files
  • e.g., manifest would be in JSON
  • structure of metadata may different
  • maybe re-think the packaging format itself
  • etc.

PWP packaging structure

A PWP Proces­sor

• PWP-s may be han­dled by ded­i­cated ap­pli­ca­tions
• But the con­cept of a WP proces­sor can be ex­tended to a PWP proces­sor, un­pack­ing con­tent on the fly
  • se­cu­rity con­sid­er­a­tions must be han­dled, though

A PWP Processor

Is it “addressing” or “identification”?

• These two “roles” are dif­fer­ent
• The usual sit­u­a­tion:
  • some form of a URI is used to (uniquely) iden­tify a re­source
  • an HTTP(S) URL is used to ad­dress (or “lo­cate”) a re­source on the Web
• In many cases the two roles co­in­cide, but not al­ways
  • e.g., for a dig­i­tal Book :
    • URN:ISBN:1-56592-521-1 iden­ti­fies the pub­li­ca­tion
    • http://www.ex.org/ex.wp ad­dresses a par­tic­u­lar copy

Is it “addressing” or “identification”?

• Iden­ti­fi­ca­tion is han­dled by a num­ber of other or­ga­ni­za­tions (like the DOI foun­da­tion, the In­ter­na­tional ISBN Agency, etc.)
• The work on (P)WP has to con­cen­trate on lo­ca­tors (i.e., ad­dress­ing)

What does a Web request return for a locator?

• Pos­si­bil­i­ties are:
  • the pack­age that in­cludes the man­i­fest at some pre­de­fined place (in case this is a Pack­aged Web Pub­li­ca­tion)
  • an HTML file with a link to a man­i­fest (through a <link> el­e­ment)
  • an HTML file with an em­bed­ded man­i­fest (through a <script> el­e­ment)
  • an HTML file with a link to a man­i­fest in the Link header of the HTTP re­sponse
• Un­clear whether all of these are nec­es­sary and/or fea­si­ble

Book read­ing needs a dif­fer­ent ap­proach to er­gon­omy

• Typ­i­cal ex­am­ple: pag­i­na­tion
  • for short text, pag­i­na­tion is not re­ally im­por­tant
  • but what about read­ing, say, the War and Peace?
  • cur­rently not avail­able or used on the Web
• Pag­i­na­tion isn’t only books: slides, cards, and other “boxes” use the same func­tion­al­ity

Per­son­al­iza­tion

• What is the level of user con­trol of the pre­sen­ta­tion?
• The Web and eBook tra­di­tions are vastly dif­fer­ent:
  • in a browser, the Web de­signer is in full con­trol
    • CSS al­ter­nate style sheets or user style sheets are hardly in use
    • some user in­ter­face as­pects can be con­trolled but only for the browser as a whole
  • in an eBook reader, there is more user con­trol
    • fore­ground/back­ground color
    • choice of fonts
• There is a need to rec­on­cile these tra­di­tions

PWP vs. EPUB3.1

• PWP does not re­place EPUB 3
• It is, in­stead, part of a fu­ture evo­lu­tion, a “vi­sion” for the years to come

Most things are the same!

• Im­por­tant to note that the con­tent of a PWP and EPUB 3 are es­sen­tially iden­ti­cal
  • usage of HTML, CSS, SVG, etc.
• The main change is the “ad­min­is­tra­tive” as­pect of the pub­li­ca­tion (lo­ca­tors, man­i­fest, etc)

De­vel­op­ment process

• EPUB was de­vel­oped at the IDPF, Web Tech­nolo­gies were de­vel­oped at the W3C
• Strong co­op­er­a­tion ex­ists since 2013 (e.g., in W3C’s Dig­i­tal Pub­lish­ing In­ter­est Group)
  • many of the ideas around PWP were de­vel­oped at W3C, with strong pres­ence of ex­perts from IDPF
• The plan is that IDPF and W3C would merge start­ing 2017
  • main­te­nance of EPUB 3.1 will con­tinue at W3C
  • fu­ture de­vel­op­ment (of PWP and oth­ers) will be part of one or more W3C Work­ing Groups

Advantage for the publishers‘ community

• The main in­ter­est of pub­lish­ers is to pro­duce, edit, cu­rate, etc., con­tent
• Pub­lish­ers would have an eas­ier ac­cess to a large au­di­ence
• Pub­lish­ers have in­vested heav­ily into tech­nol­ogy de­vel­op­ments, but the Web de­vel­op­ers’ com­mu­nity can com­ple­ment that with a wider reach and per­spec­tive

Advantage for the publishers‘ community

• Work­ing closely with Web de­vel­op­ers avoids re-in­vent­ing wheels
  • e.g., API-s to ac­cess sys­tem re­sources or util­i­ties, like index data­base, bat­tery sta­tus API, real-time com­mu­ni­ca­tion, ge­olo­ca­tion,…
• Align­ing more on OWP means that spe­cial­ized read­ers or Web sites, soft­ware, ser­vices, etc, can rely on it

Advantage for the Web community

• Pub­lish­ers have ex­pe­ri­ence in:
  • er­gonom­ics, ty­pog­ra­phy, aes­thet­ics…
  • pub­lish­ing long texts, with the right read­abil­ity and struc­ture
• Work­flow for pro­duc­ing com­plex con­tent
• In some sense, a new type of con­tent comes to the fore

Some references

Lat­est PWP Use Cases and Re­quire­ments draft:

https://w3c.github.io/dpub-pwp-ucr/

Lat­est PWP Ed­i­tors’ draft:

https://w3c.github.io/dpub-pwp/

PWP Issue list:

https://github.com/w3c/dpub-pwp/issues

Thank you for your at­ten­tion!

This pre­sen­ta­tion:

https://w3c.github.io/dpub/2016/SWIB/
(PDF is also avail­able for down­load)

My con­tact:

ivan@​w3.​org