Vincent Wartelle: Position Paper
Background
Expression of needs
The overall goal of the project is to answer the needs of producing accessible electronic books, mainly readable on mobile like smartphones for the educational market. A second goal is to build new resources from the publishing content expressed in the educational books that means modular pieces of content that could exploited for creating eLearning sites, or by teachers and learners for their own pedagogical use. All those content pieces should also be tagged semantically in order to offer better research facilities to find those new resources.
Issues: the actual FXL production
The actual production of educational books from their equivalent print materials are entirely made in EPUB 3 Fixed-Layout, with a lot of interactivity included in the digital version (images, audios, videos, exercises, etc.). Each Education Publisher add some extra features to better suit their needs in pedagogical content. Those specific features, which sometimes are not part of the EPUB standards, are then read by the specific readers each Publisher have built (Groupe Hachette, Editis, Humensis in France) just to name a few. Those actual productions of EPUB3 FXL are known to be slightly difficult to use for the accessible public with disabilities, and also not suitable for mobile learning.
Goals: the planned Reflow production
ISI research team has been thinking for three years about a new production system that solves these blocking points and provides real added value for specific reading both on mobiles and for readers with disabilities. The first step is to present content which are displayed in double-page (spread) in a new fully Reflow enrichment that benefits from specific “toolbox” bringing DYS and TTS features. To speed up the production of Reflow components from already existing Fixed-Layout conversions, we propose using AI (Artificial Intelligence) process coupled with our editorial capture technology. By capitalizing on the 3 million pages already processed by our system, and their graphic representation, we expect to find image-recognition algorithms that can help us significantly speed up the process of labeling pages.
Proposed solution
We have recently started an ambitious program, mixing Artificial Intelligence and Html capture to find unprecedented workflow solutions in producing granular and semantic content. Based on more than 1 million images from FXL conversion, conversion and a brand-new capture technology, we have set up a team of researchers, publishers, development experts and epub specialists.
W3C Tokyo Summit Presentation
I am in a position to present the goals and expectations about this project, and the roadmap that we have in mind. To treat the core of the problem (the segmentation of images), the so-called deep learning and unsupervised learning techniques will be used on the basis of a semantic categorization to be specified (e.g. title, graphic, images, paintings…). Related questions will also be addressed:
- pre-processing (data cleaning, data visualization diagnosis and descriptive statistics, automatic labeling)
- post-processing (creation of html pages)
- the question of accessibility. Moreover, it should be noted that the manual partition made by ISIcrunch which is done by successive division of images (first text vs. images + background, then in the text: titles, paragraph …) is interesting and proceeds in the same way that neural networks that also works by successive abstraction. This should therefore facilitate the choice of the method to follow.
Vincent WARTELLE
CEO and Founder of ISI
Member of IDPF since 2011