Meeting minutes
wendyreid: looks like there is the possibility of making some decisions on smil issues
Playback model for media overlays - w3c/epub-specs#2943
Hadrien: a bit of context - I started reading media overlays when we voted to add rolls - and we're implementing this in readium swift implementation
… there are a number of issues in the spec that is not completely clear and can be problematic
… we have a discord channel where we are discussing some of the issues
… looking at this first issue there is very little text about what a reading system should be doing
… one question is what happens when you follow a smil document when it moves around in the reading order between spine items
… the spec only says to play the next document after finishing
… a bigger question is that the specs says to load the next content document - in some cases this can get you stuck in a loop
… I've written about the problems in the issue including an alternate way to handle it
… in the discord channel we got into where do you start playback and how do you navigate within the publication once it's started
… we are only focusing on the second issue here - the current approach is spine-centric
… media overlays are quite different from the spec - you can move around out of reading order as they are not linear
… what I'm proposing is another way - to make a media overlays order different from the spec/spine
… it could avoid the cases of being stuck in loops
… people who have implemented media overlays in reflowable have done it differently from each other and from the spec
… the other issue is about how to handle audio playback
duga: I suspect you are right and we didn't look at the spec when we implemented for play books and we figured it out for ourselves
… but the question is whether we can find a playback model that suits everyone - children's books may be optimized and have audio be primary
… it is a hard question to solve and all the spec has is a "should" - maybe we should remove that line and just leave it to implementers
Hadrien: I don't think it is problematic for the use cases you define
… audio might be more problematic - you can have smil files without audio
… so what is the reading order and does the smil or spine define the order
wendyreid: I think the problems are real but in working on accessibility and spreads one of the things we covered is content across spreads - there are smil cases for this - but I worry about not going too far about moving around with user control
… media overlays often advance without user control - what happens to the user experience when you get moved all over the book
… there are ways to manage that - like a warning - do we need guardrails like having user interaction before this happens
Hadrien: there are no guardrails today - the current spec language doesn't handle the cases we've talked about so far
… it's common to go across spreads more than jumping around the book - the spec is not clear on how to handle spreads
… if we drop what we have and don't replace we won't have interop
wendyreid: do we have a proposal for how we would present the smil reading order?
Hadrien: I propose walking the spine and creating a set of the smil documents referenced - it's detailed in the issue
… it's not a complex way of compiling the order
<Hadrien> w3c/
Hadrien: the link is to use cases and examples
AvneeshSingh: I think this is more of an implementation guideline - one way would be to make this a note - but do we really need to change the spec?
… what change is required?
<wendyreid> https://
Hadrien: we need to change the reading systems spec
… this is more than an implementation detail - it's core logic
DaleRogers: my understanding in epub is that the markup document is the primary source of information and smil documents are subservient - they are called by the markup file
… so can a smil file be standalone and call what it wants?
Hadrien: yes, the smil has par elements that indicate the content and audio to load
… this is a technical issue but also why we have a technical spec for reading systems
ivan: I'm suprised the original smil spec doesn't say anything about this - it defines par, etc.
Hadrien: the problem isn't within one document but that we use multiple documents and tie them to the spine - it's how epub uses smil
wendyreid: smil was probably also architected more with the web in mind
ivan: no, smil allows several html document - the html documents are part of the structure but here they are adjacent
Hadrien: what we're saying is that smil is designed to drive the whole thing but the spec doesn't cover this
ivan: isn't it possible to go the whole way and let smil be the boss?
Hadrien: yes, I think it works better this way
ivan: the problem is the spec is from 2008 and we don't know how much testing it has gone through
Hadrien: no matter what we use we'll have the same issues
AvneeshSingh: one thing I'm worried about is changing the architecture of the spec late and not being able to anticipate what new issues we'll encounter
… the spine gives the reading order and smil goes back to the spine when it completes
Hadrien: I think the spec is broken and everyone has a different implementation - going back to spine won't solve it
wendyreid: hard to get my head around what this will look like
… both in spec language and reading order
… we do not tell reading systems how to do their interface - what would readium look like for example if we changed the language
… there are reading systems that care about this more than others
ivan: putting on my staff contact hat - do you have any idea of the amount and depth of work to make this happen?
Hadrien: right now it's a single paragraph but would need expanding - limited to the RS specification
ivan: more generally about all the issues you have raised
Hadrien: these two issues are mostly about reading systems
ivan: I'm looking at the charter and these are features that will have to be reviewed - will it require adding a year to our charter to handle all this
… our charter ends in less than a year and I want to avoid problems with this going on too long
Hadrien: I don't think it's big changes but I understand the time issue - it is a departure from the current spec
wendyreid: if we want to do this well we should do it right
… there is no denying this is a problem and that overlays are a powerful tool
… I don't want to push in small fixes in the hopes it scaffolds to a better experience
… I wonder if we should recharter with a specific focus on smil - push out 3.4 and then do this as a 3.4.1
mgarrish: You covered most of what I wanted to say. Everyone has implemented, but everyone has different ideas
… there will be implementations that won't conform if we change now. I worry about doing this in the current charter, we need to make sure everyone is on board
… don't want to reach a point where we make the change and someone comes and finds they don't conform
… I'm happy to make the changes around variable bit rates and smaller fixes, but this bigger change is worrying to me
… I do think this needs to be addressed, but worry about where it leads
Hadrien: one of the struggles we'll have if we do another dot release is that many people who have knowledge will not be participating in this group
… we need to think about how to get more people involved - there is a lot of interest between EAA and audiobooks taking off
… the line between audio and text is blurring - people want to implement this asap and waiting on chartering and new specs leaves people unsure what to do
… it wasn't on my radar, either, but I keep hearing about it from people
DaleRogers: I just wonder if this is a bug fix or a new feature
ivan: I don't think anyone wants to push this under the carpet but we need to optimize our manpower to get this right
… we don't have to suspend our work on this even if we move this to the next release - maybe getting 3.4 out should be a higher priority so we can start again quickly
… we could get a quick recharter if we get 3.4 out
wendyreid: I want to echo the same - by rechartering with this as a second track of work it gives us time to do the outreach
… we can talk to some of the larger platforms we don't have feedback from yet
… there might be other issues that we'll find out if we have more time to engage people
… we also have the community group to get ahold of people - discord can also be useful
… we can also improve the future spec language if people are incubating solutions to these problems
… once 3.4 is out of the way we can focus more of our attention
AvneeshSingh: the scope is larger - there are a lot of people in the daisy community who would like improvements but we haven't brought in because the scope of this revision seemed more limited
… if we have a more flexible charter to tackle media overlays that will be positive for daisy members
Hadrien: if we recharter sooner than later I will be more comfortable - we can't push this back a few years considering the state of the spec
… we have three groups of issues - the granularity issue is being done, the audio format and vbr can be tackled, the content model and reading system side, and final are the readin system issues
… if we can get the minor changes in 3.4 and the other issues in a future revision that would work but we need to keep people engaged
… we wrote specs without being sure about how implementations will work and we need to fix those problems now
ivan: I agree that the small issues are the ones we should settle in 3.4 - in parallel we should try to get a call with W3C folks to deal with the administration of whether to recharter or update our current charter
… if we do this, we have to have a hard look at what to do with annotations because we are behind on it - 3.4 is practically ready for CR absent it
Hadrien: we know that some parts are problematic - should we add a warning?
ivan: that should be okay - identifying underimplemented or unclear features is fine
wendyreid: yes, we've done this before with links to issues
Hadrien: I've also seen a recent issue with full-bleed images that is going to be problematic to solve
AOB
wendyreid: we have been asked by the web and ai interest group for feedback on how ai technologies and llm have impacted on our group/epub
… we'll discuss this on the next call
ivan: we have got some feedback about ISO and seems that the obstacles are gone and we can go through the PAS process to publish our documents