Meeting minutes
CFIs
George Kerscher: Lars talked to me at a meeting telling me how great CFI are and that he has an open source implementation
… and I am trying to understand the relationship between CFI and EPUB
… are CFI in 3.4? Are they referenced by epub? what is the w3c status
Ivan Herman: The answers are no, no, and no
… they are not even a w3c note, and it is not referenced from 3.4
… in the EPUB WG, Brady Duga and I looked at it, it seems half baked
… e.g. there is no processing model
… we talked about what to do about it, and at the time we let it alone
… we could change our mind now, that is just where we are
… CFI is very bound to xml, so we may be in a better position, since we previously had html serialization as a posibility
… but that has changed, we have decided on xml
… I think a discussion with an eye on annotations
… is a good topic. Let's not discuss now, but in the future we could
Wendy Reid: Chairs have discussed in relation to annotations
… so we likely have more to discuss
George Kerscher: Given that there is an open source impl, if two epub impls use it, does that count as two?
Avneesh Singh: Since it isn't in the spec, the 2 implementation threshold doesn't exist
… some reading system people are very into CFI. At the same time, some people hate it
Ivan Herman: I agree with Avneesh Singh, but the answer to George Kerscher's question is no, we would need another implementation
… but we need a separate full discussion on this topic
EPUB 3.4 is now in CR
<Wendy Reid> https://
Wendy Reid: We are in CR! (3.4 that is)
<Wendy Reid> https://
Wendy Reid: time to talk testing
<Wendy Reid> https://
<Wendy Reid> https://
<Wendy Reid> https://
Wendy Reid: the test suite is complete, I believe
… we added scrolled layouts, there are mixed layouts recently added
… anything else?
Ivan Herman: Not that I know, but if someone would check that it would be helpful
… we have 175 tests, inheriting old ones and adding some new ones
Wendy Reid: We still have the 3.3 test results. We have some new tests for 3.4, and some of the old tests have changed
… it was hard to get all the tests run last time, and now we have even more
… ideally, we would have everyone run all the tests, but if that is too hard what should we ask for
… ? Maybe just new tests, or not all platforms, etc
Ivan Herman: There were a number of tests we had to rename, so the old results don't match up
<Wendy Reid> ak Gregorio Pellegrino
Ivan Herman: We may want to manually update those
Gregorio Pellegrino: 3 or 4 years has past since the last test round. I think the testing was very useful for the community, so I think a full run is very useful (regression checking, etc)
<Ivan Herman> +1 to Gregorio Pellegrino
Gregorio Pellegrino: We did do some of this at TPAC last time, maybe we can do it again
Gautier Chomel_: I agree. I will be testing all the tests on our engines (reader and mobile). Even though these are the same engines, there have been forks
… Maybe we can mark the old passing tests that have not been rerun should be marked as outdated (or something)
… In terms of timing, I plan to test on thorium by dec, is that too late
Ivan Herman: We only have a minimum deadline. We said we will not go to rec until Sept, but we can go later
… our charter ends in Feb, so if we get the results in Dec, we can vote in Jan, and go to rec before Feb
… and that leaves a lot of time for others as well
Shinya Takami: we plan to test new and modified tests before tpac. If we need all the tests, we will need to discuss. I don't know how long that would take
Wendy Reid: We have reading platforms (ios, android, web, etc), and a rendering engine. We should clarify if the engine is the same, you probably don't need to retest on all platforms
… so that might take some pressure off
Brady Duga: I have a question on process, we have EPUB 3.3, it has tests, we have 2 implementations for all of those features, in EPUB 3.4, what do we need to test to get out the door for W3C's requirements?
… do we only need to test the new features?
Ivan Herman: Formally, we have to do the whole thing
… we could hand wave, but technically this is not a delta spec, so we should do the whole thing
Dale Rogers: You mention that a lot of systems use the same engine. My understanding is that on the Mac, everyone uses the same engine
Ivan Herman: The Mac comment isn't correct. On iOS it is, but not on the Mac
… We never really broke out a matrix like that
… This is not an authority that verifies things - we just believe what the reading system tells us
… We do make it clear that if two implementations use the same engine they should not both report success
Dale Rogers: Do we not test the reading systems then?
Ivan Herman: No, we can do the test ourselves if we want
… last time we thought it was cleaner and more fair if the authors of the RS do it
Wendy Reid: We have about 3 layers we need to test
… many reading systems use a browser engine underneath, we don't (and really can't) test that
… But they still have a rendering engine on top of that which does epub
… and then there may be an app layer
… So Kobo had a number of platforms, but the engine was shared
Susan Neuhaus: MobileReads has some forums for developers
… do we want to reach out to them
Brady Duga: I do not have an answer for Susan, but I wanted to talk about letting implementers do the testing
… outsiders may not know how the system works
… sideloading may be possible but it may run differently
… there's two different pathways, or more, so it's important to have the reading systems provide the information
… you also don't know about the engines or quirks, no idea what code may be shared, some may not be, and won't know
… it's difficult for anyone except the implementor
Ivan Herman: Two more things - still on the matter of the same engine on different platforms
… So we added the ability for the same company to report multiple platforms results (android, ios, web, etc)
… and the report shows the individual platforms (e.g. 3) and also a single report that shows the `or` of the reports
… and also we generate the OPDS for the entire test suite
… which could help any reading system that supports that
Hadrien Gardeur: Yes, very useful here, and widely implemented
Ivan Herman: and my answer to Susan Neuhaus is, where anywhere we have possible help, adding to tests is the best thing they could do to help
Susan Neuhaus: Thanks, that makes sense. On the other side of things, most of our books go to Kindle
… is it possible for someone else to do that?
Ivan Herman: We have failed over 10 years to get them on the group
… I would take just about any results for Kindle
<Avneesh Singh> Is Kindle considered as EPUB 3 implementation formally?
Dale Rogers: There is a question about what is behind the scenes vs what is available to users
… I would think we want to test what people can actually see
Brady Duga: It's not hidden behind a wall, there are certain publishers that can do certain things, like fixed layout, there are rules about FXL, only certain pathways are enabled for certain pipelines
… different entities might experience different pipeline, readers don't really see the differences in the end experience
Ivan Herman: We also have to understand, we are testing the spec, not implementation
… s So we care whether the spec can be implemented. So we don't care whether it is visible to end users, from a w3c CR process, it just matters that it can be dome
Hadrien Gardeur: Back to Kindle, we know they don't work with epub directly. I think that isn't really a problem, and a lot of apps also do the same thing, even though they are cosidered epub reading systems
… Also, I am collecting requirements for comics stuff. It turns out that if Amazon requires something, a lot of books follow that for all books
… they have an outsize influence
… We could set up testing for Kindle easily, and since we won't get the tests from Amazon, I think it would be interesting
Gregorio Pellegrino: I agree with Hadrien Gardeur regarding epub as b2b
… There are often lots of changes made to epub for each platform. For instance, kindle uses mathjax for mathml
… so sideloading may give different results
Ivan Herman: The reason I would be unhappy to do it ourselves is that it is ideal to have the implementor do it. But since Amazon won't we will have to settle for second best
… I am unhappy we have to do it, but that is where we are
Susan Neuhaus: Amazon doesn't like side loading for testing, they prefer an emulator they provide
Wendy Reid: One other reason we stongly prefer platforms do the testing - in the case that side loading doesn't work the same way as other ingestion methods, we might make it look like there is a bug or missing feature in the platform
… so reading system testing is best, but we take what we can get
… But even just ingesting all the books would be good
… and if they are visible we can test them
Hadrien Gardeur: Back to what Ivan Herman said, there is more than just the spec testing, Amazon has other requirements that are being implemented by authors
Ivan Herman: At the moment the results are the old results. I propose that we remove those results, so the tables would all be empty
… old results will still be archived. Objections?
Susan Neuhaus: Makes sense, just so long as there is a link somewhere to see the old results
Ivan Herman: Ok, it is there, but there is no link. But I could add a link for old tests
Susan Neuhaus: That works
Gautier Chomel_: Do we have a field to say who ran the test? For instance, was the tester from the same org as the reading system
Ivan Herman: So we can add it, but what do we do with that data? I don't think we should say who ran the test
… I am not sure if we should display a name
Wendy Reid: we don't need to show it, but we could store it
Brady Duga: It seems like we could just simply say "is representative of company" boolean? We just need to know platform a's tests were done by someone else, platform b was done by an employee
Gautier Chomel_: Agree with Brady Duga.
Ivan Herman: Should I add a flag that is official test or not?
Wendy Reid: Something like that
Ivan Herman: I will do a PR, we can discuss there
AOB
https://
Wendy Reid: reminder - tpac registration
… early bird ends soon. Invited expert don't pay, and there are fee waivers if needed
… Also, some people have vacations coming up - should we reduce meeting cadence?
Susan Neuhaus: Can we do this in slack?
Wendy Reid: maybe I will do a poll