MiniApp Testing

MiniApps WG Meeting 11 Apr

Martin Alvarez-Espinar

Resources?

home.css: CSS profile ✅
.js
- Standard ECMAScript (e.g., import...) ✅
- Limited execution (e.g., eval()) ✅
- DOM access? (e.g., .getElementById(), document, event handlers) ⚠️
.html ❓️️️ ⚠️
- Standard HTML (element subset?) ✅
- New HTML/DOM spec ❓

Structure of a miniapp directory — Structure of a MiniApp

Why testing (specifications)?

Testing every feature in each specification as soon as possible

Success Criteria

[...] In order to advance to Proposed Recommendation, each normative specification must have an open test suite of every feature defined in the specification.

There should be testing plans for each specification, starting from the earliest drafts.

Extracted from the MiniApp WG Charter

Metadata of a test

Screenshot of the test suite description — https://w3c.github.io/miniapp-tests/

Spec (pkg, mnf, lcy)
Unique ID
MUST, MAY or SHOULD
Title, description
Link(s) to the spec(s)
Link to reports

test.jsonld within app code

[example] 1) the spec reference

Let's test a Manifest feature

6.2 background_color member

The background_color member is a string that specifies the background color of the window that contains a MiniApp.

This member supports sRGB colors, and it is equivalent to the application manifest's background_color.

https://www.w3.org/TR/miniapp-manifest/#background_color-member

2) the test description

{
  [...]  // this is test.jsonld
  "@type": "earl:TestCase",
  "dc:coverage": "Manifest",
  "dc:identifier": "mnf-window-background-color",
  "dcterms:conformsTo": "must",
  "dc:date": "2022-07-11",
  "dc:title": "Window background color set",
  "dc:description": "The window's background member is set to #00FF00, 
                     so the app page must have a green background.",
  "dcterms:isReferencedBy": [
    "https://www.w3.org/TR/miniapp-manifest/#background_color-member"
  ],
  "dcterms:rights": "https://www.w3.org/Consortium/Legal/2015/copyright-software-and-document",
  "dcterms:rightsHolder": "https://www.w3.org",
  "dc:creator": [ "Martin Alvarez"],
}

3) the code

Screenshot of the Github repository with a directory that contains the files for a test with the manifest.json opened. The manifest includes the declaration of backgound color to green as part of the test — https://github.com/w3c/miniapp-tests/tests/mnf-window-background-color/

Tests in /tests
MiniApp code in /{ID}/src
Test description in /{ID}/test.json
Script to pack MiniApp as /{ID}.ma

How to test it

Get the MiniApp to test (e.g., mnf-window-background-color.ma)
Load it in the user agent (your implementation)
Annotate the result ->
Upload to /reports
[A GH action will generate the human-readable report]

{
  "name": "ACME Mini Programs",
  "ref": "https://example.org/",
  "variant" : "iOS",
  "tests": {
    "mnf-window-background-color": true,
    "mnf-window-fullscreen-default": true,
    "mnf-window-fullscreen-true": false,
    "mnf-window-orientation-default": true,
    "mnf-window-orientation-landscape": true,
    "mnf-window-orientation-portrait": false,
    "pkg-pages-same-filenames": true,
    "pkg-root-app-css-empty": true        
  }
}

Methodology & tools are ready

We can start testing right now!

Repository (test suite, instructions, and tools): w3c/miniapp-tests
Test Suite: https://w3c.github.io/miniapp-tests
Implementation reports: https://w3c.github.io/miniapp-tests/results
How to start: https://w3c.github.io/miniapp-tests/contributing

Thank you!

Questions?

Martin Alvarez-Espinar (martin.alvarez.espinar@huawei.com)
Guide for contributors