Process for adding tests to the framework & results pages

This page helps you create the additional support files needed for the results pages and the test framework. These are: BATCHNAME-batch.txt, BATCHNAME.json, BATCHNAME-data.js, all in the batches directory; and in the results directory, for each results page, FILENAME.html and FILENAME-data/translations.js.

Create the tests

Add tests in the same directories as you would submit them to WPT.

The batch file contains a list of test names, including the file extension. They may also include part or all of the directory path (see Examples, below).

If you have a test name in a batch file, the base is the missing part of path from the top of the repository (minus the trailing slash). See the examples, just below the input field.

Examples

For example, given a file at i18n-tests/css-text/text-transform/text-transform-capitalize-009.html, the batch may contain the file name text-transform-capitalize-009.html and the base would be css-text/text-transform.

Alternatively, you could put more (or all) of the base into the filename in the batch file, such as a base of css-text and filenames such as text-transform/text-transform-capitalize-009.html. This is particularly useful when the tests in a batch are spread over more than one directory.

If you put all of the path into the filenames, use . for the base.

Create a batch file

Now create a list of tests in the -batch.txt file. Order the list so that there is a logical flow as you move through the tests. (This would normally be the same order as tests will be displayed in the results file.) You should have one test file name per line, and the name should include the file extension. Blank lines are ok.

The following can help by automatically generating a list of files in a given directory for you to copy:

Create a json file

The json file contains metadata extracted from each test file, and is used by the test runner and the results page. It will be saved to batches/BATCHNAME.json.

You need to recreate the .json file each time you change the -batch.txt file, or change the metadata inside the test files themselves.

Run the tests

Run through the tests, to make sure they work.

Create a results file

A results page can reflect results for a whole batch or part of it.

Specify a filename for the results page: or create your own name here

The files that will be created are:

results/FILENAME-data/translation.js
results/FILENAME.html

Now edit the results page.

What to edit

Provide a title in the title and h1 elements.
Adapt introduction paragraphs as needed.
Arrange and name sections to suit the page, changing titles and ids to suit.
For each script element after a section:

Change the sectionid to the id of the section (3 places).
Look for "", and move each line below that to an appropriate section, replacing the text 'TESTFILENAME'.

In the #endlinks at the bottom of the page, add a fragment id for Related links, pointing to the appropriate location in the test overview page.

Run the tests and record results

Run the tests in the framework and record the results, then merge each set of results in the merge tool page.

Specify the batch name: or write the name here