Web Neural Network Models (WebNNM)

Existing neural network frameworks are huge and hard to use. WebNNM is a lightweight easy to understand high-level framework with an embedded expert system that simplifies working with neural networks for newcomers whilst offering experts fine grained control. This specification defines a high-level format for neural network models, and the API exposed by the associated JavaScript library. For an introduction and links to web-based demos, see [[WEBNNM-INTRO]].

WebNNM Expert System

WebNNM aims to make things easy for newcomers whilst giving expert users the control they seek. An expert system auto-configures hyperparameters unless explicitly overriden, along with providing warnings about potential problems.

Comparison with other frameworks

WebNNM iterates through the DAG to build a profile of the network (e.g., looking for normalization layers, dropout, and counting parameter-heavy operations). The profile is mapped against the provided hyperparams to either auto-configure settings or emit warnings. Here is a comparison with other neural network frameworks:

Feature	WebNNM	Keras (TF/JAX)	PyTorch Lightning	TensorFlow.js
Philosophy	Expert-Guided: Proactively analyzes the DAG to fix common pitfalls.	User-Centric: Provides building blocks; leaves stability to the user.	Research Abstraction: Removes boilerplate but requires manual config.	Deployment First: Focuses on running models; training is secondary.
Regularization	Capacity-Aware: Auto-enables L2 if a high-capacity model lacks Dropout.	Manual / Granular: Must be added per-layer (kernel_regularizer).	Optimizer-Linked: Usually manual via weight_decay in the optimizer.	Manual: Required per-layer; no global "auto-regularize" logic.
Gradient Stability	Automatic (DAG-based): Detects depth/norms to enable AGC or Global Norm.	Manual: User must manually add clipping to the optimizer.	Configurable: User sets flags; framework doesn't "suggest" values.	Manual: Very low-level; no "auto" scaling/clipping.
Precision (float16)	Adaptive: Auto-scales based on model data types and DAG risk.	AMP: Efficient, but doesn't "warn" about specific layer risks.	AMP Toggle: Simple toggle; no topological analysis of risk.	Semi-Manual: Requires manual casts and precision management.
Multimodal Loss	Auto-Balanced: Avoids underflow by learning relative weights automatically.	Manual: User must provide a dictionary/list of weights.	Manual: Logic is typically buried in the training_step.	Manual: Verbose implementation for multi-output loss.

Breakdown of the Heuristics

Loss Functions: Recommends the loss function based upon the final activation layer, the training label's tensor shape and data type, and an examination of a sample batch of training data. Warns when the user's choice of loss function is inappropriate due to a mismatch between the loss function and the data it will receive.
Dynamic Gradient Scaling: In WebNN, leveraging $float16$ is one of the best ways to get speedups, but the gradients can easily fall below the minimum representable positive value of $float16$ ($\approx 5.96 \times 10^{-8}$). WebNNM examines the ratio of $float16$ vs $float32$ heavy layers. If the network is primarily running its dense operations in $float16$, it auto-enables dynamic gradient scaling to protect the narrow dynamic range. If the user selects $float16$ but turns scaling off, it's almost guaranteed to stall, hence the strong warning.
Gradient Clipping (AGC vs. Global Norm): Standard deep networks benefit from Global Norm clipping. However, if the DAG scan reveals a deep network (maxDepth > 15) but no normalization layers, it recommends/sets Adaptive Gradient Clipping (AGC). This is a nod to architectures like DeepMind's NFNets (Normalizer-Free Networks), where AGC is required to train without Batch Normalization.
Regularization: The system conducts a structural audit of the DAG to deploy an intelligent regularization strategy that balances Generalization (L2) with Sparsity (L1) based on the model’s "High Capacity" profile—a metric derived from depth, width, and the count of parameter-heavy operators. If a model lacks a sufficient "safety net" (a Dropout Density below 0.2), the system automatically enables L2 Regularization to prevent weight explosion and training data memorization. Furthermore, if the system detects a "Massive Input" (over 5,000 features), it injects L1 Regularization to act as a noise-gate, pruning irrelevant dimensions through feature selection. This dual-layered approach ensures that high-dimensional data and deep architectures remain stable and compressible, providing smart defaults for newcomers while supporting the control required by experts.
Targeting Vulnerable float16 Ops: It explicitly flags layers like softmax, log, exp, and normalizations (which calculate variance via sum-of-squares). The maximum value representable in $float16$ is 65504; operations like $e^x$ easily blow past this limit, resulting in NaNs propagating backward through your DAG. Grouping the warnings prevents console spam.
NPU Gentle Nudging: By counting how many heavy parameters (matmul, conv2d) are lingering in $float32$, WebNNM can issue a single aggregate recommendation to the user. This bridges the gap between experts (who might want $float32$ precision explicitly) and newcomers (who might not realize their battery is draining because they missed an optimization opportunity).
Uncertainty-Based Loss Weighting: Balances multimodal/multi-task outputs, treating each output's relative scale as a learnable weight ($s_i$) optimized directly within the WebNN graph. This approach is essential because mismatched loss magnitudes (e.g., MSE vs. Cross-Entropy) can cause dominant tasks to "drown out" others, especially in $float16$ environments where underflow is a constant risk. By automating this balance via homoscedastic uncertainty, the model dynamically stabilizes the gradient flow without time-consuming manual tuning. This remains highly efficient for consumer NPUs as it eliminates CPU-GPU synchronization and provides per-task numerical scaling, ensuring all tasks contribute effectively to training.

For consumer grade equipment, NPUs often support $float16$ but not $float32$. To speed training and prolong battery life, it is desirable to enable WebNN to use the NPU where practical through the use of $float16$ where this doesn't detract from training stability. WebNNM supports automatic casting and switches to $float16$ for critical operations. Note that WebNNM does not support $bfloat16$ although that may be added in future if and when it is added to WebNN.

WebNNM Application Programming Interface

This section describes the API for inference, testing and training, along with the API used for datasets.

Here is an example that declares a model and then tests it using a simple dataset.

import {NNModel} from "../webnnm.js"; // import WebNNM library
import * as Iris from "./datasets/iris-dataset.js"; // import an example dataset
    
window.addEventListener("load", async () => { 
    let dataset = await Iris.Dataset.create('./datasets/iris-dataset.csv');

    const source = `
        model:name test-model;
        model:loss categoricalCrossEntropy;
        model:input shape=[4];
        model:output shape=[3];
        model:layers 
            dense(shape=[3], activation=relu),
            dense(shape=[3], activation=softmax);
        dense:layers
            matmul(w),
            add(b),
            activation();
	`;

    const config = {
        source: source,  // or set url property to load from server
        dataset: dataset,
    };

    const model = await NNModel.create(config);
    
    await model.test(); // evaluate using dataset cited in config
});

The example starts with importing the WebNNM module and a dataset module. The application logic is called when the web page has finished loading. Note that the load event handler is marked as async, which is essential for subsequent use of await within the handler. The handler creates an instance of the dataset and declares the WebNNM model. These are passed to the NNModel.create() to create an instance of the model and bind it to the dataset. Finally, the application calls model.test() to apply the test subset of data provided by the dataset. Note that the WebNNM library calls the log function to log messages. This function defaults to console.log, but can be overridden by the application to log messages to the web page.

Inference

This section describes the API for inference.

Here is an example that declares a model and and runs it.

import {NNModel} from "../webnnm.js";

window.addEventListener("load", async () => { 

    const source = `
        model:name test-model;
        model:input int32 shape=[5];
        model:output float32 shape=[5];
        model:layers
            add(1, dataType=float16), 
            pow(2);
	`;

    const config = {
        source: source,  // or set url property to load from server
    };

    const model = await NNModel.create(config);

    // get a fresh inference context with default batch size of 1
    const context = await model.createContext();
    
    // initialise model input to 1
    let input = await context.setData([1, 2, 3, 4, 5]);
    
    // run model with the inference context
    await model.run(context);
    
    const output = await context.output();
        
    log(``);
    log(`input = ${model.view(input)}`);
    log(`output = ${model.view(output)}`);
});

The example starts with int32 for the input tensor and progressively casts it to float16 and finally to float32 for the output tensor. Note the use of numeric literals for operands for the add and pow operations.

The application calls model.createContext() to create a context object for inference. This object is used to initialise the input tensor before calling model.run(context) to create and execute the inference graph. context.output() is then used to retrieve the output tensor. Note the use of model.view() as a convenience function for logging tensors.

Context object methods:

setBatchSize(batchSize): Set the inference batch size, default = 1.
async input(name): Get the named input, defaults to first input.
async output(name): Get the named output, defaults to first output.
async randomize(blockName, lower, upper): Initialise input with named block to random numbers between lower and upper. If the block name is missing, it will initialise the first input. Default is -1 for lower and +1 for upper.
setData(blockName, data): Set input for named block from an array of numbers. If the block name is missing, it will initialise the first input. The data must be an array of numbers with the number items equal to the tensor size.

Testing

This section describes the API for testing.

Here is an example that declares a model and and run the test data on it.

import {NNModel} from "../webnnm.js"; // import WebNNM library
import * as Iris from "./datasets/iris-dataset.js"; // import an example dataset
    
window.addEventListener("load", async () => { 
    let dataset = await Iris.Dataset.create('./datasets/iris-dataset.csv');

    const source = `
        model:name test-model;
        model:loss categoricalCrossEntropy;
        model:input shape=[4];
        model:output shape=[3];
        model:layers 
            dense(shape=[3], activation=relu),
            dense(shape=[3], activation=softmax);
        dense:layers
            matmul(w),
            add(b),
            activation();
	`;

    const config = {
        source: source,  // or set url property to load from server
        dataset: dataset,
    };

    const model = await NNModel.create(config);
    
    await model.test(); // evaluate using dataset cited in config
});

model.test(): runs the testing graph on the testing subset of the dataset.

Training

This section describes the API for training.

Here is an example that declares a model and and trains it on the training data.

import {NNModel} from "../webnnm.js"; // import WebNNM library
import * as Iris from "./datasets/iris-dataset.js"; // import an example dataset
    
window.addEventListener("load", async () => { 
    let dataset = await Iris.Dataset.create('./datasets/iris-dataset.csv', {});

    const source = `
        model:name test-model;
        model:loss categoricalCrossEntropy;
        model:input shape=[4];
        model:output shape=[3];
        model:layers 
            dense(shape=[3], activation=relu),
            dense(shape=[3], activation=softmax);
        dense:layers
            matmul(w),
            add(b),
            activation();
	`;

    const config = {
        source: source,  // or set url property to load from server
        dataset: dataset,
    };

    const model = await NNModel.create(config);
    
    // training using training subset of data
    await model.train({
        epochs: 5000,
        optimizer: 'rlion',
        seeds: 50,
    });
    
    await model.test(); // independent assessment using testing subset of data
});

Training Hyperparameters

Training is similar to testing, but involves a set of training hyperparameters, so called to avoid confusion with the model's trainable parameters:

model.train(hyperparameters): runs the training graph on the training subset of the dataset.

hyperparameters is an object with the following optional properties:

epochs: the maximum number of epochs to train, default = 20
lr: the maximum learning rate, where the default depends on the choice of optimizer
warmupEpochs: for cosine annealing, use 0 to auto-calculate based on total epochs, default: 0
seeds: the number of seeds to scout to evaluate potential starting points for training, default: 100
patience: the maximum number of epochs to continue after minium loss, default: 500
lossWeights: to weight outputs when computing the epoch loss, default is 'auto' for automatic weighting
optimizer: name of the learning rate optimizer: 'lion', 'rlion', 'nesterov', or 'sgdm', default: 'rlion'
weightDecay: an attribute for learning optimizers, default depends on optimizer
momentumFactor: an attribute for the sgdm optimizer, default depends on optimizer
beta1: an attribute for the lion and rlion optimizers, default depends on optimizer
beta2: an attribute for the rlion optimizer, default depends on optimizer
freeze: what proportion of the layers to freeze for transfer learning (0.0 to 1.0) default: 0.0
regularization: 'auto', L1', 'L2', ''L1+L2' or 'off', default: 'auto'
lambda1: weight for L1 regularization, default: zero
lambda2: weight for L2 regularization, default: zero
optimizeFor: 'size' or 'edge', default: 'size'
gradientScaling: 'auto', 'on' or 'off', default: 'auto'
gradientClipping: 'auto', 'global', 'agc' or 'off', default: 'auto'
scale: initial gradient scaling factor, default = 1000
growthInterval: number of batches before increasing scale factor, default: 1000

Validation, testing and learning rate optimizers

The model parameters are validated after each epoch if the dataset provides a validation subset. On detecting a minima in the loss, the model weights are saved, and training continues for a given number of epochs. The training loss is reported every 50 epochs. Applications should always test the model when training completes. Testing the model on independent data after training is essential to verify that the network has truly learned generalizable patterns rather than simply memorizing the training data.

WebNNM supports a choice of learning rate optimizer. The choices all involve tracking the gradient and its momentum for each of the model parameters:

EvoLved Sign Momentum (lion): A memory-efficient stochastic gradient descent method discovered by Google. Unlike Adam, which tracks both first and second-order moments, Lion only tracks momentum. It uses the sign operator on the momentum to determine the update direction, allowing for a smaller memory footprint and often better performance at large batch sizes.
Refined Lion (rlion): Improves upon the original Lion optimizer by replacing the discrete sign function with a continuous, non-linear bounded function, yielding more stable updates. WebNNM uses softsign in place of arctan, which isn't supported by WebNN.
Nesterov Accelerated Gradient (nesterov): An advanced momentum-based optimizer that improves upon standard gradient descent by incorporating a "look-ahead" mechanism. Instead of calculating the gradient at the current parameter position, it computes the gradient at the position where the parameters will be after applying the current momentum velocity. This anticipation allows the algorithm to correct its course more effectively, leading to faster convergence and reduced oscillations.
Stochastic Gradient Descent with Momentum (sgdm): A popular optimization algorithm that enhances standard Stochastic Gradient Descent (SGD) by adding a momentum term. This term accumulates an exponential moving average of past gradients to accelerate convergence, help navigate narrow valleys, and avoid getting stuck in shallow local minima.

Scouting for good starting points

Training can be rather time consuming, so it makes sense to avoid waisting time when the initial choice of model parameter values doesn't train well. WebNNM creates a scouting graph to evaluate a set of randomly chosen potential starting points using a few heuristics. The metrics include: Loss, Gradient Ratio, and Dead Ratio. The gradient ratio detects vanishing or exploding gradients, whilst the dead ratio detects dead neurons or overconfident layers.

Cosine Annealing

WebNNM manages training in terms of a warm up phase in which the learning rate is gradually increased to the maximum learning rate. After that the learning rate is gradually reduced using cosine annealing to a very low value when reaching the maximum number of epochs. The default value for warmupEpochs is 5% of the maximum number of epochs, or 1, whichever is larger.

Multimodal/Multi-task Models

Multimodal models combine different modalities, e.g. video and audio. Multimodal models provide superior accuracy and robustness by leveraging redundant information and cross-sensory context to resolve ambiguities that a single data source cannot. Multi-task models are trained on multiple objectives from a shared representation. WebNNM allows you to specify the relative weights of each modality (or task) to avoid one modality drowning out the learning signals for another, especially in $float16$ environments where underflow is a constant risk. WebNNM addresses this challenge using Uncertainty-Based Weighting. Each task $i$ is assigned a learnable log-variance $s_i$. The total loss $L$ is computed as:$$L = \sum_i \left( \frac{1}{2e^{s_i}} L_i + \frac{1}{2}s_i \right)$$ This allows the optimizer to dynamically attenuate noisy tasks. To ensure stability in $float16$, $s_i$ is initialized at $0.0$ and a gradient scale $S \ge 128$ is recommended.

Transfer Learning

WebNNM supports transfer learning by allowing you to freeze parameter updates for the early layers of the network. This is based upon annotating each node in the DAG with the maximum number of nodes to reach an input node, and likewise to reach an output node. The freeze hyperparameter is a number in the range 0 to 1, defaulting to 0. It describes the proportion of layers to freeze, starting from the inputs. Parameters associated with nodes closer to the outputs are updated with a fraction that goes from 0 to 1 in a sinusoidal pattern (in range 0 to $\pi/2$). This range is discretised into 5 buckets to enable kernel optimization.

Training Stability

Training involves computing gradients that risk vanishing or exploding. For models with many layers it is good practice to introduce skip connections that bypass a few layers that would otherwise overly weaken the learning signals. It is common to use float32 whilst training to provide an adequate precision for gradients. However, the hardware accelerators in consumer grade equipment may not support float32, motivating the use of a lower precision, e.g. float16. This increases the risk of underflow and overflow during gradient calculations. Another consideration is that whilst GPUs can be used for hardware acceleration of neural networks, they consume considerably more electrical power than NPUs like Apple's ANE. To speed up training in the browser and reduce power consumption, it is desirable to find ways to increase training stability when using float16.

To address this challenge, WebNNM supports techniques to boost weak gradients, to scale down overly large gradients, and ignore updates that overflow the maximum number supported for a given precision. The scaling hyperparameter controls gradient scaling, whilst clipping controls clipping. If you set these to 'auto' (the default) the library examines the width and depth of the model along with the activation functions to assess whether to enable scaling and clipping. For explicit control set these hyperparameter to 'on' or 'off' as desired.

To prevent underflow during the backward pass, WebNNM multiplies the loss $L$ by a scale factor $S$ (where $S \gg 1$) before backpropagation begins. By the chain rule, this scales every subsequent gradient in the computational graph:$$\nabla_{\theta} (L \cdot S) = S \cdot \nabla_{\theta} L$$This ensures that the intermediate gradients $g$ that would have been $10^{-7}$ (and thus zeroed out in float16) are now $g \cdot S$, keeping them within the precision range. We cannot apply the scaled gradients directly to the model's parameters, as this would be equivalent to using a learning rate that is $S$ times too large. Before the optimizer updates the parameter $\theta$, the gradients need to be unscaled back to their original magnitude:$$\theta_{t+1} = \theta_t - \eta \cdot \frac{\sum \nabla_{\theta} (L \cdot S)}{S}$$

Use the scale hyperparameter to set the initial gradient scaling factor. The scale is doubled until overflow is detected at which point it is halved. To prevent instability, the growthInterval hyperparameter sets the number of batches without overflow to wait before boosting the scale factor. The forward and backward passes may use float16 for speed, but it is better to keep the model parameters in float32, and likewise to cast from float16 to float32 for operations like softmax and batchnorm. WebNNM will do this for you unless you explicitly set the data types in the model.

Clipping is based upon the scaled global (L2) norm $||\mathbf{G}||_2$ which is defined as the square root of the sum of the squares of every individual scalar value across all gradient tensors $g_i$:$$||\mathbf{G}||_2 = \sqrt{\sum_{i=1}^{n} \sum_{x \in g_i} x^2}$$

The training graph computes the rolling mean for the scaled global norm and its absolute deviation, as a basis for dynamically setting the clipping threshold:

Update mean: $\mu_{t} = \alpha \mu_{t-1} + (1 - \alpha) G$, where G is the current scaled global norm, and $\mu_0$ is set to the initial scaled global norm
Update deviation: $\sigma_{t} = \alpha \sigma_{t-1} + (1 - \alpha) |G - \mu_t|$, where $\sigma_o$ is set to zero
Compute dynamic threshold: $Threshold = \mu_t + (k \cdot \sigma_t)$
If $G > Threshold$, scale the gradients by $\frac{Threshold}{G+\epsilon}$, where $\epsilon$ is a very small positive number used to prevent the possibility of dividing by zero

Note that $k$ is 4.0 for the first 15 batches and 2.0 after that.

Work is underway to support automated gradient clipping (AGC) and regularization on a layer by layer basis.

Datasets

This section describes the API for datasets.

Appendices

WebNNM Model Syntax

This section defines the grammar for the model syntax.

Loss Functions

WebNNM supports the following loss functions and checks that they are consistent with the activation function, label format and a sample of the training data. You can set the loss function explicitly as the loss property for the associated block(e.g. model:loss CCEL;), otherwise WebNNM will pick one for you based upon the context. Note that some loss functions have parameters, e.g. $\delta$ for Huber Loss and $\gamma$ for Focal Loss. These can be specified as block properties, e.g. model:delta 2.1;. If they are not defined, the WebNNM expert system will set them based on the standard deviation of the error observed during the initial scouting pass.

Loss Function	Mathematical Form (Loss)	Explanation	Best Suited For	WebNN Implementation Primitives
Mean Squared Error (MSE)	$\frac{1}{n}\sum(y - \hat{y})^2$	Calculates the average of the squares of the errors	Standard Regression: Best for tasks where targets are continuous values and you want to penalize large errors more heavily than small ones, e.g. prediction house prices.	`sub` $\rightarrow$ `pow(2)` $\rightarrow$ `reduceMean`
Mean Absolute Error (MAE)	$\frac{1}{n}\sum\\|y - \hat{y}$	Calculates the average of the absolute difference between targets and predictions	Robust Regression: Ideal when your dataset contains significant outliers as it is less sensitive to extreem values than MSE.	`sub` $\rightarrow$ `abs` $\rightarrow$ `reduceMean`
Binary Cross-Entropy (BCE)	$-\frac{1}{n}\sum [y\ln(p) + (1-y)\ln(1-p)]$	Measures the performance of a classification model whos output is a probability value between 0 and 1	Binary Classification: Used for single-label or multi-label "yes/no" tasks.	`log`, `mul`, `neg`, `sub`
Categorical Cross-Entropy (CCE)	$-\sum y_i \ln(\hat{y}_i)$	Measures the difference between two probability distributions	Multi-class Classification: Standard for "one-of-many" classification. Expects Softmax inputs.	`log`, `mul`, `reduceSum`, `neg`
CCE with Logits (CCEL)	$-\sum y_i \cdot {LogSoftmax}(\hat{y}_i)$	Combines Softmax activation and Cross-Entropy into a single stable step	Stability Choice: Expert system should force this if `NPU` (FP16) is detected to prevent overflow.	`LogSoftmaxNode` $\rightarrow$ `mul` $\rightarrow$ `neg`
Sparse Categorical Cross-Entropy (SCE)	$-\ln(\hat{y}_{target})$	Memory-Efficient Multi-class: Identical to CCE but accepts integer labels (e.g. 3) instead of one-hot vectors ([0,0,1,0])	Memory Efficient: For use when labels are integer indices rather than one-hot vectors.	`gather` (to select logits) $\rightarrow$ `neg`
Hinge Loss (HIL)	$\sum \max(0, 1 - y_i \cdot \hat{y}_i)$	Support Vector Machines: A "maximum-margin" loss function	Used for SVM-style classification; robust to small variations, where you want a safety margin between class boundaries.	`mul`, `sub`, `relu`
Huber Loss (HUL)	$\begin{cases} \frac{1}{2}(y-\hat{y})^2 & \text{if } \text{error} \leq \delta \\ \delta(\\|y-\hat{y}\\| - \frac{1}{2}\delta) & \text{else} \end{cases}$	General Purpose Regression: Acts as MSE for small errors and MAE for large errors	Balanced Regression: Switch to this if `scout` detects high variance in gradients.	`sub`, `abs`, `where`, `mul`
KL Divergence (KLD)	$\sum y_i \ln(\frac{y_i}{\hat{y}_i})$	Distribution Matching: Measures how a probability distribution approximates a reference distribution	Used in Variational Autoencoders (VAEs) and Knowledge Distillation	`div` (or `sub` of logs) $\rightarrow$ `mul`
Focal Loss (FOL)	$-\alpha(1 - p_t)^\gamma \ln(p_t)$	Adds a $(1 - p_t)^\gamma$ factor to the Cross-Entropy loss	Imbalanced Data: Recommended if the `dataset` class counts are skewed, where the model needs to focus on harder examples rather than on the easier ones.	`pow`, `sub`, `log`, `mul`

WebNNM Snapshot Binary Format

This section defines the binary format for models and their parameters.

WebNNM Test Harness

This section describes the test harness used to validate the WebNNM library's implementation of analytic gradients. The approach taken is to measure the gradients for each parameter and compare it to the analytic value computed by the WebNNM library for each operator. This done for repeatedly for a set of randomly initialised tensors for the parameters and the input from the previous layer. In more detail, for each operator:

Compute the tensor for the parameter's analytic gradients
Compute the tensor for the parameter's measured gradients: iteratively for each element in the tensor, compute the loss after subtracting epsilon from the element, and again after adding epsilon to the element, where epsilon is a small positive number.

Note: the analytic gradients for the WebNN recurrent operators, e.g. LSTM and GRU are computed by mapping them to the primitive operations and unrolling over time for a given sequence length.

Introduction

Conformance classes

Blocks and Layers

Directed Acyclic Graphs