The decentralized nature of DIDs has inspired a wide range of DID methods, as different innovators document novel approaches for using DIDs with different Verifiable Data Registries. At last count, there were 226 methods listed in the W3C extensions registry and countless others in use. This means that implementers are faced with the challenge of choosing which methods to support. This is true not only for components like DID Resolvers, which necessarily process method-specific features, but also for issuers and verifiers, who must decide independently with method meets their business needs for features like revocation, provenance, and cost.
How do implementers decide which methods are suitable for their use?
Applying the DID Method Rubric is one answer to that question. This document defines a coherent way to evaluate multiple DID methods on a feature-to-feature basis, tailored to the business context addressed by implementers.
The rubric presents a set of criteria which an Evaluator can apply to any DID Method based on the use cases most relevant to them. Rather than reducing this Evaluation to a single number, the rubric retains the sensibility that requirements for use tend to be multidimensional and many of the possible responses are not necessarily good or bad. It is up to the Evaluator to understand how each response in each criteria might illuminate favorable or unfavorable consequences for their needs.
While this rubric assists in the evaluation of many aspects of a DID Method, it is not intended to be exhaustive. Evaluators are encouraged to extend the framework to incorporate their own criteria (and hopefully share those criteria back to the community when possible).
A rubric is a tool used in academia to communicate expectations and evaluate performance. It consists of a set of criteria to be evaluated, possible responses for each criteria, and a scoring guide explaining how to both choose and interpret each response. The act of evaluating a rubric, which we call an Evaluation, provides basis for self-evaluation, procurement decisions, or even marketing points. Written records of an evaluation, which we'll call an Evaluation Report, document how a particular subject is measured against the criteria. For students, a rubric helps to clarify how their work will be evaluated by others. For Evaluators, a rubric provides a consistent framework for investigating and documenting the performance of a subject against a particular set of criteria.
We were inspired to develop a rubric for decentralization when discussions about the requirements for decentralized identifiers, aka DIDs, led to intractable disagreement. It became clear that no single definition of "decentralized" would suffice for all of the motivations that inspired developers of DID Methods to work on a new decentralized approach to identity. Despite this challenge, two facts remained clear:
Rather than attempt to force a definition of "decentralized" that might work for many but would alienate others, the group set out to capture the measurable factors that could enable Evaluators to judge the decentralization of DID Methods based on their own independent requirements, with minimal embedded bias.
This approach identified several useful criteria about methods that have nothing to do with decentralization. Rather than create a separate rubric for these criteria, the DID WG extended the scope of the rubric to include any criteria that might be useful for evaluators.
Pick the most important criteria for your use, ask each question, and select the most appropriate response. Do this for all of the DID Methods under consideration.
Each Evaluation should start with an explicit framing of the use under consideration. Are you evaluating the Method for use in Internet-of-Things (IoT)? For school childrens' extra-curricular activities? For international travel? The use, or set of uses, will directly affect how some of the questions are answered.
Where a given Method offers variability, such as multiple networks for the same Method, then evaluate each variant. For example, did:ethr supports Ethereum mainnet, multiple testnets and permissioned EVM-compliant networks such as Quorum. To apply a criteria to did:ethr, you will evaluate it against all the variations that matter to you. Each variation should get its own Assessment. This applies to Level 2 Networks that can operate on multiple Level 1 Networks as well as DID Methods that directly offer support for multiple underlying DID registries.
When creating an Evaluation Report, we recommend noting both the Evaluator and the date of the Evaluation. Many of the criteria are subjective and all of them may evolve. Tracking who made the Evaluation and when they made it will help readers better understand any biases or timeliness issues that may affect the applicability of the Evaluation.
Be selective and acknowledge the subjective. Evaluations do not need to be exhaustive. There is no requirement to answer all the questions. Simply answer the ones most relevant to the use contemplated. Similarly, understand that any recorded Evaluation is going to represent the biases of the Evaluator in the context of their target use. Even the same Evaluator, evaluating the same Method for a different use, may come up with slightly different answers—for example, that which is economically accessible for small businesses might not be cost-effective for refugees, and that could affect how well-suited a given Method is for a specific use.
Finally, note that the rubric is not just about decentralization. There are security, privacy, and economic concerns that are considered. However, no such list of criteria could ever be exhaustive and none should be considered "complete". We encourage Evaluators to use this rubric as a starting point for their own work rather than the final say in the merit of any given Method.
In short, use this rubric to help understand if a given DID Method is appropriate for your needs, whatever your use cases may be.
To record and report an Evaluation, we recommend two possible formats, either comprehensive or comparative.
A comprehensive Evaluation applies a single set of criteria to just one Method. This set is chosen by the Evaluator; it need not be all possible criteria, but it is all relevant criteria as judged by the Evaluator.
A comparative Evaluation includes multiple Methods in the same table to easily compare and contrast two or more different Methods. This may include any number of criteria.
In addition to the selected criteria, we recommend each report specify
We have grouped our criteria into several categories:
Evaluators should consider criteria from all groups, as best fits your use cases.
Three categories cover how a given Method is governed: Rulemaking, Operations, and Enforcement. Our approach parallels the same separation of authority embodied in the tripartite governments.
Rulemaking addresses who makes the rules and how. (This is the legislative branch.)
Operations addresses how those rules are executed and how everyone knows that they are carried out. (This is the executive branch.)
Enforcement addresses how we find and respond to rule breaking. (This is the judicial branch in the US.)
This mental model is key to understanding the criteria of each section as well as why we included some criteria and not others.
The remaining categories each covers different areas worth considering when evaluating DID Methods.
Design addresses the method as designed. In other words, the output of the rulemaking: what rules apply to this DID method?
Adoption & Diversity covers questions related to how widely a DID Method is used.
Security influences overall trust in the ecosystem. Different DID methods offer different security guarantees, or guarantees of different strengths.
Privacy addresses the ability of a DID method to ensure various privacy mechanisms. When DIDs are used as identifiers for people, it becomes important to consider what tools a DID method offers to operate at different levels of privacy.
When evaluating the governance of DID Methods, three potentially independent layers should be considered: the specification, the network, and the registry.
For Rulemaking, the criteria should be evaluated against all three of the above layers.
For Operations, the criteria should be evaluated against the network and the registry. The specification is taken as a given (it is the core output of Rulemaking).
For Adoption, the criteria should be evaluated for each major software component: wallet, resolver, and registry.
For the examples in the rest of this document we refer to a set of Methods that are familiar to the authors and exhibit interesting characteristics for Evaluation. See the tables below.
The following sources provided example evaluations for this rubric. These are not presented as objective fact, but rather as attempts by contributors to illustrate noteworthy differences using their subjective judgement. Additional contributions that flow into the registry should include a section that documents their source with an additional row in the table.
This is not a comprehensive list (see the DID Specification Registries for a current list of registered Methods). Rather, this section aggregates the methods used in Example Assessments of current criteria for easy reference.
The term "criteria" is often treated as both a singular and a plural noun. In the singular, we say "The most important criteria is the buyer's age". This singular use of criteria has been in use for over half a century https://www.merriam-webster.com/dictionary/criterion#usage-1 In the plural we say "That proposal doesn't meet all of the criteria." Sometimes people use both in the same sentence: "Select one criteria from the list of criteria."
However, for formal use, "criteria" is more broadly accepted as plural while "criterion" is singular. By this style rule, the last sentence in the previous paragraph should be "Select one criterion from the list of criteria."
As editors of a Note published by the World Wide Web Consortium, we are torn. We would prefer to use rigorous grammar and be consistent in doing so. At the same time, we find attempts to enforce the formal rule sometimes leads people to using the inverse, such as "Select a criteria from the list of criterion." This is the exact opposite of our desired outcome.
In our own work with implementers and developers of the DID Core specification, we have found that the singular "criteria" is readily accepted and understood and leads to no confusion when used, even alongside plural usage. Since the DID Method Rubric is, at its core, a set of criteria with ample reason to refer to, for example, "Criteria #23", we find the combined singular and plural use is cleaner (just stick with "criteria"), less confusing, and more aligned with common usage among our audience.
As such, throughout the DID Method Rubric, we use the term "criteria" to refer to both singular instances of criteria and plural sets of criteria.
This registry — the DID Method Rubric Registry — provides a public vehicle for publishing updated DID Method Rubric criteria. In order to add or update a criteria, a submitter MUST submit a modification request for this registry, as a pull request on the repository where this registry is hosted, where the modification request adheres to the following policies:
The primary components managed by this registry are criteria for evaluating DID Methods, with as many as eight subcomponents: name, id, version, question, assessmentTemplate, response, relevance, exampleAssessments, and, optionally, a source. In addition, the DID Method Rubric maintains a list of cited DID methods and evaluations. The criteria are independently identified and versioned (see those sections for details) while references cited in the criteria themselves link to their full citation in the appropriate list.
rubric/criteria/ directory. The file
name MUST be the criteria's number (from its
id, e.g., 1 for criteria-1) followed by a
kebab-case slug of the criteria name and the
.json file extension (e.g.,
1-open-contribution-participation.json).
id to the appropriate category in
rubric/rubric-outline.json. The rubric
outline determines the organization and ordering of
criteria within the rendered document. A criteria file
that is not referenced in the outline will not appear
in the Rubric.
The following is an example of a complete criteria JSON file:
{
"name": "Offline creation",
"id": "https://www.w3.org/TR/did-rubric#criteria-42",
"version": "1.0.0",
"source": "https://didcriteria.com/criteria/7",
"question": {
"question": "Does the Method require network communications to create a DID?"
},
"response": {
"type": "multipleChoice",
"possibleResponses": [
{
"label": "A",
"meaning": "No. Creation is expected to be off-line."
},
{
"label": "B",
"meaning": "Yes. Creation requires network coordination with a single party."
},
{
"label": "C",
"meaning": "Yes. Creation requires network coordination with multiple parties."
}
]
},
"assessmentTemplate": {
"columns": [
{
"heading": "Method",
"type": "method",
"propertyRef": "method"
},
{
"heading": "Spec.",
"type": "enhancedLetter",
"propertyRef": "spec"
},
{
"heading": "Notes",
"type": "note",
"propertyRef": "note"
}
]
},
"relevance": "Communication is costly, with increasing costs the more parties are involved.",
"exampleAssessments": [
{
"id": "a-29",
"method": "did:v1.testnet",
"spec": "A",
"note": "Veres One DID creation is a local cryptographic process.",
"evaluationCitation": "#eval-3"
},
{
"id": "a-31",
"method": "did:web",
"spec": "B+",
"note": "The DID controller must be able to communicate with the web server.",
"evaluationCitation": "#eval-4"
}
]
}
Each criteria needs a human-friendly, short, descriptive name that captures the essence of the criteria as concisely as possible.
The criteria id must be explicit, unique, and persistent. See the section on Identifiers for details. Criteria ids under the DID Method Rubric namespace will be assigned by the rubric maintainers.
The criteria version must increment, as appropriate, any time the criteria, or any of its subcomponents, is updated. See the section on Versioning for details.
The key question for evaluating the criteria. This property is REQUIRED.
Any additional instructions for how the evaluator should interpret or apply the question when making their assessment. This property is OPTIONAL.
An ordered array of column definitions. Each column
specifies a heading, a type, and a property reference.
This property is REQUIRED. The columns array MUST
include a column with a type of method as
the first entry, which identifies the specific DID
method being assessed.
The display label for the column in the assessment table. This property is REQUIRED.
The type of data expected in this column. This property is REQUIRED. The following types are currently defined, but additional types MAY be introduced as needed:
method
enhancedLetter
multipleChoice
note
The property name used in example assessments to provide the value for this column. This property is REQUIRED.
This subcomponent defines the options an evaluator has for responding to the criteria's question.
The type of response expected from evaluators. This property is REQUIRED. Currently defined types include:
multipleChoice
An ordered array of the possible responses available to
evaluators. This property is REQUIRED when the response
type is multipleChoice.
A short identifier for the response. Labels MUST start with "A" and progress sequentially through the English alphabet. This property is REQUIRED.
A description of what the label represents. This property is REQUIRED.
This subcomponent explains why this criteria is useful. Readers should be able to understand the extent to which this particular criteria is applicable to their situation.
This subcomponent lists example assessments of different DID methods against this criteria. Each example assessment is an evaluation of a specific DID method, structured according to the criteria's assessmentTemplate. The set of example assessments is presented as a table for easy comparison across methods.
This OPTIONAL subcomponent tracks the provenance of the criteria. Criteria in the Rubric may be independently versioned and developed externally under their own identifiers. When such iterations are folded back into the Rubric, the source MUST point to the external origin so that the versioning history is preserved.
The value MUST be either:
"."
If no source is provided, the default value is
".".
Each criteria must be explicitly, uniquely, and persistently identified using incremental numbers. New criteria should use the next available increment based on the highest numbered identifier in the current publication.
Previous numbered identifiers MUST NOT be re-used, even if the criteria so identified are no longer published; this maintains the persistent linkability of previously published versions. Such "retired" criteria MAY be listed in a Prior Criteria section linking directly to the latest published Rubric containing the retired criteria; this Prior Criteria section MUST also contain an anchor with the canonical id.
The registry shall maintain an entry with the "next available" number. Additions should use the "next available" number to construct an identifier for the new criteria, and update the "next available" entry at the same time. Editors will manage any sequencing errors when accepting PRs.
Identifiers must be constructed by appending that numerical value to the string "criteria-" and incorporated in the ID of the heading element for that criteria, which when placed in the Rubric appears as something like (note the version in its own span after the permalink):
<h4 id="criteria-1">Open contribution (participation)</h4>
<a href="#criteria-1">https://www.w3.org/TR/did-rubric#criteria-1</a> <span>1.0.0</span>
This allows permanent links to citations based on the publication date of a given Rubric: https://www.w3.org/TR/2021/NOTE-did-rubric-20210826/#criteria-32 as well as version-free links that resolve to the latest version of the criteria, and if retired, to an entry in the Prior Criteria list which itself links to the last valid presentation of the criteria. So that, for example, https://www.w3.org/TR/did-rubric/#criteria-32 points to the current criteria-32 in the latest published document. When that criteria is retired, that same URL points to the entry in the Prior Criteria list, which links to its last versioned publication, e.g., https://www.w3.org/TR/2021/NOTE-did-rubric-20210826/#criteria-32
Criteria versions allow for incremental improvements while retaining long-term referenceability.
Every example evaluation cited in the Notes entry in a assessment MUST link to a proper citation in this DID Evaluations Cited table.
This is not a comprehensive listing of DID Method Rubric evaluations. It is expected that such evaluations are developed and published elsewhere, in support of various methods and use cases. Only those evaluations cited as Example Assessments in current criteria will be listed.
Evaluation entries defined by creating a JSON file under the rubric/evaluationCitations folder in the github repository. This JSON file MUST include:
Every DID method cited in an Example Assessment MUST link to a proper citation in the Methods Considered table. The table MUST only contain methods that are explicitly cited in the Example Assessments.
The table is generated from individual JSON files under
rubric/methodsConsidered/. To add or
update a method, edit the corresponding JSON file rather than
this table directly.
Each entry SHOULD contain the following:
The Editors of the DID Method Rubric Registry MUST consider all of the policies above when reviewing additions to the registry and MUST reject registry entries if they violate any of the policies in this section. Entities registering additions can challenge rejections first with the W3C DID Working Group and then, if they are not satisfied with the outcome, with the W3C Staff. W3C Staff need not be consulted on changes to the DID Method Rubric Registry, but do have the final authority on registry contents. This is to ensure that W3C can adequately respond to time sensitive legal, privacy, security, moral, or other pressing concerns without putting an undue operational burden on W3C Staff.
Submissions to the registry that meet all the criteria listed above will be considered for inclusion, however the editors retain the responsibility of curating the content to ensure the broadest applicability of the DID Method Rubric with the goal of enabling effective evaluations of any DID Method for any legitimate use case.
The following criteria have been retired. We link to their last canonical publication for historical provenance. Many of these criteria have evolved into newer criteria, others have simply been removed from this Rubric for lack of completeness, currency and/or relevance.
This DID Method Rubric one framework for evaluating DID Methods. It offers a set of criteria which can be used selectively by Evaluators to better understand and document their considerations when deciding to support or adopt a given DID Method.
This Note is a derivative work of [[[DID-DEC-RUBRIC]]], a collaborative paper written at Rebooting the Web of Trust IX by Joe Andrieu, Shannon Appelcline, Amy Guy, Joachim Lohkamp, Drummond Reed, Markus Sabadello, and Oliver Terbu.