9.1 Example

This example serves as a basis for elaboration that will be described in subsequent sections. It helps to illustrate how best practices may be applied.

9.2 Metadata

The Web is an open information space, where the absence of a specific context, such a company's internal information system, means that the provision of metadata is a fundamental requirement. Data will not be discoverable or reusable by anyone other than the publisher if insufficient metadata is provided. Metadata provides additional information that helps data consumers better understand the meaning of data, its structure, and to clarify other issues, such as rights and license terms, the organization that generated the data, data quality, data access methods and the update schedule of datasets.

Metadata can be of different types. These types can be classified in different taxonomies, with different grouping criteria. For example, a specific taxonomy could define three metadata types according to descriptive, structural and administrative features. Descriptive metadata serves to identify a dataset, structural metadata serves to understand the structure in which the dataset is distributed and administrative metadata serves to provide information about the version, update schedule etc. A different taxonomy could define metadata types with a scheme according to tasks where metadata are used, for example, discovery and re-use.

   :dataset-001
       a dcat:Dataset ;
       dct:title "Bus timetable of MyCity" ;
       dcat:keyword "transport","mobility" ,"bus" ;
       dct:issued "2015-05-05"^^xsd:date' ;
       dct:modified "2015-05-05"^^xsd:date' ;
       dcat:contactPoint <http://example.org/transport-agency/contact>;
       dct:temporal <http://reference.data.gov.uk/id/year/2014>;
       dct:spatial <http://www.geonames.org/3399415>;
       dct:publisher :transport-agency-mycity  ;
       dct:accrualPeriodicity <http://purl.org/linked-data/sdmx/2009/code#freq-A>  ;       
       .

   :dataset-001
       a dcat:Dataset ;
       dct:title "Bus timetable of MyCity" ;
       dcat:keyword "transport","mobility" ,"bus" ;
       dct:issued "2015-05-05" ;
       dct:modified "2015-05-05" ;
       dcat:contactPoint <http://example.org/transport-agency/contact>;
       dct:temporal <http://reference.data.gov.uk/id/year/2014>;
       dct:spatial <http://www.geonames.org/3399415>;
       dct:publisher :transport-agency-mycity  ;
       dct:accrualPeriodicity <http://purl.org/linked-data/sdmx/2009/code#freq-A> ; 
       dct:language <http://lexvo.org/id/iso639-3/eng> ;
       dct:language <http://lexvo.org/id/iso639-3/por> ;
       .

9.3 Data Licenses

A license is a very useful piece of information to be attached to data on the Web. As defined by the Dublin Core Metadata Initiative [DC-TERMS], a license is a legal document giving official permission to do something with the data with which it is associated. According to the type of license adopted by the publisher, there might be more or fewer restrictions on sharing and re-using data. In the context of data on the Web, the license of a dataset can be specified within the data, or outside of it, in a separate document to which it is linked.

   :dataset-001-csv
       a dcat:Distribution ;
       dcat:mediaType "text/csv";
       dct:license <http://creativecommons.org/licenses/by-sa/3.0/>;
       .

9.4 Data Provenance

Provenance originates from the French term "provenir" (to come from), which is used to describe the curation process of artwork as art is passed from owner to owner. Data provenance, in a similar way, is metadata that allows data providers to pass details about the data history to data users. Provenance becomes particularly important when data is shared between collaborators who might not have direct contact with one another either due to proximity or because the published data outlives the lifespan of the data provider projects or organizations.

The Web brings together business, engineering, and scientific communities creating collaborative opportunities that were previously unimaginable. The challenge in publishing data on the Web is providing an appropriate level of detail about its origin. The data publishers may not necessarily be the data provider and so collecting and conveying this corresponding metadata is particularly important. Without provenance, consumers have no inherent way to trust the integrity and credibility of the data being shared. Data publishers in turn need to be aware of the needs of prospective consumer communities to know how much provenance detail is appropriate.

   :dataset-001
       a dcat:Dataset , prov:Entity;
       dct:title "Bus timetable of MyCity" ;
       dcat:keyword "transport","mobility" ,"bus" ;
       dct:issued "2015-05-05" ;
       dct:modified "2015-05-05" ;
       dcat:contactPoint <http://example.org/transport-agency/contact>;
       dct:temporal <http://reference.data.gov.uk/id/year/2014>;
       dct:spatial <http://www.geonames.org/3399415>;
       dct:publisher :transport-agency-mycity  ;
       dct:accrualPeriodicity <http://purl.org/linked-data/sdmx/2009/code#freq-A>  ; 
       dct:language <http://id.loc.gov/vocabulary/iso639-1/en>  ;
       prov:wasAttributedTo :john;  
       .
       
  :john
       a foaf:Person, prov:Agent;
       foaf:givenName "John";
       foaf:mbox ;
       prov:actedOnBehalfOf :transport-agency-mycity;
       .
  :transport-agency-mycity
       a foaf:Organization, prov:Agent;
       foaf:name "Transport Agency of Mycity";
       .

9.5 Data Quality

Data quality is commonly defined as “fitness for use” for a specific application or use case. It can affect the potentiality of the application that use data, as a consequence, its inclusion in the data publishing and consumption pipelines is of primary importance.

Usually, the assessment of quality involves different kinds of quality dimensions, each representing groups of characteristics that are relevant to publishers and consumers. Measures and metrics are defined to assess the quality for each dimension. There are heuristics designed to fit specific assessment situations that rely on quality indicators, namely, pieces of data content, pieces of data meta-information, and human ratings that give indications about the suitability of data for some intended use.

9.6 Data Versioning

Data on the Web often changes over time. Many datasets are updated on a scheduled basis, such as census data or funding data that changes every fiscal year. Other datasets are changed as improvements in collecting the data make updates worthwhile. Still other data changes in real time or near real time. All these types of data need a consistent, informative approach to versioning, so data consumers can understand and work with the changing data.

In order to deal with changes over time, multiple versions may be created for a single dataset. To illustrate this let us consider a simple example: a dataset that collects data about weekly weather forecast for MyCity. The following figure shows the relation between the dataset (Dataset001) and its different versions (Dataset001_W1, Dataset001_W2, Dataset001_W3 and Dataset001_W4), where each version corresponds to the weather forecast of a week of May 2015. Each dataset version has two different distributions: one in CSV and one in JSON. For example, the version Dataset001_W1 has two distributions: Dataset001_W1_csv and Dataset001_W1_json.

The following best practices address issues that arise in tracking and managing different versions of datasets.

curl -L -I http://dbpedia.org/resource/Paris

http://mementoarchive.lanl.gov/dbpedia/timegate/http://dbpedia.org/page/Paris

http://dbpedia.mementodepot.org/memento/20120515/http://dbpedia.org/page/Paris

curl -H "Accept-Datetime: Sat, 16 Jun 2012 00:00:00 GMT" -L -I http://mementoarchive.lanl.gov/dbpedia/timegate/http://dbpedia.org/page/Paris

9.7 Data Identification

Identifiers are simple conventions of labels that allow us to distinguish what is being identified from anything else. Identifiers are used extensively in every information system, making it possible to refer to any particular element. The Web is predicated on a uniform system of identifiers that are globally unique and can be looked up by dereferencing them over the Internet. There are three terms in common use for these identifiers and, although they are often used interchangeably, there are differences.

• URI (Uniform Resource Identifier) is an identifier for anything including those not available over the Internet such as people, buildings and mountains. There are many URI schemes, not all of which can be looked up over the Internet. For example, doi:10.1103/PhysRevD.89.032002 is as URI but cannot be looked up (directly) on the Internet. For data on the Web, only HTTP(s) URIs are relevant. HTTP URIs are a subset of URIs which are a subset of IRIs.
• IRI (Internationalized Resource Identifier) are conceptually identical to URIs but allow the use of non-ASCII characters to which URIs are limited.
• URL (Uniform Resource Locator) is the location of a resource on the Web. URLs are a subset of URIs.

Of these three, the term URL is by far the most commonly used. The term URI, and even more so, IRI, may cause confusion among some audiences, however, in the context of data on the Web, URI is more appropriate since data points and datasets very often refer to real world objects and phenomena. The term IRI is used where necessary.

Data discovery, usage and citation on the Web depends fundamentally on the use of HTTP (or HTTPS) URIs.

It is perhaps worth emphasizing some key points about URIs in the current context.

1. URIs are 'dumb strings', that is, they carry no semantics. Their function is purely to identify a resource.
2. Although the previous point is accurate, it would be perverse for a URI such as http://example.com/datset.csv to return anything other than a CSV file. Human readability is helpful.
3. When de-referenced (looked up), a single URI may offer the same resource in more than one format. http://example.com/dataset may offer the same data in, say, CSV, JSON and XML. The server returns the most appropriate format based on content negotiation .
4. One URI may redirect to another.
5. De-referencing a URI triggers a computer program to run on a server so that the URI acts as a call to an API. The server may therefore do something as simple as return a single, static file, or it may carry out complex processing. Precisely what processing is carried out, i.e. the software on the server, is completely independent of the URI itself.

  http://weather.example.com/temperature/UK/London/noon
  http://weather.example.com/temperature/UK/London/noon/today
  http://weather.example.com/temperature/UK/London/noon/2015-06-03  

9.8 Data Formats

The formats in which data is made available to consumers are a key aspect of making that data usable. The best, most flexible access mechanism in the world is pointless unless it serves data in formats that enable use and reuse. Below we detail best practices in selecting formats for your data, both at the level of files and that of individual fields. W3C encourages use of formats that can be used by the widest possible audience and processed most readily by computing systems. Source formats, such as database dumps or spreadsheets, used to generate the final published format, are out of scope. This document is concerned with what is actually published rather than internal systems used to generate the published data.

9.9 Data Vocabularies

Data is often represented in a structured and controlled way, making reference to a range of vocabularies, for example, by defining types of nodes and links in a data graph or types of values for columns in a table, such as the subject of a book, or a relationship “knows” between two persons. Additionally, the values used may come from a limited set of pre-existing values or resources: for example object types, roles of a person, countries in a geographic area, or possible subjects for books. Such vocabularies ensure a level of control, standardization and interoperability in the data. They can also serve to improve the usability of datasets. Say, a dataset contains a reference to a concept described in several languages. Such reference allows applications to localize their display of their search depending on the language of the user.

According to W3C, vocabularies define the concepts and relationships (also referred to as “terms”) used to describe and represent an area of concern. Vocabularies are used to classify the terms that can be used in a particular application, characterize possible relationships, and define possible constraints on using those terms. Several categories of vocabularies have been coined, for example, ontology, controlled vocabularies, thesaurus, taxonomy, semantic network.

There is no strict division between the artifacts referred to by these names. “Ontology” tends however to denote the vocabularies of classes and properties that structure the descriptions of resources in (linked) datasets. In relational databases, these correspond to the names of tables and columns; in XML, they correspond to the elements defined by an XML Schema. Ontologies are the key building blocks for inference techniques on the Semantic Web. The first means offered by W3C for creating ontologies is the RDF Schema [RDF-SCHEMA] language. It is possible to define more expressive ontologies with additional axioms using languages such as those in The Web Ontology Language [OWL2-OVERVIEW].

On the other hand, “controlled vocabularies”, “concept schemes”, “knowledge organization systems” enumerate and define resources that can be employed in the descriptions made with the former kind of vocabulary. A concept from a thesaurus, say, “architecture”, will for example be used in the subject field for a book description (where “subject” has been defined in an ontology for books). For defining the terms in these vocabularies, complex formalisms are most often not needed. Simpler models have thus been proposed to represent and exchange them, such as the ISO 25964 data model [ISO-25964] or W3C's Simple Knowledge Organization System [SKOS-PRIMER].

9.10 Sensitive Data

Sensitive data is any designated data or metadata that is used in limited ways and/or intended for limited audiences. Sensitive data may include personal data, corporate or government data, and mishandling of published sensitive data may lead to damages to individuals or organizations.

To support best practices for publishing sensitive data, data publishers should identify all sensitive data, assess the exposure risk, determine the intended usage, data user audience and any related usage policies, obtain appropriate approval, and determine the appropriate security measures needed to taken to protect the data. Appropriate security measures should also account for secure authentication and use of HTTPS.

At times, because of sharing policies sensitive data may not be available in part or in its entirety. Data unavailability represents gaps that may affect the overall analysis of datasets. To account for unavailable data, data publishers should publish information about unavoidable data gaps.

9.11 Data Access

Providing easy access to data on the Web enables both humans and machines to take advantage of the benefits of sharing data using the Web infrastructure. By default, the Web offers access using Hypertext Transfer Protocol (HTTP) methods. This provides access to data at an atomic transaction level. However, when data is distributed across multiple files or requires more sophisticated retrieval methods different approaches can be adopted to enable data access, including bulk download and APIs.

One approach is packaging data in bulk using non-proprietary file formats (for example tar files). Using this approach, bulk data is generally pre-processed server side where multiple files or directory trees of files are provided as one downloadable file. When bulk data is being retrieved from non-file system solutions, depending on the data user communities, the data publisher can offer APIs to support a series of retrieval operations representing a single transaction.

For data that is streaming to the Web in “real time” or “near real time”, data publishers should publish data or use APIs to enable immediate access to data, allowing access to critical time sensitive data, such as emergency information, weather forecasting data, or published system metrics. In general, APIs should be available to allow third parties to automatically search and retrieve data published on the Web.

On a further note, it can be observed that data on the Web is essentially about the description of entities identified by a unique, Web-based, identifier (a URI). Once the data is dumped and sent to an institute specialised in digital preservation the link with the Web is broken (de-referencing) but the role of the URI as a unique identifier still remains. In order to increase the usability of preserved dataset dumps it is relevant to maintain a list of these identifiers.

9.12 Data Preservation

9.13 Feedback

Publishing data on the Web enables data sharing on a large scale, providing data access to a wide range of audiences with different levels of expertise. Data publishers want to ensure that the data published is meeting the data consumer needs and user feedback is crucial. Feedback has benefits for both data publishers and data consumers, helping data publishers to improve the integrity of their published data, as well as to encourage the publication of new data. Feedback allows data consumers to have a voice describing usage experiences (e.g. applications using data), preferences and needs. When possible, feedback should also be publicly available for other data consumers to examine. Making feedback publicly available allows users to become aware of other data consumers, supports a collaborative environment, and allows user community experiences, concerns or questions are currently being addressed.

From a user interface perspective there are different ways to gather feedback from data consumers, including site registration, contact forms, quality ratings selection, surveys and comment boxes for blogging. From a machine perspective the data publisher can also record metrics on data usage or information about specific applications consumers are currently relying upon. Feedback such as this establishes a line of communication channel between data publishers and data consumers. In order to quantify and analyze usage feedback, it should be recorded in a machine-readable format. Blogs and other publicly available feedback should be displayed in a human-readable form through the user interface.

This section provides some BP to be followed by data publishers in order to enable data consumers to provide feedback about the consumed data. This feedback can be for humans or machines.

9.14 Data Enrichment

Data enrichment refers to a set of processes that can be used to enhance, refine or otherwise improve raw or previously processed data. This idea and other similar concepts contribute to making data a valuable asset for almost any modern business or enterprise. It also shows the common imperative of proactively using this data in various ways.

This section provides some advice to be followed by data publishers in order to enable data consumers to enrich data.