W3C Workshop on Web and Machine Learning

Hello everybody, my name is Wolfgang Maß.

I am at the German Research Center for Artificial Intelligence.

And I'm working on smart service engineering, in industrial settings, healthcare settings and other domains.

I want to show what we are doing with respect to what we call collaborative learning in the next couple of slides.

So, a background motivation for work is that we are investigating smart services, which can be considered as software systems that adapt to user needs and are adaptive to various kinds of contexts in the real world.

And smart services apply AI technologies, processing box data, but also streaming data and smart services shall be interoperable and shall collaborate over the network in particular the world wide web.

There are some goals that we investigate on one hand side: we want to investigate smart services that only live in a browser and not so much in data centers.

Smart services shall be quickly built and shall be based on reusable software modules.

We do not want to exclude cloud based AI services, but want to merge locally executed AI services and cloud based AI services.

And another goal is that smart services are self-descriptive, and in this sense of self-containing.

And then it shall be self maintaining to a certain extent, while this at the moment is rather a goal than that we actually have implementations on that.

Furthermore, we do not want to confine ourselves toward very specific architecture, but by adopting an agent based approach, agents shall communicate with each other, they should also dedicate roles to agents such as becoming a server role or communication partner role, and that should be done dynamically in negotiation between agents.

We have built a platform called Tucana, whose architecture you see on the right hand side of the slide.

And Tucana is a peer to peer architecture, where different kinds of environments what we call Tucana environments T-ENV, can communicate with one another.

In each Tucana environment, several smart services can be executed.

They can communicate locally, but they can also communicate via peer to peer networks based on WebRTCWebRTC stands for Web Real-Time Communications, a set of browser APIs and protocols that enable the exchange of audio, video and data in near real-time.

Each smart service as you can see on the second image on this slide, consists of different kinds of minions and these minions are software modules with dedicated tasks.

We distinguish between perceiver minions, thinker minions, communicator minions, but also actuator minions.

The basic minions according to a three layer architecture are perceiver minions (PMINs) and thinker minions and communicator minions.

Perceiver minion collects data and stores the data in the local database of the browser.

Thinker minion accesses the data and processes by leveraging data analytical software, in particular TensorFlowTensorFlow is a Python framework to build Machine Learning systems-based kind of analytics but could be any kind of JavaScript machine learning software or customized software, while the communicator minion either communicates with other smart services locally via peer to peer network globally, or with the end user via the browser interface.

We are using JavaScript for all our implementation, WebRTC.

For access control we use OpenAuthentication 2.0, we are able to export the data that we collect into JSON-LD, one implementation is using BabelNet as the transformer basis.

We also leverage TensorFlow and we are compliant with the specification of GAIA-X and the International Data Space which is a widely used kind of specification for Industry 4.0 applications, yes.

So, we specify the smart services in JSON format, and we use these JSON formats in order to bind the software configuration, the smart service configuration to pre-installed minions.

Probably more interesting for this workshop is the thinker minion.

I have here an example that we use for activity recognition in the shopping context.

You also see here that the minions again have an activate method, a notify method and a terminate method.

But in between we also have a method for predictPartially the data that is coming in and streams for the acceleration and the geolocation is used to make predictions.

And we are using a model that was pre-trained in the server side.

And in during the activate method, we load that model and we use it to make predictions.

In this case it is a very straightforward and simple prediction - simply to say that a person is tired or it's not tired.

And the integration of TensorFlow JS works quite nicely.

On the right hand side, you see the model topology that we use here is based on LSTM layers, and the fully connected layer and we integrate it all together and push that into our TMIN.

Here is an example for the whole smart service for shopping, and as you can see on the left hand side we have a configuration file for our smart service that says we have two perceiver minions, one for geolocation and the other for linear acceleration.

We have one TMIN for activity prediction and three CMINs.

The one for communicating with the user for shopping deal that might be appropriate in that situation.

We have a pure communicator that can pass information to internal locally defined or via peer to peer connection also to external smart services.

And we have another communication minion that shows the acceleration also to the user.

And this configuration in JSON is very handy and allows us very similar to the architectural and topology of a machine learning model to define smart service designs.

Okay, so let's wrap it up.

What are our lessons learned?

And where do we see potential for further development and specifications?

First, it would be interesting to have agent-oriented architecture, better being supported by current systems in JavaScript, by WebRTC and by other technologies.

That's one element.

Then, on the machine learning side, it would be really great if also on the JavaScript side, we would have some support for federated learning, distributed learning and reinforcement learning, this is currently missing.

In WebRTC, I know that the signaling service and the protocol is outside of webRTC.

However, I would appreciate and I think the community would also appreciate a general and optimized signaling protocol, maybe also an environment for testing for signaling services.

And last but not least, OpenAuthentication 2.0 - it requires always on at the moment, if you are offline authentication is a pain.

So, you have to switch it on and off.

A buffering service for authentication that supports offline services would be really appreciated.

...