This specification defines the syntax and semantics of a language for defining the behavior of interactive dialogue. It formalizes the representation of states (i.e. fluents), how they change (i.e. actions), and which are preferred (i.e. utility). These three components characterize a task, so we call DMPL a task-oriented programming language. The interpreter of the language resolves the task by identifying and executing actions that maximize utility.

Introduction

This specification aims to decouple the representation of dialogue from its execution. For example, conversational interfaces employ artificial intelligence in different ways: finite state machines, path planning, or reinforcement learning systems. Regardless of the execution framework, the logical representation may be shared. An agreed upon representation of dialogue allows content writers to author and share conversational experiences without being distracted by the underlying runtime.

Scope

This section is informative.

This document is limited to specifying the syntax and semantics for a task-oriented language for interactive behavior. One example application of DMPL is an intermediate represenation of content written in authoring tools for non-technical users. Another use-case of DMPL is exporting and importing dialogue content authored by writers from different web platforms. Technologies closely related to these use-cases are in scope.

Goals and Motivation

This section is informative.

DMPL is a task-oriented programming language, which may be better explained by extending Microsoft's Functional Programming vs. Imperative Programming table, shown below.

Imperative Functional Task-Oriented
Programmer Focus How to perform tasks (algorithms) and how to track changes in state What information is desired and what transformations are required Why an action is taken and how information changes
State Changes Important Non-existant (for purely functional languages) Important
Order of Execution Programmer is mostly responsible Hybrid resposibility Compiler is mostly responsible
Loops For/While, Recursion Recursion None, loops are implicit
Primary Manipulation Unit Instances of structures or classes Functions Utility

The runtime shall evaluate DMPL in an infinite loop, similar to game-loops inherent in most game engines.

Language Syntax

This section is normative.

This section of the document defines the types used within this specification using EBNF notation with corresponding railroad diagrams.

Atomic Types

literal

A literal is a terminal atom that may be assigned to a variable, used in a dictionary, or used as an operand to build an expression.

	                literal ::= string | number | boolean | 'null'
                
Literal rail-road diagram
Literal

boolean

A boolean may take on values true or false.

	                boolean ::= 'true' | 'false'
                
Boolean rail-road diagram
Boolean

name

A name is a sequence of characters conforming to those in the [[XML-NAMES]] standard.

	                name ::= [ http://www.w3.org/TR/xml-names/#NT-NCName ]
                
Name rail-road diagram
Name

quote_start

A quote_start is the starting quotation mark, which indicates the beginning of a string.

	                quote_start ::= '"' '`'
                
Quote-start rail-road diagram
Quote-start

quote_close

A quote_close is the ending quotation mark, which indicates the termination of a string.

	                quote_close ::= '`' '"'
                
Quote-close rail-road diagram
Quote-close

string

A string literal represents a finite sequence of characters. It is surrounded by starting and ending quotation marks.

	                string ::= quote_start name quote_close
                
String railroad diagram
String
                "`hello`"
            

digit

A digit represents a real valued integer in the range [0-9].

	                digit ::= [0-9]
                
Digit railroad diagram
Digit

number

A number literal may represent either a positive or negative decimal value.

	                number ::= '-'? ( digit+ ( '.' digit* )? | '.' digit+ )
                
Number railroad diagram
Number
                0.2
            

Container Structures

variable

A variable is a named reference to the result of evaluating an expression.

                        variable :: = '"' name '"'
                    
Variable railroad diagram
Variable
                "num_correct_answers" // => 3
            
A variable should not be confused with a string, which uses backticks (`) like so:
                "`num_correct_answers`" // => "num_correct_answers"
            

dictionary

A dictionary is a structure that maps data in key-value pairs. The keys and values are all evaluated. The keys can be either strings or variables that evaluate to strings.

                    dictionary : : = '{' ( string | variable ) ':' expression
                                     ( ',' ( string | variable ) ':' expression )* '}'
                
Dictionary railroad diagram
Dictionary
                {"`age`": "x", "`score`": 10, "`name`": "`Joe`"}
            

expression

An expression may be evaluated, and that result is stored in memory to be referenced later. An expression may be denoted by a JSON array [[RFC7159]], in which case it is also called a function. The first element of the array represents the operator of the function. The operator can be either a string or a variable that evaluates to a string. All remaining elements of the array constitute the operands.

	                expression ::= literal
                              | variable
                              | dictionary
                              | '[' ( literal | variable ) ( ',' expression )* ']'
                
Expression railroad diagram
Expression
Return Type Operator Example
string
"to_str"
["to_str", 4] // => "4"
string
"+"
["+", "`hello `", "`world`"] // => "hello world"
number
"+"
["+", 1, 2] // => 3
number
"-"
["-", 5, 3] // => 2
number
"*"
["*", 2, 4] // => 8
number
"/"
["/", 1, 2] // => 0.5
number
"//"
["//", 1, 2] // => 0
number
"%"
["%", 168, 2] // => 0
number
"to_num"
["to_num", "`4`"] // => 4
number
"len"
["len", ["", 1, 2, 3]] // => 3
number
"floor"
["floor", 1.5] // => 1
list
""
["", 1, 2, 3, 4, 5] // => [1, 2, 3, 4, 5]
list
"++"
["++", ["", 1, 2], ["", "`hello`"]] // => [1, 2, "hello"]
list
"range"
["range", 1, 5] // => [1, 2, 3, 4]
["range", 0, 8, 2] // => [0, 2, 4, 6]
list
"map"
                            ["map", "to_str", ["", 1, 2, 3]] // => ["1", "2", "3"]
                        
                            ["map", "+", ["", 1, 2, 3], ["", 4, 5, 6]] // => [5, 7, 9]
                        
list
"foldl"
                            ["foldl", "+", 0, ["", 1, 2, 3]] // => 6
                        
list
"sort"
                            ["sort", ["", 1, 3, 2]] // => [1, 2, 3]
                        
list
"shuffle"
                            ["shuffle", ["", 1, 2, 3]] // => [1, 3, 2] randomly shuffled
                        
list
"reverse"
                            ["reverse", ["", 1, 2, 3]] // => [3, 2, 1]
                        
boolean
"=="
["==", 1, 2] // => false
boolean
"!="
["!=", 1, 2] // => true
boolean
">"
[">", 2, 3] // => false
boolean
">="
[">=", 2, 3] // => false
boolean
"<"
["<", 2, 3] // => true
boolean
"<="
["<=", 2, 3] // => true
boolean
"&&"
["&&", true, false] // => false
boolean
"||"
["||", true, false] // => true
boolean
"!"
["!", false] // => true
boolean
"in"
["in", 3, ["", 1, 2, 3]] // => true
boolean
"input"
["input"] // => true if there's an input
["input", "`yes`"] // => true if there's an input and it equals "yes"
boolean
"return"
["return"] // => true if there's an return result from callee
boolean
"?"
["?", "`is_greeted`"] // => true if variable defined
dynamic
"get"
["get", 0, ["", 1, 2, 3]] // => 1
["get", "`a`", {"`a`": 1}] // => 1
dynamic
"pick"
["pick", ["", 1, 2, 3]] // => 1, 2, or 3 uniformly at random
dynamic
"patch"
["patch", {"`a`": 1, "`b`": 2}, {"`a`": 2}] // => {a: 2, b: 2}
Reference: RFC 7386
dynamic
"edit"
["edit", {"`a`": 1}, 2, "`a`"] // => {a: 2}
dynamic
"from_list"
["from_list", ["", ["", "`a`", 1], ["", "`b`", 2]]] // => {a: 1, b: 2}

Statements

statement

A statement may be a non-terminal such as do or fork, or a terminal such as effect. Optionally, statements may contain condition, await, or once flags.

	                statement ::= '{' ( condition ',' )?
                               ( await ',' )?
                               ( once ',' )?
                               ( effect | do | fork ) '}'
                
Statement railroad diagram
Statement

condition

A condition is a boolean expression that runs the statement if the expression evaluates to true. A condition check takes priority over other parts of the statement. If a statement does not explicitly contain a condition, then that statement's condition is trivially true.

	                condition : : = '"if"' ':'  expression
                
Condition railroad diagram
Condition
                "if": [">", "num_wrong_answers", 3]
            

await

An await blocks execution until a boolean expression evaluates to true.

	                await ::= '"await"' ':' expression
                
Await railroad diagram
Await
                "await": ["input"]
            

once

A once flag specifies whether the statement can only be run once. The statement is then ignored in all subsequent instances. By default, the once flag is set to false.

	                once ::= '"once"' ':' boolean
                
Once railroad diagram
Once
                "once": true
            

effect

An effect is a terminal node, meaning it does not produce more statements. There are 6 types of effects.

	                effect ::= act | set | def | run | use | pop
                
Effect railroad diagram
Effect

act

An act represents an action for the client to realize. The expression is evaluated before being sent out to the client. For example, the payload of the act statement may be represented using Behavior Markup Language (BML). The client is then responsible for realizing the behavior defined in the act message.

	                act ::= '"@act"' ':' expression
                
Act railroad diagram
Act
                "@act": {
                  "`object`": "`tutor`",
                  "`action`": "`say`",
                  "`params`": {"`intent`": "`greeting`"}
                }
            

set

A set updates the value of a variable, where the names of the variables and the updated values are specified by evaluating the corresponding expressions. The expression specifying the names of the variables must evaluate to a string, a list consisting of only strings, or a dictionary consisting of only strings.

	                set ::= '"@set"'  ':' expression ',' '"val"' ':' expression
                
Set railroad diagram
Set
                "@set": "`is_user_greeted`", "val": true
            
                "@set": ["", "`is_user_greeted`", "`num_questions`"], "val": ["", true, 7]
            

def

A def defines a new expression operator that can be used later in the code. The signature of the new expression operator is specified by an expression that must evaluate to a list of strings, which maps to the operator name followed by the argument names. These argument names are valid only for this scope, and may shadow global variables. The result is returned by a pop statement.

When running a user-defined operator, a copy of its surrounding environment (e.g. variables/operators) is coupled to the newly defined operator. In contrast, functions in Javascript keep references to the surrounding state.

Custom operators support closure, since the operator is enclosed with a copy of its surrounding environment. After an operator is defined, the copy of the enclosing scope does not change. When an operator is called, def and set statements inside the body of the operator will only create local symbols that shadow symbols in the enclosing scope.

	                def ::= '"@def"' ':' expression ',' '"val"' ':' statement
                
Def railroad diagram
Def
                "@def": ["", "`inc`", "`x`"], "val": {"@pop": ["+", 1, "x"]}
            
                "@def":["","`custom`","`divisor`"], "val":{
                    "@do":[
                      {"@def":["","`mod`","`num`"], "val": {"@pop":["%","num","divisor"]}},
                      {"@pop":"mod"}
                    ]
                }
            

run

A run calls DMPL code from another component, optionally passing in arguments. The name of the called component is specified by an expression that must evaluate to a string. The current variables stored in memory are pushed to a stack before the call, and popped back after the call.

The passed in arguments is a JSON array [[RFC7159]] accessible inside the called component, stored in a built-in variable called _args. Note that _args is null by default if the component is not called by another component. If a component is called with no passed in arguments, _args will be an empty array.

	                run ::= '"@run"' ':' expression ( ',' '"args"' ':' expression )?
                
Run railroad diagram
Run
                "@run": "`Outro`"
            
                "@run": "`Question`", "args": ["", "`What's the largest planet?`", "`Jupiter`"]
            

use

A use imports variables and expression operators defined in another component. The name of the imported component is specified by an expression that must evaluate to a string. The variable and expression operator names can be optionally specified, or all variables and expression operators will be imported. It's similar to the Python notation: from os import path.

	                use ::= '"@use"' ':' expression ( ',' '"import"' ':' expression )?
                
Use railroad diagram
Use
                "@use": "`MathExpressions`", "import": ["", "`inc`", "`square`", "`exp`"]
            

pop

A pop quits execution on the current DMPL code, pops the variables from the stack, and resumes execution from the previous run location.

	                pop ::= '"@pop"' ':' expression
                
Pop railroad diagram
Pop
                "@pop": true
            

do

A do statement specifies a list of statements to be executed.

	                do ::= '"@do"' ':' '[' ( statement ( ',' statement )* )? ']'
                
Do railroad diagram
Do
                "@do": [
                  {"@act": "a"},
                  {"@act": "b"},
                  {"@act": "c"}
                ]
            

fork

A fork statement specifies a branch in program-execution for the list of statements that follow. Only one is chosen, and the strategy to pick one may be provided using the scheme attribute. By default, a greedy scheme is used, meaning the first statement whose condition is met will be chosen. More complicated schemes, such as depth-first-search or Monte Carlo Tree Search may be indicated, leaving the implementation up to the interpreter.

	                fork ::= '"@fork"' ':' '[' ( statement ( ',' statement )* )? ']' ( ',' scheme )?
                
Fork railroad diagram
Fork

When no scheme is provided, a fork is essentially the traditional if/elif/else logical flow:

                "@fork": [
                  {"if": "is_user_greeted", "@act": "a"},
                  {"if": [">", "num_wrong_answers", 3], "@act": "b"},
                  {"@act": "c"}
                ]
            

Providing a scheme allows stochastic behavior to take place:

"scheme": {"depth": 3}, "@fork": [
  {"@set": "`do_action_1`", "val": "true"},
  {"@set": "`do_action_2`", "val": "true"},
  {"@set": "`do_action_3`", "val": "true"},
  {"if": "is_entry_condition_met", "@set": "`do_action_4`", "val": "true"},
  {"if": false, "@act": "`never reached`"}
]
            

scheme

A scheme is associated with a fork statement, and it describes the fork branch resolution strategy.

	                scheme ::= '"scheme"' ':' expression
                
Scheme railroad diagram
Scheme
                {"depth": 3}
            

Examples

This section is informative.

This section details DMPL conforming example programs.

The following sends "hello world" to the action realizer.

        {
          "@act": "`hello world`"
        }
    

Two actions can be sent sequentially.

        {
          "@do": [
            {"@act": "`hello`"},
            {"@act": "`world`"}
          ]
        }
    

Fork statements allow branching logic.

        {
          "@fork": [
            {"if": ["==", ["+", 2, 2], 4], "@act": "`this statement is run`"},
            {"@act": "`never reached`"}
          ]
        }
    

Variables can be set and accessed like so.

        {
          "@do": [
            {"@set": "`a`", "val": 1},
            {"@set": "`b`", "val": 2},
            {"@set": "`c`", "val": ["+", "a", "b"]}
          ]
        }
    

Here we define a new expression called inc which takes an argument called x.

        {
          "@do": [
            {"@def": ["", "`inc`", "`x`"], "val": {"@pop": ["+", 1, "x"]}},
            {"@act", ["`inc`", 5]}
          ]
        }
    

The set statement allows unpacking a list.

        {
          "@do": [
            {"@set": "`content`", "val": ["", "`What's the biggest planet?`", "`Jupiter`"]},
            {"@set": ["", "`Prompt`", "`Correct-Answer`"], "val": "`content`"}
          ]
        }
    

Run an external file and pass in arguments. The return value can be considered in the following fork statement.

        {"@do":
          {"@run": "`Question`", "args": ["", "`What's the biggest planet?`", "`Jupiter`"]},
          {"await": ["return"], "@fork": [
            {"if": ["return", true], "@set": "`score`", "val": 100},
            {"@set": "`score`", "val": 0}
          ]}
        }
    

Expressions and variables may be imported from other files.

        {
          "@do": [
            {"@use": "`MathExpressions`", "import": ["", "`square`", "`exp`"]},
            {"@act": ["==", ["`square`", 6], ["`exp`", 6, 2]]}
          ]
        }
    

To halt until a user input is supplied, the await flag on a fork may be used.

        {
          "@do": [
            {"@act": "`are you ready?`"},
            {"await": ["input"], "@fork": [
              {"if": ["input", "`yes`"], "@act": "`great`"},
              {"if": ["input", "`no`"], "@act": "`no problem`"}
            ]}
          ]
        }
    

More immersive dialogue can be achieved by setting and checking variables, such as greeted.

        {
          "scheme": {"depth":  2}, "@fork": [
            {"if": ["!", ["exists", "`greeted`"]], "@set": "`greeted`", "val": false},
            {"if": ["!", "greeted"], "@do": [
              {"@act": "`hello`"},
              {"@set": "`greeted`", "val": true}
            ]},
            {"if": "greeted", "@do": [
              {"@act": "`how are you?`"},
              {"@set": "`asked_feelings`", "val": true},
              {"await": ["input"], "@fork": [
                {"if": ["input", "`good`"], "@act": "`great`"}
              ]}
            ]}
          ]
        }
    

More complete question and answering scenarios may be constructed.

        {
          "@fork": [
            {"if": ["!", ["exists", "`Init`"]], "@do": [
              {"@set": "`Init`", "val": true},
              {"@set": "`Wrong-Counter`", "val": 0},
              {"@set": "`Is-Hint1-Given`", "val": false},
              {"@set": "`Is-Hint2-Given`", "val": false},
              {"@set": "`Is-Answer-Given`", "val": false},
              {"@set": "`Num-Hints-Given`", "val": 0},
              {"@set": "`Question`", "val": "`What's the biggest planet?`"},
              {"@set": "`CA`", "val": "`planet.jupiter`"},
              {"@set": "`CA-Response`", "val": "`Exactly!`"},
              {"@set": "`WA1`", "val": "`planet`"},
              {"@set": "`WA1-Response`", "val": "`Nope. That's not the biggest`"},
              {"@set": "`WA2`", "val": "`nonplanet`"},
              {"@set": "`WA2-Response`", "val": "`That's not a planet`"},
              {"@set": "`Hint1`", "val": "`It has a big red spot`"},
              {"@set": "`Hint2`", "val": "`It's name begins with the letter J`"},
              {"@set": "`Answer`", "val": "`The biggest planet is Jupiter`"},
              {"@set": "`Hint-Announcement`", "val": "`Here's a hint`"}
            ]},
            {"@fork": [
              {"if": ["&&", ["<=", "Wrong-Counter", 3], ["!", "Is-Answer-Given"]], "@do": [
                {"@act": "Question"},
                {"@fork": [
                  {"if": ["input", "CA"], "@do": [
                    {"@act": "CA-Response"},
                    {"@set": "`Is-Answer-Given`", "val": true}
                  ]},
                  {"if": ["input", "WA1"], "@do": [
                    {"@act": "WA1-Response"},
                    {"@set": "`Wrong-Counter`", "val":  ["+", 1, "Wrong-Counter"]}
                  ]},
                  {"if": ["input", "WA2"], "@do": [
                    {"@act": "WA2-Response"},
                    {"@set": "`Wrong-Counter`", "val":  ["+", 1, "Wrong-Counter"]}
                  ]},
                  {"@do": [
                    {"@set": "`Wrong-Counter`", "val":  ["+", 1, "Wrong-Counter"]}
                  ]}
                ], "await": ["input"]}
              ]},
              {"if": ["&&", ["!", "Is-Hint1-Given"], ["==", "Wrong-Counter", 1]], "@do": [
                {"@act": "Hint-Announcement"},
                {"@act": "Hint1"},
                {"@set": "`Is-Hint1-Given`", "val": true}
              ]},
              {"if": ["&&", ["!", "Is-Hint2-Given"], ["==", "Wrong-Counter", 2]], "@do": [
                {"@act": "Hint-Announcement"},
                {"@act": "Hint2"},
                {"@set": "`Is-Hint2-Given`", "val": true}
              ]},
              {"if": ["&&", ["!", "Is-Answer-Given"], ["==", "Wrong-Counter", 3]], "@do": [
                {"@act": "Answer"},
                {"@set": "`Is-Answer-Given`", "val": true}
              ]}
            ], "scheme": {"depth": 3}}
          ]
        }
    

Here's how to handle global interruptions from the user.

      {"scheme": {depth: 1}, "@fork": [
        {"@do": [
          {"@act": "`What's your favorite color?`"},
          {"await": ["input"], "@fork": [
            {"if": ["input", "`red`"], "@act": "`That's my favorite color, too!`"}
            {"@act": "`I've never heard of it`"}  
          ]}
        ]},
        {"if": ["input", "`want to quit`"], "@act": "bye"}
      ]}
    

Related Technologies

This section is informative.

Artificial Intelligence Modelling Language (AIML)

AIML tightly couples dialogue logic with natural language processing (NLP). As a result, authoring complicated dialogue systems in AIML requires authors to create an extensive knowledge-base, requiring a large number of rules that need to be crafted in order to imitate a natural conversation. This often requires domain expertise in order to capture the intricate nuances associated with open-ended dialogue. DMPL, on the other hand, is a general representation for dialogue flow and control without text pattern matching rules.

ChatScript

ChatScript incorporates stronger pattern matching than AIML by using NLU to find the best topic match instead of the best pattern match for an input. To incorporate NLU, ChatScript packages a large database of concepts, which are sets of synonyms. Defining custom concepts is easily accomplished, but requires careful design in order to avoid being matched with already existing concepts, and to avoid having disjoint inputs being matched with the same concept. DMPL modularizes the NLU, relieving content-writers from having to write complex pattern matching expressions.

RiveScript

RiveScript mainly concerns itself with fetching a user's input message, and leaving everything else to the content writer. It's designed to extend and simplify AIML but still has the overhead of having to write complex pattern matching expressions limiting the variability and scope of the dialogue that can be produced. DMPL decouples the need to write complicated pattern matching rules, and instead uses utility functions to both drive dialogue and respond to the user's input accordingly.

Planning Domain Definition Language (PDDL)

One of the first languages, Planning Domain Definition Language (PDDL) represents tasks by two files: a domain file, which lists available actions, and a corresponding problem file, which defines the initial fluents and goal specifications. Each action defined in the domain file has an entry-condition and a deterministic effect. Deterministic planning is a major limitation, since we’re modeling tasks in interactive or unpredictable environments. PDDL v3.1 (the latest version as of April 2019) incurs the following limitations:

Probabilistic PDDL (PPDDL)

Probabilistic PDDL (PPDDL) addresses PDDL's limitation by introducing stochastic effects, so that planning may account for probabilistic outcomes of each action. The PPDDL inference engine can solve for an action sequence that optimizes a user-defined utility function. Through some careful organization and design of the domain file, it is also possible, but not intuitive, to support hierarchical planning within PPDDL. On the other hand, DMPL is designed with hierarchical planning from the get-go, by pushing fluents to a stack once entering a new subroutine.

Probabilistic Programming

Unlike PPDDL, WebPPL and Stan are general purpose probabilistic programming languages that offer a rich selection of statistical models and efficient MCMC sampling. These languages are used to specify general statistical models, which can model an autonomous agent in a stochastic environment. Since they’re general purpose, like DMPL, WebPPL and Stan support user-defined functions and external interfaces to user-defined business logic.

Decision Theoretic Languages

Languages designed specifically for decision problems include IBAL and Markov logic decision networks (MLDN), which come with approximate inference solvers. DTProbLog introduces exact solvers to compute the optimal strategy in a decision-theoretic probabilistic language, using Algebraic Decision Diagrams (ADD). However, DTProbLog is less concerned with sequential decision problems. These languages leverage the Markov Decision Process (MDP) framework, which requires a definition of immediate reward of transitioning from one state to another. Strategically, DMPL disallows users from defining MDP-like rewards and limits the utility function to be defined indirectly through listing pairs of preferred situations. Doing so alleviates the guess-work from the user, and reduces buggy code (most common being infinite-loops due to ill-defined rewards).

Missing from most probabilistic programming languages is the runtime of the agent. In other words, once an action-sequence is resolved, instructions for following through with the chosen action on an agent is run elsewhere, usually defined in a separate codebase. DMPL borrows ideas from DTGolog, where both the planner (simulation engine) and evaluator (runtime) consult the same user-written abstract syntax tree. The DMPL code is used both to plan by sampling possible actions and to execute by traversing the same abstract syntax tree.