Advice to authors for the preparation of camera ready contributions Now We Are Talking! Flexible and Open Goal-Oriented Dialogue Systems for Accessing Touristic Services Umutcan Şimşek and Dieter Fensel STI Innsbruck, Department of Computer Science, University of Innsbruck Technikerstrasse 21a 6020 Innsbruck {umutcan.simsek, dieter.fensel}@sti2.at Abstract Goal-oriented dialogue systems have drawn interest from the academia and industry since the 1950s. The application areas vary between the likes of well-defined customer service processes and complex planning tasks. The practical applications are mostly developed with predefined dialogues and backend systems, which makes them challenging to extend with new services as well as new dialogues. In this research note, we propose an approach for flexible and open goal-oriented dialogue systems based on schema.org annotations. We extend the traditional concept of dialogue systems with a semantic task manager that generates task related dialogues dynamically by processing semantic descriptions of lightweight web services. Such a system can improve e-tourism processes especially for online marketplaces offering heterogeneous services (e.g. destination management organizations, DMO), where new services can be included mostly with the cost of having semantically annotated structured data and web services. Keywords: e-tourism, schema.org, goal-oriented dialogue systems, lightweight semantic web services 1 Introduction The web is evolving into a platform to which people have other ways than browsing to access the data. The search engines are turning into question answering engines that give the answer of the question a user asks, instead of just listing the links that may contain it. Naturally, for machines, it is hard to understand the unstructured content (e.g. text, images) that is primarily for human consumption. The structured data embedded in the web pages help to make the content understandable also for machines, by describing it with a shared vocabulary. The de-facto standard for such a vocabulary is schema.org. Not only the search engines but also other automated agents can utilize the structured data on the web. A concrete example of these agents is the intelligent personal assistants (IPA), for instance, Google Now 1 and Microsoft Cortana 2 . These conversational agents utilize schema.org annotations in emails and extract information like flight reservations and products. In this research note, we present our idea for another kind of conversational agent, namely a flexible and open dialogue system, that can utilize structured data on a set of web sites. The proposed dialogue system extracts goals from the user utterances and 1 https://developers.google.com/gmail/markup/google-now 2 https://docs.microsoft.com/en-us/cortana/data-markup/structured-data-markup matches with the task related dialogues generated dynamically based on lightweight semantic web service descriptions. We argue that such a dialogue system can be especially useful for a highly fragmented industry like tourism, where various service providers offer heterogeneous services. Thanks to the semantic web service descriptions done with a shared vocabulary, new services can be added to the knowledge base of the dialogue system without a significant development effort. Common marketplaces for touristic services such DMOs can benefit from it by offering a new access modality to the services in a touristic region without heavy technical development and maintenance costs. The rest of the paper is structured as follows: In Section 2 we review the related work and explain our motivation. We describe our methodology for developing the complete system in Section 3. Afterwards, in Section 4 we give a possible scenario for the usage of the dialogue system. Finally, we conclude the paper with a summary and remarks on future work in Section 5. 2 Related Work and Motivation The research on dialogue systems goes all the way back to the 1950s when the first effort on artificial intelligence emerged (McTear, 2002). Several domains such as customer service and transportation information systems (Jokinen & McTear, 2009). Very recently, Facebook launched the bot framework for its Messenger product, which allowed businesses to create their own chatbots. These business oriented dialogue systems are rather question answering systems that have predefined dialogues that communicate with a single backend. Although these systems work well as closed systems (i.e. with predefined well-curated data and dialogue structures), adapting them to new domains as well as different backend systems (e.g. databases, web services) requires heavy development. Additionally, dialogues need to be defined manually in a static manner, which is a costly process. As for the tourism sector, dialogue systems have been mostly used for accessing tourism information, connected to geographic information systems (Janarthanam et al., 2012) or pre-defined databases and web services (Niculescu et al., 2014). Interesting applications like dialogue based group recommendation systems have also recently emerged (Nguyen & Ricci, 2017). A recent example that utilizes schema.org annotations is Mayley, a tourism information service for Mayrhofen region. (Akbar et al., 2017). To take it one step beyond only question-answering and to complete tasks like booking a hotel room or a restaurant table, the dialogue system should utilize semantic descriptions of web services to find, compose and invoke them. The services are highly heterogeneous (e.g. accommodation, gastronomy, tours) and the e- commerce of these services is done through either website of individual providers or through organizations such as DMOs websites. Developing a dialogue system to provide access to these services is a costly endeavour, since the varying data structures of different backend solutions, therefore the dialogue systems should be tailored for each of them. Another challenge is the dialogues for each task since each task requires different input and is part of a different workflow. By having a common vocabulary for describing data and services, the task related dialogues can be automatically generated from the service descriptions. Accessing web services with dialogue systems has been studied in academic works (Gatius & Gonzalez, 2007), as well as using semantic web services in the context of question answering (Sonntag et al., 2007). However, these systems work on a small set of well-curated services. We aim to achieve openness by utilizing schema.org as a web service annotation vocabulary. 3 Methodology We aim to develop a flexible and open goal oriented dialogue system that can be used in sectors like tourism as a new user interface to the data and services. The flexibility and openness will be achieved by extending a traditional dialogue system architecture with a semantic task manager that retrieves relevant service descriptions annotated with schema.org actions and builds appropriate task related dialogues (Figure 1). Our methodology consists of methods from lightweight semantic web services field (Fensel, Facca, Simperl, & Toma, 2011), as well as, NLP techniques such as entity linking (Rao, McNamee, & Dredze, 2013), classification of utterances by semantic similarity (e.g. wordnet similarity) (Slimani, 2013) and natural language generation from RDF graphs. As soon as a website is registered to the dialogue system, the data extractor component extracts schema.org annotations with actions and stores them in a triple store as an RDF graph with reasoning support. Schema.org offers an action vocabulary 3 that provides a set of concepts which can be defined as actions on entities. The implicit semantics of these action names will be later used by the language understanding module to classify user intent and to find the suitable services. Once a user sends an utterance, the language understanding unit classifies the relevant intent by considering several factors like the semantic similarity of the verb in the utterance to the extracted actions and entities linked to collected instances from the registered websites. After the user intent and related entities are determined, the 3 https://schema.org/docs/actions.html Fig. 1. The overview of the dialogue system language understanding unit sends these data to the dialogue manager, which in return creates an initial dialogue state4. The dialogue manager then sends the intent to the semantic task manager, which in return searches for the suitable services in the triple store. The relevant web service description indicates expected input and promised output of the service invocation which can be also used for the composition of multiple services if it is required for achieving the user goal. The order of the web service calls is implicit in the resource hierarchy of the annotated RESTful API 5. The semantic task manager sends the required input back to the dialogue manager. The dialogue manager updates the dialogue state and communicates with the natural language generator to create natural language statements from triples6. 4 Use Case A typical use case for the proposed dialogue system would be buying a ticket for an event in a certain region. The DMO annotates the events on its webpage as well as the search engine that lists events against a query with SearchAction. When a user wants to buy a ticket for a certain event, the semantic task manager looks for the action annotations that produce Event or Ticket instances. Once the relevant actions are found, the required input is asked and collected from the user. Ideally, the responses returned from service invocations also contain potential actions on them to complete the task (e.g. BuyAction on the event offer returned after invocation of the search service). If that is not the case, the semantic task manager tries to find another service description that takes the selected event by the user and completes the buying task by composing two services. Since the dialogue manager memorizes the context of the conversation, if the user asks for an accommodation, the process goes faster given that some of the required input (e.g. location) is already known from the event task. The semantic task manager can also benefit from reasoning, for instance, if the user wants a room, a clarification question regarding the type of the room (e.g. hotel or meeting room) can be generated automatically based on subsumption reasoning. Currently a dialogue system that can realize the aforementioned scenario with the data of DMO Seefeld7 is being implemented. Their existing annotations are stored in a triple store. They currently have a dialogue system implementation that uses statically generated dialogues. In the next step the annotations will be enriched with actions that will facilitate the dynamic creation of the dialogues. 5 Conclusion and Future Work In this research note, we explained our idea for a flexible and open dialogue system based on schema.org annotations of data and services. The dialogue system can 4 The details of the dialogue management technique that may be adopted are described in (Traum & Larsson, 2003). 5 When a request is sent to a resource, the response is also annotated with schema.org may contain the next potential action. 6 For instance, an adapted version of https://github.com/AKSW/SemWeb2NL/wiki/Triple2NL 7 http://seefeld.com eliminate the burden of defining dialogues for each task, which can be very beneficial for touristic service aggregators like DMOs. Even though schema.org actions are sufficient for describing functional properties which can help us to generate task related dialogues, the description of non-functional properties (e.g. reliability, security) is still an open question. For the future, we will focus on the technical details of the methods explained in Section 3 and complete the implementation and evaluate our working prototype. References Akbar, Z., Kärle, E., Panasiuk, O., Simsek, U., Toma, I., and Fensel, D. (2017). Complete Semantics to empower Touristic Service Providers. In OnTheMove Federated Conferences & Workshops - ODBASE. (To appear). Fensel, D., Facca, F. M., Simperl, E., and Toma, I. (2011). Lightweight Semantic Web Service Descriptions. In Semantic Web Services, pages 279-295. Springer Berlin Heidelberg, Berlin, Heidelberg. Gatius, M. and Gonzalez, M. (2007). A multilingual Dialogue System for Accessing the WEB. 3rd International Conference on Web Information Systems and Technologies, pages 1-3. Janarthanam, S., Lemon, O., Liu, X., Bartie, P., Mackaness, W., Dalmas, T., and Goetze, J. (2012). Integrating location, visibility, and question-answering in a spoken dialogue system for pedestrian city exploration. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 134-136. Association for Computational Linguistics. Jokinen, K., and McTear, M. (2009). Spoken dialogue systems. Synthesis Lectures on Human Language Technologies, 2(1), 1–151. McTear, M. (2002). Spoken Dialogue Technology: Enabling the Conversational User Interface. ACM Computing Surveys, 34(1):90-169. Nguyen, T. N. and Ricci, F. (2017). A Chat-Based Group Recommender System for Tourism. In Information and Communication Technologies in Tourism 2017, pages 17-30. Springer International Publishing, Cham. Niculescu, A. I., Jiang, R., Kim, S., Hui Yeo, K., D, L. F., Niswar, A., and Banchs, R. E. (2014). SARA: Singapore's Automated Responsive Assistant, A Multimodal Dialogue System for Touristic Information. In 11th International Conference on Mobile Web and Information Systems, volume 8640, pages 153-164, Barcelona. Springer. Rao, D., McNamee, P., and Dredze, M. (2013). Entity Linking: Finding Extracted Entities in a Knowledge Base. pages 93-115. Springer, Berlin, Heidelberg. Slimani, T. (2013). Description and Evaluation of Semantic Similarity Measures Approaches. International Journal of Computer Applications, 80(10):25-33. Sonntag, D., Engel, R., Herzog, G., Pfalzgraf, A., Peger, N., Romanelli, M., and Reithinger, N. (2007). SmartWeb Handheld - Multimodal Interaction with Ontological Knowledge Bases and Semantic Web Services. Artificial Intelligence for Human Computing, pages 272-295. Traum, D. R. and Larsson, S. (2003). The Information State Approach to Dialogue Management. pages 325-353. Springer, Dordrecht