Special Focus Paper—Sustainability, Social Impact, Learning and Training Innovation in Online Exp… Sustainability, Social Impact, Learning and Training Innovation in Online Experimentation https://doi.org/10.3991/ijim.v11i5.7065 Mario A. Bochicchio!"!#, Lucia Vaira University of Salento, Lecce, Italy mario.bochicchiounisalento.it Abstract—Experimentation is an essential ingredient in any learning strate- gy about Sciences and Technologies (SciTech in the following). That is why the research on online experimentation is paramount to help confront the skill shortage in SciTech disciplines, by stimulating the development of more effec- tive online learning and training approaches. Nonetheless, in this sector "func- tional" aspects are often overrated by researchers and no specific attention is paid to the sustainability of their proposals or to the possibility to analyze and improve the achieved social impact. A better integration among these aspects is then essential to move online experimentation research out of its infancy, to improve its perceived value and to increase its diffusion. Keywords—online experimentation, sustainability, social impact, online learn- ing, online training. 1 Requirements for Online Experimentations In systems engineering, functional requirements define how systems react to a set of inputs to produce the desired outputs. However, functional correctness is necessary but not sufficient to define effective and successful systems, or even suitable ones. Horses, for example, are functionally perfect for transportation services, but not suita- ble and not effective in modern urban environments. This concept applies well to the current status of online experimentation research [1] (OER from now on), which is able to exhibit the functional correctness and feasi- bility of a large number of valuable online experiments. However, nothing or little is said about their suitability, effectiveness or successfulness. Several concepts and tools, borrowed from the business sector, can be usefully ex- ploited to find which non-functional aspects must be considered to transform "func- tioning" online experiments into effective and sustainable ones. For example, refer- ring to the suitability of the (possibly emerging) technologies adopted for a specific online experimentation, very useful conclusions can be drawn from the Hype Cycle [2]. Gartner's Group introduced it for providing a graphical representation of the ma- turity level, adoption degree and social application of the most relevant technologies at a given time. From the analysis of Figure 1, for example, we can see that "4D printers" are less mature and widespread than "volumetric displays" to represent some 6 http://www.i-jim.org Special Focus Paper—Sustainability, Social Impact, Learning and Training Innovation in Online Exp… phenomena, and that both of them are in a much less mature phase with respect to virtual reality, which is almost ready for large scale adoption and diffusion in 2016. Fig. 1. Hype Cycle for Emerging Technologies. Gartner. July 2016 It should be observed that Figure 1 represents a simplified view of the whole 2016's report [3], which includes more than 2000 technologies. A detailed explanation about how to use such business report in innovation initia- tives, is out of the scope of this introductive paper but, in the opinion of the authors, this kind of considerations must become an essential part in any successful OER pro- posal. In this sense, referring to the specific topic of the "Experiment@ International Workshop 2016", which was "The Emerging Technologies on the Internet of Every- thing" [4] (IoE in the following), a second relevant source of information about non- functional requirements for online experimentation research is provided by the: "Nav- igating the IoE Roadmap of Challenges" report [5], published by "TM Forum" in 2016, which is a global association for digital business. As we can easily deduct from the table of contents shown in Figure 2, this report is oriented to managers and not to researchers, but in order to produce effective and sustainable IoE experiments, the latter cannot overlook the main concerns (monetiza- tion, brand management, trust, vision, etc.) typically expressed by managers working in the IoE field. Not considering these concerns, researchers could produce solutions which turn out to be not suitable/acceptable for the common user, not compatible with the most widespread IoE devices or simply not available out of their lab. iJIM ‒ Vol. 11, No. 5, 2017 7 Special Focus Paper—Sustainability, Social Impact, Learning and Training Innovation in Online Exp… Fig. 2. TM Forum Report on Internet of Everything 2016. Table of Contents 2 The Relevance of Business Models As stated in the same TM Forum report, a third specific aspect to be considered for evaluating the sustainability of an OER solution, is the "business model" or, at least, a rough version of it. In research scenarios, this concept is not to be interpreted literally, i.e. by including earnings, profits, etc. Rather, it can be useful to define and describe the main involved stakeholders; the scenario (e.g. school, home, etc.) in which the solution should be considered; the main cost drivers (e.g. connectivity, maintenance cost, consumables, etc.) needed to deploy/operate it in a real context; the main goals to achieve; some ideas about competitors (if any) or alternative approaches to achieve the same goals. Particular attention should be focused also on how to build up a capa- ble and motivated partnership that is interested in the proposed solution. It is worth to mention that even great OER solutions can fail, or remain socially ir- relevant, if these aspects are not taken into account. Again, a complete description of how to design and implement a successful busi- ness model for OER solutions is out of the scope of this paper but it is worth to men- tion two methodologies, among others, named "business model canvas" [6] and "E3Value" [7], because of the quantity and quality of good literature and case descrip- tions available online in the field of technology innovation. The former methodology, initially proposed by Alexander Osterwalder [6] and based on his earlier work on Business Model Ontology, can be defined as a strategic management and lean startup template for developing or documenting business models. It is based on a visual chart with elements describing the firm's or product's value proposition, its infrastructure, customers and finances. An example is given in Figure 3, where a simplified version of the Google business model is represented. 8 http://www.i-jim.org Special Focus Paper—Sustainability, Social Impact, Learning and Training Innovation in Online Exp… Fig. 3. Business model canvas: simplified version of the Google business model The latter methodology [7] stems from the conceptual modeling research field and it has been designed to help defining how value is created and exchanged within a network of actors, which can be very helpful for researchers working with online experimentation. 3 The Role of Social Innovation As shown in Figure 4, around 40% of employers in Europe have experienced diffi- culties finding employees with the required skills [8]. The problem is even more pro- nounced in Japan and in India, where the need for technical skills is stringent. This lack of professional profiles with technical skills is commonly referred to as "skill shortage": it has a strong negative impact on our Society and requires to be confront- ed. For these and other relevant reasons, public funding programs are often adopted by advanced countries to stimulate the development of innovative learning and train- ing approaches, including online experimentation. On the other hand, these approach- es are likely to fail without synergic activities of social innovation [9]. As stated by Phills, Deiglmeier and Miller in their article for the Stanford Social Innovation Re- view, social innovation is: “a novel solution to a social problem that is more effective, efficient, sustainable, or just than existing solutions and for which the value created accrues primarily to society as a whole rather than private individuals”. NESTA [10] defines social innovation as: “innovation that is explicitly for the social and public good. It is innovation inspired by the desire to meet social needs which can be ne- glected by traditional forms of private market provision and which have often been poorly served or unresolved by services organized by the state”. iJIM ‒ Vol. 11, No. 5, 2017 9 Special Focus Paper—Sustainability, Social Impact, Learning and Training Innovation in Online Exp… Fig. 4. Skill Shortage in 2014 In this sense, online experimentation researchers should consider social innovation as an opportunity to design not only new learning/training activities, but also new social contexts in which such activities can be effectively carried out and can produce positive outcomes. The Social Innovation Learning Model, adopted by the Stanford Center for Social Innovation, gives useful hints and examples in this direction but, obviously, it should be noted that a research project integrating social innovation and new learning/training approaches can be really challenging to define and deploy, even for strong research groups. Charter schools, in US, represent a real-life example in this sense, but there is a strong debate about their effectiveness and their overall results. 4 Conclusion Online experimentation represents a stimulating and increasingly interesting re- search field, with several opportunities, even for small research groups. A non-negligible risk is to produce good research results with irrelevant social im- pact. To overcome this problem, a suitable set of non-functional elements should be considered to integrate the functional characteristics of online experiments and to support their adoption in real-life contexts. Combining social innovation with online experimentation research, in this scenar- io, can be challenging, but results can be worth the effort. 10 http://www.i-jim.org Special Focus Paper—Sustainability, Social Impact, Learning and Training Innovation in Online Exp… 5 References [1] M.T.Restivo and A.Cardoso. Exploring Online Experimentation. International Journal of Online Engineering iJOE, Vol 9 (2013), pp. 4-6. [2] F. Jackie and H.LeHong. "Hype cycle for emerging technologies, 2011" Gartner, July (2011). [3] http://www.gartner.com/newsroom/id/3412017. [4] http://expat.org.pt/other-events/workshop16/ [5] https://www.tmforum.org, "Navigating the IoE Roadmap of Challenges", May 2016 Edi- tion. [6] A.Osterwalder,“The Business Model Ontology - A Proposition in a Design Science Ap- proach”. PhD thesis University of Lausanne, (2004). [7] J. Gordijn and H. Akkermans.Value based requirements engineering: Exploring innovative e-commerce ideas Requirements Engineering Journal, vol. 8, no. 2, pp. 114-134, (2003). [8] http://www.keepeek.com/Digital-Asset-Management/oecd/employment/getting-skills- right-assessing-and-anticipating-changing-skill-needs_9789264252073-en#page1 [9] "Defining Social Innovation". Stanford Graduate School of Business. https://www.gsb.stanford.edu/faculty-research/centers-initiatives/csi/defining-social- innovation. [10] NESTA. Social Innovation: New approaches to transforming public services. January 2008. http://www.nesta.org.uk/sites/default/files/social_innovation.pdf 6 Authors Mario A. Bochicchio (corresponding author) and Lucia Vaira are with the De- partment of Innovation Engineering, University of Salento, via Monteroni, sn - 73100, Lecce, Italy (mario.bochicchio@unisalento.it, lucia.vaira@unisalento.it). Article submitted 27 April 2017. Published as resubmitted by the authors 06 June 2017. iJIM ‒ Vol. 11, No. 5, 2017 11 iJIM – Vol. 11, No. 5, 2017 Sustainability, Social Impact, Learning and Training Innovation in Online Experimentation