SHORT PAPER ENGAGING STUDENTS WITH A MOBILE GAME-BASED LEARNING SYSTEM IN UNIVERSITY EDUCATION Engaging Students with a Mobile Game-Based Learning System in University Education http://dx.doi.org/10.3991/ijim.v8i4.3991 A. Bartel and G. Hagel Kempten University of Applied Sciences, Germany Abstract—In this contribution we present a game-based learning concept which is based on mobile devices. It focuses a joyful stabilization of knowledge and the engagement of students using the Gamification approach and its game mechanics. Previous findings how to promote students’ motivation are adapted in the mobile context and discussed. A pre-evaluation of the prototype is described with its find- ings. Index Terms—Game-Based Learning; Gamification; Mobile Learning; Motivation; University Education I. INTRODUCTION Due to the mobile diffusion, learning is not a matter of a specific time of occurrence or a particular location any more [1]. This fact offers possibilities to revolutionize the way education is by now. To use these possibilities suc- cessfully for teaching, combining mobile learning and game-based learning approaches with Gamification seems a proper chance. This is what Extended Mobile Gaming Education (eMgage) is about. As a quiz-based mobile learning application it is extended with game mechanics to increase students' motivation and engagement, possibly some of the most important prerequisites for learning [2]. II. CONCEPT We had the vision to build a system which motivates students to stabilize and extend their knowledge in a sus- tainable and joyful way with the use of recent mobile technology. After tailoring an agile software engineering process to break down and operationalize these thoughts into appropriate aims as well as prioritizing them, we came to the following system architecture as a result of the requirements analyses. A. Architectural view The system consists of three parts, which interact to- gether using the internet or a local intranet (see Fig. 1.). The system components are loose coupled and use a re- quest-response model for their interaction. Due to the fact that we wanted the system components not only to com- municate using plain text commands, we had to define a very flexible and extendible protocol. It is based on TCP/IP and the encoding and decoding of the communica- tion is done manually. This provides the freedom in mes- sage contents as we use plain bytestreams for communica- tion. In addition, possible problems with (un-) marshalling [3] which can bring along non-expected amounts of data, can be ignored. This makes the communication among the system components clearly arranged and fast. Figure 1. Overview of the learning system and relevant components. Each system component follows a defined multilayered architecture [4]. Accordingly for example the infrastruc- ture layer which the communication functionality is as- signed to can be found in the three system components likewise. The functionalities of each system component is described in the following in more detail. B. Features The mobile server (MS) is responsible for providing da- ta to the clients and channeling the communication and interaction between them. It accesses a MySQL database using a persistence layer which integrates the O/R map- ping framework OrmLite [5]. It owns the business logic of the whole system. The supervising client (SC), a standalone Java client, controls the server, its data and provides instant feedback on mobile clients connected to the mobile server and their actions. The learning applica- tion eMgage is a mobile application client (MC), based on Android, which is used by students during university lec- tures or in non-university contexts (e.g. at home). It is constructed and build as a thin client since it primarily acts a presentation tier. The application provides the following main features (see Fig. 2 main menu): • Learnsessions capsulate different types of questions as a learning unit to a specific topic which can be ed- ited by students. • Personal profile to inform students about the recent achievements and knowledge state (see Fig. 2 profile page). • Leaderboard to compare and compete with other par- ticipants. • News area to inform students about upcoming events and learnsessions. • Question builder to send in new suggestions for ques- tions from students to the lecturer. 56 http://www.i-jim.org SHORT PAPER ENGAGING STUDENTS WITH A MOBILE GAME-BASED LEARNING SYSTEM IN UNIVERSITY EDUCATION Figure 2. Screenshot main menu (left); overview profile page (right) C. Supporting the individual learning process In order to create sustainability in learning, student per- formances and related benchmarks have to be integrated in a broader process. As a temporal limitation of the pro- cess, the beginning and the end of a university course can be applied. The learning application eMgage offers the possibility to support this process by providing volunteer benchmarks on knowledge units as milestones of the pro- cess, which – not depending on the grade of success – gives students feedback on their knowledge. Each mile- stone is connected with the processing of a learnsession by each student, either during lecture or in non-university contexts. Learnsessions deal with a defined topic, for example nonfunctional requirements in the context of software engineering. They contain different types of questions (single choice, multiple choice and textual input) as well as their related answers. Each question is rated with a score and can be supported with different types of media (pictures, video streams and podcasts). Students have different types of feedback tools when using eMgage. They derive from the Gamification ap- proach and are summarized under the term “game me- chanics” [6]. D. The use of game mechanics in eMgage Gamification is defined as the concept of applying game-design thinking into none-game applications to make them more fun, engaging and change user’s behav- ior. Game mechanics are “the mechanics of a gamified system which are made up of a series of tools, that when used correctly, promise to yield a meaningful response from the players” [6, p. 36]. Each feature of eMgage is connected to at least one game mechanic. The learning application integrates the following game mechanics: point systems, leaderboards and badges. Point systems are described as an “absolute requirement for all gamified systems” [6, pp. 36]. According to the classification in [6] the applied point system in eMgage is based on experience points. This type provides the possi- bility to observe a learner, benchmark and lead him out of the view of a lecturer. Lecturers could observe the under- standing of topics due to the average scores achieved in Figure 3. Meta model of game mechanics in eMgage the related learnsessions. They can react respectively this average score for example by revisiting a topic with a different learning method. A comparable way of instant feedback is already successfully implemented in other learning scenarios like JiTT (Just-in-Time Teaching) [7, 8]. Experience points are expressed and compared in lead- erboards. The type of leaderboard implemented in eMgage is described as local (limited to a university course) and non-infinite (limited to the users who participate in that course) [9]. It is used to increase the peer-group pressure in order to let students compete with each other. Due to the fact that every student is rated with the same metric system, performances are comparable. In case a partici- pant recognizes a deficit in a topic he is able to appeal to another participant with a higher score, referring to the leaderboard. That is why leaderboards are often combined with a known and social environment [9]. Badges can be found in everyday life: sticky on the back of a car, as an award on clothes or as a virtual rating on e-commerce platforms like eBay. Besides the aim to signal status and social standing, badges are often used to control or guide a progress [9]. In eMgage there are two different types of badges: performance badges and fun badges. As the term indicates, performance badges are related to student performances hence to a specific metric. In order to achieve both seriousness and fun, special fun badges are implemented. They also use a metric for awarding but are primarily focused on non-performance achievements. An example awarding of a fun badge is the submitting of a suggestion for a question. Fig. 3 summarizes the use of game mechanics in eM- gage in a meta model using a UML class diagram. It indi- cates, that defining a particular metric each game mechan- ic is an important part of the design [9]. Additionally to the features the learning application of- fers, there are some more areas of impact which can posi- tively influence students’ interest in a topic and their learning motivation. III. MOTIVATIONAL AREAS OF IMPACT Learning motivation itself is hard to foster and de- scribes the willingness of students to address sensory, cognitive and motoric capacities in a predefined learning iJIM ‒ Volume 8, Issue 4, 2014 57 SHORT PAPER ENGAGING STUDENTS WITH A MOBILE GAME-BASED LEARNING SYSTEM IN UNIVERSITY EDUCATION situation and to coordinate in a way suitable for achieving a clearly defined objective [2]. However, we found four areas to be very effective in increasing students’ motiva- tion and their interest in a topic: arouse curiosity, promote independence, adapt difficulty and create incentives [2]. These can be adapted in the mobile context. A. Arouse curiosity A study of the University of Hampshire has shown that more than 70 percent of the students participating in a course use their smartphone more than three times during a course [10]. These findings might not be representative for every university but indicate a high significance of mobile devices for students and their interest in this tech- nology. Hence, most students are familiarized with this medium. In addition human beings have got a natural and “objective interest for everything new” [11, p. 229]. The familiarization with this medium and our natural interest are used to arouse students’ curiosity. This is the reason why eMgage is based on mobile devices. B. Promote independence Learning processes are personalized and can differ in time (when to learn), place (where to learn) and context (under which circumstances) [12, pp. 18]. Due to the use of mobile devices these constraints are almost compen- sated and provide learners increasing independence to structure their personal learning process. In addition, the access to learning material and its sharing gets more com- fortable and context specific information is provided just in time. The learning application eMgage profits from these aspects as it uses recent mobile technology. C. Adapt difficulty “Learners will only have a joyful learning and develop motivation when they consider themselves to be efficient” [2, p.1303]. An imperative prerequisite to achieve this state is to have a flexible level of learning. It is character- ized by the parameters that transfer of learning and level of learning are adequate but still there is something new to discover for learners [2, p. 1304]. The three parameters cannot be ideal provided for each type of learner, but different forming can at least support plenty of different learner types. Therefore eMgage uses a simple but effec- tive algorithm to rate each question on the basis of these parameters (lectures rate on their personal estimation): • Evaluate each parameter with a value in the range from 1, which means low, to 3, what expresses high. • Build the arithmetic mean of the three values. • Ceil the whole arithmetic expression. • Compute. The result of this calculation is the score that can be achieved when the question is answered correctly by a student. Certainly there are additional parameters and aspects which have to be taken into account when con- structing a learning exercise as described in [13]. In this case the main focus of attention should not be the overall competence-oriented construction of learning exercises, rather the learner-centric and structured sensitization what knowledge can be expected from learners. D. Create incentives Incentives like prices or rewards can be used to increase students’ short-term learning motivation [2]. Most of them are limited to the “action-reaction-principle” and are not embedded in a broader process. An example could be the evaluation of a student’s performance during semester by a lecturer where the action is the performance itself and the evaluation result is the reaction in an educational short-term context. A challenge here is to keep students learning motivation high in long-term situations beyond finished student performances. Therefore it is necessary to create sustainability which is not related to knowledge of students in the first place, rather addresses their learning motivation. That is related to the step before learning: promoting the interest in a topic by creating incentives when dealing with it. IV. LIMITS AND CHALLENGES Game mechanics derived from the Gamification con- cept as well as our findings offer possibilities to promote the engagement of students. Actually there are some limi- tations which have to be taken into account when transfer- ring these approaches into university education. One of the biggest challenges is to combine these ap- proaches in a way that the learning motivation of students does not only depend on the awards which can be achieved. This would imply that an existing intrinsic mo- tivation shifts to a one-sided extrinsic motivation which is not driven by curiosity anymore [14]. In addition a reward approach can suffer under the circumstance that achieved awards do not satisfy students anymore when using them over a long period. Thus incentives have to be dynamic and steadily improved or changed, otherwise the learning motivation is in danger to decrease. Another important aspect is the kind of person a univer- sity is dealing with. Students cannot be compared with each other across-the-board in their personalities. Each student personality is individual and the motivation to use and play with such an application can differ widely. Con- sequently competing with other students can work well or will just have no effect on some individuals [15]. None- theless many researchers support the thesis, that mobile learning can actually revolutionize learning issues: “Mo- bile devices really make a difference in education. Their possibilities allow de!ning new kinds of scenarios that improve motivation, increase retention, enhance creativi- ty, facilitate more "exibility“ [16, p. 5]. V. EMGAGE AT THE KEMPTEN UNIVERSITY OF APPLIED SCIENCES A. Basic situation At the faculty of computer science at Kempten Univer- sity of Applied Sciences the total number of students in our software engineering courses is about 50 – 100 each semester [17]. The software engineering education is placed in the bachelor programs in computer science and business informatics. B. Pre-evaluation of the prototype 1) General setup and research method We tested eMgage in our labs and pre-evaluated the prototype in order to receive the feedback of potential users and consider their ideas in the development of the application. The pre-evaluation took place in a software engineering practice with computer science students being in the third semester. Each participant (n=10) was asked to follow a defined workflow after they installed the app. 58 http://www.i-jim.org SHORT PAPER ENGAGING STUDENTS WITH A MOBILE GAME-BASED LEARNING SYSTEM IN UNIVERSITY EDUCATION The workflow contained to create an account, log in with that account, check the profile page, proceed two learnses- sions, check the profile page again, then check the leader- board and the news area. Finally each of them was prompted to send in a question and related answers as a suggestion and finalize the work by logging out. After completing the workflow the participants were asked to answer some questions in an evaluation sheet. The evalua- tion sheet contained 34 items which where standardized and thematically structured in 5 areas: the usability of eMgage (15 items), additional functionalities (4 items), possible scenarios (10 items) of usage, general opinion on eMgage (3 items), and personal data (2 items). A marked five-point answer scale (Likert scale) was used, so the participants decided whether they confirm the statements “strongly agree“, “agree”, ”disagree”, ”strongly disagree”, “undecided”. In addition each area contained free space for open comments from the participants. The evaluation sheets where processed with EvaSys to gather the results of this pre-evaluation. 2) Results The results revealed that the participants were confident with the usability of the prototype. For example they 80% strongly agreed that eMgage reacts fast on user input. This was one of our basic quality attributes when determining the requirements. Also 80% of the participants strongly agreed that the depth of the feature structure is adequate which positively influences the search time for certain features. Only the text size was seen as too small by 50% of the participants, a fact that can be changed quite easily. The free text comments confirmed the results in this area and did not come up with new findings. In the area additional functionalities the participants were discordant about the configuration possibilities the app should offer. Exemplary the statement “the back- ground color should be individually adjustable” was strongly agreed by 20%, agreed by 20%, disagreed by 30% and strongly disagreed by 20% while one participant was undecided. This variance also appears in the results of the other items in this area. This could be explained that the personal taste and preferences could differ a lot from student to student. Three participants stated in the free text comments that there could be a button to log out from the application which should be placed in the header next to the eMgage logo (see Fig. 2). This would reduce the click amount from currently two clicks to one single click. The following area contained items related to possible scenarios of usage. Fortunately almost all participants were in favor of the idea to use eMgage at home (80% strongly agreed), during waiting or travel time (50% strongly agreed, 30% agreed) or directly in university courses (50% strongly agreed, 30% agreed). Even 90% percent strongly agreed or agreed using eMgage for exam preparation while also 90% strongly agreed that the app supports their personal learning process. One participant argued that “eMgage helps a lot for exam preparation but does not replace it since software engineering is not only about plain factual knowledge”. The next area was about the general opinion on eM- gage. For example it was asked if one would recommend eMgage to other students which was answered by 100% of the participants strongly positive. Also the statement “I would be appreciated to apply eMgage in software engi- neering education” was strongly agreed by 60%, and agreed by 30% while one participant was undecided. Two participants argued that it would be better to have repeti- tive learnsessions. This would mean that learnsessions could be processed how often a user wants to. Actually this thought was one we had at the beginning of the pro- ject. We decided not to offer this repetitive structure in the first place since we want to increase the willingness to compete with each other by comparing gained experience points. As the significance of this feature was shown through these comments one could think about a solution which has two different modes: training in which the learnsessions are repeatable and competition. In general this pre-evaluation confirmed our idea of eMgage and showed that we are on track. We also gained valuable feedback and suggestions for future improve- ments although the number of participants was quite small and not representative. That is why an evaluation has to follow these findings once the implementation and testing phases of the learning system are completed. VI. SUMMARY In this contribution we presented a concept for a mobile game-based learning system. We described the features of eMgage and their realization with game mechanics and furthermore the relation between them. Our previous find- ings in increasing the learning motivation of students were adopted in the mobile context. Possible areas of motiva- tional impact and their limitations were shown and dis- cussed. Hence we presented the current state and a pre- evaluation of the application in one of our software engi- neering courses. ACKNOWLEDGMENT The authors would like to thank all the participants of the pre-evaluation for their time and the constructive statements. REFERENCES [1] M. Amberg and M. Lang, Innovation durch Smartphone und Co. - Die neuen Geschäftspotenziale mobiler Endgeräte. Düsseldorf: Symposium Publishing GmbH, 2011. [2] P. Figas, G. Hagel, and A. Bartel, The Furtherance of Motivation in the Context of Teaching Software Engineering, in IEEE EDUCON - Special Track "Technical Didactic Software Engi- neering" (TDSE), Berlin: IEEE Computer Society, 2013. [3] G. Coulouris, J. Dollimore, T. Kindberg, Verteilte Systeme – Konzepte und Design, München: Pearson Studium, 2002. [4] F. Buschmann, R. Meunier, H. Rohnert, P. Sommerlad, M. Stal, Pattern-Oriented Software Architecture, Volume 1, A System of Patterns, Chichester: Wiley, 1996. [5] G. Watson, ORMLite [online]. Available: http://ormlite.com [June 25, 2014] [6] G. Zichermann and C. Cunningham, Gamifcation by Design: Implementing Game Mechanics in Web and Mobile Apps. Sebas- topol: O’Reilly Media, 2011. [7] G. M. Novak, E.T. Patterson, A.D. Gavrin and W. Christian, Just- in-Time Teaching: Blending Active Learning with Web Technol- ogy, Upper Saddle River, NJ: Prentice-Hall, 1999. [8] G. Hagel, J. Mottok and M. Müller-Amthor, Drei Feedback- Zyklen in der Software Engineering-Ausbildung durch erweitertes Just-in-Time-Teaching, in Software Engineering im Unterricht der Hochschulen (SEUH), 2013, pp. 17-28. [9] K. Kapp, The Gami!cation of Learning and Instruction: Game- based Methods and Strategies for Training and Education, San Francisco: Wiley, 2012. [10] L. Alfano, M. Horowitch and N. Kitchen. (2010). Cell Phone Use and Concentration During Class [online]. Available: iJIM ‒ Volume 8, Issue 4, 2014 59 SHORT PAPER ENGAGING STUDENTS WITH A MOBILE GAME-BASED LEARNING SYSTEM IN UNIVERSITY EDUCATION http://www.unh.edu/news/docs/2010CellphoneUsageStudy.pdf [June 25, 2014]. [11] H. Roth, Pädagogische Psychologie des Lehrens und Lernens, Hannover: Schroedel, 1983. [12] J. Breuer, Spielend lernen? Eine Bestandsaufnahme zum (Digital) Game-Based Learning, in LfM-Dokumentation, vol. 41. Düssel- dorf: Landesanstalt für Medien Nordrhein-Westfalen, 2010. [13] P. Figas, M. Müller-Amthor, A. Bartel, A. Baumgärtner and G. Hagel, Man wächst mit seinen Aufgaben - Über die kompetenz- orientierte Konstruktion von Lernaufgaben in der Hochschullehre am Beispiel von Software Engineering, in GfD Fachtagung, Dortmund: Gesellschaft für Fachdidaktik e.V., 2013, in press. [14] F. Rheinberg, Verhaltenslenkung in Lehr- und Erziehungssituatio- nen, in Motivationsförderung im Schulalltag. Psychologische Grundlagen und praktische Durchführung, F. Rheinberg and S. Krug, Eds. Göttingen: Hogrefe, 2005, pp. 15-23. [15] A. Bartel, P. Figas and G. Hagel, Mobile Game-Based Learning in University Education. in 21st Century Education: Constructing meaning and building knowledge in technology supported learning environments, S. Feller and I. Yengin, Eds. Benjamins, 2014, in press. [16] C.D. Kloos, C. Fernández-Panadero, B. Ibáñez; M. Muñoz and A. Pardo, M-learning will Disrupt Educational Practices, in IEEE EDUCON, Marrakesh: IEEE Computer Society, 2012. [17] J. Abke et al., EVELIN. Ein Forschungsprojekt zur systematischen Verbesserung des Lernens von Software Engineering, in Tagungs- band ESE-Kongress, Sindelfingen: Elektronikpraxis, 2012, pp. 653-658. AUTHORS A. Bartel and G. Hagel are with the Faculty Computer Science, Kempten University of Applied Sciences, Ger- many ({alexander.bartel, georg.hagel}@hs-kempten.de). This work is part of the EVELIN project which is supported by the Bundesministerium für Bildung und Forschung with the project number 01PL12022C. Submitted 30 June 2014. Published as resubmitted 14 October 2014. 60 http://www.i-jim.org iJIM – Vol. 8, No. 4, 2014 Evaluation of Augmented Reality Frameworks for Android Development Engaging Students with a Mobile Game-Based Learning System in University Education