Microsoft Word - BRAIN_vol_9_issue_special1_2018_v1l.doc 110 A Critical Analysis of Mobile Applications for Learning. Study Case: Virtual Campus App Victor Holotescu University Politehnica of Timisoara, Romania victor.holotescu@yahoo.ro Radu Vasiu University Politehnica of Timisoara, Romania radu.vasiu@cm.upt.ro Diana Andone University Politehnica of Timisoara, Romania diana.andone@cm.upt.ro Abstract During the last years, the worldwide education has been challenged and innovated by the online and blended learning approaches, and the openness towards Social Media (SM), Open Educational Resources (OERs) and Massive Open Online Courses (MOOCs). University Politehnica of Timisoara, Romania is one of the pioneering higher education institution in the country, adopting an open education strategy with an important impact on the quality of education. The changing engine is represented by the Center of eLearning, which implemented the Virtual Campus for supporting the academic programs, and also the Unicampus MOOC platform. The aim of the paper is twofold. Firstly, to present and evaluate the Virtual Campus mobile application comparing it with mobile applications accompanying MOOC platforms. Then, to produce the requirements for an enhanced version of this application and the principles for the development of a mobile application for Unicampus. We hope that the paper conclusions regarding quality applications for mobile learning will be useful for teachers and developers designing open and mobile learning environments and applications. Keywords: Open education, online learning, mobile learning, mobile applications, MOOCs. 1. Introduction Over the last years, the adoption of emerging technologies, Open Educational Practices (OEPs), Open Educational Resources (OERs) and Massive Open Online Courses (MOOCs) are challenging and transforming the educational processes, bringing the openness of teaching and learning. Open Education means access to content, courses, support, assessment and certification in ways that are flexible, and accommodate diverse needs (Inamorato dos Santos et al., 2016). Also, new opportunities in learning are implied by the rapid diffusion of mobile technologies, mobile learning becoming a driver both in traditional and continuing education. Mobile learning means “learning across multiple contexts, through social and content interactions, using personal electronic devices” (Crompton, 2013). 2. UPT Virtual Campus platform and its mobile application University Politehnica of Timisoara (UPT), Romania is one of the pioneering higher education institution in the country, adopting an open education strategy with an important impact on the quality of education (Vasiu and Andone, 2014). The changing engine is represented by the Center of eLearning, which implemented the Virtual Campus in 2009, for supporting the academic programs, and also the Unicampus platform in 2014, for delivering MOOCs. V. Holotescu, R. Vasiu, D. Andone - A Critical Analysis of Mobile Applications for Learning. Study Case: Virtual Campus App 111 The Virtual Campus platform is a customized and extended version of the open-source Learning Management System Moodle, and hosts online spaces for blended courses at undergraduate, Master and Doctoral levels, also for teacher training and research projects. The academic and administrative support for all faculties, and also for distance learning, provides students and teachers with multiple delivery models, and connections with social media and collaborative platforms (Ciuclea et al., 2016, Ternauciuc et al., 2018). Mobile learning is enabled through the Virtual Campus (VC) application, developed by the first author and launched in March 2017, as a native Android mobile app, built based on the Web Services API from Moodle, which is lacking a few plugins broadly used in the courses available on the platform (https://docs.moodle.org/dev/Web_service_API_functions). The VC app was downloaded and used by 350 students, representing a percentage of 5% of the total number of the students enrolled on the platform courses. The application has features corresponding to the desktop version of the platform. In developing it, there were applied principles related to mobile learning usability and design, trying to provide enhanced possibilities for learning on-the-go and also specific notifications (Machun et al., 2012; Harrison et al., 2013; Mocofan, 2017). Figure 1 presents a series of screenshots of the VC app. After signing in the app, a student can view and visit all the courses in which he or she is enrolled (Fig.1a), from where can manually download materials to study offline (Fig.1b). Also, the user can visit the discussions forums to see what is new (Fig.1e), query the calendar (Fig.1c), to display upcoming exams or homework deadlines (Fig.1d), and also look up for any homework on a specific screen. Figire 1. Screenshots of the Virtual Campus mobile application (a, b, c) BRAIN – Broad Research in Artificial Intelligence and Neuroscience, Volume 9, Special Issue on Educational Psychology (April, 2018), ISSN 2067-8957 112 Figure 2. Screenshots of the Virtual Campus mobile application (d, e) 3. Evaluation of the Virtual Campus mobile application In May 2017, two months after the release, the Virtual Campus application has been the subject of usability testing of a group of 15 students enrolled in the Master studies of UPT. The objective of the usability testing was to find out how good the user interface and experience are, and which improvements are suggested by the students for an increased learning satisfaction. The students have followed the complete steps for a usability test, which consisted of (HHS, 2006):  Signing an end-user agreement for not disclosing any information the subject has been witnessing;  Completing a pre-questionnaire to find out how familiar the students are with web and mobile technologies and what role should they be assigned during the tests;  Each subject has taken part in 4 types of usability testing;  At the end, subjects were offered a thank you gift. We applied the moderated technique called concurrent think aloud. The usability testing was conducted by a moderator who greeted the subjects and helped them to follow the steps of the tests. In order to get as many information as possible, the setting of the environment consisted of a room with a table, chair, and a smartphone to be used during the test. Besides those, two cameras were used, one facing the smartphone on the table and one facing the subject in order to monitor the facial expressions. There were also 6 different observers:  2 time observers who had the role to register the time taken for each task to be completed;  2 error observers, registering the number of errors the subjects have encountered during a task;  2 observers writing comments they found useful during each individual task. V. Holotescu, R. Vasiu, D. Andone - A Critical Analysis of Mobile Applications for Learning. Study Case: Virtual Campus App 113 After all the subjects have went through the moderated user testing and made themselves familiar with the mobile app, in a focus group interview, they have been asked different questions about improvements and remarks. This testing method was also recorded. To obtain a complete feedback from the subjects, as a final test, the users have been asked to fill in a final questionnaire, in order to assess their satisfaction regarding the app been using for a couple of hours. The most important findings are listed below:  During the pre-questionnaire we found out that most of the users considered themselves having medium knowledge regarding using mobile apps for learning, using them occasionally, only 2 of the students considering themselves as power users.  During the moderated usability testing, most of the subjects had problems with the fact that some pages had to open in a web browser and also having to log in multiple times, sometimes even encountering errors while doing so. Besides this, students find it a little difficult to add events to their calendar for different course work deadlines or exams.  During the focus group, subjects found useful to have instant access to the course content they are looking for, compared to the web version of the platform. The subjects considered necessary to have a better interface for the calendar, because here the pace of the course is given.  The questionnaire showed that students are overall satisfied with the mobile application and that they will be using it in the future, with the remark that login issues should be fixed. We found out that our mobile app is an overall good app, with minor bug fixes that need to be solved, but with the major issue that some platform modules are missing, with the disadvantage that users have to log in a web view again to access them. A question we have been asking ourselves after the usability testing was how important the calendar was on the Virtual Campus, and how should we remake the user experience to bring the calendar in front, while also keeping the course contents in the main screen. We found out that notifications are very useful in learning apps, in order to keep the user engaged in the courses, reminding them not to overdue deadlines or about updates in the course contents. Some new features suggested by students were: a search option through courses. Also, in order to integrate more the mobile apps in the academic life, notifications related to different events and projects in university could be sent. The conclusion of the user testing was that with small improvements we can make the app more useful for the students. Because we didn’t have the possibility to bring all the necessary data inside the app, due to the missing modules in API, we concluded to give up the native Android and iOS development and to adapt the open-source hybrid Ionic app, developed by Moodle, for an enhanced VC mobile application. 4. Critical analysis of mobile applications for learning In order to enlarge the specifications and requirements for the improvements to be brought by the next version of the Virtual Campus mobile application, also to prepare those for the Unicampus mobile application, we realized a critical analysis of a few mobile applications for learning, consisting in a number of applications integrated with MOOC platforms, and one for professional training – LinkedIn Learning (https://www.linkedin.com/learning). The VC app has close features because it assures the access to multiple courses in which a student is enrolled in an academic year, also the number of students participating in a course is usually between one and two hundred. We have selected five mobile applications corresponding to the MOOC platforms: Coursera (http://coursera.org), EdX (http://edx.org), Udacity (http://udacity.com), Canvas (http://canvas.net), and Udemy (http://udemy.com), starting from the top of the first 33 MOOC platforms, ranked based BRAIN – Broad Research in Artificial Intelligence and Neuroscience, Volume 9, Special Issue on Educational Psychology (April, 2018), ISSN 2067-8957 114 on the number of courses offered (Class Central, 2017) and the list of MOOC providers (Class Central, 2018). Most of the other MOOC platforms are country-specific (China, Taiwan, Italy, Germany, Spain, France, Mexico, Russia, Ukraine, Thailand, Jordan, Japan, Korea, Indonesia) or/and don't have mobile applications. The specific features of the mobile applications which are different of those of the desktop version will be highlighted. Also, there will be described the features for mobile browsers for the FutureLearn European platform, which is not accompanied by a mobile application. The list of compared functionalities for mobile applications were defined after consulting similar studies (Mihaescu, 2016), being grouped in categories defined by the Mobilegogy model (Machun et al., 2012): Location, Technical, Culture, Satisfaction and Design. Because of the limited space of this article, we will present only the most innovative features for mobile learning of the analysed applications. All the above mobile applications are free, while the one for edX is also open-source, corresponding to Open edX (https://open.edx.org). 20% of Coursera learners complete their courses using exclusively the mobile app. The mobile app has the following specific features: taking notes and bookmarking videos, also reminders settings for work at the course. For offline work there are available: download videos and readings, and access them from the course dashboard; also complete surveys while offline and submit when online, thus upgrading the course progress (Samuelson, 2017). The edX mobile app permits the course completion on the go too. It has facilities for new accounts creation, and also video uploading by course creators (edX, 2017). While the other mobile apps are designed for students, not having administrative features, Canvas came with two separate apps for students and teachers. For students, it assures the communication between users across courses and groups, also sends notifications for new grades and course updates. For teachers, the app allows them to facilitate their courses on the go, providing quick access to three of the most frequent course facilitation tasks: grading - browse submissions and provide feedback, communicating - send announcements and messages, and updating - change due dates, publish assignments and update course content (Canvas, 2015). Starting from the idea that most students use at least occasionally mobile devices to access a MOOC platform, FutureLearn was built in 2012 using Mobile First, Responsive Web Design, meaning that its pages adapt to whatever device used to display them. The Mobile First requires a new approach to planning, UX design, and development, starting with the design for the smallest screen and working the way up (Gremillion, 2017). The experience on FutureLearn is consistent and the same, no matter the browser is open on phone, tablet or desktop, so it is no need for a mobile app. Also the product is accessible using screen readers and other assistive technologies, such as cognitive aids (FutureLearn, 2013). Thus, this platform came with a different approach than the others, which at the beginning built the platform for desktop usage, and after a few months or years started to design a mobile application. The analysis depicted many features for offline activities using mobile applications. To accomplish these, a solution is to integrate the eLearning standard Experience API – xAPI in native mobile applications in order to record locally learners' experiences and progress data, as Learning Record Stores, and then data is synchronized with the platform when network connection is available (xAPI, 2016). xAPI has hundreds of adopters, one being Moodle (2017). 5. Specifications for Virtual Campus app Following the suggestions for improvements resulted from the students’ evaluation and the mobile applications analysis presented in the previous sections, a new version of the Virtual Campus application is in developing starting from the Moodle Mobile open source code found on GitHub. Both iOS and Android versions will be available and will have all the plugins necessary to contain the same information as the web version. V. Holotescu, R. Vasiu, D. Andone - A Critical Analysis of Mobile Applications for Learning. Study Case: Virtual Campus App 115 The user will be able to stay logged in inside the app and to receive different types of notifications. The application will have the ability to be set up for working offline, while users have the possibility to change the application language, which can be useful for foreign or Erasmus students. Additionally, users will be able to customize the app based on some preferences available in a settings screen, check their grades, send direct messages to other students or professors and also save important documents under “My files” section. During the beta phase of development, a focus group with teachers will be organized in order to present the new apps and to understand what new features are needed to support the courses facilitation on the go. Figure 3. Screenshots of the new VC app 6. Implementation details The new version is built with AngularJS using the Ionic Framework (https://ionicframework.com/docs) and is currently in alpha version, soon to be released and replace the old application. The Ionic Framework is used to build hybrid mobile applications using HTML5, which are basically small websites that work inside a web view, inside the application and have access to the native platform layer. The main advantage of using Ionic is the ability to write cross platform applications which can work on both Android and iOS, with the downside that sometimes can be a little sluggish, which adds up to not have the same user experience, not being able to be as immersive as a native application. BRAIN – Broad Research in Artificial Intelligence and Neuroscience, Volume 9, Special Issue on Educational Psychology (April, 2018), ISSN 2067-8957 116 7. Conclusions In order to increase the adoption rate of the Virtual Campus apps, we plan to follow actions such as:  on the Virtual Campus platform to place the links for downloading the mobile apps;  to better make known the apps to teachers, by organizing training sessions and webinars and providing concrete scenarios for using apps in the teaching process. As future research we plan to evaluate the effectiveness of mobile learning and how it is influenced by factors such as teaching and facilitation methods, subject disciplines, collaborative work, peer and social interactions, but also how these educational issues are supported by the updated mobile application features. Also the degree in which mobile learning produces digital skills improvement and changes in the processes of information access both for students and teachers (Wong, 2018). The Unicampus app will permit us to extend the evaluation to wider groups of learners. References Canvas. (2015). Student and Teacher Mobile Applications Guides. Retrieved from https://community.canvaslms.com/community/answers/guides/mobile-guide. Ciuclea, C., Prostean, G., Andone, D., & Draghici, A. (2016, August). Integrating E-learning Platforms in Teaching/Learning. A Critical Review Upon the Virtual Campus Usage in UPT. In International Workshop Soft Computing Applications (pp. 225-234). Springer, Cham. Class Central. (2017). Massive List of MOOC Providers Around the World Where to Find MOOCs: The Definitive Guide to MOOC Providers. Retrieved from https://www.class- central.com/report/mooc-providers-list. Class Central. (2018). MOOC Providers. Retrieved from https://www.class-central.com/providers. edX. (2017). Mobile Features. Retrieved from https://open.edx.org/features-roadmap/mobile. Crompton, H. (2013). A historical overview of mobile learning: Toward learner-centered education. In Berge, Z. L. & Muilenburg, L. Y. (Eds.), Handbook of mobile learning (pp. 3–14). Florence, KY: Routledge. FutureLearn. (2013). Being Responsive. Retrieved from https://about.futurelearn.com/blog/being- responsive. Gremillion, B. (2017). A Hands-On Guide to Mobile First Responsive Design. Studio by UXPin. Retrieved from https://www.uxpin.com/studio/blog/a-hands-on-guide-to-mobile-first-design. Harrison, R., Flood, D., Duce, D. (2013). Usability of mobile applications: literature review and rationale for a new usability model. Journal of Interaction Science 2013, Vol 1(1). HHS. (2006). The Research-Based Web Design & Usability Guidelines. Running a Usability Test. Retrieved from https://www.usability.gov/how-to-and-tools/methods/running-usability- tests.html. Inamorato dos Santos, A.I, Punie, Y. & Mũnoz, J.C. (2016). Opening up Education: A Support Framework for Higher Education Institutions (No. JRC101436). Directorate Growth & Innovation and JRC-Seville, Joint Research Centre. Retrieved from http://publications.jrc.ec.europa.eu/repository/bitstream/JRC101436/jrc101436.pdf. Mihaescu, V. (2016). The Development and Implementation of a MOOC Platform Model from a Technical and Educational Perspective. PhD Thesis. UPT. Editura Politehnica Timisoara, Romania. Mocofan, M. (2017). Improvement of courses interactivity using web and mobile technologies for instant feedback. Proceedings of The 11th International Technology, Education and Development Conference Valencia, Spain. 6-8 March, 2017. Moodle. (2017). Moodle, the LRS? Getting Started with xAPI Learning Record Stores. Retrieved from https://www.moodlenews.com/2017/moodle-the-lrs-getting-started-with-xapi-learning- record-stores. V. Holotescu, R. Vasiu, D. Andone - A Critical Analysis of Mobile Applications for Learning. Study Case: Virtual Campus App 117 Samuelson, J. (2017). New mobile features: Transcripts, notes, and reminders. Retrieved from https://blog.coursera.org/new-mobile-features-transcripts-notes-reminders. Ternauciuc, A., Andone, D., Ciuclea, A. (2018). Development of a Moodle Extension for Improving the Management of the Educational Process. In The International Scientific Conference eLearning and Software for Education, Bucharest, Romania, April, 2018, Vol.4, p.363. Vasiu, R., & Andone, D. (2014, November). OERs and MOOCs—The Romanian experience. In Web and Open Access to Learning (ICWOAL), 2014 International Conference on (pp. 1-5). IEEE. Wong, B.T.M. (2018) Success in Mobile and Ubiquitous Learning: Indicators of Effectiveness. BRAIN - Broad Research in Artificial Intelligence and Neuroscience, Vol 9, Special Issue on Educational Psychology. xAPI. (2016). A Mobile Device’s Story. Retrieved from https://xapi.com/a-mobile-devices-story. Victor HOLOTESCU is a student in the Multimedia Technologies Master program of UPT, in the second year. He has a 5 years experience in mobile application development. In his portfolio there are the TPark parking applications, also the ArtEncounters, ArtTM, and UPT Virtual Campus apps. He has won many awards in profile competitions and hackathons. Also he participated in an Erasmus internship at the University of Oulu, Finland. Radu VASIU is Professor at the Faculty of Electronics and Telecommunications Engineering of the Politehnica University of Timisoara. His latest research interests are in e-learning, multimedia, web technologies, open data and smart cities. He is the President of Politehnica University Senate and Director of the Multimedia Research Centre. He published 12 books and over 100 papers presented at different international conferences. He was involved in 30 research or international cooperation projects and is part of 7 international professional associations. Currently, Prof. Vasiu acts as President of the International Association for e-Science (IAFeS), that promotes at international level the use of ICT in science and technology. Dr. ing. Diana ANDONE is the Director of the eLearning Center of Politehnica University of Timisoara, responsible for planning and implementing eLearning. Diana Andone holds a PhD in Designing eLearning Spaces for Digital Students with University of Brighton, UK. The publication list comprises 17 books and over 80 papers presented at international conferences. In the last years she was involved in over 30 EU funded projects, recently with a focus in open education, publishing and technologies. She was co-ordinator of the LLL ViCaDiS (Virtual Campus for Digital Students) project and national responsible on m-commerce, SKILL2E, CBVI, i2Agora, TafCity, ESIL, e-Taster, e-report projects, as well as the national DidaTec project (Training in blended-learning and new educational technologies for university academic staff). She is now leading the UniCampus project with the goal of creating the first Romanian MOOCs. She actively promotes Open Educational Resources (OER), Open Knowledge (OKF) and Massive Open Online Courses (MOOCs) as part of her everyday activities or through her work in different associations or task-force.