International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 14, No. 13, 2020 Short Paper—Virtual Teachers’ Toolbox (VTT-Box) The Experience of the Costa Adeje… Virtual Teachers’ Toolbox (VTT-BOX) The Experience of the Costa Adeje International School and the University of La Laguna https://doi.org/10.3991/ijim.v14i13.14109 David Pérez-Jorge (*), María del Carmen Rodríguez Jiménez University of La Laguna, Canary Islands, Spain dpjorge@ull.edu.es Nuria Marrero Rodríguez, Sergio Pastor Llarena, María Dolores Mateos Peñas Costa Adeje International School, Canary Islands, Spain Abstract—We present the results which arise from the development of the Virtual Teachers’ Toolbox (VTT-Box) project. The main aim of the project was to develop a virtual web-based service for teachers (toolbox) to support them best with a sophisticated tool to use the pedagogical framework based on an innova- tive learner-centered approach, including a quality enhancement framework. The project was designed for students and teachers of the 4th grade of the Obligatory Secondary Education (4th ESO) and Bachillerato, and was focused on the students’ needs with the aim of increasing their motivation and learning success. The development and practical implementation of an innovative moti- vating self-evaluation tool for students called Self-evaluation Mandala was in- cluded. The Mandala is an easy to handle graphic tool which defines learning results based on competences, evaluates student’s success, motivates them and makes them aware of the improvement achieved throughout their learning pro- cess, increasing motivation and academic success. This work analyses the results of the development of the project with the stu- dents of the 4th ESO and Bachillerato (n=227), and the opinions of the teachers who implemented the project at the Costa Adeje International School. Moreover, the effect of the training with the Mandala self-evaluation tool was evaluated by teachers in training of the University of La Laguna (n=25). The evaluation made by the teachers in training and the students invites to keep working on the devel- opment of training events like the one carried out at the VTT-Box project. The dissemination of the project entailed a global training experience that has allowed the enhancement not only of participant teachers and students, but also of teach- ers from several countries, including teachers in training of the University of La Laguna. Keywords—Digital competence, innovation, motivation, collaborative learn- ing. iJIM ‒ Vol. 14, No. 13, 2020 221 Short Paper—Virtual Teachers’ Toolbox (VTT-Box) The Experience of the Costa Adeje… 1 Introduction This work is the result of the collaboration between the Costa Adeje International School and the University of La Laguna. The results want to motivate readers to reflect on the effects of the development of innovative experiences in the frame of the training of university students. The so-called Virtual Teachers’ Toolbox (VTT-Box) [7] developed in collaboration with other four European countries (Sweden, Greece, Austria and Italy) has been an outstanding experience to train not only secondary students of Costa Adeje Interna- tional School but also undergraduates of the Education School of the University of La Laguna. 2 Background The main goal of the VTT-Box project was to develop a virtual toolbox for teachers which allowed the development of open distance learning courses using a learner cen- tered methodology and including a quality framework [2][4]. These courses should pro- mote the enhancement of digital competences of teachers with the aim of encouraging open and innovative educational practices, encouraging the acquisition of skills and competences of the students, promoting their motivation and avoiding school failure and early school leaving. The project was designed for students and teachers of the 4th ESO and Bachillerato, and used learning materials and tools focused on students’ needs with the aim of in- creasing their motivation and learning success. This project was adapted as a training workshop for teachers in training of the University of La Laguna. One of the main challenges for teachers nowadays is to offer alternative teaching approaches, which arouse interest in learning and are attractive to students [5][6][1]; [3]. The evaluation criteria reflect the teaching philosophy. When a student knows what he is learning and he is aware of his progress and achievements, he learns in a better way [5][6]. Self-evaluation is a strategy that encourages reflective processes in the act of learning. The VTT-Box project developed and implemented an innovative and motivating self-evaluation tool called the Self-evaluation Mandala. The Mandala is an easy to han- dle graphic tool which defines learning results based on competences, evaluates stu- dents success, motivates them, and makes them aware of their own learning process, their improvement and the goals achieved throughout the training process [4] The edu- cational goals that were addressed were specifically: a) to provide teachers with the necessary tools for a methodological change so as to promote open and innovative ed- ucational practices; b) to promote the use and improvement of digital competence, both in students and teachers; c) to promote students’ motivation through multimedia and interactive activities that allow them to acquire skills and promote long-life learning (generalization of learning), and d) to encourage academic success as a basic strategy to avoid school failure and early school leaving. For the implementation of the experi- ence, the evaluation criteria and specific learning standards of the subjects (Biology, 222 http://www.i-jim.org Short Paper—Virtual Teachers’ Toolbox (VTT-Box) The Experience of the Costa Adeje… Physics and English) were specified. Regardless of the subject, the following compe- tences were taken into account: a) digital competence (the courses required the use of electronic devices for the creation of content in different formats (text, photo, video, etc.) and manipulation of interactive activities); b) linguistic competence (these distance learning courses promoted the development of both communication skills and strategies for reading and writing, as well as skills related to the treatment of information); c) social and civic competence (all courses contained interactive activities in which stu- dents should interact with other classmates according to rules based on mutual respect and democratic convictions, students should also show tolerance, and express and un- derstand different points of view), and d) learn to learn (the courses required reflection and awareness of the learning process itself, an example of this was the use of the Self- Evaluation Mandala). All interactive materials that teachers could use for the development of this kind of distance learning courses are available on the project website and are freely available (https://www.vtt-box.eu/project/). The experience has been a good example of collaboration between a Secondary Ed- ucation School (Costa Adeje International School) and a University (University of La Laguna). 3 Objectives This study aims to assess the impact and effectiveness of the VTT-Box project. For this we have tried to know to what extent the project: • Enables a methodological change and promotes open and innovative educational practices. • Promotes generic skills in students. • Improves digital competence in both students and teachers. • Promotes students’ motivation. • Promotes academic success. 4 Methodology 4.1 Sample A total of 227 students of Obligatory Secondary Education (ESO) (NESO = 183) and Bachillerato (NB = 44) participated in this study, together with the 25 teachers in training in the first-year of Pre-primary Teacher’s Degree (NMI = 12) and Primary Teacher’s Degree (NMP = 13) who voluntarily participated in the VTT-Box training sessions. The demographic and experiential characteristics are shown in Table 1: iJIM ‒ Vol. 14, No. 13, 2020 223 Short Paper—Virtual Teachers’ Toolbox (VTT-Box) The Experience of the Costa Adeje… Table 1. Demographic and experiential characteristics of the sample N % Gender Male 96 38.1 Female 156 61.9 Age Less than 18 years 227 72.7 More than 18 years 25 13.2 The sample of this study was mainly female (61.9%) under the age of 18 (72.7%). In addition, 3 interviews were carried out with teachers who developed the project at the Costa Adeje International School (N=2, women), and (N=1, men). The answers obtained have provided qualitative information that has supported and clarified the in- formation obtained in the questionnaires. 4.2 Instruments and techniques A questionnaire of 12 evaluation items was used to carry out this study. This is an Ad hoc test which aimed to investigate the general level of satisfaction of the VTT-Box experience and the value attributed to the Mandala Self-assessment tool. It is a unipolar Likert scale of 4 response levels where value 1 expresses the lowest degree of agree- ment and 5 the highest degree of agreement. The interview collected 7 general assessment questions: 1. What general evaluation do you make of the VTT-Box experience?, 2. To what extent has the experience con- tributed to improve your teaching quality?, 3. Has the experience improved students’ digital, linguistic, social and civic and learn to learn competences? Make a comment 4. Do you think the Mandala Self-Evaluation tool has been useful for students? How has it helped you? 5. Level of overall satisfaction with VTT-Box experience, 6. Degree of motivation that the VTT-Box project has aroused in you, 7. Other aspects that you wish to highlight about the VTT-Box experience. Table 2. For the identification of the responses of each participant, the code “Pn” was estab- lished where “P” indicates that it is a teacher and the “n” number of the participant (see table 2 for the profile of the interviewees).Profile of the interviewees Group 1: P1, P2 & P3: Teachers in training Group 2: P4, P5 & P6: Teachers from the Costa Adeje International School who implemented the VTT-Box project 4.3 Procedure In order to obtain the data of this study, questionnaires were given to both, the stu- dents of the Costa Adeje International School and the teachers in training. In order to carry out the interviews, appointments were arranged with the different teachers in training. In the case of the teachers of the Costa Adeje International School who carried out the experience, the questionnaires were sent via email for their response and subse- quent return. 224 http://www.i-jim.org Short Paper—Virtual Teachers’ Toolbox (VTT-Box) The Experience of the Costa Adeje… 5 Results and Discussion To facilitate the reading of the results, a P is indicated at the end of each item when it refers to the project and an M when it refers to the Self-Evaluation Mandala. Regard- ing the assessment made by students (VA) and teachers in training (VPF) who partic- ipated in the experience, it should be noted that in general terms, they consider that the project favors academic success and prevents early school leaving at ESO and Bachil- lerato (VA = 80%; VPF = 90%); it encourages students’ motivation, (multimedia and interactive activities) that promotes the acquisition of competences and the generaliza- tion of learning (VA = 100%; VPF = 90%); it promotes the use and the improvement of digital competence in students (VA = 80%; VPF = 95%) and it is suitable for the development of linguistic, social and civic and learning to learn competences in stu- dents. (VA = 90%; VPF = 100 %). They also value the Self-Evaluation Mandala as a tool that: can be easily edited (download, color and reload easily) (VA = 80%; VPF = 90%); it allows comparison (before-after activity) and it makes you aware of the learning improvements (VA = 60%; VPF = 80%); it enables teacher’s assessment (comparison of before-after results) (VA = 50%; VPF = 75%). Regarding the rest of the aspects evaluated, the discrepancies between students and teachers in training have been more evident in relation to: the ability to describe competences and show what should be learned (VA = 45%; VPF = 80%); the understanding of the level of competence achievement (VA = 40%; VPF = 75%); the evaluation of the teacher's ability to teach (VA = 40%; VPF = 60%). Despite the level of motivation and interest aroused by the Self-Evaluation Mandala, they considered, both students and teachers in training, that the graphic representation chosen is more appropriate for younger students (VA = 90%; VPF = 90%). Regarding the opinion of the teachers who carried out the project, it was observed that they agreed (100%) on most aspects evaluated in relation to both the project and the Self-Evaluation Mandala (see table 3). There is a discrepancy between the evaluation of teachers in training and students with regard to the opinion of teachers who carried out the project. The teachers who carried out the project considered the graphic representation of the Self-Evaluation Mandala suitable for the ages of ESO and Bachillerato while the teachers in training and the students considered the graphic representation unsuitable or childish. From the analysis of the answers given by the teachers who carried out the VTT- Box project and the teachers in training, it was observed that in general terms the level of satisfaction with the experience was very high, and as they stated, allowed them: "to discover new methodological tools that will be of great help to break the routine in the classroom" (P4), "to know about new resources" (P1), "to modify their usual method- ology and show enthusiasm when they developed activities which were different from the ones they were used to" (P5), "to think of other ways of doing and teaching" (P2). iJIM ‒ Vol. 14, No. 13, 2020 225 Short Paper—Virtual Teachers’ Toolbox (VTT-Box) The Experience of the Costa Adeje… Fig. 1. Assessment of the project and the Self-Evaluation Mandala by students and teachers in training They all agreed that the experience has had a positive impact on the improvement of educational quality. With regard to teachers, they stated that the experience allowed them: “to know about methodological alternatives” (P4), “to see different ways of teaching” (P2), “to discover the necessary tools for a methodological change and im- prove digital skills” (P5), “to use technological resources” (P3); “to know and share experiences from other educational systems […] to modify some of the educational habits that may be obsolete”(P6),“to know what is currently being done at school” (P1). With regards to students, they stated that: "it promoted the motivation of students" (P4), "I think it can interest and motivate children more" (P2), "it encouraged motivation through multimedia and interactive activities that allowed students to acquire skills and increase their educational success" (P5), "it is more interesting to students because they are used to new technologies" (P1). 226 http://www.i-jim.org Short Paper—Virtual Teachers’ Toolbox (VTT-Box) The Experience of the Costa Adeje… Table 3. Opinions of teachers who carried out the VTT-Box project Disagree Agree It describes competences correctly and shows what should be learned (M) 100% It helps to understand the level of competence achievement (M) 100% The graphic representation is more suitable for younger students (M) 100% It enables teacher’s assessment (it compares before-after results) (M) 25% It evaluates teacher’s ability to teach (M) 75% It can be easily edited (download, color and reload easily) (M) 100% The comparison (before and after the activity) makes students aware of their learning improvements (M) 100% It provides teachers with tools for methodological change (promotes open and innovative educational practices) (P) 100% It promotes students’ linguistic, social and civic and learning to learn competences (P) 100% It promotes the use and improvement of digital competence in students (P) 100% It encourages students’ motivation (multimedia and interactive activities) that promotes the acquisition of competences and generalization of learn- ing (P) 100% It promotes academic success and prevents early school leaving (P) 100% They also said that students had improved their digital, linguistic, social and civic, and learning to learn competences. The autonomy given to students, the change of role of the student and the nature of the activities themselves justified the development of competences. In this sense, they stated that: “students acquire these competences in an autonomous way without the teacher having to intervene constantly” (P4), “they al- ready know how to use applications and programs and it is not necessary to be teaching everything, they are more independent” (P3), "the challenges posed by the tasks made most of the students involved in providing something different and original" (P6), "the tasks are attractive and arouse curiosity" (P1). They also valued the Self-Evaluation Mandala as a useful and positive tool for students, thus they considered that it had helped them: “to reflect before and after the learning process […] assess their degree of achievement of the objectives” (P4), “to see how and how much they have learnt” (P2); “the Self-Evaluation Mandala makes students aware of their progress, reflect on what they have done and analyse the complexity of true learning and not just purely rote” (P6), “students become aware at every moment of where they are in regards to what they should learn” (P3). In the same way they considered that it is not suitable for courses such as Bachillerato because “in Bachillerato, whether you like it or not, teach- ers have the University-access test (EBAU) in their heads, and the way it is configured today, affects teaching a lot because we must keep close to the EBAU”(P6). In relation to the motivation generated by the VTT-Box project, all participants ex- pressed a very high level of satisfaction when they confirmed improvements derived from the development of the project not only in terms of individual satisfaction but also in terms of professional satisfaction due to the improvement of the quality of teaching and the interest and motivation of the students. This was pointed out by the participants who stated: “I like the tools developed and I will continue using them because I think they are great to reinforce and consolidate knowledge in a different way” (P4); “it is the iJIM ‒ Vol. 14, No. 13, 2020 227 Short Paper—Virtual Teachers’ Toolbox (VTT-Box) The Experience of the Costa Adeje… first time I see how to transfer the mandalas to learning experiences, very interesting” (P1), “I have discovered other ways of organizing my teaching units, including more attractive activities for students” (P5), “it does not take so much to make learning at- tractive, I feel that I have been programmed as a future teacher to do things in just one way, and I think it is essential to be original and creative. This experience has helped me to think in an alternative way” (P2), and “I have improved and now I am more efficient and effective” (P6),“I think that developing this kind of projects can make us enjoy more as teachers and can also make students enjoy more because they find more sense in what they do in this way” (P3). Other highlighted aspects were the enrichment of experiences from other countries, the experience of the development of the project from approaches and points of view of the different disciplines and educational fields, and the relational and cooperative atmosphere that was developed during the implementation of the VTT- Box project. 6 Conclusion From the experience of application and implementation of the VTT-Box project, it can be concluded that: • It provides teachers with the necessary tools for a methodology change so that it promotes open and innovative educational practices. • It promotes the development of following competences in students: linguistic, social and civic, and learn to learn. • It develops digital competences in both students and teachers. • It promotes students’ motivation with multimedia and interactive activities that al- low them to acquire competences and encourage long-life learning (generalization of learning). • It encourages learning success as a basic strategy to avoid early school leaving. • The Self-Evaluation Mandala allows to compare previous knowledge with the ac- quired and achieve knowledge, which enables reflection and awareness of the learn- ing progress in a simple way. • Students found learning materials very useful and felt satisfied, since the use of tools such as forums allowed them to share opinions with their peers and to be aware of different points of view. They also learnt ideas, techniques and strategies from their peers, which helped them in the study and understanding of the subject. • The Self-Evaluation Mandala allows students to know the skills to acquire and rec- ognize their own educational achievements in a simple way. To sum up, it has been an experience that has enabled training improvement not only of the participating students and teachers of the Costa Adeje International School, but also of teachers from all over the world, including teachers in training of the University of La Laguna. 228 http://www.i-jim.org Short Paper—Virtual Teachers’ Toolbox (VTT-Box) The Experience of the Costa Adeje… 7 Acknowledgement We are deeply grateful to Costa Adeje International School and the University of La Laguna for their cooperation. 8 References [1] Gago, Rodrigo, and Peris-Valencia, Alexandra. Autoevaluación y evaluación compartida a través del uso de rúbricas. Una experiencia en educación primaria. Revista Infancia, Educa- ción y Aprendizaje, vol. 5, no. 2, pp. 40-47, 2019. https://doi.org/10.22370 /ieya.2019.5.2.1499 [2] Mazohl, Peter, Ossiannilsson, Ebba, Makl, Harald, Ampartzaki, Maria and Kalogiannakis, Michail. Virtual Teachers’ Toolbox – An Innovative Tool to Assist the Creation of High- Quality Open Distance Learning Courses. In Proceedings of the 10th International Confer- ence on Computer Supported Education (CSEDU), no. 2, pp. 555-560, 2018. https://doi.org/10.5220/0006820005550560 [3] Méndez, Vicente, and Magaña, Ernesto. La autoevaluación como herramienta de evalua- ción: percepciones del proceso de aprendizaje de los estudiantes en prácticas del Grado en Educación Primaria. Revista Prácticum, vol. 4, no. 1, pp. 37-54, 2019. https://doi.org/10.5944/reec.34.2019.23524 [4] Ossiannilsson, Ebba, Ampartzaki, Maria, Kalogiannakis, Michail and Mazohl, Peter. The VTT-BOX, Pedagogical and Quality Considerations. In Proceedings of the 11th Interna- tional Conference on Computer Supported Education (CSEDU), no. 2, pp. 654-659, 2019. https://doi.org/10.5220/0007876506540659 [5] Pérez-Jorge, David. Reforma versus innovación. Educadores: Revista de la Federación Es- pañola de Religiosos de Enseñanza, vol. 41, no. 192, 443-455, 1999. [6] Pérez-Jorge, David. (2010). El profesor motivador: El profesor que estimula e implica a los alumnos en proyectos para el desarrollo de la competencia cultural y artística. Olga Alegre, (Ed.), Capacidades docentes para atender a la diversidad, pp. 139-153, 2010. https://doi. org/10.2307/j.ctt2050wjh.10 [7] Virtual Teachers’ Toolbox Project, 2018. Retrieved January 13, 2020 de https://www.vtt- box.eu/project/. 9 Authors David Pérez-Jorge is presently, teacher and researcher at the Didactic and Educa- tional research Department in the University of La Laguna. María del Carmen Rodríguez-Jiménez is presently, teacher and researcher at the Didactic and Educational research Department in the University of La Laguna. Nuria Marrero Rodríguez is presently, secondary school teacher at the Costa Adeje International School. Sergio Pastor Llarena is presently, secondary school teacher at the Costa Adeje International School. María Dolores Mateos Peñas is presently, secondary school teacher at the Costa Adeje International School. Article submitted 2020-03-06. Resubmitted 2020-04-21. Final acceptance 2020-04-21. Final version pub- lished as submitted by the authors. iJIM ‒ Vol. 14, No. 13, 2020 229