Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach on Enhancing Computational Thinking https://doi.org/10.3991/ijim.v15i21.23143 Sutiwat Supaluk(), Jintavee Khlaisang, Noawanit Songkram Chulalongkorn University, Bangkok, Thailand Sutiwat.S@student.chula.ac.th Abstract—In each country, skills development in the 21st century plays a crucial role in education at all levels. One of these skills is computational thinking (CT), which is the ability to systematically think and solve problems in daily life. This research aims to ascertain the benefits of using a mobile cloud-based learning system with a peer-to-peer reverse engineering approach in enhancing CT skill. In this study, a cloud-based learning system and learning management plans were employed as research instruments, and a CT test was used as a data gathering instrument. The data were analyzed using the mean and standard devi- ation, repeated-measures analysis of variance (ANOVA), and t-test. The find- ings suggested that the students’ scores were higher after using the cloud-based learning system, at a 0.05 level of significance. After performing the repeated- measures ANOVA, the students’ scores were significantly higher at a 0.05 level in each round of the CT test. At the same time, peer learning increased the positive attitude of the learners toward the system and promoted the effectiveness of working in a team. This study provides a better understanding of the current research trends in the field of CT education, the perception of students, and the learning strategies and technology involved in CT education. Keywords—cloud-based learning, reverse engineering, computational thinking, peer learning 1 Introduction Computer technology and communication have been developed and employed as tools for accelerating work, education, and learning. Technology learning has insuf- ficiently supported life in the era of the digital economy, when knowledge and skills are needed to solve problems in real life. Innovation development facilitates human lives. Moreover, technology information and communication are used to create knowl- edge and value. The educational authorities of each country throughout the world have realized that developing learners’ skills, which allows them to obtain quality of life in the 21st century, is essential. Therefore, the authorities have promoted computational iJIM ‒ Vol. 15, No. 21, 2021 67 https://doi.org/10.3991/ijim.v15i21.23143 mailto:Sutiwat.S@student.chula.ac.th Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… thinking (CT), which encourages learners to develop their critical skills to solve problems systematically and to become individuals with morals and values who apply science and technology with creativity [1], [2]. As this knowledge is important, it has been included in the fundamental curriculum in many countries [3] where course syllabuses and teaching methods have been designed to provide students with effective teaching that is compatible with the levels of the learners. According to the JRC Science for Policy Report of the European Union [2], it is important to include CT in the compulsory curriculum because it can help prepare students for an innovative society where people can use technology to facilitate their lives. In the United States, the International Society for Technology in Education and the Computer Science Teachers Association [4] gave precedence to CT by creating a learning program, developing learning resources, teaching media, activities, and sheets, and allocating them to teachers at all levels. So, Jung, and Liu [5] found that some Asian countries, such as Korea, Chinese Taipei, Chinese Hong Kong, and China, have strong positions in the ICT industry. In these countries, national curricula were reformed to strengthen the CT skills of students in K-12, who would become their human capital. Moreover, school executives, teachers, and stakeholders in the Asia-Pacific region are encouraged to be ready. Each region around the world considers the development of CT teaching important because it can help prepare human capital for the digital era. CT is considered a highly valuable skill for the 21st century. Based on computing concepts and systematic work plans that can be applied in everyday life, it involves problem solving, data analysis, product design, and product development [6]. Com- puting is important in the era of data complexity and a massive amount of data. People are unable to perform efficient data analysis without the involvement of a computer; therefore, people and technology need to be integrated to create effective data manage- ment [7]–[9]. As CT is essential, many scholars and researchers have become interested in studying the factors and methods that can promote this skill. From 2009 to 2013, Lye and Koh [10] conducted a systematic review on the teaching and learning of CT through programming to gather data and present guidelines on classroom design for K-12 students. Papadakis and Kalogiannakis [11] applied the ScratchJr program to CT and the science education teaching of pre-service kindergarten teachers. The results showed that the teachers understood the concept and had a positive attitude toward the teaching of programming and science education. Papadakis [12] analyzed robots and robotics kits that help the development of children in early childhood and students during their primary education. The results showed that class arrangements with the application of learning tools drew the attention of students, encouraged them to learn, and led them to achieve the learning outcomes. Wu, Hu, Ruis, and Wang [13] used quan- titative ethnography to analyze collaborative programming activities to promote CT. Bedar and Al-Shboul [6] demonstrated how the STEAM approach could develop the CT of high school learners. Zaharin, Sharif, and Mariappan [14] introduced CT through plugged or unplugged activities to promote problem-solving skills and to allow learners to perceive knowledge well. Most of these studies were conducted on indi- vidual learners and groups of learners. Some studies were conducted on peer learning to motivate learners to help each other solve problems, decrease the number of mis- takes, and promote teamwork skills [15]. The researchers of this study considered that peer learning could improve strategies for the development of thinking skills. They also investigated the different learning skills of the learners in groups. When a class is 68 http://www.i-jim.org https://www.semanticscholar.org/author/Nur-Lisa-Zaharin/151157707 https://www.semanticscholar.org/author/Sabariah-Sharif/35665267 https://www.semanticscholar.org/author/M.-Mariappan/11372029 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… arranged using this model, appropriate technology and strategies are required to obtain the goals of learning. In education, CT allows students to create knowledge and exchange ideas with their peers. However, teachers must ensure the availability of learning resources and technology so that students can access information anytime and anywhere. This helps knowledge to be constructed through mutual interaction. A technology that can help promote learning skills is cloud computing. Cloud com- puting is suitable for all solutions because it consists of useful features, such as data communication and transmission via networking, storage, and data service resources. [16]–[19]. Consequently, mobile cloud technology [20] has been integrated into some schools to promote collaborative learning among students. Unsurprisingly, it allows students to communicate easily with their peers within and outside the classroom [21], [22]. Reverse engineering (RE) is a mobile cloud technology used in the learning pro- cess to enhance students’ CT. The assessment of students’ outcomes usually occurs upon the completion of this learning process. The RE approach has been integrated into cloud-based learning to provide students with a real-world experience, develop their cognitive and problem-solving skills, teach them how to identify cause-and-effect rela- tionships, and select the best possible solutions or create better development outcomes. The types of content stored in this learning system should be the ones developed by computer programs, such as content related to software design and development, dig- ital games, websites, technological media, and materials/items or anything created by computer programming [23]–[25]. Learning activities in a cloud system should allow students to share their learning experiences and engage them in a cooperative manner by working toward common goals. In this study, the researchers used the peer-assisted learning technique, in which opportunities for students are created to exchange knowledge between and among them and to work in collaboration. This technique helps students gain valuable insights into their skills and correct their mistakes. Peer-assisted learning enables students to improve their potential by working and solving problems alongside their peers. Addi- tionally, it helps students develop emotional intelligence, which is the gateway to living a more fulfilled and happy life [26], [27]. In this paper, the researchers aimed to investi- gate the results of peer learning in collaboration with RE using cloud technology. They also examined whether the differences between students who worked in pairs and those who learned independently had an effect on the students’ CT. Therefore, this study aimed to develop a mobile cloud-based learning system using a peer-to-peer (P2P) RE approach to enhance CT. The other specific objectives are as follows: i) to determine how a mobile cloud-based learning system fosters play and learning in a system and ii) to determine the effects of a mobile cloud-based learning system on learning outcomes when used for play and learning in a system. The research questions are as follows: i) How does a mobile cloud-based learning system foster play and learning in a system? ii) What are the effects of a mobile cloud-based learning sys- tem on learning outcomes when used for play and learning in a system? This paper is organized as follows. Section 2 reviews the background and related works, and Section 3 presents the conceptual framework. Section 4 discusses the learn- ing system analysis and design, and Section 5 explains the experimental research design, instrument, participants, and study procedures. Section 6 evaluates and validates the learning system. Section 7 summarizes the study with a discussion and conclusions, iJIM ‒ Vol. 15, No. 21, 2021 69 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… and Section 8 presents the research findings for future works. Lastly, Section 9 provides the research limitations and some suggestions. 2 Background and related works As 21st-century learning skills play a crucial role, many scholars have investigated learning skills based on context, environment, and educational technology to conduct research appropriately and effectively. The theoretical framework of this research is illustrated in Figure 1. Social Constructivism Computational thinking Mobile cloud- based learning Peer learning & Reverse engineering 21st-Century Learning Skills Technology- Enabled Learning Fig. 1. Theoretical framework • Twenty-first-century learning skills are essential for living in a world of technology, economy, and trade. As technology facilitates human lives, connects people, and allows them to access news and information anywhere and anytime, human techno- logical skills should be strengthened so that they can learn and smartly adapt them- selves. Moreover, when these skills are strengthened, people become well informed and can obtain leadership. The reinforced skills can also help people get jobs and become good citizens of the world. There are three skills that should be strengthened among students: learning and innovation skills; information, media, and technology skills; and life and career skills. In this study, learning and innovation skills for CT were investigated [28], [29]. • Social constructivism, which is based on constructivism theory proposed by Lev Vygotsky, states that social interaction plays an important role in the development of cognitive skills. Vygotsky believes that students can create knowledge while having social interactions with other people, such as parents, teachers, and friends. Therefore, the classroom is designed to encourage students to interact with others. For example, students are assigned to work in groups and create projects. They are encouraged to use technology to support their learning while studying with their friends. In this article, information about peer learning and the RE technique, which promotes learning, was collected and analyzed [30]. • Technology-enabled learning allows technology to promote teaching and studying in the classroom and learning in students’ lives to solve problems, such as differences 70 http://www.i-jim.org Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… among students, basic knowledge, processing speed, and learning style. The use of technology-enabled learning can develop a cognitive process composed of the fol- lowing significant factors: perceiving stimulus, discriminating stimulus and creating a concept, associating a concept and creating rules with inductive approach, applying rules with deductive approach, and make generalizations. Moreover, technology pro- motes the development of problem-solving capability, as it helps search, analyze, and synthesize information to find solution. It also creates knowledge for students. In this research, the researchers investigated other studies and gathered information about the advantages, conditions, and limitations of technology-enabled learning. An appropriate technology, that is, a mobile cloud-based system, was selected to develop learning [31]. For the present study, information was gathered from reliable resources to examine the importance, meaning, strategies, and techniques required to effectively promote this skill among students. The collected information is presented as follows: 2.1 Computational thinking CT is an approach used to solve problems or respond to the needs of working life. The components of CT [28] are (1) decomposition, which is the process of problem analysis or the occurrence of elements; (2) pattern recognition, which is the process of recognizing patterns using information that already exists; (3) abstraction, which is the ability to generalize a pattern based on specification; and (4) algorithm, which is the process of using acquired knowledge to perform sequences of actions, followed by the development of a computer processing system. The fundamental concepts of technol- ogy involve computer data processing. Therefore, CT is drawn from an understanding of how computers can help humans create innovative products and solve problems, from the easiest to the hardest levels, and in ways that humans are unable to do so [32]. Researchers have conducted studies on how to enhance CT skills. Kalelioğlu [33] discussed how to encourage K-12 students to develop their CT using Code.org, a pro- gramming aid tool and a drawing tool for cartoon graphic images. Kalogiannakis and Papadakis [11] used ScratchJr to support pre-service kindergarten teachers in teaching CT and science education. The results showed that the lecturers understood the concept and possessed a positive attitude toward teaching programming and science education. In 2020, Papadakis [34] developed strategies for teaching in col- laboration with the App Inventor programming environment to enhance programming skills in a CS course. Papadakis [35] then reviewed empirical studies that applied four coding apps (ScratchJr, Kodable, Lightbot, and Daisy the Dinosaur) and developed an approach for the design and development of innovative learning tools to support young children’s learning of CT and coding skills. So, Jung, and Liu [5] reported the current research trend in the field of CT education and teachers’ perceptions, pedagogical design issues, and gender issues associated with CT education in the Asia-Pacific region. 2.2 Mobile cloud-based learning How applications are accessed and developed has greatly improved with mobile cloud computing technologies [36]. With an innovative infrastructure, current cloud iJIM ‒ Vol. 15, No. 21, 2021 71 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… applications can be used as an online service. In this study, the researchers demonstrated a solution for a virtual, yet interactive, learning environment by integrating mobile cloud computing technologies into teaching. The environment not only aimed to design and monitor educational content but also to build platforms for idea exploration and discovery. This learning system provides opportunities for educational exchange and the integration of various teaching and learning techniques. Cloud applications offer educators access to both free/low-cost services and expen- sive productivity tools, which can be beneficial for teaching and learning. Recently, many educational institutions have considered replacing or supplementing their tra- ditional instruction with cloud-based learning [31]. They chose mobile cloud-based learning because it helps reduce the cost of expensive information technology while offering services that are easy to access anytime and anywhere through various tech- nological devices. On the cloud, users can customize their learning features to meet their needs and preferences. They can increase or decrease connections, interact, and exchange knowledge in a cloud learning environment, thus promoting learning effec- tiveness [16]. Mobile learning is defined as the use of applications installed in a mobile device (tablet, smartphone, or laptop) for learning purposes in educational institutions, such as schools or universities. Mobile cloud-based learning has intriguingly facilitated the learning process, representing the mobile learning model and providing users with use- ful learning materials [37]. Compared with other forms of learning, learning through mobile devices is more private and has more potential for connected collaborative activities, despite being in the same learning environment [17], [19]. 2.3 Reverse engineering RE is the process of extracting knowledge or information from anything man-made, designing out of the existing system, redesigning the product, analyzing it for opti- mization, and maintaining the system for product quality improvement. RE plays an important role in prototypes creation [38]. Its process is widely used in many industries, such as electronics, software, and automotive spare parts. The advantages of using the RE process are time/cost savings and the ability to detect the weaknesses of the system [39]–[41]. Some studies have shown that during product design and creation, the possi- bility of RE application increases in parallel with hardware and software development. 2.4 Peer learning Based on social constructivism and cognitive theories [42], peer learning, or P2P learn- ing, is a collaborative learning strategy that highlights a student-activating approach. Therefore, it focuses mainly on student learning, while teachers act as facilitators [30]. With this learning strategy, students learn and progress by giving each other feedback on their tasks or assignments. Specifically, the peer learning model refers to when two or more students prepare, discuss, solve problems, and reflect on their structured tasks by themselves before asking for help from an instructor [43]. Peer learning [20] 72 http://www.i-jim.org Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… is a learning strategy suitable for both inside and outside classrooms. It includes peer tutoring, group discussions (small and large groups), and feedback-based learning. Stu- dents can effectively grasp the learning techniques provided by their peers and com- prehend the learning content better with these methods. In this study, the instructor implemented the peer learning technique to determine its effectiveness, as it has several advantages in learning, such as easing discussion among peers and improving students’ learning perception. 3 Conceptual framework In the conceptual framework of this study, three components were included in the design of the learning mode: cloud-based learning system, P2P learning technique, and the RE approach (Figure 2). Mobile Cloud- Based Learning System P2P Learning RE Approach CT - Decomposition - Pattern Recognition - Abstraction - Algorithm Mobile Cloud-Based Learning System Using P2P RE Approach Fig. 2. Conceptual framework of a mobile cloud-based learning system 4 System analysis and design 4.1 Stakeholders and process diagram The design of a cloud learning system consists of three elements: 1) Administrator – a person who manages the system operation, which includes system installation; configuring user access control and permission, submodule, and data- base management; adding/deleting data users; and providing learning aid tools on a cloud platform. 2) Teacher – a person who plans and delivers instructions in the learning system. The teacher uses modules and tools to support teaching, present education-related announcements to students, prepare educational content, explain learning and teach- ing processes, plan weekly activities, evaluate students’ progress, give feedback to students, and check students’ learning behaviors. 3) Student – a person who takes an active role in learning either independently or in group. A learning system for students comprises a personal information display, personalized learning space for individuals and collaborative teams, and indicators for their own learning progress and group progress. iJIM ‒ Vol. 15, No. 21, 2021 73 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… 4.2 System architecture The proposed learning system was developed using cloud-based services. Compute Engine, a virtual machine running on a Google data center, was used in combination with other rental services, such as storage and database services. The researchers also used an app engine called Platform as a Service, in which the researchers installed LMS Moodle 3.5 in the infrastructure system. A plug-in function was added to the system to verify user identity and connect to cloud services. A user tracking system was installed using Google Analytics to monitor students’ learning behaviors. Figure 3 illustrates the architecture of the cloud-based learning system. When a user (student, teacher, or administrator) logs into the system through the cloud domain name system, the signal is sent to the Google Cloud Platform using the HTTP load balancer. The HTTP load balancer works as a device to signal and control access to the app engine in which the LMS Moodle is installed. The app engine works together with the compute engine in the input, displays, and sending and receiving information from the database in cloud storage. The app engine also works with cloud tools in the cloud system. Tracking and learning analytical tools were installed in the system for further use in this study. The researchers designed an architectural structure of a cloud-based learning system that allows users to log into web browsers using a personal cloud server. The struc- ture consists of two nodes: a web application node and a database node. The system works by verifying user identity, including teachers, students, and administrators, while accessing the personal cloud server. Users can also log into a public cloud to request services for learning or activity management. Cloud Platform Fig. 3. Architectural design of the cloud system 74 http://www.i-jim.org Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… Fig. 4. Structure of the cloud-based learning system As shown in Figure 4, the structure of the system’s working process consists of five layers. Each layer has elements and functions as follows: Interface layer: This layer has features that allow users to interact directly with the system, while the webpage acts as a platform for learning management. Each window contains membership, news, content, weekly activity, contact, tutorial, support service, evaluation, and back office. Learning system layer: The second layer of interface combines with learning mod- ules, including the membership module, news module, management module, tutorial or learning support service module, and contact and work module. Delivery layer: This layer is under the learning system layer and is responsible for allowing data and tools to move between a personal cloud and a public cloud to the learning system. The delivery layer also controls communication from its point of ori- gin to the final destination and provides the proper levels of consistency and accuracy for sending and receiving data. Content and tools management layer: This layer is located in the cloud in both per- sonal and public systems and is used for content management or collaboration, learning support, learning tracking, and learning object database. Database management layer: This is the lowest layer of the system used to man- age the storage of data, including learning content, membership, learning tracking, and learning objects stored in a database. 5 Research methodology 5.1 Participants In this study, 40 undergraduate students were chosen as the experimental subjects using the purposive sampling method. The participants were third-year students in iJIM ‒ Vol. 15, No. 21, 2021 75 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… the Department of Educational Communication and Technology at King Mongkut’s University of Technology Thonburi, Thailand. 5.2 Instruments of the study Questionnaires were used to gather the necessary data. A CT skill test, which con- tains seven questions, was created to test the students individually. A CT skill test with 12 questions was also developed to test the students during the course. Synthesized and developed from Computing at School [44], the tests were initiated based on the four aspects of CT, and a rubric assessment was employed. The validity of the tests was assessed by experts in terms of content, and evaluation. The tests were distributed to students whose characteristics were similar to those of the participants. The scale had good reliability, as the rater agreement index was 0.80. 5.3 Research design A repeated-measures design [45] was used in this study. This research design involves multiple measures of the same variable, and the only change in each measure is the difficulty level of the learning content. The experimental results were measured, observed, and compared in each round to examine the students’ learning progress. The reliability of the experiments was also analyzed. If the results of the latter experiments were better than those of the previous ones, the students or the experimental subjects could be inferred to have improved their learning progress. The research design is shown in Table 1. Table 1. Repeated-measures design E OPre X O1 X O2 X O3 E: Experimental group X: Treatment OPre: Pretest observation O1, O2, O3: Posttest observation (Rounds 1, 2, and 3) 5.4 Study procedure Figure 5 shows the research process presented in detail in Table 2. Fig. 5. Cloud-based learning activities 76 http://www.i-jim.org Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… Table 2. Details of the research methodology No. Topic Details 1 Preparation prior to the use of the mobile cloud-based learning system The teacher explained the overall functions of the system, the mobile cloud-based learning tools, and the agenda of the learning activities and assessment. The students were instructed to answer seven questions on the subjective pre-test individually. 2 Learning process 2.1 The teacher explained goal setting and student grouping. The teacher also assigned roles to students as an individual and as a group. 2.2 Planning: The students planned together and worked collaboratively toward shared goals. 2.3 Pattern selection: Each group selected a pattern based on its chosen principles, theories, and task structures. They set the expected outcomes for their group project. 2.4 Collaborative analysis: The students in each group worked together on the chosen topic. They decomposed or separated the elements of the selected pattern. They also analyzed the sub-elements for a better understanding of how internal and external structures work. For example, they examined the object-oriented structures of the elements and sub- elements. After the decomposition step, the students reconsidered or re-analyzed each element carefully and classified the elements into groups. This part relates to pattern recognition. Then, they searched for abstraction or the ability to generalize a pattern based on specification. In this step, the students used cloud-based learning tools to help facilitate their learning. 2.5 Collaborative design: The students used their acquired knowledge to design their group projects by reconsidering the original pattern. They reported their progress and made sure that their project design was accurate and complete. The students were instructed to answer 12 questions in the test during the course. 2.6 Feedback and summary: The teacher observed the students while they were working on their tasks. In this part, the teacher gave feedback and encouragement to the students regarding their projects. The teacher also answered and clarified related questions to help the students develop their skills. At the end, the students in each group had an opportunity to reflect on their learning. They presented their discoveries and limitations while working on their projects. They also shared principle-based or theory- based suggestions for project improvement. 3 Experiment repetition for students’ learning progress assessment After completing all six steps of the mobile cloud-based learning process in step 2, the students participated in the same experiment twice. The only difference is that the difficulty level of the content was higher or harder in each round. The teacher observed and collected information for further analysis. 4 Summative assessment The assessment after the use of the mobile cloud-based learning system involved the students answering the seven questions in the subjective post-test. 5. Conclusions An assessment was conducted after three rounds of experiments. In this step, the teacher assessed the students to determine whether the students were successful according to the learning objectives and indicators written in the learning management plans. The results revealed the students’ scores before and after using the cloud-based learning system in each experimental round. iJIM ‒ Vol. 15, No. 21, 2021 77 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… 6 Results 6.1 Examples of learning outcomes The researchers helped facilitate the students’ learning by providing them with learn- ing support tools for use in the cloud-based learning system. For example, the students could use mind mapping to extract the elements of a pattern or to visualize the work process using a flowchart, as shown in Figure 6. Fig. 6. Examples of how a mobile cloud-based learning system can be used 6.2 Mobile cloud-based learning system testing Test scores before and after using the cloud-based learning system. The students’ scores before and after using the mobile cloud-based learning system were compared in the pre-test and post-test using the dependent T-test method. Table 3 shows the results. Table 3. Comparison of CT scores before and after mobile cloud-based learning engagement using the dependent t-test n M SD t p Pre-test 40 7.35 5.31 −52.374 .000* Post-test 40 59.58 6.20 *p < .05 78 http://www.i-jim.org Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… As shown in Table 3, the students’ CT score was higher after using the cloud system at a 0.05 significance level (t = −52.374, Sig. = .000*). Comparison of the findings from the first, second, and third experiments in group work activities. The students’ overall mean scores in group work activities continuously increased. The mean and standard deviation of the group work activities obtained from the first, second, and third experiments were 15.5 (3.32), 33.9 (2.93), and 35.9 (2.70), respectively. Table 4. Result of Mauchly’s test of sphericity Within- Subjects Effect Mauchly’s Approx. Chi-Square df p value Epsilon Greenhouse– Geisser Huynh– Feldt Lower- bound Round .345 40.306 2 .000 .605 .614 .500 Table 4 shows the results of Mauchly’s test of sphericity, which is the test of the assumption of repeated-measures ANOVA called sphericity. The chi-square value and its associated P value indicate that the significance level was below 0.05 (< 0.0001), which was not consistent with the assumption. Thus Greenhouse–Geisser correction, which has an already adjusted statistical value, was used. Table 5. Test of the within-subject effect on the experiments Source Type||| Sum of Squares df Mean Square F value p value Round 9118.601 1.209 7540.088 552.197 .000 Error (round) 644.019 47.165 13.655 Table 5 shows the results of the analysis based on the average scores of the individ- ual couples in the sample analyzed for three rounds using the Bonferroni correction. The results were statistically different at .05 (F = 552.197, p = .000). This shows that the sample had a CT score that was different in least one round of the assessment. Table 6. Two-group comparison of CT activity engagement from three experimental rounds using one-way ANOVA with repeated-measures analysis Round(I) Round(J) Mean Difference(I-J)(*) Std. Error Sig. 95% Confidence Interval for Difference Lower Bound Upper Bound 1 2 −16.855 .792 .000 −18.836 −14.874 3 −19.780 .726 .000 −21.597 −17.963 2 1 16.855 .792 .000 14.874 18.836 3 −2.925 .290 .000 −3.650 −2.200 3 1 19.780 .726 .000 17.963 21.597 2 2.925 .290 .000 2.200 3.650 *The mean difference is significant at the 0.05 level. iJIM ‒ Vol. 15, No. 21, 2021 79 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… Table 6 presents the post-test comparison results showing where the differences occurred if an overall significant difference was achieved. This table shows which spe- cific means were different. A significant difference was observed in all scores from round 2 onwards compared with the baseline visit (round 1). The results suggested that the scores taken from the three experiments were different in at least one pair of group. Additionally, the mean scores derived from the third experiment on group work activities were higher than the mean scores in the first experiment. The mean scores in the third experiment on students’ CT were higher than those in the first experiment at a 0.05 significance level. Figure 7 shows the results. 15.5 33.9 35.9 0 10 20 30 40 Round 1 Round 2 Round 3 Estimated Marginal Means of Score Fig. 7. Average score of individual couples of the sample in three rounds 7 Discussion and conclusions This study presented the effects of a mobile cloud-based learning system using a P2P RE approach to enhance CT. The results showed that the system was able to enhance CT and encourage the students to have a positive attitude toward learning, as the system allowed them to work together and exchange knowledge. Moreover, the interesting tasks and the RE strategy enabled them to understand the structure of their work and learn new things right away. The average pre-test CT score of the sample, which was individually calculated, was significantly statistically higher than the post-test score at .05 (t = –52.374, Sig. = .000*). The average score of the individual groups after three rounds, which was calculated using the Bonferroni correction, revealed an increase in the value of the CT score in all rounds [15.5 ± 3.32, 33.9 ± 2.93, and 35.9 ± 2.70), respectively], with P < 0.05. The results were consistent with Johnson’s [37] study, which found that integrating cloud technology into learning helped increase the scores at a significance level of .05. Moreover, the study showed that cloud technology integration helped develop students’ potential for learning and solving problems collaboratively and promoted students’ higher-order thinking skills. Shana and Abulibdeh [31] reported that after the imple- mentation of cloud technology in higher education institutes, students had a positive attitude toward cloud technology because of its simple and convenient functions and because it could help them achieve academic success. In conclusion, the cloud sys- tem promoted the exchange of knowledge among students, which led them to CT. 80 http://www.i-jim.org Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… Moreover, the system responded to Google Analytics, which was employed to check the system users and support the students working in teams. Google Analytics also collected data showing the records of the students’ system access. The collected data can be analyzed and managed to improve teaching and learning [46]. Positive learning outcomes result from an effective learning system. The pro- posed learning process was developed based on the concepts and principles of the RE approach. According to Orta et al. [47], RE-based learning can help improve students’ abilities through factor analysis, pattern recognition, distinctive point extraction, and project design. In terms of the learning methodology for enhancing CT, the students were required to work in pairs, share ideas, and solve problems with their peers to minimize errors and save time on their projects, consistent with the study of Williams [15]. Williams applied the P2P learning method to a software company and found that it could help increase productivity and save time in software development and that working with peers was much better than working alone. Although working together can make team members feel more connected, it is important to realize that each member must take on his or her responsibilities. All members are expected to complete their specific tasks. More importantly, they should listen to their peers’ points of view, as this is another factor that leads to successful learning. Therefore, applying P2P learning to learning or working can create knowledge, improve skills, and promote the interaction of students. This technique is based on social constructivism theory [30], which promotes social and communication skills. It creates a learning environment in which collaboration and exchange of ideas are encouraged. Students must learn how to articulate their ideas clearly and collaborate to effectively manage tasks by sharing ideas in the group. There- fore, students must contribute their efforts and exchange ideas in an appropriate social manner. This can lead students to succeed in the real world, where they are required to work in collaboration with others and gain diverse experiences. This research included four empirical studies that contributed to a better understand- ing of the current research trend in the field of CT education and learners’ percep- tions, pedagogical design issues, and learning systems associated with CT education. However, several issues remain unclear and require further investigation to improve CT education under the conditions and limitations of learners, teachers, environments, and changes in education policy in different countries [8]. 8 Research findings Following the objective of this study, which is to examine peer learning through a mobile cloud-based learning system using the RE approach to enhance CT, the research- ers determined the basic conditions of the students and student groups based on their desire to work together effectively. The researchers also analyzed the data recorded in Google Analytics and found the following peer learning concepts. The researchers observed the students during their mobile cloud-based learning engagement through the learning analytics feature in the cloud system and reviewed their scores each time they used the system. After considering the students’ CT scores at iJIM ‒ Vol. 15, No. 21, 2021 81 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… the end of the experimental round, the researchers divided the students into two types: high-performing and low-performing students. These types are defined as follows: • High-performing students earn grades higher than average in all subjects calculated together compared with the average grade of the class. • Low-performing students earn a lower average grade than the average grade of the entire class. In the experiment, the students’ performance was grouped into three categories: 1) Group of high-performing students. The students in this group helped one another during collaborative learning and took great responsibility for their roles, leading to positive learning outcomes and very high CT scores. 2) Mixed group of high-performing and low-performing students. Two important points were found in this category. 2.1) The high-performing students taught the low-performing students about learn- ing techniques, how to work cooperatively and independently, and how to search for information efficiently, etc. They also supervised the group as a whole and managed tasks and responsibilities within the group. They helped low-performing students when facing problems or difficulties. 2.2) The low-performing students taught the high-performing students. In this case, the low-performing students learned alongside the high-performing students. The researchers determined this by tracking the students’ learning behaviors in the cloud system. The interviews with the low-performing students showed that the content of the cloud system influenced their learning motivation. 3) Group of low-performing students. The researchers observed two important points in this category. 3.1) Some low-performing students lacked the motivation to study, resulting in poor academic performance. 3.2) According to the observation from the tracking system, some low-performing students attempted to learn together, leading to positive learning outcomes and higher CT scores. The researchers interviewed these low-performing students and discovered that the content of the cloud learning system helped them focus on what they were learning. The research findings showed a correlation between the high-performing/low- performing students and the use of a mobile cloud-based learning system. According to research question ii, the researchers discovered that a cloud system and its content were essential components that helped develop students’ learning skills. This finding is con- sistent with Becker’s [23] study, which stated that game design and developing content could promote a growing mindset in students and make their learning more meaningful. Content stored in cloud systems can help students accomplish their learning goals and progress more quickly. Additionally, the arrangement of cooperative learning groups 82 http://www.i-jim.org Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… greatly affected students’ learning outcomes and was part of the learning achievement. Nevertheless, teachers should work hard to increase students’ learning motivation by providing them excellent learning resources, such as intriguing content, supporting them when needed, complimenting them, or rewarding them with extra points. In future work, the researchers are interested in developing CT in higher education and learning the style of students. They also intend to conduct qualitative research to obtain in-depth information that can help develop students’ skills. 9 Research limitations and suggestions The proposed learning system was a collaborative system. Class activities were arranged to encourage students to use CT in other regular courses in the curriculum. Teachers who want to apply the results of this study to their lectures should realize the appropriateness and compatibility between the system and courses, such as program- ming, software design and development, digital games, websites, technological media, and material/item design. In addition, the content of the system was a crucial factor related to game design. The students were required to analyze a game model that they were familiar with. As a result, all students were drawn to participate in the activities, creating motivation and a positive attitude. This conformed to Settle’s study [48], which showed the development of a game design course that created CT in higher education. Moreover, Papadakis [34] investigated the result of the application of an App Inventor programming environment to process game development. The results of the programming study were good, and they created motivation for studying computer science [49]. This study used the repeated-measures design approach. According to the first exper- iment, the results indicated that the students’ CT scores were higher at a 0.05 level of significance. Therefore, a mobile cloud-based learning system can be applied for future use with one single test. However, for better score results, it is necessary to do more than one experiment. Moreover, the difficulty levels should be more complicated to increase the challenge in the study. Nevertheless, before the system is applied, a period of teaching must be undertaken. The system should be applied in three rounds to be the most effective; therefore, it takes a long time to apply it. If there are many students in class, the lecturer should have an assistant to review the students’ work and help them when needed. As learning takes place using electronic devices, the screen of the devices should be 7 inches or more. The internet must have a strong connection to access and operate the system easily and effectively. 10 Acknowledgments The researchers would like to thank the 90th Anniversary of the Chulalongkorn University Fund and the Ratchadaphiseksomphot Endowment Fund for funding this research. iJIM ‒ Vol. 15, No. 21, 2021 83 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… 11 References [1] Williamson, B. (2016). Political computational thinking: policy networks, digital gover- nance and ‘learning to code’. Critical Policy Studies. Vol. 10(1) , 39–58. https://doi.org/10. 1080/19460171.2015.1052003 [2] Bocconi, S., Chioccariello, A., Dettori, G., Ferrari, A., & Engelhardt, K. (2016). Developing Computational Thinking in Compulsory Education. Implications for policy and practice. JRC Working Papers JRC104188 from Joint Research Centre (Seville site). [3] Heintz, F., Mannila, L. & Färnqvist, T. (2016). A review of models for introducing com- putational thinking. computer science and computing in K-12 education. In Frontiers in Education. Conference (FIE), IEEE, 1–9. https://doi.org/10.1109/FIE.2016.7757410 [4] ISTE & CSTA. (2014). Computational thinking for all. Retrieved Dec 1, 2017 from, https:// www.iste.org/explore/Solutions/Computa tional-thinking-for-all [5] So, H., Jong, M. S., & Liu, C. (2020). Computational Thinking Education in the Asian Pacific Region. The Asia-Pacific Education Researcher. 29, 1–8. https://doi.org/10.1007/ s40299-019-00494-w [6] Bedar, R. A. H., & Al-Shboul, M. (2020). The Effect of Using STEAM Approach on Devel- oping Computational Thinking Skills among High School Students in Jordan. International Journal of Interactive Mobile Technologies (iJIM), 14(14). https://doi.org/10.3991/ijim. v14i14.14719 [7] Wing, J. M. (2006). Computational Thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215 [8] Denning, P. J. (2009). The Profession of It Beyond Computational thinking. Communica- tions of the acm, 52(6), 28–30. https://doi.org/10.1145/1516046.1516054 [9] Lu, J., & Fletcher, G. H. L. (2009). Thinking About Computational Thinking. ACM SIGCSE Bulletin - SIGCSE '09, 41(1), 260–264. https://doi.org/10.1145/1539024.1508959 [10] Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12?. Computers in Human Behavior, 41, 51–61. https://doi.org/10.1016/j.chb.2014.09.012 [11] Papadakis, S., & Kalogiannakis, M. (2020). Learning computational thinking development in young children with Bee-Bot educational robotics. In Handbook of research on tools for teaching computational thinking in P-12 education (pp. 289–309). IGI Global. https://doi. org/10.4018/978-1-7998-4576-8.ch011 [12] Papadakis, St. (2020). Robots and Robotics Kits for Early Childhood and First School Age. International Journal of Interactive Mobile Technologies (iJIM), 14(18), 34–56. https://doi. org/10.3991/ijim.v14i18.16631 [13] Wu, B., Hu, Y., Ruis, A., & Wang, M. (2019). Analysing computational thinking in collab- orative programming: A quantitative ethnography approach. Journal of computer assisted learning. 35(3), 421. https://doi.org/10.1111/jcal.12348 [14] Zaharin, N. L., Sharif, S., & Mariappan, M. (2018). Computational Thinking: A Strategy for Developing Problem Solving Skills and Higher Order Thinking Skills (HOTS). The Inter- national Journal of Academic Research in Business and Social Sciences. 8(10), 1265–1278. https://doi.org/10.6007/IJARBSS/v8-i10/5297 [15] Williams, L., Wiebe, E., & Yang, K. et al. (2002). In Support of Pair Programming in the Introductory Computer Science Course Computer Science Education. 12(3). https://doi. org/10.1076/csed.12.3.197.8618 [16] González-Martínez, J. A., Bote-Lorenzo, M. L., Gómez-Sánchez, E., & Cano-Parra, R. (2015). Cloud computing and education: A state-of-the-art survey. Computers Education, 80, 132–151. https://doi.org/10.1016/j.compedu.2014.08.017 84 http://www.i-jim.org https://doi.org/10.1080/19460171.2015.1052003 https://doi.org/10.1080/19460171.2015.1052003 https://doi.org/10.1109/FIE.2016.7757410 https://www.iste.org/explore/Solutions/Computa tional-thinking-for-all https://www.iste.org/explore/Solutions/Computa tional-thinking-for-all https://doi.org/10.1007/s40299-019-00494-w https://doi.org/10.1007/s40299-019-00494-w https://doi.org/10.3991/ijim.v14i14.14719 https://doi.org/10.3991/ijim.v14i14.14719 https://doi.org/10.1145/1118178.1118215 https://doi.org/10.1145/1516046.1516054 https://doi.org/10.1145/1539024.1508959 https://doi.org/10.1016/j.chb.2014.09.012 https://doi.org/10.4018/978-1-7998-4576-8.ch011 https://doi.org/10.4018/978-1-7998-4576-8.ch011 https://doi.org/10.3991/ijim.v14i18.16631 https://doi.org/10.3991/ijim.v14i18.16631 https://doi.org/10.1111/jcal.12348 https://doi.org/10.6007/IJARBSS/v8-i10/5297 https://doi.org/10.1076/csed.12.3.197.8618 https://doi.org/10.1076/csed.12.3.197.8618 https://doi.org/10.1016/j.compedu.2014.08.017 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… [17] Naik, N. V., & Madhavi, K. (2015). Cloud Computing Architecture for Collaborative E-learning System. International Conference on Applied and Theoretical Computing and Communication Technology (iCATccT), 58–62. https://doi.org/10.1109/ICATCCT.2015 .7456855 [18] Riahi, G. (2015). E-learning systems based on cloud computing: a review. J Procedia Com- puter Science, 62, 352–359. https://doi.org/10.1016/j.procs.2015.08.415 [19] Wang, J., Lu, Y., Jou, M., & Zhang, J. (2016). Research on the effects of cloud-based ped- agogy for creative talents: A case study on Chinese High School, 63, 229–239. https://doi. org/10.1016/j.chb.2016.05.038 [20] Blilat, A., & Ibriz, A. (2020). Design and Implementation of P2P Based Mobile App for Collaborative Learning in Higher Education. International Journal of Interactive Mobile Technologies (iJIM), 14(7). https://doi.org/10.3991/ijim.v14i07.13167 [21] Kultawanich, K., Koraneekij, P., & Na-Songkhla, J. (2015). A Proposed Model of Con- nectivism Learning Using Cloud-based Virtual Classroom to Enhance Information Literacy and Information Literacy Self-efficacy for Undergraduate Students.” Procedia—Social and Behavioral Sciences, 191, 87–92. https://doi.org/10.1016/j.sbspro.2015.04.394 [22] Zhao, K., Yang, Q., & Ma, X. (2017). Exploration of an Open Online Learning Platform Based on Google Cloud Computing. International Journal of Emerging Technologies in Learning (iJET), 12(7). https://doi.org/10.3991/ijet.v12i07.7249 [23] Becker, K. (2007). Digital game based learning once removed: Teaching teachers. British Journal of Educational Technology, 38(3), 478–488. https://doi.org/10.1111/j. 1467-8535.2007.00711.x [24] Costa-Soria, C., Llavador, M., & del Carmen Penades, M. (2009). An approach for teaching software engineering through reverse engineering. Paper presented at the EAEEIE Annual Conference. https://doi.org/10.1109/EAEEIE.2009.5335495 [25] Telea, A., Byelas, H., & Voinea, L. (2009). A framework for reverse engineering large C++ code bases. Electronic Notes in Theoretical Computer Science, 233, 143–159. https://doi.org/ 10.1016/j.entcs.2009.02.066 [26] Auttawutikul, S. (2008). Development of a knowledge sharing process using computer- supported collaborative learning based on peer assisted learning approach to enhance knowledge creation behaviors of graduate students. PhD thesis, Department of Educational Technology and Communications, Faculty of Education, Chulalongkorn University. [27] Hongsrichinda, C. (2014). Development of a personal knowledge management system using learning contract and peer assisted learning to enhance academic task performance. PhD thesis, Department of Educational Technology and Communications, Faculty of Education, Chulalongkorn University. [28] Computational thinking for educators. Computational Thinking. Available: https://compu- tationalthinkingcourse.withgoogle.com/course?use_last_location=true [Accessed 17/09/ 2017]. [29] Rungrangtanaopl, N. & Khlaisang, J. (2021). Development of a Teaching Model in Virtual Learning Environment to Enhance Computational Competencies in the 21st Century. Inter- national Journal of Interactive Mobile Technologies(iJIM), 15(13), 93–107. https://doi. org/10.3991/ijim.v15i13.21791 [30] Elshami, W., Abuzaid, M., & Abdalla, M. E. (2020). Radiography students’ perceptions of Peer assisted learning. Radiography, 26 ,109–113. https://doi.org/10.1016/j.radi.2019.12.002 [31] Shana, Z., & Abulibdeh, E. S. (2017). Cloud Computing Issues for Higher Education: The- ory of Acceptance Model. International Journal of Emerging Technologies in Learning (iJET), 12(11). https://doi.org/10.3991/ijet.v12i11.7473 [32] Kalelioglu, F., Gülbahar, Y., & Kukul, V. (2016). A Framework for Computational Thinking Based on a Systematic Research Review. Baltic J. Modern Computing, 4(3), 583–596. iJIM ‒ Vol. 15, No. 21, 2021 85 https://doi.org/10.1109/ICATCCT.2015.7456855 https://doi.org/10.1109/ICATCCT.2015.7456855 https://doi.org/10.1016/j.procs.2015.08.415 https://doi.org/10.1016/j.chb.2016.05.038 https://doi.org/10.1016/j.chb.2016.05.038 https://doi.org/10.3991/ijim.v14i07.13167 https://doi.org/10.1016/j.sbspro.2015.04.394 https://doi.org/10.3991/ijet.v12i07.7249 https://doi.org/10.1111/j.1467-8535.2007.00711.x https://doi.org/10.1111/j.1467-8535.2007.00711.x https://doi.org/10.1109/EAEEIE.2009.5335495 https://doi.org/10.1016/j.entcs.2009.02.066 https://doi.org/10.1016/j.entcs.2009.02.066 https://computationalthinkingcourse.withgoogle.com/course?use_last_location=true https://computationalthinkingcourse.withgoogle.com/course?use_last_location=true https://doi.org/10.3991/ijim.v15i13.21791 https://doi.org/10.3991/ijim.v15i13.21791 https://doi.org/10.1016/j.radi.2019.12.002 https://doi.org/10.3991/ijet.v12i11.7473 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… [33] Kalelioğlu, F. (2015). A new way of teaching programming skills to K-12 students: Code. org. Computers in Human Behavior, 52, 200–210. https://doi.org/10.1016/j.chb.2015.05.047 [34] Papadakis, S., & Kalogiannakis, M. (2019). Evaluating the effectiveness of a game-based learning approach in modifying students’ behavioural outcomes and competence, in an introductory programming course. A case study in Greece. International Journal of Teaching and Case Studies, 10(3), 235–250. https://doi.org/10.1504/IJTCS.2019.10024369; https:// doi.org/10.1504/IJTCS.2019.102760 [35] Papadakis, S. (2021). The impact of coding apps on young children Computational Think- ing and coding skills. A literature review. Frontiers in Education, 6, 657895. https://doi. org/10.3389/feduc.2021.657895 [36] Todoranova, L., Nacheva, R., Sulov V., & Penchev, B. (2020). A Model for Mobile Learning Integration in Higher Education Based on Students’ Expectations. International Journal of Interactive Mobile Technologies (iJIM), 14(11). https://doi.org/10.3991/ijim.v14i11.13711 [37] Johnson, L. D. (2017). Exploring Cloud Computing Tools to Enhance Team-Based Prob- lem Solving for Challenging Behavior. Topics in Early Childhood Special Education, 37(3), 176–188. https://doi.org/10.1177/0271121417715318 [38] Raja, V., & Fernandes, K. J. (2007). Reverse Engineering An Industrial Perspective. Springer- Verlag London. https://doi.org/10.1007/978-1-84628-856-2 [39] Ali, M. R. (2005). Why teach reverse engineering?. In ACM SIGSOFT Software Engineer- ing Notes, 30(4), 1–4. https://doi.org/10.1145/1082983.1083004 [40] Dempere, L. A. (2009). Reverse engineering as an educational tool for sustainability. Paper presented at the Sustainable Systems and Technology. ISSST'09. IEEE International Symposium on. https://doi.org/10.1109/ISSST.2009.5156748 [41] Eilam, E. (2005). Reversing: Secrets of Reverse Engineering. Wiley Publishing, Inc. [42] Zwedberg, S., Alnervik, M., & M. Barimani, M. (2021). Student midwives’ perception of peer learning during their clinical practice in an obstetric unit: A qualitative study. Nurse Education Today. https://doi.org/10.1016/j.nedt.2021.104785 [43] Polkowski, Z., Jadeja, R., & Dutta, N. (2020). Peer Learning in Technical Education and it’s Worthiness: some facts based on implementation. Procedia Computer Science, 172, 247–252. https://doi.org/10.1016/j.procs.2020.05.039 [44] Computing at School. A subdivision of the British Computer Society (BCS). Computational Thinking: A Guide for Teachers. Available: http://community.computingatschool.org.uk/ files/6695/original.pdf [Accessed 17/09/2017]. [45] Tadsuwan, J. (2002). Statistical Analysis. Retrieved from http://cmuir.cmu.ac.th/handle/ 6653943832/33482 [46] Google Analytic Academy. Google Analytic for Beginners. Available: https://analytics.goo- gle.com/analytics/academy/course/6 [Accessed 20/01/2020]. [47] Orta, P., Mendoza, R. R., Elizalde, H., & Guerra, D. (2006). Use of Reverse Engineering As a Teaching Tools In Mechanical Engineering Education. American Society for Engineering Education. [48] Settle, A. (2011). Computational thinking in a game design course. SIGITE' 11 ACM Special Interest Group for Information Technology Education Conference. https://doi.org/10.1145/ 2047594.2047612 [49] Papadakis, S., Vaiopoulou, J., Sifaki, E., Stamovlasis, D., & Kalogiannakis, M. (2021). Attitudes towards the Use of Educational Robotics: Exploring Pre-Service and In-Service Early Childhood Teacher Profiles. Education Sciences, 11(5), 204. https://doi.org/10.3390/ educsci11050204 86 http://www.i-jim.org https://doi.org/10.1016/j.chb.2015.05.047 https://doi.org/10.1504/IJTCS.2019.10024369 https://doi.org/10.1504/IJTCS.2019.102760 https://doi.org/10.1504/IJTCS.2019.102760 https://doi.org/10.3389/feduc.2021.657895 https://doi.org/10.3389/feduc.2021.657895 https://doi.org/10.3991/ijim.v14i11.13711 https://doi.org/10.1177/0271121417715318 https://doi.org/10.1007/978-1-84628-856-2 https://doi.org/10.1145/1082983.1083004 https://doi.org/10.1109/ISSST.2009.5156748 https://doi.org/10.1016/j.nedt.2021.104785 https://doi.org/10.1016/j.procs.2020.05.039 http://community.computingatschool.org.uk/files/6695/original.pdf http://community.computingatschool.org.uk/files/6695/original.pdf http://cmuir.cmu.ac.th/handle/6653943832/33482 http://cmuir.cmu.ac.th/handle/6653943832/33482 https://analytics.google.com/analytics/academy/course/6 https://analytics.google.com/analytics/academy/course/6 https://doi.org/10.1145/2047594.2047612 https://doi.org/10.1145/2047594.2047612 https://doi.org/10.3390/educsci11050204 https://doi.org/10.3390/educsci11050204 Paper—Effects of a Mobile Cloud-Based Learning System Using a P2P Reverse Engineering Approach… 12 Authors Sutiwat Supaluk (Author) is PhD. Student at Department of Educational Technol- ogy and Communications, Faculty of Education, Chulalongkorn University, Thailand, 10330 (e-mail: Sutiwat.S@student.chula.ac.th, thearming@gmail.com). Jintavee Khlaisang (Corresponding Author) is a Professor at the Department of Educational Technology and Communications, Faculty of Education and Educational Invention and Innovation Research Unit, Chulalongkorn University, Bangkok, Thailand (jintavee.m@chula.ac.th). Noawanit Songkram is a Professor at Department of Educational Technology and Communications, Faculty of Education, Chulalongkorn University. (e-mail: noawanit_s @hotmail.com) Article submitted 2021-04-05. Resubmitted 2021-08-06. Final acceptance 2021-08-08. Final version published as submitted by the authors. iJIM ‒ Vol. 15, No. 21, 2021 87 mailto:Sutiwat.S@student.chula.ac.th mailto:thearming@gmail.com mailto:jintavee.m@chula.ac.th mailto:noawanit_s@hotmail.com mailto:noawanit_s@hotmail.com