International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 16, No. 02, 2022 Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian Primary School https://doi.org/10.3991/ijim.v16i02.27677 Liew Chuang Tong1, Mohd Shafie Rosli2(*), Nor Shela Saleh3 1Ministry of Education, Malaysia 2School of Education, Universiti Teknologi Malaysia, Malaysia 3Department of Social Sciences, Universiti Tun Hussein Onn Malaysia, Johor, Malaysia shafierosli@utm.my Abstract—Learning in the 21st century emphasize on development of higher order thinking skills such as problem-solving skills with substantial highlight was given toward student-centred approach. Digital game and problem-solving activities are an ideal combination of student-centred approach which facilitate active participation for students to practice and develop their thinking skills. This study was conducted to investigate the effect and the process of using digital game and problem-based learning skills to enhance higher order thinking skills in primary school student in the context of Malaysia. A preliminary study was conducted to 50 samples to collect data about their current higher order thinking skills level. The result shows samples are good in identify problem and solu- tion, but weak in decision making and reasoning. To study the effect of digital game and problem-based learning combined in enhancing higher order thinking skills, a pre-test and post-test was conducted to 35 samples before and after three sessions of digital game problem-based learning section. After running a paired sample t test upon the pre-test and post-test, the result show that the combination of digital game and problem-based learning have significant effect in enhancing higher order thinking skills. To understand samples’ learning process, reflection journal was used after each learning section and an interview was conducted following the post-test. The data gathered indicate samples do enjoy learning with digital games and problem-based learning, and they agreed the approaches help them to learn better. However, they expected for more exciting games and group activities in the future. This study opens a gateway for developing student-centred and fun learning approach that beneficial toward student’s higher order thinking skills. Keywords—problem-based learning, digital game, higher order thinking skills 1 Introduction Malaysia had introduced her National Education Blueprint that had laid founda- tion for the nation educational development for 2013 until 2025. The blueprint had highlighted six aspirations as a manoeuvre to develop human resources for Malaysia to compete at global stage in term of innovation and economic competitiveness. One of iJIM ‒ Vol. 16, No. 02, 2022 101 https://doi.org/10.3991/ijim.v16i02.27677 mailto:shafierosli@utm.my Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… the upmost goals of this blueprint is to improve the mastery of Higher Order Thinking Skills (HOTS) among students nation-wide. In the context of Malaysian schooling system, HOTS is defined as student ability to apply knowledge, skill, and value in reasoning, reflecting, and solving problem, making decision, and innovating a new idea [1]. Multiple approaches were introduced to improve HOTS in education. Gaming seems like a promising medium to bridge the curriculum, content, and persuasive ele- ment of the technology into a unified medium of effective education. Throughout the years, studies had been done upon the effect of gaming on academic achievement [2], psychological health [3], educational outcome [4], mathematics [5] [6], science [7] [8], and language [9]. The integration of game into teaching and learning of various sub- jects had been discussed for many years and lately it becomes a new practice among teachers. This led to a new question, how effective is game-based learning in promoting HOTS? Therefore, the researchers are looking forward to study the effect of digital game-based learning in promoting HOTS among primary school students. In education, beside the learning approach, a well devised pedagogical approach is needed in safeguarding that the engagement will lead to productive knowledge con- struction. For years, problem-based learning was highlighted as an effective approach that suit the dynamic needs of 21st century education. The literatures show the effect of problem-based learning on creative thinking [10], mathematical literacy [11], metacog- nition [12], analytical thinking [13], mathematical problem solving [14], and science problem solving skills [15]. However, what is the effect of problem-based learning in promoting HOTS? Therefore, this study is going to study the effect of digital game- based learning and problem-based learning combination toward sample’s HOTS. 2 Literature review Bloom had come up with the concept of Bloom’s Taxonomy which divided learning outcome into six categories which range from lower thinking skills to higher thinking skills. The taxonomy was later revised by [16]. As digital age of education had come into reality, HOTS becoming more imperative. It is not only in Malaysia, but HOTS had become mainstream in the neighbouring Singapore with daily classroom and out of the classroom activities are centred on instilling HOTS into learning [17]. The similar development is also taking place in the neighbouring Indonesia. HOTS exists in nexus with critical thinking and problem-solving skills. However, the main concern till today is how to enhance HOTS? According to [18] HOTS skills are composed of critical thinking, creative think- ing and problem solving. HOTS process is a mental activity that solve complex non-structural problem. On the other hand, Bloom stated three domains for HOTS which are cognitive, affective, and psychomotor [19]. The learning and teaching pro- cess of HOTS will be more effective if it follows the steps from prerequisite, bridge to HOTS [19]. Educators need to act as facilitators to facilitate students to bridge that knowledge. These scaffolding steps help to lighten the cognitive burden of learners and provide an appropriate learning challenge to learners [20]. 102 http://www.i-jim.org Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… The benefits of applying HOTS in teaching and learning are being portrayed by various of studies such as enhancing learning abilities, motivate students and speed up learning effectively [19]. Sadly, the teaching trend is still skew toward the traditional rote memorizing method in the 21st century. Educators prefer to practice knowledge memorization more than thinking and applying [21]. [22] also review the same problem in the teaching of chemistry as chemistry educators’ tents to teach facts and skills com- pare to instilling HOTS into teaching and learning process with practical or projects. Therefore, most students only possess lower thinking skills. [23] defined digital games as a planned system with rules which equip with variable and quantifiable outcome where player will pay effort to influence the outcome and have attachment toward the outcome gained. [24] concluded that the reasons why dig- ital games are gaining trust are its ability to motivate learner in embarking and staying engage with learning as well as it allows certain learning paradigm being implemented with its attributes. Digital game had been proven to bring ample of benefits to the world of education by stimulating interest and motivation [25]. Digital games have proved as improving student’s retention of information [26]. The digital role-playing was found to be an influential factor in learning performance, self-efficacy, and motivation of its users [27]. However, it still prone toward some drawbacks such as lack of educational focus, the need of gadget and ineffective time management. Thus, study on how digital game best benefits the education ecosystem seems needed. 3 Problem statement There are still certain schools that are still unable to achieve good or distinguish achievement in HOTS-based assessment. The ministry is currently put intensive efforts and initiatives to facilitate these schools to achieve higher performance in HOTS. The educational blueprint set had given substantial focus for HOTS in national standardized examination in Malaysia such as UPSR (primary school) and SPM (secondary school). Yet, it is not an easy to improve HOTS achievement at a very large scale as a method- ology and approach that help to promote and nurture HOTS among students are largely still under study. Digital game had shown positive and great feedback in education. It is also a prom- ising approach. Therefore, digital game is one of the prospective solutions that might improve thinking skills among the learners in Malaysia. Even though digital game is a promising solution, it still prone toward disadvantages in learning. Study by [28] reported that students might focus on the fun elements of games than learning that eventually reduced the effectiveness of the learning process. It is relatively difficult to balance the benefit and its advantages. The same goes to problem-based learning as if its construction of knowledge was made without robust practical and theoretical foundation which render ineffective learning [29]. [30] had reported several weaknesses of problem-based learning such as time inadequacy, loaded curriculum and difference in implementation. iJIM ‒ Vol. 16, No. 02, 2022 103 Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… For this study, researcher wishes to identify the effect of using digital game integrat- ing with problem-based learning approach to help enhancing student’s ability to apply problem solving skills which required HOTS. Students are expected to be able to come up with better and more effective solution with the help of digital games and prob- lem-based learning approach. In conclusion, the problems stated led to the construction of the following research objectives: 1. To study the current level of HOTS among students 2. To study the effect of the combination of problem-based learning and digital game toward HOTS among students 3. To study the learning process of HOTS construction during the engagement with problem-based learning and digital game 4 Methodology This research is quasi-experimental research due to the limitation of access to samples and the intact nature of classroom in a formal education setting globally. Since the study is studying about the effect of digital games integrating of problem-based learning in enhancing HOTS, quasi-experimental will provide strong evidence and sup- port toward the impact of the related pedagogies. This research was conducted in two phases which are the preliminary study and the experimental treatment. The samples are primary school students who are learn- ing English, and English was made as the subject domain for this intervention. In the preliminary study, the current HOTS level of the samples was identified. During the experimental treatment, new samples had engaged the designed treatment for four weeks. The treatment was around 10 hours in total with pre-test and post-test in between of the treatment sessions. The preliminary had sampled 50 samples using random sampling technique and another 35 were samples for the experimental treat- ment using cluster random sampling technique. Selection of school was made using the non-probability sampling method of purposive sampling. Sampling is bound to the willingness of the school and samples to get involve into the research voluntarily as well as the familiarity of the samples with digital technologies either in classroom or out of classroom context. The instruments are preliminary test, pre-test, post-test, and journal reflection. The preliminary test, pre-test and post-test instrument and its marking rubric were devel- oped based on the discussion of the researchers with the panel of experts from schools and university in Malaysia. These are an open-ended question engineered to exhibit HOTS mastery by samples with maximum mark of 12 with total number of 3 ques- tions. There are three pictures in the test for the respondents to respond to. In each picture, there are different situation that required the respondents to analyze and think critically to find the needed answer. For each picture, there were three questions that respondents need to answer. Samples are required to answer each question in a short paragraph written by them. No words limit for each paragraph given. The grammar and spelling mistakes were not penalized by the marker. The instruments were marked by two experts with a high inter-rater value for each answer. To ensure the validity of the 104 http://www.i-jim.org Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… tests, researchers had consult with 3 English teachers from primary school for purpose of face and content validity. The reliability of these instruments was tested with 15 samples that shows its reliability at the value of 0.6. Reflection journal was distributed to the samples using Google Form. It focuses on the reflection of learning process during the engagement with the treatment sessions. Samples are allowed to write their reflection journal after each session of treatment or during and after the end of school session. The reflection journal consists of two questions as guidance for samples to write their reflection which are how they learn individually and with peers during the treatment sessions. 5 Result The result from preliminary study shows that for the first question the mode was at 11. The skewness of the histogram as in Figure 1 is toward the middle with higher score at right that shows samples able to answer the question well. It concluded that samples could identify the problem and solution to the complex situation given. Fig. 1. Histogram for preliminary study The full mark for the question 2 is 7 marks. This question required samples to make decision for choose the solution they prefer the most to handle the problems they are facing in the situation given. From Figure 1, it shows that most of the samples’ scores iJIM ‒ Vol. 16, No. 02, 2022 105 Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… are gather around the mode and the medium of the marks with is 4 marks. Most of the samples acquired marks that close to the medium mark which is 4 and very few sam- ples get the very low marks and very high marks. This indicated that the ability among samples in decision making is average; there are no extreme good or bad among their ability in decision making. The average performance of samples in decision making also reflected their level of applying higher order thinking skills such as critical and creative thinking skills, as well as their problem-solving skills. In question 3, samples are required to give reason for the answer they choose in question 2. Samples need to come up with more than 1 reasonable and rational reason to explain their decision in question 2. According to [31], the fundamental skill for thinking is by learning to reason. According to the definition, if samples able to explain and reason well about the choice they made, they are showing high level thinking skill with good quality which able to benefit them. However, samples in this test do not illustrate thinking process with good quality. The full marks for question 3 are 11 marks. The highest marks samples able to obtain is only 9 marks and only 1 person obtain 9 marks. From Figure 1, most of the samples obtain very low marks and the distribution of the marks is skewed toward the low marks. The mode of the marks obtain by samples is 1 mark. Samples perform poorly in question 3. To reason is to make connection between knowledge and information by using analysis, synthesis, and evaluation skills [32]. Therefore, it is not an easy task. As [16] had suggested, analysis, synthesis and evaluation are HOTS, it infers that samples not able to make reasoning based on the decision they made and not able to achieve the desired HOTS yet. The result of pre-test and post-test of samples engaged with the treatment is as in Table 1. The pre-test and post-test result of the 35 samples were compared using paired sample t-test. The significant level, α is set at 95% with the value α = 0.05 for this paired sample t-test. The result was separated based on questions in the pre-test and post-test to gain better insight into the effect of treatment to samples’ HOTS. Table 1. Paired sample t-test for pre-test and post-test Mean SD t df Sig. (2-tailed) Question 1: pre-post –0.60 1.17 –3.04 34 .005 Question 2: pre-post –1.37 1.06 –7.66 34 .000 Question 3: pre-post –6.09 1.90 –6.74 34 .000 Based on the table above, the p value for the comparison of pre-test and post-test for question 1 is 0.005 which is less than the α value set at 0.05. Therefore, there is a significant different between the result of pre-test and post-test. For questions 2 and 3, the p value is 0.000 which are also below the α value which is 0.05. Therefore, there is a significant different between the result of pre-test and post-test for these two questions as well. As in Table 1, the three questions are having mean with negative value that shows samples’ score are higher in the post-test rather that pre-test. It clearly shows that the treatment is giving positive effect to the HOTS level of samples. The data from the journal were analysed qualitatively using thematic analysis. The themes produced are as in Figure 2. 106 http://www.i-jim.org Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… Fig. 2. Theme of the reflection journal Based on the reflection journal, samples respond toward the digital game prob- lem-based learning section is positive after the first learning section. They feel that digital game problem-based learning is fun and interesting compared to normal teach- ing and learning section in the classroom. They say they will no longer be boring in learning and studying section if learning is carried out in digital game problem-based learning section. The journal from the second treatment session divided in to both positive and neg- ative feedback from the samples. The most repeated theme in the samples’ journal is still fun and interesting. Most of them still engage and intrigue by the digital game inte- grates with problem solving approach. Moreover, the request for longer gaming section and less working section is also repeated frequently in the journal for second learning section with digital game problem-based learning. The negative feedback also repeated in samples’ journal. One of the most repeated negative feedback is the game start to bore them. However, this feedback was only repeated in a few of the samples only. Another negative feedback from the samples is about playing the game individually and asking samples to write down solution for problem individually. The statement if “not wanting individual works” is repeated numbers of time in samples’ journal. They are statement like “I feel happier play and write with friends”, “I hope teacher allow us to play and share answer” and “I don’t want to play and write alone”. Samples request for more group work and group play time is based on the desire of group learning goal. In the further reflection journal, the feedback from samples’ journal is mostly nega- tive. There is still repetition on the statement related on fun and interesting among sam- ples. However, the repetition on helping in learning does not appear in samples’ journal for the third section of learning. The request for group playing time and complete task in group also appear in the third digital game problem-based learning section. Besides, many samples also mention that the section with digital game started to bore them. The reason for samples become bored is clearer because repetition on request for different kind of game in samples’ journal. Samples also request on game from different genre. One of the samples stated that “I like playing game in class, but I am boring playing iJIM ‒ Vol. 16, No. 02, 2022 107 Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… the same game.” Another sample also stated that “I wish to play others game, the same game boring me.” Samples’ interested span cannot withstand the same game for the three sections. Another negative feedback is samples start to show concern that too much playing may cause them to divert from the learning and become less concentrate in learning. 6 Discussion To understand the current level of HOTS among primary school students, the data was gathered to indicate their thinking skills level by using a test instrument. The data turn out to show that samples have their strong aspect in HOTS and weak aspect in HOTS too. [33] run a study to try to find out the factor that influence primary school students in HOTS. The result of their research does show different between the boys and the girls where female students preform slightly better that the male. However, after testing the different using Mann-Whitney U test, they found out the differences is not significant enough to conclude that gender does affect the differences in primary school students’ HOTS [33]. There are two studies that found out that the learning environment does affect the decision-making skills of pupils in the learning process. [34] carried out a study by using questionnaire and simulated-patient-encounter situation to test on medical stu- dents’ level of decision making. After gather the result from different situation, he tested the result by using ANOVA test and it turn out that when medical students were place in different scenario and environment, the decision-making level and process will be different. The next study is done with teacher trainee in learning democracy approach in classroom. When [35] carry out the case study for the democracy education, he does aim to collect data about learners’ decision-making process. Decision-making would be better if participants could be facilitated by the facilitator carefully [35]. This indicate that decision making need facilitation and scaffold. [21] tried to find out the relationship between different learning style and techni- cal student’s HOTS. They reported that reasoning is the among the weakest perceived HOTS based on the data collected from 375 technical students using Marzano HOTS. After running MANOVA test, they conclude that learning style will affect learners’ level of HOTS such as their perceived level of reasoning. Recent studies shown that digital game is effective in developing 21st century skills including HOTS either in Malaysia or out of Malaysia [36]. Besides, digital games also influence attitude toward learning in positive magnitude and it would affect their learning performance [37], [38]. There is research support that digital game promote creativity [39] and problem solving [40]. Research about the effect of digital games in improving HOTS are extensive [37]. The positive impact of digital games is overwhelming such as for students with learning difficulties [41], increase competitive growth [42] and fostering knowledge, attitudes, and skills of the students with instructor guidance [43]. Therefore, future research on the impact of digital games toward learning and education is highly suggested. We suggest that learning using the unique combination of problem-based learning and digital game still have room for improvements. For examples, to strengthen the 108 http://www.i-jim.org Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… positive effects of this intervention, teachers and educational policy makers should look into how the combination of these approaches should be unified to gain a fun and joyful learning session without compromising the learning elements. The second improve- ment is that, teachers could use a variety of games to support the problem-based learn- ing. As we found that the samples feel bored using the same game repeatedly. Finally, we would suggest that teachers or instructors to actively facilitate the digital games problem-based learning so that it would enable the students to make better decision that might impact their learning outcomes and success. 7 Conclusion Studying the mechanism of improving HOTS using digital games is not a new topic. However, an effective mechanism of how it can be done is still puzzling around. This research had tried the combination of digital games and problem-based learning as a new mechanism of enhancing HOTS. This study has proven that the integration of these two approaches is effective in enhancing primary school students’ HOTS. Sam- ples shows good perspective toward the application and generate a dynamic process of learning. Hopefully, more future studies can be done upon the integration of these two approaches to help foster HOTS among our young learners. 8 Acknowledgment Authors would like to thank Ministry of Education and Universiti Teknologi Malaysia for sponsoring this paper through UTM Fundamental Research (UTMFR) grant with Project Number Q.J130000.2553.21H23. 9 References [1] Ministry of Education Malaysia. (2015). Malaysia Education Blueprint 2015–2025 (Higher Education). www.moe.gov.my [2] Anthony, W. L., Zhu, Y., & Nower, L. (2021). The relationship of interactive technology use for entertainment and school performance and engagement: Evidence from a longitudinal study in a nationally representative sample of middle school students in China. Computers in Human Behavior, 122, 106846. https://doi.org/10.1016/j.chb.2021.106846 [3] Mukherjee, S., Mallick, A., & Biswas, S. (2021). Effects of massively multiplayer online role-playing games on psychological health and inter-personal relationships of the male youth: A cross-sectional study in sub-urban Kolkata, West Bengal, India. Anthropological Review, 84(2), 163–179. https://doi.org/10.2478/anre-2021-0010 [4] Adžić, S., Al-Mansour, J., Naqvi, H., & Stambolić, S. (2021). The impact of video games on Students’ educational outcomes. Entertainment Computing, 38, 100412. https://doi. org/10.1016/j.entcom.2021.100412 [5] Fraga-Varela, F., Vila-Couñago, E., & Rodríguez-Groba, A. (2021). Serious Games and Mathematical Fluency: A Study from the Gender Perspective in Primary Education. Sus- tainability, 13(12), 6586. https://doi.org/10.3390/su13126586 iJIM ‒ Vol. 16, No. 02, 2022 109 http://www.moe.gov.my https://doi.org/10.1016/j.chb.2021.106846 https://doi.org/10.2478/anre-2021-0010 https://doi.org/10.1016/j.entcom.2021.100412 https://doi.org/10.1016/j.entcom.2021.100412 https://doi.org/10.3390/su13126586 Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… [6] Nurfatanah, N., Yudha, C. B., Marini, A., & Sumantri, M. S. (2021). Development of matemathic media games education based on e-learning in the Planting of Basic Con- cepts in Numeracy. Journal of Physics: Conference Series, 1869(1), 012127. https://doi. org/10.1088/1742-6596/1869/1/012127 [7] Lameras, P., Arnab, S., de Freitas, S., Petridis, P., & Dunwell, I. (2021). Science teachers’ experiences of inquiry-based learning through a serious game: a phenomenographic perspec- tive. Smart Learning Environments, 8(1), 7. https://doi.org/10.1186/s40561-021-00152-z [8] Obery, A., Lux, N., Cornish, J., Grimberg, B. I., & Hartshorn, A. (2021a). Competitive games as formative assessment in informal science learning: Improvement or hindrance? TechTrends, 65(4), 454–463. https://doi.org/10.1007/s11528-021-00619-3 [9] EL Rahman, S. A. (2021). A Game-Based Learning for Teaching Arabic Letters to Dys- lexic and Deaf Children. In K. and H. A. E. Al-Emran Mostafa and Shaalan (Ed.), Recent Advances in Intelligent Systems and Smart Applications (pp. 337–361). Springer Interna- tional Publishing. https://doi.org/10.1007/978-3-030-47411-9_19 [10] Albar, S. B., & Southcott, J. E. (2021). Problem and project-based learning through an investigation lesson: Significant gains in creative thinking behaviour within the Australian foundation (preparatory) classroom. Thinking Skills and Creativity, 41, 100853. https://doi. org/10.1016/j.tsc.2021.100853 [11] Maslihah, S., Waluya, S. B., Rochmad, K., Karomah, N., & Iqbal, K. (2021). Increasing mathematical literacy ability and learning independence through problem-based learning model with realistic mathematic education approach. Journal of Physics: Conference Series, 1918(4), 042123. https://doi.org/10.1088/1742-6596/1918/4/042123 [12] Izzati, L. R. (2021). The effect of problem-based learning to improve students’ metacogni- tion skills in solving mathematical problems based on cognitive style. Journal of Physics: Conference Series, 1918(4), 042073. https://doi.org/10.1088/1742-6596/1918/4/042073 [13] Sujatmiko, Y. A., Isnaeni, W., Ridlo, S., & Saptono, S. (2021). The development of andro-webcomic media based on problem-based learning to improve analytical thinking ability and scientific attitude. Journal of Physics: Conference Series, 1918(5), 052023. https://doi.org/10.1088/1742-6596/1918/5/052023 [14] Musna, R. R., Juandi, D., & Jupri, A. (2021). A meta-analysis study of the effect of Problem-Based Learning model on students’ mathematical problem-solving skills. Journal of Physics: Conference Series, 1882(1), 012090. https://doi.org/10.1088/1742-6596/1882/1/ 012090 [15] Monsang, P., Srikoon, S., & Wichaino, N. (2021). The effects of problem-based learning for enhancing science problem solving skills. Journal of Physics: Conference Series, 1835(1), 012016. https://doi.org/10.1088/1742-6596/1835/1/012016 [16] Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Richard, R. E., Pintrich, P., Raths, J., & Wittrock, M. C. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman. [17] Chiong, C., & Lim, L. (2020). Seeing families as policy actors: Exploring higher order thinking reforms in Singapore through low-income families’ perspectives. Journal of Edu- cation Policy, 1–21. https://doi.org/10.1080/02680939.2020.1777468 [18] Kwangmuang, P., Jarutkamolpong, S., Sangboonraung, W., & Daungtod, S. (2021). The development of learning innovation to enhance higher order thinking skills for students in Thailand junior high schools. Heliyon, 7(6), e07309. https://doi.org/10.1016/j.heliyon.2021. e07309 [19] Nguyễn, T. M. T., & Nguyễn, T. T. L. (2017). Influence of explicit higher-order thinking skills instruction on students’ learning of linguistics. Thinking Skills and Creativity, 26, 113–127. https://doi.org/10.1016/j.tsc.2017.10.004 110 http://www.i-jim.org https://doi.org/10.1088/1742-6596/1869/1/012127 https://doi.org/10.1088/1742-6596/1869/1/012127 https://doi.org/10.1186/s40561-021-00152-z https://doi.org/10.1007/s11528-021-00619-3 https://doi.org/10.1007/978-3-030-47411-9_19 https://doi.org/10.1016/j.tsc.2021.100853 https://doi.org/10.1016/j.tsc.2021.100853 https://doi.org/10.1088/1742-6596/1918/4/042123 https://doi.org/10.1088/1742-6596/1918/4/042073 https://doi.org/10.1088/1742-6596/1918/5/052023 https://doi.org/10.1088/1742-6596/1882/1/012090 https://doi.org/10.1088/1742-6596/1882/1/012090 https://doi.org/10.1088/1742-6596/1835/1/012016 https://doi.org/10.1080/02680939.2020.1777468 https://doi.org/10.1016/j.heliyon.2021.e07309 https://doi.org/10.1016/j.heliyon.2021.e07309 https://doi.org/10.1016/j.tsc.2017.10.004 Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… [20] Dubas, J. M., & Toledo, S. A. (2016). Taking higher order thinking seriously: Using Marzano’s taxonomy in the economics classroom. International Review of Economics Edu- cation, 21, 12–20. https://doi.org/10.1016/j.iree.2015.10.005 [21] Yee, M. H., Yunos, J. Md., Othman, W., Hassan, R., Tee, T. K., & Mohamad, M. M. (2015). Disparity of learning styles and higher order thinking skills among technical students. Procedia – Social and Behavioral Sciences, 204, 143–152. https://doi.org/10.1016/j. sbspro.2015.08.127 [22] Wun, K. P., & Harun, J. (2017). The Effect of Scenario-Epistemic Game on Higher Order Thinking Skills among High School Chemistry Students in Malaysia. 2017 International Conference on Learning and Teaching in Computing and Engineering (LaTICE), 16–22. https://doi.org/10.1109/LaTiCE.2017.10 [23] Juul, J. (2003). The game, the player, the world: looking for a heart of gameness. DiGRA Conference. [24] All, A., Nuñez Castellar, E. P., & van Looy, J. (2016). Assessing the effectiveness of digital game-based learning: Best practices. Computers & Education, 92–93, 90–103. https://doi. org/10.1016/j.compedu.2015.10.007 [25] Bağ, H., & Çalık, M. (2021). Designing an argumentation-based educational digi- tal game to teach the subject of force. Physics Education, 56(3), 035002. https://doi. org/10.1088/1361-6552/abdae8 [26] Apostolellis, P., & Bowman, D. A. (2016). Audience Involvement and Agency in Digital Games: Effects on Learning, Game Experience, and Social Presence. Proceedings of the The 15th International Conference on Interaction Design and Children, 299–310. https:// doi.org/10.1145/2930674.2930700 [27] Zou, D., Zhang, R., Xie, H., & Lee, W. F. (2021). Digital game-based learning of information literacy: Effects of gameplay modes on university students’ learning perfor- mance, motivation, self-efficacy and flow experiences. Australasian Journal of Educational Technology, 37(2), 152–170. https://doi.org/10.14742/ajet.6682 [28] Bolliger, D. U., Mills, D., White, J., & Kohyama, M. (2015). Japanese students’ perceptions of digital game use for English-language learning in higher education. Journal of Educa- tional Computing Research, 53(3), 384–408. https://doi.org/10.1177/0735633115600806 [29] Bessa, B. R., Santos, S. C. dos, & Fonseca, L. da. (2017). Using a virtual learning environ- ment for problem-based learning adoption: A case study at a high school in India. 2017 IEEE Frontiers in Education Conference (FIE), 1–7. https://doi.org/10.1109/FIE.2017.8190642 [30] Eryilmaz, A., & Ates, O. (2010). Strenghts and weaknesses of problem-based learning in engineering education: Students’ and tutors’ perspectives. Buca Egitim Fakultesi Dergisi, 28, 40–58. [31] Lin, M., Preston, A., Kharrufa, A., & Kong, Z. (2016). Making L2 learners’ reasoning skills visible: The potential of Computer Supported Collaborative Learning Environments. Think- ing Skills and Creativity, 22, 303–322. https://doi.org/10.1016/j.tsc.2016.06.004 [32] Siwawetkul, W., & Koraneekij, P. (2020). Kasetsart journal of social sciences. In Kaset- sart Journal of Social Sciences (Vol. 41, Issue 1). https://so04.tci-thaijo.org/index.php/kjss/ article/view/229138#.YQcIdo-VUqU.mendeley [33] Kashefi, H., Yusof, Y. M., Ismail, Z., Men, O. L., Lee, T. J., & Joo, T. K. (2017). Gen- der and Mathematics Performance of Primary Students in Higher Order Thinking Skills. 2017 7th World Engineering Education Forum (WEEF), 808–811. https://doi.org/10.1109/ WEEF.2017.8467086 [34] Jendyk, R. M. (2016). Decision making in different care settings: Do undergraduate students already care? Health Professions Education, 2(2), 114–120. https://doi.org/10.1016/j. hpe.2016.01.008 iJIM ‒ Vol. 16, No. 02, 2022 111 https://doi.org/10.1016/j.iree.2015.10.005 https://doi.org/10.1016/j.sbspro.2015.08.127 https://doi.org/10.1016/j.sbspro.2015.08.127 https://doi.org/10.1109/LaTiCE.2017.10 https://doi.org/10.1016/j.compedu.2015.10.007 https://doi.org/10.1016/j.compedu.2015.10.007 https://doi.org/10.1088/1361-6552/abdae8 https://doi.org/10.1088/1361-6552/abdae8 https://doi.org/10.1145/2930674.2930700 https://doi.org/10.1145/2930674.2930700 https://doi.org/10.14742/ajet.6682 https://doi.org/10.1177/0735633115600806 https://doi.org/10.1109/FIE.2017.8190642 https://doi.org/10.1016/j.tsc.2016.06.004 https://so04.tci-thaijo.org/index.php/kjss/article/view/229138#.YQcIdo-VUqU.mendeley https://so04.tci-thaijo.org/index.php/kjss/article/view/229138#.YQcIdo-VUqU.mendeley https://doi.org/10.1109/WEEF.2017.8467086 https://doi.org/10.1109/WEEF.2017.8467086 https://doi.org/10.1016/j.hpe.2016.01.008 https://doi.org/10.1016/j.hpe.2016.01.008 Paper—Enhancing HOTS using Problem-Based Learning and Digital Game in the Context of Malaysian… [35] Harell, K. F. (2019). Deliberative decision-making in teacher education. Teaching and Teacher Education, 77, 299–308. https://doi.org/10.1016/j.tate.2018.10.015 [36] Ahmad, M., Mansor, N. R., Rashid, R. A., Ain, N., Zakaria, C. R., & Sung, C. M. (2021). Implementation of digital games in advancing students’ higher-order thinking skills: A review. Journal of Physics: Conference Series, 1793(1), 012069. https://doi.org/ 10.1088/1742-6596/1793/1/012069 [37] Giannakos, M. N. (2013). Enjoy and learn with educational games: Examining factors affect- ing learning performance. Computers & Education, 68, 429–439. https://doi.org/10.1016/j. compedu.2013.06.005 [38] Srisawasdi, N., & Panjaburee, P. (2019). Implementation of game-transformed inqui- ry-based learning to promote the understanding of and motivation to learn chemistry. Journal of Science Education and Technology, 28(2), 152–164. https://doi.org/10.1007/ s10956-018-9754-0 [39] Hall, J., Stickler, U., Herodotou, C., & Iacovides, I. (2020). Expressivity of creativity and creative design considerations in digital games. Computers in Human Behavior, 105, 106206. https://doi.org/10.1016/j.chb.2019.106206 [40] Ke, F. (2019). Mathematical problem solving and learning in an architecture-themed epis- temic game. Educational Technology Research and Development, 67(5), 1085–1104. https:// doi.org/10.1007/s11423-018-09643-2 [41] Meishar-Tal, H., & Kesler, A. (2021). “If I create a game I’ll learn”: Online game creation as a tool to promote learning skills of students with learning difficulties. Interactive Learning Environments, 1–12. https://doi.org/10.1080/10494820.2021.1919146 [42] Kapustina, L. v., & Martynova, I. A. (2020). Training Employees in the Digital Economy with the Use of Video Games. In S. Ashmarina, A. Mesquita, & M. Vochozka (Eds.), Digital Transformation of the Economy: Challenges, Trends and New Opportunities (pp. 444–454). Springer International Publishing. https://doi.org/10.1007/978-3-030-11367-4_44 [43] Nino, M., & Evans, M. A. (2015). Fostering 21st-Century Skills in Constructivist Engi- neering Classrooms With Digital Game-Based Learning. IEEE Revista Iberoamericana de Tecnologias Del Aprendizaje, 10(3), 143–149. https://doi.org/10.1109/RITA.2015.2452673 10 Authors Liew Chuang Tong is working as a teacher and had graduated from Universiti Teknologi Malaysia with Master of Education (Educational Technology). He had explored the mechanism of HOTS enhancement in the context of Malaysia primary school. This paper was written based on his master project report. He had collected the data and make the fundamental writing for this paper. Mohd Shafie Rosli is a Malaysian active researcher in HOTS and online learning environment with more than a decade of research experience in technological assisted cognitive augmentation. He had supervised a significant number of postgraduate research on the application of online learning environment in the field of education. Nor Shela Saleh is serving Universiti Tun Hussein Onn Malaysia. She is an expert in social sciences statistics and research methodology, contributing to data analysis and literature support in this paper. Article submitted 2021-10-19. Resubmitted 2021-11-04. Final acceptance 2021-11-04. Final version published as submitted by the authors. 112 http://www.i-jim.org https://doi.org/10.1016/j.tate.2018.10.015 https://doi.org/10.1088/1742-6596/1793/1/012069 https://doi.org/10.1088/1742-6596/1793/1/012069 https://doi.org/10.1016/j.compedu.2013.06.005 https://doi.org/10.1016/j.compedu.2013.06.005 https://doi.org/10.1007/s10956-018-9754-0 https://doi.org/10.1007/s10956-018-9754-0 https://doi.org/10.1016/j.chb.2019.106206 https://doi.org/10.1007/s11423-018-09643-2 https://doi.org/10.1007/s11423-018-09643-2 https://doi.org/10.1080/10494820.2021.1919146 https://doi.org/10.1007/978-3-030-11367-4_44 https://doi.org/10.1109/RITA.2015.2452673