International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 16, No. 06, 2022 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach to Computational Thinking https://doi.org/10.3991/ijim.v16i06.26281 Chong Sin Yoon, Mas Nida Md. Khambari(*) Faculty of Educational Studies, Universiti Putra Malaysia, Serdang, Malaysia khamasnida@upm.edu.my Abstract—This paper presents the design, development, and evaluation of the Robobug board game, which was developed to foster computational thinking (CT) among primary school pupils in Malaysia, particularly those who encoun- ter computer and Internet access issues. Utilizing Richey and Klein’s design and development (DDR) methodologies, this research serves as a primer by first concentrating on needs analysis for the design, then converting it into the development of the board game. The focus of this study was on how the design aspects of Robobug might help children grasp CT principles and foster CT prac- tices. The Robobug board game was initially assessed remotely via questionnaire by a panel of eight subject matter experts from Malaysian institutions. Using the questionnaire and evaluation rubrics, experts were asked to offer insights with the following objectives: (i) to evaluate the design features; (ii) to exam- ine the computational concepts; and (iii) to examine the computational practices of the Robobug board game. The research found that the Robobug board game has significant potential for use as an unplugged tool and alternative technique for instructors to enhance CT abilities outside the typical classroom. Addition- ally, it can be utilized to incorporate collaborative and game-based learning into teachers’ instructional strategies to boost student engagement and achievement. The Robobug board game’s shortcomings and suggestions for improvement are discussed. Keywords—computational thinking, board game, design and development research, evaluation 1 Introduction Education 4.0, which aligns with the Industrial Revolution 4.0 (IR 4.0), has a major impact on teachers’ instructional approaches [18][21]. To successfully educate students for future life and work as envisioned by IR 4.0, they must first learn how to communi- cate with computers [36]. Computational thinking (CT) is a phrase coined by Jeannette Wing in 2006 [3] to describe successful communication along these lines [33]. CT literacy, as defined by IR 4.0 [22][36][41], is viewed as critical for students’ preparation for personal and professional survival in the future. Since then, numerous educators iJIM ‒ Vol. 16, No. 06, 2022 41 https://doi.org/10.3991/ijim.v16i06.26281 mailto:khamasnida@upm.edu.my Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… and scholars have emphasized the critical significance of incorporating computational thinking as an essential 21st-century literacy at all educational levels [17][28][35][39]. 1.1 Computational thinking in the Malaysian curriculum As a result, many governments worldwide have recognized the critical nature of CT and have included it as a required subject in their curricula. Some countries have designated CT as a national program, while others have developed new instruc- tional materials and textbooks for CT [17]. In 2017, Malaysia’s Ministry of Education (MOE) announced the inclusion of CT skills in the Standard Based Curriculum for Primary Schools [24]. According to Wahab et al. [41], the MOE has partnered with the Malaysian Digital Economy Commission (MDEC) to train teachers on how to inter- pret and incorporate CT into their lessons. MOE undertook these efforts to enable a smooth transition to CT in schools [41], following scholars’ emphasis on the signifi- cance of educators’ perspectives, attitudes, and technical competencies in incorporating CT abilities into curricula [31][32]. Additionally, research indicates that teachers are unable to implement CT activities in their daily classroom practices due to a lack of expertise, self-confidence, resources, official support, and training [30][31][32][40]. Specifically, Ung et al. [40] reported that Malaysian educators face the following barri- ers in incorporating CT into teaching and learning: (i) a lack of CT understanding, (ii) a lack of instructional materials and tools, and (iii) a lack of infrastructure. Recognizing the challenges faced by Malaysian educators, we proposed the development of Robo- bug, an unplugged tool for teaching computer literacy in a formal or informal setting. 1.2 Research on computational thinking using robotics The use of educational robotics to foster CT has been a popular topic in the educa- tion research literature in recent years [2][11][17][20][29][32][39][41]. Robotics has been proven to be an efficient technique to introduce CT, since students are introduced to computational principles such as sequencing, pattern recognition, decomposition, and loops while designing, creating, and programming robots [36]. For instance, it was observed that Bee-bot, which was used to introduce young children to coding and CT, aided in the development of CT skills among those children [2][29][32]. Additionally, Angeli and Valanides’ research found that preschool kids are capable of coping with the complexity of learning activities by decomposing tasks into manageable subtasks using Bee-bot [2]. 1.3 Research on computational thinking using board games In recent years, several approaches and interventions have been developed to pro- mote CT among students [2][5][17][19][21][28]. Of these, one strategy that has been emphasized for teaching CT is the unplugged method using board games, which involves pupils developing CT abilities without the use of computers [21]. Bayeck [4] discovered that board games assist individuals not only in acquiring new abilities but also in keeping those already gained. Board games also help participants grasp and 42 http://www.i-jim.org Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… demonstrate difficult concepts, such as computational reasoning. Harris’s [16] research further established the validity of a non-coding, non-computer-based educational method for CT. His research found that teachers who interacted with the Robot Turtles board game saw statistically significant increases in their confidence as teachers of CT. There are numerous advantages of using board games to deliver CT lessons in an unplugged manner. According to Scirea and Valente [34], board games are a superior option compared to digital games for teaching CT in the classroom, as the latter suffer from a high level of inaccessibility, single-player attention, and difficulties in adapting/ modifying them for the classroom environment. Additionally, Berland and Lee’s [5] research established that complex CT develops spontaneously when non-traditional, non-computational media like strategic board games are used. Yu and Roque [19] demonstrated that some computational kits encourage the exploration of specific com- putational concepts and practices, concluding that there is room to broaden the manner in which these concepts can be supported by kits such as board games. Utilizing a CT board game may ultimately help boost students’ interaction and enthusiasm to study, thereby enhancing their academic performance [21]. 1.4 The design and development of Robobug as an unplugged tool for cultivating computational thinking This paper presents the design, development, and evaluation of Robobug, a board game designed to help primary school pupils learn computational skills. The findings of this study can be utilized to direct future research towards the development and innovation of an unplugged tool to teach CT literacy abilities to children in elementary schools. Due to resource constraints, educators struggle to incorporate game-based and collaborative learning into CT lessons [40]. Additionally, because the majority of edu- cators have just a cursory understanding of CT, they are bewildered as to how to apply it with computers. As a few studies have examined how CT can be contextualized, we created the Robobug board game to assist educators working in non-computer envi- ronments [3]. This unplugged tool will benefit Malaysian primary school students and teachers in learning CT, especially those in remote rural locations with limited access to electricity, let alone computers. The study’s findings thus provide a way to improve traditional teaching and learning CT skills in Malaysian schools. To summarize, this research offers Malaysian teachers a student-centered, theoretically sound, and field- tested CT board game. 2 Methods By focusing on needs analysis for the design, and subsequently transforming the design into the creation of the board game, this research serves as a primer for Richey and Klein’s design and development (DDR) techniques [33]. In particular, this study uti- lized design and development research approaches with a special emphasis on product development [33]. Needs analysis was carried out in a series based on assessments of several types of CT kits, both academic and commercial. We then proceeded to design the Robobug board game. Finally, the results of the needs analysis were used as the basis for translating the design concepts into a product in the development phase [33]. iJIM ‒ Vol. 16, No. 06, 2022 43 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… 2.1 Data collection and sampling We adopted and modified expert sampling, a purposive sampling technique, to request the opinions of the board game experts. When determining the viability of novel research ideas, expert sampling is an effective method [9][14]. Accordingly, a panel of eight subject matter experts was formed to provide insight into the following objectives: (i) to evaluate the design features of the Robobug board game; (ii) to ana- lyze the computational concepts in the Robobug board game; and (iii) to analyze the computational practices in the Robobug board game. The experts were chosen based on the criterion that he or she should be a gamification and/or computer science expert. To acquaint the expert panel members with the operation of the Robobug board game, we provided them two sets of demonstration videos along with a user manual to peruse, all of which were shared on Google Drive. We then distributed questionnaires, which included evaluation rubrics, to the experts via email. Guided by the videos and user manual, they examined and evaluated the board game over a two-week period before emailing the questionnaire back to the research team. We also corresponded with the experts via email after they returned their evaluations. This strategy was adopted to circumvent mobility restrictions and institution closures in response to the Covid-19 out-break [6][7]. 2.2 Data analysis In this study, Microsoft Excel was used to code, classify, and analyze the data. According to Bree and Gallagher [8], Microsoft Excel is a cost-effective tool for the analysis of themes and the triangulation of qualitative data gathered through ques- tionnaires. The data from the experts was therefore transferred to a Microsoft Excel worksheet, following which it was analyzed to determine the theme areas. The analyt- ical process continued with data categorization, reflection and synthesis, compression for reporting ease, and interpretation of the data and results—all of which occurred prior to writing the final report [8]. 3 Design and development of the Robobug board game The Robobug board game was inspired by the way insects hunt for food to survive. Our goal was to develop a board game that supports CT learning for two to four players aged six and up. The Robobug board game was designed and developed in two stages: (i) designing the board game’s features, and (ii) integrating computational concepts and practices into the board game. 3.1 Stage 1—designing the Robobug board game’s features The design of the Robobug board game began with the creation of the board and its corresponding sets of characters, goals, and instruction cards. Bayeck [4] asserts that learning occurs as a result of learners’ participation, engagement, and interaction with a game and their environment. Accordingly, the researcher developed six boards 44 http://www.i-jim.org Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… that gamers can rearrange according to their level of interest. Four Robobug characters and four Robobug goals were added. To move their Robobug around the board, play- ers must build action sequences. A ‘Loop’ card repeats a sequence of action orders, whereas an ‘Undo’ card debugs possible game steps that were performed incorrectly. Figure 1 in Table 1 illustrates the board game’s components, which include six boards, four Robobug character cards, four ‘Goal’ cards, 60 ‘Go Straight’ cards, 60 ‘Turn Right’ cards, 60 ‘Turn Left’ cards, 16 ‘Loop’ cards, and four ‘Undo’ cards. Numerous studies have proven that game-based learning is a good method for improving cognitive processes and problem-solving abilities [6][12][38]. Based on this evidence, the board game’s card deck was expanded to increase players’ interest while making it more thrilling and engaging [1][13]. In particular, for the development of CT skills, additional sets of cards that enhance gamification were included in the board game, as illustrated in Figure 3 in Table 1. These consist of 20 ‘Obstacle’ cards, 20 ‘Power’ cards, 40 ‘Flower’ cards, 15 large ‘Treat’ cards, 12 small ‘Treat’ cards, 15 large ‘Trick’ cards, and 12 small ‘Trick’ cards. To supplement the game cards, ‘Computational Thinking Challenge’ cards were created based on a Bebras task that requires students to transfer and project their CT abilities to solve ‘real-world’ situations. Substantial research has shown that Bebras tasks are effective for assessing CT skill transfer and may be used as measurement instruments or CT teaching and learning tools in meaningful ways [10][11][12][15] [23]. Therefore, we incorporated the CT challenge aspect into the Robobug board game. 3.2 Stage 2—integrating computational concepts and practices into the board game In Stage One, we discussed the design of the board game’s features. Next, Stage Two involved bringing computational concepts and practices into the Robobug board game by creating a deck of cards that may be used to create an algorithm. The design of the card deck was adapted from Scratch, a block-based programming language. To win the game, players must arrange the deck of cards to represent the sequences of steps taken by the Robobug to accomplish its mission. The collection of cards displayed in Figure 2 in Table 1 enables players to create their own algorithm using all the instruction cards utilized by the Robobug during its mission. There are a total of four ‘When Robobug Moved’ cards, 40 ‘Turn Left Degrees’ cards, 40 ‘Turn Right Degrees’ cards, 40 ‘Move Forward Steps’ cards, 10 ‘If Then’ cards, 10 ‘Repeat’ cards, and four ‘Get Ice Cream’ cards. Table 3 gives examples of the computational concepts, while Table 5 summarizes the primary methods for promoting each of the five computational practices. 4 Results and discussion This section discusses the expert panel’s evaluation of the Robobug board game. Three aspects were evaluated, namely (i) the design features, (ii) the board game’s potential to nurture CT concepts, and (iii) CT practices. For the assessment of the Robobug board game, the researcher created the evaluation criteria and rubrics based on Yu and Roque’s work [19]. iJIM ‒ Vol. 16, No. 06, 2022 45 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… 4.1 Evaluating the design features of the Robobug board game A computational kit’s design and features considerably impact how players perceive and use it [19][37]. Both physical and graphical interfaces, according to Sullivan et al. [37], encourage diverse forms of learning. As a result, our evaluation shed light on the various components and features that went into the design of this board game, result- ing in new and purposeful design options. The design features’ codebook is given in Table 1, which was adapted from Yu and Roque’s [19] research. Table 1. Codebook of design features Theme Description Examples and Sample Picture Components What does the kit consist of? Fig. 1. Components of Robobug board game The Robobug board game includes four Robobug characters, instructions cards, goal cards, Trick and Treat cards, Obstacle cards, Power cards, Flower cards, and six boards that can be assembled. Coding blocks What are the design features of the coding blocks? Fig. 2. Coding blocks of Robobug board game The Robobug coding blocks are made up of a combination of instruction and algorithm cards. The steps taken by Robobug to reach its destination are represented by instruction cards. (Continued) 46 http://www.i-jim.org Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… Theme Description Examples and Sample Picture Supporting materials What are the materials provided to help scaffold children’s play activities? Fig. 3. Supporting materials of Robobug board game To scaffold the play experience, the Robobug board game includes Trick and Treat cards, Obstacle cards, and Power cards. Additionally, there are Computational Thinking Challenge cards, a user manual, and algorithm cards. Algorithm cards are used to represent all Robobug’s instructions for reaching its destination. The rubrics employed to evaluate the design features of the Robobug board game consisted of the following nine categories: (i) creativity component, (ii) attractiveness component, (iii) coding blocks, (iv) coding block design, (v) supporting materials, (vi) Computational Thinking Challenge cards, (vii) Trick and Treat cards, Obsta- cles cards, Power cards, Flower cards, (viii) user manual, and (ix) Algorithm cards. Meanwhile, the standards set for each category were (i) exceptional, (ii) admirable, (iii) marginal, and (iv) unacceptable. The evaluation of the design features and their summarized results are presented in Table 2. Table 2. Evaluation of design features and their summarized results Category Exceptional Admirable Marginal Unacceptable Creativity component 75% 12.5% 12.5% 0% Attractiveness component 12.5% 87.5% 0% 0% Coding blocks 75% 25.0% 0% 0% Coding block design 37.5% 50% 12.5% 0% Supporting materials (scaffolding of children’s play activities) 75% 25% 0% 0% Supporting materials (support and guidance to play the game) 37.5% 50.0% 12.5% 0% Table 1. Codebook of design features (Continued) (Continued) iJIM ‒ Vol. 16, No. 06, 2022 47 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… Category Exceptional Admirable Marginal Unacceptable Supporting materials (Computational Thinking Challenge cards) 100% 0% 0% 0% Supporting materials (Trick and Treat cards, Obstacles cards, Power cards, Flower cards) 75% 12.5% 12.5% 0% Supporting materials (user manual) 0% 87.5% 12.5% 0% Supporting materials (Algorithm cards) 100% 0% 0% 0% The majority of expert panel members agreed that the creativity component of this board game is exceptional and well-suited for school children to explore CT in an unplugged setting. The expert panel felt that great effort had been taken to ensure that the game is engaging and enjoyable to play, as seen by the game’s unique questions, game pieces, and game board. This is in line with Yu and Roque’s finding that the design characteristics and components of a computational kit significantly influence how children view and use the kit for learning [19]. The following quotations from the expert panelists supported this viewpoint: Expert 1: “As a technical expert and board/card game enthusiast, I applaud the effort to approach the teaching of programming using board games. This effort would provide equal opportunity and close the digital divide faced by students.” Expert 3: “Exceptional. Judging from your video, I can tell that you put a lot of thought into making this board game. I am a programmer myself, and I am amazed by the inclusion of ‘loop’ and ‘if’ functions in the board game, which are useful in real programming. I believe the level of play for the board game is suitable for school children.” Expert 7: “The game creator has displayed a creative selection of components to make the game more interesting while sticking to the objective of the game, which is to cultivate computational thinking (CT) skills among young kids.” While the majority of expert panels (87.5 percent) assessed the component’s appeal as excellent, they also gave ideas for improvement. Three experts believe that cooper- ating with a game artist or employing images from online resources would improve the Robobug board game’s overall design. This is consistent with the findings of Nabhani et al. [26] that a game with a bad interface design discourages players from participat- ing. The expert panels further advised that 3D images be used to enhance the design components of the board game. Additionally, an expert proposed that the researcher provide an alternate design approach for visually impaired students. Expert 1: “The use of color, design, and space is admirable; however, to com- pete with digital games and other board games, collaboration with a game artist would enhance the attractiveness of Robobug.” Expert 2: “There can be some room for improvement, but at this stage, it is already a nice design. I’d suggest you look for some examples of designs from other board games before you turn this into a commercial product.” Table 2. Evaluation of design features and their summarized results (Continued) 48 http://www.i-jim.org Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… Expert 5: “The choice of color, space, and design is appropriate. However, from what I feel, it lacks depth and plumpness. I would suggest getting a pro- fessional graphic designer to look at your board and offer suggestions for you. Designing and building 3D models might be unnecessary but that would really add depth to the overall look of your board game.” Expert 8: “My suggestion would be to keep it to one artist to draw all the char- acters so that the styles and coloring stay the same.” Seventy-five percent of the expert panel agreed that the code blocks support the primary goal of honing CT skills. However, some experts advised that more compli- cated code blocks, such as “Backward” instruction cards, be included to increase the game’s difficulty and excitement. One expert also proposed including a more elaborate loop, such as an “If.…Then ... Else” block, which converts numerous stages into a single loop. Expert 1: “The inclusion of If … Then and If … Then … Else cards would complete the basic building blocks of all programming languages. Coding blocks are made of cards. I suggest the following modification to be consistent with pro- gramming languages terminology: loop, if … then, if … then … else, which are referred to as ‘control structures. Such ‘Control cards’ are more appropriate than ‘Algorithm cards’.” Expert 2: “I think making the ‘Backward’ block as part of the coding block can make the game more challenging. Players can be challenged to think of certain steps that require them to go backward. Another suggestion is to provide a more complicated loop that requires multiple steps turning into a single loop.” Apart from that, another expert proposed classifying the coding block cards into three unique categories: beginner, intermediate, and advanced. Additionally, she pio- neered the concept of players ‘unlocking features’ as they advance through the game’s levels. These views align with those of Almeida and Machado [1], who stated that game difficulty is a necessary component of instructional game design. According to Yunus’ study [25], “Challenge” is a critical component of game-based learning that integrates CT concepts. Expert 7: “The coding blocks introduced are appropriate in exposing the players to the concept of creating an algorithm. The level of difficulty can also be increased based on the creativity of the instructor as well as the level of students’ understanding.” Eighty-five percent of the experts agreed that the coding blocks are feasible for creating an algorithm, and complimented the set of coding blocks for covering all the fundamental programming skills. In particular, four of them noted that the architecture of the code blocks is suitable and accessible for creating an algorithm following the gameplay. One of the experts proposed that we examine additional CT kits to improve the design of the coding blocks. This is line with the work of Scirea and Valente [34], who suggested that analyzing myriad CT kits could aid CT teachers and game designers in adapting existing games for the classroom and generating more CT-supporting board games. The following is an excerpt from the expert panels’ comments: iJIM ‒ Vol. 16, No. 06, 2022 49 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… Expert 6: “Being from a computer science background, I think it is easy to cre- ate an algorithm from the sequence of cards that have been built during the game. Easy to understand the concept of the game.” All expert panels agreed that CT challenge cards and algorithm cards had evolved greatly, stating that products like Bebras CT tasks help students develop computational skills. The Bebras Computational Thinking Challenge is indeed crucial for monitoring CT development [10][11][12][15][27]. González et al. [15] claim that Bebras CT tasks offer a realistic and important assessment tool to teach and learn CT. The algorithm cards in the Robobug board game also satisfy all Berland and Lee’s [5] requirements for the five domains of coding in CT. The experts recommended that schools incorporate rubrics for measuring and grading students’ CT abilities in the board game kit. Accord- ing to previous studies, players should be given feedback on their progress after play- time [1]. To this end, the next step of this board game’s design and development entails creating a set of rubrics for instructors to utilize in evaluating students’ CT abilities. With 75 percent of the experts rating the other supporting materials as excellent, the flexibility and customizability of the game’s levels gained positive feedback from the experts. The opportunity for players to control the gameplay in Robobug is in accor- dance with suggestions made by Almeida and Machado [1] that design elements do not only assist in developing educational games but also increase players’ enjoyment when they are able to control the gameplay. Such control allows the player to be in the zone of their own character’s action and manage the interface and input devices—all of which empower players to use their own strategies and take their own actions to achieve the game’s goal. This is evident in the following excerpts: Expert 4: “The customizability of the current board game enables scaffolding learning.” Expert 7: “Since the difficulty levels of the game can be adjusted by the instructor, this board game has huge potential to be an interesting educational game for kids.” The majority of supplementary materials received excellent feedback from the panel professionals. However, the user manual as well as the support and playing instructions require additional modifications. We will hence reformat the user manual information to reflect the experts’ recommendations. To begin, instructions for play- ing the game will be simplified to make them more age-appropriate, as most experts discovered that the user manual uses complex language, creating an impression of difficulty for the players. Second, the experts called for the researchers to develop dis- tinct user manuals for instructors, parents, and students. Additionally, a comprehensive guidebook should be produced for parents who are new to CT. Third, video tutorials should be included as part of the user manual’s content. Finally, gamers should be able to play without consulting the handbook. 4.2 Evaluating the computational concepts in the Robobug board game To evaluate the computational concepts integrated into the Robobug game, we used a survey form with an open-ended comment section derived from Yu and Roque’s 50 http://www.i-jim.org Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… work [19]. Five computational concepts were evaluated in this study: (i) loops, (ii) sequence, (iii) events, (iv) parallelism, and (v) data. Table 3 presents a codebook of the computational ideas examined in this study, while Table 4 summarizes the findings of the evaluation of CT concepts. Table 3. Codebook of computational concepts Theme Description Example and Sample Picture Loops Running the sequence of steps for a task The player can use the Loop function to replace some repeated steps. Sequence Creating a sequence of steps for a task The player can use the direction cards (Turn right, turn left, move forward, etc.) to move the Robobug to get to its goal. Event parallelism One thing causing another thing to happen. Making things happen at the same time The player can decide whether to use a Power card to remove the obstacles or move around them. (Continued) iJIM ‒ Vol. 16, No. 06, 2022 51 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… Theme Description Example and Sample Picture Data Storing, retrieving, and updating values The player can use the ‘Undo’ button to undo a step of the Robobug. Table 4. Evaluation of CT concepts and their summarized results Description Agree Disagree The player can develop Loops (computational concept) by using the Loop function card to replace repeated steps while creating the algorithm. 100% 0% The player can develop a Sequence (computational concept) by using direction cards to move the Robobug to get its Ice Cream/Treat. 100% 0% The player can develop a Sequence (computational concept) by using algorithm cards to recreate the algorithm to move the Robobug to get its Ice Cream/Treat. 100% 0% The player can develop Event Parallelism (computational concept) by deciding whether to use a Power card to remove an obstacle or move around it. 75.0% 25.0% The player can develop Data (computational concept) by using the ‘Undo’ button to undo a step of the Robobug. 62.5% 37.5% The player can develop Data (computational concept) by using algorithm cards to create a set of instructions (algorithm) based on the steps taken to get the Robobug’s Ice Cream/Treat. 87.5% 12.5% The results in Table 4 indicate that all experts concurred that players will be able to develop computational concepts such as loops and sequences while playing the game. When it comes to event parallelism concepts, the majority agreed that players can build this concept by determining whether to utilize a Power card to eliminate or circumvent Table 3. Codebook of computational concepts (Continued) 52 http://www.i-jim.org Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… barriers. However, one expert had a different viewpoint. He claimed that utilizing a Power card to remove or shift impediments is connected more closely to control struc- tures or decision making. The following is his comment: Expert 1: “Parallelism refers to performing several jobs at the same time. Power cards are more related to control structures or decision making.” Players demonstrate parallelism by searching for sequences in two dimensions to maximize their points. Parallelism is also aided by controlling multiple characters’ movements simultaneously or by programming a single figure’s motion, light, and sound effects [32][40]. Consequently, the Robobug board game does not support event parallelism. Parallelism is an important CT concept because it encourages players to monitor their opponents’ planned programming and use it to their advantage [34]. Therefore, to support the CT concept of event parallelism, the researchers will modify the gameplay of Robobug. Over half (62.5%) the experts agreed that users can create data by redoing a Robobug step; however, 37.5 % disagreed. Expert 1 felt that it is difficult to relate ‘Undo’ to data, and that data is tied to coding blocks in the Robobug board game. Another expert advised gamifying the ‘Undo’ option. Data is the process of storing, retrieving, and updating values. Changing a parameter or variable, such as the distance travelled or the number of loop repeats, typically achieves this [19][34]. As a result of this finding, the ‘Undo’ button will get a variable wherein players can reverse actions if needed. With this modification, the ‘Undo’ button will exhibit the data concept. Most experts (87.5 percent) stated that players could master the data concept by using algorithm cards to build a set of instructions (algorithm) based on the steps taken by the Robobugs to get treats. However, one expert held a contrary position, explaining that players might establish the concept of sequence rather than data when using algo- rithm cards in this manner. Sequences are implemented by writing a series of actions for a robot or sprite [19][34], which in this case, is the series of Robobug-specific actions or instructions. Hence, we will take the experts’ feedback into account in the next phase of building the Robobug board game. 4.3 Evaluating the computational practices in Robobug board game Five computational practices in the Robobug board game were reviewed and eval- uated by the experts: (i) experimenting and interacting, (ii) testing and debugging, (iii) abstracting and modularizing, (iv) pattern recognition, and (v) algorithm. Table 5 shows Robobug’s computational practices, which were adapted from the codebook of Yu and Roque [19]. The findings of the evaluation of CT practices are summarized in Table 6. iJIM ‒ Vol. 16, No. 06, 2022 53 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… Table 5. Codebook of computational practices Theme Description Example and Sample Picture Experimenting and interacting Developing a little bit, then trying it out, then developing more Robobug allows the player to keep adding or deleting steps for the Robobug Testing and debugging Making sure things work—and finding and solving problems when they arise The player can test if they can control the Robobug’s movement to the target square (ice cream) using as few steps as possible with or without deploying the Undo/Power cards. Abstracting and modularizing Exploring connections between the whole and the parts Robobug allows the player to use Power cards to simplify steps by keeping only the essential steps and avoiding/ overcoming unnecessary obstacles. (Continued) 54 http://www.i-jim.org Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… Theme Description Example and Sample Picture Pattern Recognition Observing the movements of Robobug throughout game time and recognizing its pattern. The player can reach the ice cream (goal) with as few steps as possible by recognizing the pattern. Algorithm Developing step-by- step instructions to complete all levels in as few steps as possible. The player can use the loop function to create repeatable patterns. Table 6. Evaluation of CT practices and their summarized results Description Agree Disagree The player can practice CT (experimenting and interacting) by adding or deleting steps for the Robobug. 100% 0% The player can practice CT (experimenting and interacting) by combining the coding blocks to create an algorithm. 100% 0% The Undo/Power cards enable the player to practice CT (testing and debugging) by controlling the Robobug’s movement to the target square (ice cream) in as few steps as possible. 75% 25% The player can practice CT (testing and debugging) by rearranging the coding blocks to form a set of instructions. 100% 0% Table 5. Codebook of computational practices (Continued) (Continued) iJIM ‒ Vol. 16, No. 06, 2022 55 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… Description Agree Disagree The Power card enables the player to practice CT (abstracting and modularizing) by simplifying the steps and keeping only the essential steps to avoid/overcome unnecessary obstacles. 87.5% 12.5% The player can practice CT (abstracting and modularizing) by using the Loop card to replace repeated steps. 100% 0% The player can practice CT (pattern recognition) by recognizing the pattern so that the bugs can reach the ‘ice cream’ with as few steps as possible. 87.5% 12.5% The player can practice CT (algorithm) by using the coding blocks to represent the steps of the Robobug after the game. 100% 0% In terms of CT practices, most of the experts agreed with the survey items that encompass the five computational practices fostered by the Robobug board game, as shown in Table 5. Based on the table, only a fraction of the experts disagreed on three items, namely that (i) the Undo/Power cards enable players to practice testing and debugging (25%), (ii) the Power card enables players to practice abstracting and modu- larizing (12.5%), and (iii) players can practice pattern recognition by using as few steps as possible for the bugs to reach the target. Expert 5 opposed the notion that the usage of the Undo/Power cards allows players to exercise CT (testing and debugging) by directing the Robobug to go as swiftly as possible to the target square (ice cream). He expressed that incorporating obstacles into the board game facilitates users in improving their debugging and critical thinking abilities. Because Expert 6 was unfamiliar with the Power Card’s function, she was doubtful about whether it will develop players’ CT (abstracting and modularizing). Given this situation, the researchers intend to update the manuals with extra details and examples to help users understand the board game’s Power card function. Expert 2 also felt that pattern recognition is more about intricate loops than it is about completing an objective in the fewest possible steps. Additional modifications will thus be made in the future stage, in response to the experts’ suggestions and evaluations. 5 Conclusion A panel of specialists evaluated the design and features of the Robobug board game, as well as the extent to which these features facilitate the exploration of computational concepts and practices. The expert panel’s reviews revealed new paths and opportunities for improving the Robobug game’s design and features to more effectively develop primary school kids’ CT abilities. Due to the fact that this was a preliminary study, its conclusions are limited to the expert panel’s reviews and analyses of the game’s design and development. However, these findings inform future evaluations from the standpoint of consumers, such as students, teachers, and parents. Additionally, they may help developers bridge the knowledge divide between their perspectives and users’ needs. The outcomes of this Table 6. Evaluation of CT practices and their summarized results (Continued) 56 http://www.i-jim.org Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… study further assist instructional designers in developing aspects of board games that are specifically geared to CT, based on the experts’ viewpoints. This board game should next be assessed by students and instructors in a school context as well as by parents in an informal context. This is consistent with one of the expert’s recommendations that “experts of quality control should be employed to test and beta-test the board game with actual school children and teachers in order to uncover any problems.” Given that the Covid-19 pandemic has resulted in school closures and movement restrictions, evalu- ation by prospective consumers will take place after the board game has been updated as per the study’s findings. We hope that educators will be able to use the Robobug board game to foster collab- orative and game-based learning while teaching CT skills. Both strategies have been demonstrated to improve student interaction and learning outcomes in formal and infor- mal settings [5][17][21][25][34]. This unplugged approach may also be beneficial in remote areas where schools lack computers and equipment. Overall, this study serves as an introduction to the full development of the Robobug board game. It is critical for improving the Robobug’s design and functionality as an unplugged tool and alternative technique for fostering CT among students. 6 References [1] Almeida, J.L.F. & Machado, L.D.S. (2021). Design requirements for educational seri- ous games with focus on player enjoyment. Entertainment Computing, 38. https://doi. org/10.1016/j.entcom.2021.100413 [2] Angeli, C. & Valanides, J. (2019). Developing young children’s computational thinking with educational robotics: an interaction effect between gender and scaffolding strategy. Comput- ers in Human Behavior, https://doi.org/10.1016/j.chb.2019.03.018 [3] Anuar, N., Suraya, M.F., Minoi, & Lynn, J. (2020). Contextualizing computational thinking: a case study in remote rural sarawak borneo. International Journal of Learning, Teaching and Educational Research, 19, 98–116. https://doi.org/10.26803/ijlter.19.8.6 [4] Bayeck, R.Y. (2020). Examining board gameplay and learning: a multidisci- plinary review of recent research. Simulation & Gaming, 51(4), 411–431. https://doi. org/10.1177/1046878119901286 [5] Berland, M. & Lee, V.R. (2011). Collaborative strategic board games as a site for distrib- uted computational thinking. International Journal of Game-Based Learning. 1(2), (2011) 65–81. https://doi.org/10.4018/ijgbl.2011040105 [6] Braun, V., Clarke, V., & Gray, D. (2017). Innovations in qualitative methods. In the palgrave handbook of critical social psychology (pp. 243–266). Palgrave Macmillan UK. https://doi. org/10.1057/978-1-137-51018-1_13 [7] Braun, V., Clarke, V., Boulton, E., Davey, L., & McEvoy, C. (2020). The online survey as a qualitative research tool. International Journal of Social Research Methodology, 1–14. [8] Bree, R.T. & Gallagher, G. (2016). Using microsoft excel to code and thematically anal- yse qualitative data: a simple, cost-effective approach. All Ireland Journal of Higher Education, 8(2). [9] Brenner, M. (1985). The research interview: uses and approaches. London: Academic Press. [10] Choi, H. & Kim, M.S. (2017). A complementary approach of three methods for computa- tional thinking assessment. Journal of the Korean association of information education, 21(6), 639–646. https://doi.org/10.14352/jkaie.21.6.639 iJIM ‒ Vol. 16, No. 06, 2022 57 https://doi.org/10.1016/j.entcom.2021.100413 https://doi.org/10.1016/j.entcom.2021.100413 https://doi.org/10.1016/j.chb.2019.03.018 https://doi.org/10.26803/ijlter.19.8.6 https://doi.org/10.1177/1046878119901286 https://doi.org/10.1177/1046878119901286 https://doi.org/10.4018/ijgbl.2011040105 https://doi.org/10.1057/978-1-137-51018-1_13 https://doi.org/10.1057/978-1-137-51018-1_13 https://doi.org/10.14352/jkaie.21.6.639 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… [11] Chiazzese, G., Arrigo, M., Chifari, A., Lonati, V., & Tosto, C. (2019). Educational robotics in primary school: measuring the development of computational thinking skills with the Bebras tasks. In Informatics, 6(4), 43. https://doi.org/10.3390/informatics6040043 [12] Dagiene, V. & Stupuriene, G. (2016). Bebras—a sustainable community building model for the concept based learning of informatics and computational thinking. Informatics in Edu- cation, 15(1), 25–44. https://doi.org/10.15388/infedu.2016.02 [13] De Almeida, J.L.F. & Dos Santos Machado, L. (2021). Design requirements for educational serious games with focus on player enjoyment. Entertainment Computing, 38. https://doi. org/10.1016/j.entcom.2021.100413 [14] Etikan, I., Musa, S.A., & Alkassim, R.S. (2016). Comparison of convenience sampling and purposive sampling. American journal of theoretical and applied statistics, 5(1), 1–4. https://doi.org/10.11648/j.ajtas.20160501.11 [15] González, M., Moreno-León, J., & Robles, G. (2019). Combining assessment tools for a com- prehensive evaluation of computational thinking interventions. In Computational thinking education (pp. 79–98). Springer, Singapore. https://doi.org/10.1007/978-981-13-6528-7_6 [16] Harris, C. (2018). Computational thinking unplugged: comparing the impact on confidence and competence from analog and digital resources in computer science professional devel- opment for elementary teachers: a education doctoral dissertation in executive leadership. Retrieved from library in St. John Fisher College (Paper 374). https://fisherpub.sjfc.edu/ education_etd/374 [17] Hsu, T.C., Chang, S.C., & Hung, Y.T. (2018). How to learn and how to teach computa- tional thinking: Suggestions based on a review of the literature. Computer & Education, 126, 296–310. https://doi.org/10.1016/j.compedu.2018.07.004 [18] Hussin, A.A. (2018). Education 4.0 made simple: Ideas for teaching. International Journal of Education and Literacy Studies, 6(3), 92–98. https://doi.org/10.7575/aiac.ijels.v.6n.3p.92 [19] Yu, J., & Roque, R. (2019). A review of computational toys and kits for young children. International Journal of Child-Computer Interaction, 21, 17–36. https://doi.org/10.1016/j. ijcci.2019.04.001 [20] Jamal, N.N., Abang Jawawi, D.N., Hassan, R., & Mamat, R. (2021). Conceptual model of learning computational thinking through educational robotic. International Journal of Emerging Technologies in Learning (iJET), 16(15), pp. 91–106. https://doi.org/10.3991/ijet. v16i15.24257 [21] Kuo, W.C. & Hsu, T.C. (2019). Learning computational thinking without a computer: how computational participation happens in a computational thinking board game. The Asia-Pacific Education Researcher, 29(1), 67–83. https://doi.org/10.1007/s40299-019-00479-9 [22] Lockwood, J. & Mooney, A. (2018). Developing a computational thinking test using bebras problems. In: CC-TEL 2018 and TACKLE 2018 Workshops. [23] Menon, D., Romero, M., & Viéville, T. (2019). Computational thinking development and assessment through tabletop escape games. International Journal of Serious Games, Serious Games Society, 6. https://doi.org/10.17083/ijsg.v6i4.319 [24] Ministry of Education. (2017). “Teknologi Maklumat dan Komunikasi Enam,” B.P. K. K. P. MALAYSIA, Ed., Pusat Pentadbiran Kerajaan Persekutuan. [25] Yunus, E., & Zaibon, S.B. (2021). Connecting computational thinking (CT) concept with the game-based learning (GBL) elements. International Journal of Interactive Mobile Technol- ogies (iJIM), 15(20), pp. 50–67. https://doi.org/10.3991/ijim.v15i20.23739 [26] Nabhani, S., Harrap, N., Ishtiaq, S., Ling, V., Dudzinski, M., Greenhill, D., Caton,H., Philip, N., Wells, J., & Kayyali, R. (2020) Development and evaluation of an educational game to support pharmacy students. Pharmacy Teaching and Learning, 12(7), 786–803. https://doi.org/10.1016/j.cptl.2020.02.006 58 http://www.i-jim.org https://doi.org/10.3390/informatics6040043 https://doi.org/10.15388/infedu.2016.02 https://doi.org/10.1016/j.entcom.2021.100413 https://doi.org/10.1016/j.entcom.2021.100413 https://doi.org/10.11648/j.ajtas.20160501.11 https://doi.org/10.1007/978-981-13-6528-7_6 https://fisherpub.sjfc.edu/education_etd/374 https://fisherpub.sjfc.edu/education_etd/374 https://doi.org/10.1016/j.compedu.2018.07.004 https://doi.org/10.7575/aiac.ijels.v.6n.3p.92 https://doi.org/10.1016/j.ijcci.2019.04.001 https://doi.org/10.1016/j.ijcci.2019.04.001 https://doi.org/10.3991/ijet.v16i15.24257 https://doi.org/10.3991/ijet.v16i15.24257 https://doi.org/10.1007/s40299-019-00479-9 https://doi.org/10.17083/ijsg.v6i4.319 https://doi.org/10.3991/ijim.v15i20.23739 https://doi.org/10.1016/j.cptl.2020.02.006 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… [27] Palts, T. & Pedaste, M. (2017, June). Tasks for assessing skills of computational thinking. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (pp. 367–367). https://doi.org/10.1145/3059009.3072999 [28] Papadakis, S. & Kalogiannakis, M. (2019). Evaluating the effectiveness of a game-based learning approach in modifying students’ behavioural outcomes and competence, in an introductory programming course. A case study in Greece. International Journal of Teach- ing and Case Studies, 10(3), 235–250. https://doi.org/10.1504/IJTCS.2019.10024369 [29] Papadakis, S. (2020). Robots and robotics kits for early childhood and first school age. International Journal of Interactive Mobile Technologies (iJIM), 14(18), pp. 34–56. https:// doi.org/10.3991/ijim.v14i18.16631 [30] Papadakis, S. & Kalogiannakis, M. (2020). Exploring preservice teachers’ attitudes about the usage of educational robotics in preschool education. In Handbook of Research on Tools for Teaching Computational Thinking in P-12 Education (pp. 339–355). IGI Global. https:// www.igi-global.com/gateway/chapter/257125#pnlRecommendationForm; https://doi.org/ 10.4018/978-1-7998-4576-8.ch013 [31] Papadakis, S., Vaiopoulou, J., Sifaki, E., Stamovlasis, D., Kalogiannakis, M., & Vassilakis, K. (2021). Factors that hinder in-service teachers from incorporating educational robotics into their daily or future teaching practice. In CSEDU (2) (pp. 55–63). https://doi. org/10.5220/0010413900550063 [32] Papadakis, S. & Kalogiannakis, M. (2020). Learning computational thinking development in young children with Bee-Bot educational robotics. In Handbook of research on tools for teaching computational thinking in P-12 education (pp. 289–309). IGI Global. https://doi. org/10.4018/978-1-7998-4576-8.ch011 [33] Richey, R. & Klein, J.D. (2007). Design and development research: methods, strategies, and issues. Lawrence Erlbaum Associates. [34] Scirea, M. & Valente. A. (2020). Board games and computational thinking: how to identify games with potential to support CT in the classroom. Proceedings of the 15th International Conference on the Foundations of Digital Games, 114, 1–8. https://doi. org/10.1145/3402942.3409616 [35] Shahroom, A.A. & Hussin, N. (2018). Industrial revolution 4.0 and education. International Journal of Academic Research in Business and Social Sciences, 8(9), 314–319. https://doi. org/10.6007/IJARBSS/v8-i9/4593 [36] Shute, V.J., Chen, S., & Clarke, J.A. (2017). Demystifying computational thinking. Educa- tional Research Review, 22, 142–158. https://doi.org/10.1016/j.edurev.2017.09.003 [37] Sullivan, A., Bers, M., & Pugnali, A. (2017). The impact of user interface on young chil- dren’s computational thinking. Journal of Information Technology Education: Innovations in Practice, 16(1), 171–193. https://doi.org/10.28945/3768 [38] Tatar, C. & Eseryel, D. A literature review: fostering computational thinking through game- based learning in K-12. The Proceedings of AECT’s Convention, 288. [39] Tzagkaraki, E., Papadakis, S., & Kalogiannakis, M. (2021). Exploring the use of edu- cational robotics in primary school and its possible place in the curricula. In Educa- tional Robotics International Conference (pp. 216–229). Springer, Cham. https://doi. org/10.1007/978-3-030-77022-8_19 [40] Ung, L., Saibin, Tammie, Labadin, J., & Aziz, N. (2018). Preliminary investigation: teach- ers’ perception on computational thinking concepts. Journal of Telecommunication, Elec- tronic and Computer Engineering, 9, 23–29. [41] Wahab, N.A., Talib, O., Razali, F., & Kamarudin, N. (2021). The big why of implementing computational thinking in stem education: a systematic literature review. Malaysian Journal of Social Sciences and Humanities (MJSSH), 6(3), 272–289. https://doi.org/10.47405/mjssh. v6i3.706 iJIM ‒ Vol. 16, No. 06, 2022 59 https://doi.org/10.1145/3059009.3072999 https://doi.org/10.1504/IJTCS.2019.10024369 https://doi.org/10.3991/ijim.v14i18.16631 https://doi.org/10.3991/ijim.v14i18.16631 https://www.igi-global.com/gateway/chapter/257125#pnlRecommendationForm https://www.igi-global.com/gateway/chapter/257125#pnlRecommendationForm https://doi.org/10.4018/978-1-7998-4576-8.ch013 https://doi.org/10.4018/978-1-7998-4576-8.ch013 https://doi.org/10.5220/0010413900550063 https://doi.org/10.5220/0010413900550063 https://doi.org/10.4018/978-1-7998-4576-8.ch011 https://doi.org/10.4018/978-1-7998-4576-8.ch011 https://doi.org/10.1145/3402942.3409616 https://doi.org/10.1145/3402942.3409616 https://doi.org/10.6007/IJARBSS/v8-i9/4593 https://doi.org/10.6007/IJARBSS/v8-i9/4593 https://doi.org/10.1016/j.edurev.2017.09.003 https://doi.org/10.28945/3768 https://doi.org/10.1007/978-3-030-77022-8_19 https://doi.org/10.1007/978-3-030-77022-8_19 https://doi.org/10.47405/mjssh.v6i3.706 https://doi.org/10.47405/mjssh.v6i3.706 Paper—Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach… 7 Authors Chong Sin Yoon is a postgraduate student at the University Putra Malaysia’s Faculty of Educational Studies. She is also an Inspiros International School ICT teacher where she primarily teaches KS2 and KS3 Computing subject. She has competed in and won a number of national competitions for innovation in teaching and learning. (email: happysinyooon@yahoo.com). Mas Nida Md. Khambari is a senior lecturer in Learning Technologies and Innovation at the Department of Foundations of Education, Faculty of Educational Studies. Her research interest and specialization include instructional design, inno- vation and emerging technologies in teaching and learning, gamification, alternative assessment and the scholarship of teaching and learning. She is currently the Exec- utive Committee of the prestigious Asia-Pacific Society for Computers in Education (APSCE). At the university level, she is appointed as the Fellow at the Center for Academic Development (CADe). Mas Nida has actively participated and won sev- eral medals at innovative teaching and learning competitions at both the national and international level. She is currently the head of the University’s Virtual and Augmented Immersive Learning Hub (UN+VAIL) which promotes the interest-driven philosophy in learning (email: khamasnida@upm.edu.my). Article submitted 2021-08-15. Resubmitted 2021-12-06. Final acceptance 2021-12-14. Final version published as submitted by the authors. 60 http://www.i-jim.org mailto:happysinyooon@yahoo.com mailto:khamasnida@upm.edu.my