61 Learning Mathematics Formulas by Listening and Reading Worked Examples Wahyuni Eka Maryati1, Endah Retnowati1*, Ng Khar Thoe2 1Department of Mathematic Education, Universitas Negeri Yogyakarta, Indonesia 2Seameo Recsam, Malaysia *Correspondence: E-mail: e.retno@uny.ac.id A B S T R A C T S A R T I C L E I N F O This study aims to examine whether there is a significant difference between the effectiveness of worked examples with voice notes and worked examples without voice notes and their relations with computational thinking skills. Both learning strategies were implemented in the derivative of polynomial algebraic function learning. This quasi- experimental study involved 62 students and employed a pre-test and post-test non-equivalent control group design via WhatsApp group. Data were analyzed using ANCOVA with student initial ability measured during the pretest as the covariate. The study empirically proved that there is a significant difference in terms of the effectiveness of both learning strategies viewed from student cognitive load. Worked example without voice notes was more effective and makes students have less cognitive load during learning. In addition, there was no significant difference in the effectiveness of learning strategies in terms of computational thinking skills. This study showed that adding voice notes may lead to redundancy effects, hence the use voice notes with worked examples should be thoroughly considerated. Β© 2022 Universitas Pendidikan Indonesia Article History: Received 15 Jan 2022 Revised 20 Feb 2022 Accepted 04 Mar 2022 Available online 07 Mar 2022 ____________________ Keyword: Cognitive load, Computational thinking skill, Voice note, WhatsApp group, Worked example. Indonesian Journal of Teaching in Science Journal homepage: http://ejournal.upi.edu/index.php/ IJOTIS/ Indonesian Journal of Teaching in Science 2(1) (2022) 61-74 IJOTIS http://ejournal.upi.edu/index.php/%20IJOTIS/ Maryati et al., Learning Math Formulas by Listening and Reading | 62 DOI: http://dx.doi.org/10. 17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 1. INTRODUCTION The spread of the COVID-19 virus in all countries in the world has an impact on many aspects including education. The distance learning has become the best mode of learning during the pandemic, and though it has ended, the learning mode is still applied using many choices of online platforms. Online learning is conducted using certain learning methods that are under the conditions and situations where the students live albeit in distance. Several examples of online learning have been reported. Alaby (2020) suggested online learning applications and platforms support teaching-learning processes, such as WhatsApp, Edmodo, Google Meet, Zoom, Google Classroom, etc. Sahidillah and Miftahurrisqi (2019) stated that WhatsApp groups are one of the main choices used by several schools to carry out effective and affordable learning activities. This application provides various features that support online learning activities such as group discussions and chat features that allow us to send voice recordings. In the WhatsApp application, users may send texts, pictures, videos, messages, voice messages, and documents in several formats (pptx, docx, xlsx, pdf). Fauziah and Sukasno (2015) pointed out that the changes in paradigms in the learning process provide opportunities for teachers to use and develop various approaches with the orientation of developing students' abilities and skills. The learning activities are expected to involve students actively. It also should be oriented to the core of the learning process and the acquisition of the mathematics concepts. Evenmore, varied activities should be provided to improve students' higher-order thinking skills. It is widely agreed that the understanding of concepts in the cognitive aspects is one of the aims of the mathematics teaching-learning process. Students should have a meaningful understanding of concepts and be active in gaining new knowledge from experiences and knowledge possessed (Sweller, 2020). The indicators of understanding are students' abilities to restate the concepts of a series of words, identify or give examples of a concept, and apply/use the right concept in various conditions. Worked examples efficiently provide problem-solving schemes that might not be stored yet in student’s long-term memory and thus, have a significant impact on lowering cognitive load. Nurazizah et al. (2020), Chen et al. (2020), and Retnowati et al. (2010) indicate that worked examples strategy may reduce the difficulties in learning mathematics as this strategy may support initial cognitive acquisitions and give a preview of concept implementation and mathematics theories. In addition, Mufidah (2019) states that during teaching-learning processes using worked examples, mathematics understanding ability may improve and be in the high category as students get used to solve their problems independently, have initiative, and be not dependent on other people. Worked example is a cognitive load theory-based learning design that can be applied to teach problem-solving (Sweller, 2020). Students' mathematical understanding ability in worked example learning is higher than their ability when taught using guided discovery and scientific learning. Worked examples may provide opportunities for early learners to improve the quality of knowledge obtained from the examples. This is in line with Azizah et al. (2020) who stated that in work examples, students do not need to do trial-error activities because guided discovery and scientific learning trial-error activity cause a massive cognitive load on the working memory. Moreover, Sweller (2020) stated that cognitive load theory emphasizes instructional design to reduce the complexity of unnecessary information to reduce the load on working memory when acquiring new knowledge. Sweller explained that the total cognitive load, consisting of intrinsic and extraneous cognitive content, should not exceed the http://dx.doi.org/10.%2017509/xxxx.xxxx 63 | Indonesian Journal of Teaching in Science, Volume 2 Issue 1 March 2022 Hal 61-74 DOI: http://dx.doi.org/10.17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 capacity of working memory. If the total cognitive load is too high, information processing lags, and learning may stop. If the intrinsic cognitive load is high, the level of extraneous cognitive load can be critical. Working memory storage and processing capabilities alternate with each other. When the memory load increases above the limit, one's performance may decrease. Understanding a concept is not merely understanding information, but students can interpret and then change and manage the information received into a series of words or other forms so that students are supported in solving problems that are considered difficult. However, according to Khotimah et al. (2016), there were found still many students who find difficulties in understanding mathematical concepts. Furthermore, Agustin et al. (2022) analysed that there were many students who has lack of transfer ability of their pre-acquired mathematics concepts. These students are not only failed able to redefine concepts with their own set of words and understand the mathematical concept but also to use them into different contexts. Apriliyanto (2019) stated that conceptual errors in problem-solving are very dominant as students tend to memorize rather than comprehensively understand the concepts to relate them with previously owned concepts. Hasanah (2019) stated that the factors that cause difficulties in understanding and solving advanced problems such as in derivative algebraic functions. This topic is related to arithmetic mastering however many students are not easily able to analyze algebraic function derivation techniques, because do not use the basic concepts of derivatives, and the flow of solving algorithms correctly, make mistakes in completing the reduction procedure, and rush in working out the tasks. Then, some further skills are difficult to possess because students do not understand the mathematics operations well, and there might also mistakes in counting numbers and variables. Moreover, students’ lack of information is caused by mistakes when reading and interpreting questions, making mathematics models properly or manipulating algebraic, and implementing the concepts of the correct derivative. Some mistakes made students might be unable to get into the algorithm level in computational thinking. Supiarmo et al. (2021) argued that scaffolding can help the computational thinking process through questions, instructions, reminders, directions, or encouragement. The worked examples might be seen as scaffolding since theyprovides an overview of possible ways of solving problems in complete, algorithmic, and logical structure. The worked example instruction askes students to study the worked example first, and then to solve the pair problems independently. However, most worked example studies, the learning mode is conducted face-to-face. Such modification in the activity which in this present study is examined is that when the worked examples are studied online through a popular chatting platform, namely WhatsApp group. In this platform, a voice note tool is avaliable and being frequently used among students. The current study compared the worked examples with or without voice notes. With voice nore, students receive learning materials both written a worked example and the verbalised version of the worked example. Moreno and Mayer (2002) stated that verbal redundancy is the simultaneous presentation of texts and narratives containing the same information in different modes of presentation. Multimedia design should provide verbal explanation via text, audio (voice recorder), or both. An effective technique for presenting learning material is to use auditory and visual media simultaneously. Then, according to Moreno and Mayer, both unrequired musical and voice effects may interfere with the auditory canal and reduce effective working memory capacity, thus leading to core material acquisition. This also leads to less working memory to be built to develop coherent verbal representation and connect with other representations. ttp://dx.doi.org/10.17509/xxxx.x Maryati et al., Learning Math Formulas by Listening and Reading | 64 DOI: http://dx.doi.org/10. 17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 therefore, learning materials should complement each other. Specifically, they focused on relevant words from presented texts and narratives, as well as relevant pictures and illustration. They also covered some activities, such as arranging words into coherent verbal coherently, arranging pictures into coherent coherently, and integrating figures and words with possessed knowledge to eliminate redundancy. Voice notes may become a choice to present materials to students through WhatsApp groups as the voice note feature may help students focus on one screen display that contains learning media, such as pictures, file handouts, and student worksheets. This feature may avoid split attention. Sweller (2020) mentioned the effect of split attention occurs when two or more information sources are processed simultaneously to understand the information presented. Although mathematics formulas might be difficult, when the formulas read by teachers, it is hypithesised that then the formulasmight be easily understood. Moreover, voice notes in the learning platform may help students during learning, as the recording can be replayed in their own time. Indeed the Whatsapp group also provides a video call tool for more interactive communication between teacher and students, however students may have problems with unstable connection and more quota for online learning (Yulistyanti et al., 2021). As a result, the teaching-learning processes may not be optimally conducted. This study aims to examine whether there is a significant difference in terms of the effectiveness of worked examples without voice notes and worked examples with voice notes viewed from students' cognitive load and computational thinking skills. After the difference between worked examples with or without voice, notes is revealed, teachers may decide which strategy is better applied. It is important that teachers are informed empirical data of the right strategy to be implemented in classes by considering both cognitive load and computational thinking skills. 2. METHODS 2.1. Research Design This quasi-experimental research used an authentic classroom that had used Whatsapp platform for the main communication during the study was conducted. This study was conducted in one public senior high schools in Yogyakarta, Indonesia (. The teaching schedule and number of participanst is presented in Table 1. Table 1. Learning schedule. Phase Day/Date Time Class Number of participants 1. Introductory Friday, 5 February 2021 07.30-09.00 XI MIPA 1 XI MIPA 2 36 26 2. Acquisition Friday, 19 February 2021 07.30-09.00 XI MIPA 1 XI MIPA 2 36 34 2.3. Research Subject The participating school in this study had six classes consisting of four Mathematics and Natural Sciences and two Social Studies classes. As many as 210 students were in the school, and by the expected error rate is 10%, the number of samples calculated using the Slovin formula is presented in Equation [1]. http://dx.doi.org/10.%2017509/xxxx.xxxx 65 | Indonesian Journal of Teaching in Science, Volume 2 Issue 1 March 2022 Hal 61-74 DOI: http://dx.doi.org/10.17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 𝑛 = 210 210(0.1)2+1 = 67.74193 β‰ˆ 70 (1) Based on the calculations, the expected number of participants is 70. Using a random sampling, two among six classes was recruited. As many as 36 students were selected from XI MIPA 1, and they were taught without voice notes, and 34 students selected from XI MIPA 2 were taught using worked examples with voice notes. However, eight students from XI MIPA 2 did not participate in all learning phases, so there were only 26 students. The number of participants in this study was 62. 2.4. Research Procedure This study went through three stages, namely preparation, implementation, and report development. In terms of operation, this study made use of the pre-test – post-test non- equivalent control group design (see Table 2). Description for Table 2 is in Table 3. This research was conducted following the lesson plan designed. We carried out the pretest to recall students’ knowledge of the limits of polynomial algebra function material. The experiment was initially conducted for only six hours, but then the school provided four hours for assignments done at home. The first meeting was held in 2 meeting hours [2Γ—45 minutes] for the pre-test, learning prerequisite material (limits of polynomial algebraic functions) and presenting definitions of derivatives of polynomial algebraic functions, as well as assignments to finish home [2Γ—45 minutes]. The second meeting was held in 2 hours [2Γ—45 minutes] to discuss derivative properties of polynomial algebraic functions, second derivatives of polynomial algebraic functions, and enrichment material to be completed at home [2Γ—45 minutes]. These materials were provided in a worksheet consisting of worked examples and paired problem solvings, and a handout consisting of some explanations abouth the concepts with figures, and video links on YouTube. These materials were sent through the whatsapp groups, either only in written messages or with voice notes. The third meeting was a post-test on the derivatives of polynomial algebraic functions conducted in 2 hours [2Γ—45 minutes]. Table 2. The experiment design. Class Treatment Pre-test Post-test 𝐸𝐴 𝐴 π‘Œπ΄1 π‘Œπ΄2 𝐸𝐡 𝐡 π‘Œπ΅1 π‘Œπ΅2 Table 3. Description of the symbol used in Table 2. Symbol Note 𝐸𝐴 The experimental class taught using worked examples without voice notes [XI MIPA 1] 𝐸𝐡 The experimental class taught using worked examples with voice notes [XI MIPA 2] 𝐴 Worked example learning strategy without voice note 𝐡 Worked example learning strategy without voice note π‘Œπ΄1 Pre-test of basic of polynomial algebra function material [XI MIPA 1] π‘Œπ΅1 Pre-test of basic of polynomial algebra function material [XI MIPA 2] π‘Œπ΄2 Post-test on computational thinking skills in limits of polynomial algebra function material [XI MIPA 1] π‘Œπ΅2 Post-test on computational thinking skills in limits of polynomial algebra function material [XI MIPA 2] ttp://dx.doi.org/10.17509/xxxx.x Maryati et al., Learning Math Formulas by Listening and Reading | 66 DOI: http://dx.doi.org/10. 17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 Learning activities were carried out through two different WhatsApp groups. In experimental class 1 WhatsApp group and online chat features were used. Then, experimental class 2 was taught using WhatsApp group and voice notes, in which the online chat and voice notes were sent to students in the form of instructions and explanations. Each experimental class focused on one learning media only:experimental class 1 used the online written chat while experimental class 2 used voice notes only. This was done to make students stay focused on the messages. In addition, because learning activities in both classes were carried out at the same time, there were two instructors who monitored the teaching-learning process and sent the learning instructions in sequence as prepared by the researchers including that the voice notes were recorded in advanced prior the experiment was conducted. When a student asked a question in experimental class, the instructors would provide an answer or explanation of the material that will be sent back to the student in the form of written only or voice notes. The chart of the learning procedure is presented in Figure 1. Figure 1. The scheme of research procedure . 2.5. Data Collection Technique The instruments to collect the data in this study are explained below. (i) Observation sheets were made and distributed to each mathematics teachers during learning. This instrument included several statements on teacher activity and students during the class, including the introduction, core, and closing activities. The observation data was used to oversee whether the learning procedure is applied as planned. (ii) The test contained essay questions about the limits of polynomial algebraic functions (for the pre-test) and the derivatives of polynomial algebraic functions (for the post-test). http://dx.doi.org/10.%2017509/xxxx.xxxx 67 | Indonesian Journal of Teaching in Science, Volume 2 Issue 1 March 2022 Hal 61-74 DOI: http://dx.doi.org/10.17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 The hypothesis testing was performed using ANCOVA and when interaction effects are found, independent samples t-test was used. The decisions taken based on ANCOVA results are explained as follows. H0 is rejected if the value of Sig. learning strategy is less or equal than 0.05, meaning there is a significant difference in the effectiveness of the two types of learning strategies. When there are significant differences, the means and standard of deviation scores were used to determine which learning strategies are more effective. There were two dependant variables measured: (i) Cognitive load. A subjective rating (asking the level of difficulty) in a 9-point Likert’s scale was given on every problem solving during the tests. If the mean value of the cognitive load is lower and the standard deviation is lower in one experimental class, the class implements a more effective learning strategy. The low value of cognitive load indicates that students do not experience any cognitive overload. (ii) Computational Thinking Skill. All questions in the computational thinking test may be categorised as higher cognitive level than those given during the acquisition phase. Six components in each answer were scored: includes (i) problem formulation by utilizing technology to solve it, (ii) logically compiling and analyzing data, (iii) abstraction through data representation, models, and simulation, (iv) automation of algorithms, (v) improving planning through efficient steps, and (vi) generalizing the problem-solving process. If the mean value of computational thinking skills measured in the post-test is higher and the standard deviation is smaller in one of the experimental classes, then that class has a more effective learning strategy. H0 is accepted if the value of Sig. learning strategy is greater than 0.05 so there is no significant difference in the effectiveness of the two types of learning strategies. Since there is no significant difference, it is not necessary to review the mean and standard deviation of the results of descriptive cognitive load analysis and computational thinking ability tests. A partial eta squared (Ξ·_p^2) which has a great influence on student cognitive load and computational thinking skills were calculated by controlling the students' pretest learning achievement. Table 4 is the category of the effect size as suggested by Cohen (Lakens, 2013). Assumptions tests were also run using the Levene's test for equality of variance. Table 4. The category of effect size. Interval Category 0.01 β‰₯ πœ‚2 > 0.06 Low 0.06 β‰₯ πœ‚2 > 0.14 Intermediate πœ‚2 β‰₯ 0.14 High 3. RESULTS The observation informed that learning implementation using the worked example strategy with WhatsApp voice note in the XI MIPA 1 class and learning implementation using the worked example strategy without WhatsApp voice notes in the XI MIPA 2 class was 100% met the lesson planning. The percentages show that all learning activities in both experimental classes have been done following the plan. 3.1. The Cognitive Load Levels There is a difference in the average cognitive load score, in which worked example strategy without voice note (M = 3.244; SD = 0.797) is lower than the worked example with voice note ttp://dx.doi.org/10.17509/xxxx.x Maryati et al., Learning Math Formulas by Listening and Reading | 68 DOI: http://dx.doi.org/10. 17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 (M = 3.831; SD = 1.250). Then, to test whether the difference in the average cognitive load is significant, a statistical test is performed using ANCOVA. The following is a table of IBM SPSS Statistics 26 output results (see Table 5). Based on Table 5, the Sig. value for strategy is 0.037. It is smaller than 0.05. This means that H0 is rejected, so it can be concluded that there is a significant difference in the effectiveness of the two types of learning strategies in terms of cognitive load with an effect of 7.1% in the intermediate category. Because there is a significant difference, determining a more effective learning strategy may be done by comparing the results of descriptive analysis of cognitive load in terms of the mean and standard deviation values between the two experimental classes. Table 5. Test of between-subjects effects cognitive load. Source Type III sum of square df Mean square F Sig. Partial eta squared Corrected model 5.379 2 2.690 2.598 0.083 0.081 Strategi 4.702 1 4.702 4.542 0.037 0.071 The mean value of cognitive load scores of students learning using the worked example without voice note (M = 3.244; SD = 0.797) is lower than the mean value of the cognitive load of students learning using worked example with voice note (M = 3.831, SD = 1.250). Therefore, the worked example strategy without voice notes tends to be more effective. The smaller mean value of the cognitive load score means that students do not experience cognitive overload and are easy to follow in learning activities. Then, to show whether pre-test data have an effect or not, an independent samples t-test is carried out. In Levene's test for equality of variance, the Sig value obtained is 0.094, which is greater than 0.05. It indicates that the variance of the pre-test data in the two experimental classes is homogeneous. Based on the independent samples t-test, the value of t-count is -1.361. This shows that the t-count is negative because the average value of the pre-test strategy of worked example without voice note (84.72) is lower than the average value of the pre-test strategy of worked example with voice note (87.92). Then, the obtained value of Sig. (2-tailed) is 0.178 > 0.05; it means that H0 is accepted. Therefore, there is no significant difference between the average value of the strategy class worked example without voice notes and worked example with voice notes. Thus, it can be said that the pre-test data independently does not affect the results of the ANCOVA cognitive load test. The "significant difference" obtained from the results of the ANCOVA cognitive load test is not due to the difference in the pre-test score scale (1-100) with the cognitive load scale (1-9), but it is due to the original treatments of cognitive load data with no influence of pre-test data. 3.2. The Computational Thinking Skills Levels To test whether the difference in the mean value of the computational thinking ability test was significant, a statistical test was conducted using ANCOVA. The following is a table of IBM SPSS Statistics 26 output results (see Table 6). Based on Table 6, the pretest data function as the covariate, and it is found that the Sig. pre-test value is 0.111 (greater than 0.05). This means that H0 is accepted, and it can be concluded that there is no linear relationship between the results of the pretest and computational thinking skills. Then, it is obtained that the value of Sig. Strategy is 0.298 (greater than 0.05). http://dx.doi.org/10.%2017509/xxxx.xxxx 69 | Indonesian Journal of Teaching in Science, Volume 2 Issue 1 March 2022 Hal 61-74 DOI: http://dx.doi.org/10.17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 This means that H0 is accepted, and there is no significant difference in the effectiveness of the two types of learning strategies in terms of computational thinking ability with an effect size of 1.8% (low). Since there is no significant difference, it is not necessary to review the mean and standard deviation of the results of the descriptive analysis of the computational thinking ability test. Table 6. Test of between-subjects effects of computational thinking skills. Source Type III sum of square df Mean square F Sig. Partial eta squared Corrected model 114.584 2 57.292 1.617 0.207 0.052 Pre-test 92.968 1 92.968 2.624 0.111 0.043 Strategi 30.039 1 30.039 1.102 0.298 0.018 4. DISCUSSION The ANCOVA tests revealed a significant difference in the effectiveness of the two types of learning strategies in terms of cognitive load with an effect size of 7.1% (intermediate). Because there was a significant difference, determining the effective strategy can be done by comparing the results of descriptive analysis of cognitive load (mean and standard deviation) between the two experimental classes. The average cognitive load of worked examples without voice notes was smaller than the same strategy with voice notes added. Therefore, worked examples without voice notes are considered more effective. The initial ability of students regarding the limit of polynomial algebraic functions is stored in long-term memory. When students processed new information about the derivatives of polynomial algebraic functions, which was more complex, the working memory will automatically recall or retrieve the possessed knowledge related to the derivatives of polynomial algebraic functions including the basic limits of polynomial algebraic functions, algebraic arithmetic operations, factoring, elimination, and substitution steps. Then, with the knowledge they already have, students recognize and construct new schema between existing knowledge and new knowledge received. The initial ability in the form of prerequisite knowledge has an important role for students to build new knowledge. If students do not able to remember, connect or do not have any initial knowledge which is now a prerequisite, working memory will use random search to try the problem solutions and even struggle in making meaning of new concepts being acquired, and consequently students might experience cognitive overload. This study prooved that the worked example strategy with online written chat only was more effective because it can minimize redundancy and split attention so that students do not experience cognitive overload. Worked examples provide a significant difference when online learning activities using written media was used in terms of the cognitive load. Students are motivated to pay attention to written messages and instructions. The worked example helps them prepare for the assigned followed-up problem solving activities. Students are more active in thinking since the worked examples direct their way of constructing the new concepts. This process encourages students to understand the content of problem- solving acccordingly, the learning activities become more directed and effective. The worked example with voice notes seems likely requires visual senses to read the worked examples and auditory to listen to voice notes. These two psychological aspects were proposed so that the worked example works more effectively. However, the data indicated ttp://dx.doi.org/10.17509/xxxx.x Maryati et al., Learning Math Formulas by Listening and Reading | 70 DOI: http://dx.doi.org/10. 17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 that students who use voice notes experience a higher cognitive load than students who do not use voice notes. It might possibly due to redundancy where students need to think twice to match both media used. As suggested, redundancy hinders learning (Moreno & Mayer, 2002; Sweller, 2020). The voice notes were given along with the sections in the student worksheets and handouts for derivatives of polynomial algebraic functions.. Students need to press the play button and listen to the voice note first, and then they should match the information provided by us in the voice note with the parts on the student worksheets so that they can follow the learning activities properly. This activity somehow makes the students put less focus on the lesson, but perhaps, more on activities irrelevant to learning, such as matching the content between the given learning resources. Besides, there is a possibility that mathematics symbols and formulas for derivatives are very interactive yet abstract, causing higher complexity. This might have added that the voice notes provided not better than mere written texts which was prepared systematically presented to be easily understood. Another technical aspects, though not surveyed during the experiment, some students might have insufficient storage space in their cellphones, thus the voice notes might not be downlowded in full. This study suggets that teachers should be careful when deciding the use of voice notes with worked examples in learning on the derivative material of polynomial algebraic functions using online chat platforms. The ANCOVA test results indicated no significant difference in the effectiveness of the two types of learning strategies in terms of computational thinking ability In line with the results of this study, Sung et al. (2017) stated that there is no difference in the initial knowledge of students who do not have previous experience with programming tools or those who know programming concepts. Cahdriyana and Rido (2020) stated that in learning mathematics, computational thinking skills can be applied by giving questions to students for practice. By doing so, students are used to thinking logically and coherently with clear steps and procedures (algorithms) to implement. Moreover, they may become familiar with calculations (computing) as well as determining the right strategy, and they are oriented to problem-solving. The ability to think computationally in solving problems requires systematic coherence so that no steps are missed in the compiled algorithm. Wing (2008) stated that computational thinking skills refer to the use of an approach to solve problems, design systems, and understand the behavior of students by referring to the basic concepts of computing. The current study informed that the computational skills might be developed using both learning strategies, worked examples with or without voice notes. Guidelines for classifying computational thinking skills by Lestari & Annizar (2020) are presented in Table 7.Based on the results of the descriptive analysis, the average score of the pretest in the class taught using voice notes is 87.92, and the class's computational thinking skill average score is 91.69 (high). Then, based on the descriptive analysis, it is found that the average score of pretest in the class taught using chat only is 84.72 and its’ computational thinking skill average score is 92.89 (high). Based on the score category in the pretest and computational thinking skills in each experimental class, there is a significant improvement in terms of computational thinking skills when worked examples strategy is used either with or without voice notes. Noorfitriani and Rosyid (2020) stated that there is an increase in students' understanding ability in the high category. when taught using worked examples Figure 2 shows the students’ works in each experimental class. http://dx.doi.org/10.%2017509/xxxx.xxxx 71 | Indonesian Journal of Teaching in Science, Volume 2 Issue 1 March 2022 Hal 61-74 DOI: http://dx.doi.org/10.17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 Table 7. The classification of computational thinking skills. Range Classification 66.7 ≀ π‘π‘–π‘™π‘Žπ‘– ≀ 100 High 33.4 ≀ π‘π‘–π‘™π‘Žπ‘– ≀ 66.6 Moderate 0 ≀ π‘π‘–π‘™π‘Žπ‘– ≀ 33.3 Low It was found that the students can break down the problem into a simpler form. The students can identify the function of f(x) and f(x+h) from the questions presented as a source of searching for the first derivative by the definition given. Then, the students can distinguish how to determine the first derivative of a polynomial algebraic function from its’ definition and the way the derivative properties of polynomial algebraic functions are marked by writing the first derivative formula. The students also show the right steps to solve the problem. They are also able to use previous knowledge by limiting polynomial algebraic functions, breaking down the form of algebraic factors, completing calculations to final results, and making conclusions. This shows that students solve problems coherently, and the algorithm used is clear. These has added more empirical data to those previously found (Sweller, 2020) that the worked example based instructions, particularly for studying advanced mathematics material, could assist students to construct new computational skills. Figure 2. The result of the post-test conducted in the experimental class (class without voice notes). ttp://dx.doi.org/10.17509/xxxx.x Maryati et al., Learning Math Formulas by Listening and Reading | 72 DOI: http://dx.doi.org/10. 17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 Figure 3. The result of the post-test conducted in the experimental class (class with voice notes). 5. CONCLUSION Worked examples without voice notes may improve learning an advanced mathematics. In this case, learning by reading the mathematics worked examples and listening to the voice notes simultaneousy may cause redundancy and significant higher cognitive load. Although the long display of chat may result in split attention as students are required to frequently scroll the information up to reread the information given, the instructions, worked example and paired problem solving, assist students in learning. This study suggests us to be more considerable when adding verbal notes to written worked examples, particularly when the learning material is full of mathematics symbols and operations. Nevertheless, the study might direct an important followed-up research to be done that is how should advanced mathematics material could be learned more effectively when students and the teacher is in distance. 6. AUTHORS’ NOTE The authors declare that there is no conflict of interest regarding the publication of this article. The authors confirmed that the paper was free of plagiarism. http://dx.doi.org/10.%2017509/xxxx.xxxx 73 | Indonesian Journal of Teaching in Science, Volume 2 Issue 1 March 2022 Hal 61-74 DOI: http://dx.doi.org/10.17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 7. REFERENCES Agustin, A., Retnowati, E., and Ng, K. (2022). The Transferability Level of Junior High School Students in Solving Geometry Problems. Journal of Innovation in Educational and Cultural Research, 3(1), 59-69. Alaby, M. A. (2020). Media sosial whatsapp sebagai media pembelajaran jarak jauh mata kuliah ilmu sosial budaya dasar (ISBD). Jayapangus Press, 3(2), 273-289. Apriliyanto, B. (2019). Analisis kesalahan siswa dalam pemecahan masalah turunan fungsi aljabar. Jurnal Komunikasi Pendidikan, 3(2), 117-125. Azizah, N., Rosyid, A., and Noorfitriani, I. (2020). Perbandingan kemampuan pemahaman matematis siswa melalui penerapan pembelajaran guided discovery, worked example, dan scientific. Supremum Journal of Mathematics Education, 4(2), 159-169. Cahdriyana, R. A. and Richardo, R. (2020). Berpikir komputasi dalam pembelajaran matematika. Jurnal Ilmu Pendidikan, 11(1), 50-56. Chen, O., Retnowati, E. and Kalyuga, S. (2020), Element interactivity as a factor influencing the effectiveness of worked example–problem solving and problem solving–worked example sequences. British Journal of Educational Psychology, 90, 210-223. Fauziah, A. and Sukasno, S. (2015). Pengaruh model missouri mathematics project (MMP) terhadap kemampuan pemahaman dan pemecahan masalah matematika siswa SMA N I Lubuklinggau. Jurnal Ilmiah Program Studi Matematika STKIP Siliwangi Bandung, 4(1), 10-21. Hasanah, H. (2019). Analisis kesulitan mahasiswa dalam menyelesaikan soal turunan fungsi aljabar. Jurnal Industri dan Teknologi Terpadu, 2(1), 76-84. Khotimah, K., Yuwono, I., and Rahardjo, S. (2016). Kesulitan siswa dalam menyelesaikan soal perbandingan trigonometri. Prosiding Seminar Nasional Pendidikan Matematika, Indonesia, 1, 46-52. Lakens D (2013) Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontier in Psychology, 4, 863. Lestari, A. C. and Annizar, A. M. (2020). Proses berpikir kritis siswa dalam menyelesaikan masalah PISA ditinjau dari kemampuan berpikir komputasi. Jurnal Kiprah, 8(1), 46–55. Moreno, R. and Mayer, R. E. (2002). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology, 94(1), 156–163. Mufidah, S. (2019). Upaya meningkatkan kemandirian belajar siswa dalam pembelajaran matematika menggunakan worked example pada siswa kelas VIII J SMP Negeri 5 Yogyakarta tahun pelajaran 2018/2019. Prosiding Sendika Bidang Pendidikan Matematika, Indonesia, 5, 346-351. Noorfitriani, I. and Rosyid, A. (2020). Peningkatan kemampuan pemahaman matematis siswa menggunakan worked example. Jurnal Matematika dan Pendidikan Matematika, 5(1), 26-36. ttp://dx.doi.org/10.17509/xxxx.x Maryati et al., Learning Math Formulas by Listening and Reading | 74 DOI: http://dx.doi.org/10. 17509/xxxx.xxxx p- ISSN 2776-6101 e- ISSN 2776-6152 Nurazizah, M., Santosa, C. A. H. F., and Pamungkas, A. S. (2020). Pengaruh model pembelajaran numbered heads together dengan strategi worked-example terhadap kemampuan penalaran siswa. Majalah Ilmiah Kependidikan, 4(1), 1-7. Retnowati, E., Ayres, P., & Sweller, J. (2010). Worked Example Effects in Individual and Group Work Settings. Educational Psychology, 30(3), 349-367. Sahidillah, M. W. and Miftahurrisqi, P. (2019). Whatsapp sebagai media literasi digital siswa. Varia Pendidikan, 31(1), 52-57. Sung, W., Ahn, J., and Black, J. B. (2017). Introducing computational thinking to young learners: Practicing computational perspectives through embodiment in mathematics education. Technology, Knowledge, and Learning, 22, 443-463. Supiarmo, M. G., Mardhiyatirrahmah, L., and Turmudi, T. (2021). Pemberian scaffolding untuk memperbaiki proses berpikir komputasi siswa dalam memecahkan masalah matematika. Jurnal Pendidikan Matematika, 5(1), 368-382. Sweller, J. (2020). Cognitive load theory and educational technology. Education Technology Research and Development, 68, 1-16. Wing, J. M. (2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 366(1881), 3717–3725. Yulistyanti, D., Farkhatin, N., and Mustari, D. (2021). Penggunaan aplikasi sebagai media e- learning remaja di karang taruna. Journal of Empowerment, 2(1), 89-100. http://dx.doi.org/10.%2017509/xxxx.xxxx