BCP Social Sciences & Humanities ECSS 2022 Volume 19 (2022) 352 Wheelspinning frequency and reason in Robotutor Yuzhe Wu Vrije University Amsterdam, Amsterdam, Netherlands y19.wu@student.vu.nl Abstract. 20 March 2022. This paper describes wheelspinning happened in Robotutor team. Wheelspinning, which is a state that a student cannot make any effort although this student has worked hard. Since wheelspinning happens frequently on university student due to higher level of education, the theme is to analyze whether there is the same phenomenon on children and what is the reason for wheelspinning. The analytical tools are Rstudio and Excel. The main method for analyzing is the classification. The main table is split by columns, having “CHILD_ID”, “MATRIXNAME”, “ACTIVITIES”, “TOTALPROBLEMS”, “ACTIVITY_DURATION”, “FIRST_ATTEMPT_ PERCENTAGE_CORRECT”. After the analysis, the result shows that wheelspinning happens frequently on children, proved by a low percentage correct of all activities. Also, an initial reason of a low percentage correct is considered as children’s low interest of activities due to plenty of “BACKBUTTON” and short-time acitvities]. However, representing abstract definitions by numbers might be inaccurate], which is one of the limitations. Meanwhile, insisting for revision of definition representation is included in the future work. Keywords: Wheelspinning; Correction percentage; Six aspects; Numbers. 1. Introduction Wheelspinning is an infrequent topic around the world research projects. In these projects, the factors and how to detect wheelspinning are discussed at most. Since wheelspinning is influential for students, which might decrease students’ interest of studying in the long term. As a result, this project writes wheelspinning happened in one specific field, which is Robotutor, and hasincluded a whole process of detecting details that how to judge wheelspinning and factors [1-10]. There are two definitions that should be stated. Robotutor is an application with different online resources and assists children between 7 and 10 years without proper education, in the aspects like writing, math and oral abilities. In this report, the main analysis is based on each activity done by 93 children with more than 65 thousand activities. For wheelspinning, obviously, it makes cars stay motionless, despite both of the engine and the wheels have put great efforts. Metaphorically, wheelspinning represents people that try hard but cannot have any progress in the related works [1]. To define wheelspinning, a percentage correct less than 70 is regarded as wheelspinning. 70% in this report can be equal to the cut-off score in the common exam [8]. 2. RESEARCH QUESTION The research question is about testing different total correction rates in six aspects to find out whether wheelspinning occurs in Robotutor. Does wheelspinning happen frequently on children in Robotutor and what is the reason? 3. EXPERIMENT/METHOD For the data analysis, the tools Rstudio and Excel are used. Excel is used for showing the data, drawing and filtering some unrelated data while Rstudio is applied for importing tables and analyzing the data. The standard of wheelspinning is based on two classifications in the table, which are Percentage Correct that should be over 70% and Promotion that should be “PROMOTION”, which is shown BCP Social Sciences & Humanities ECSS 2022 Volume 19 (2022) 353 below. The leftmost column is Promotion, and the rightmost column is Percentage Correct. Actually, there are two states for Promotion, “PLACEMENT” and “PROMOTION”. The former decides which level to start kid at, while the latter decides whether to promote kids to the next activity. Due to the definition of wheelspinning, activities should be started at first and put efforts. As a result, “PLACEMENT” state is excluded from analyzing. Percentage_Correct implies the extent children engage for activities. The more Percentage_Correct is, the more progress children make while the less probability that wheelspinning happens. 3.1 Measurements The samples are split into six different parts, “By Student Number”, “by MatrixName”, “By Activities”, “By TotalProblems”, “By Duration” and “By First Attempt Percentage Correct”. The examples are shown above. The Unique_Child_ID is to differentiate children. MatrixName is the fields, and KC(Subtest) is the quizzes for activities which can represent skills mastered by children. TotalProblems, Activity_Duration and First_Attempt_Percentage are leveled each twenty problems, one hundred thousand milliseconds and ten percent respectively. By Child_ID By filtering, 93 children’s activities are recorded in the table. Since 5 children have less than 10 activities, they are excluded from analyzing. By analyzing, the top student has the mean of 57.2% Percentage_Correct, which is lower than the standard Percentage_Correct. The last student has the mean of 7.94% Percentage_Correct. The pie chart [figure 2] is the last student (CPVSTK_375) and another pie chart [figure 1] is the top student (SPARE_452). The pie charts are based on Completion column in the table. An obvious difference is that the BACKBUTTON occupies 75% for the last child and COMPLETE has the most proportion in the chart. The meaning of BACKBUTTON may be confused. It is a state that has no progress and backward. By Matrixname There are 4 fields in this part, literacy, math, songs and stories. From the graph [figure 3], stories have the best performance, however, it does not exceed the standard Percentage_Correct and is highly gapped with 70%. The field of songs is 0% and the related table is shown above. Many zeros are included in the table and the Completion state is almost BACKBUTTON. In consequence, wheelspinning happens most frequently around songs. By Activities There are a total number of 18 skills included, however, the skill Addition Level 3 has less than 10 samples, which has been excluded. From the graph [figure 4], the best skill is Addition Level 2, with 70 Percentage_Correct. The last skill is Writing a Sentence, with 0 Percentage_Correct. By Totalproblems The column TotalProblems is leveled each 20 problems. The height of each problem number in the graph represents the magnitude of Percentage_Correct. In the graph [figure 5], the first bar has the maximum proportion and the last bar is too small to demonstrate. Although wheelspinning happens frequently by the split, there is a trend that the probability wheelspinning occurs is inversely proportional to the increasing TotalProblems. By Acticitiy_Duration The part of duration is split into 11 sections, each with one hundred thousand milliseconds, and the last level is all activities over ten hundred thousand milliseconds. From the graph [figure 6], the second and third bars have more than 70 Percentage_Correct. For the remaining bars except the first one, there are small differences between them and they all exceed 50 Percentage_Correct, which means less wheelspinning happen for activities more than 100000 milliseconds. This is because from the previous table, activities with 0 millisecond duration tend to BCP Social Sciences & Humanities ECSS 2022 Volume 19 (2022) 354 have 0 Percentage_Correct, when these zeroes are excluded, the mean of Percentage_Correct will increase. By First_Attempt_Percentage_Correct First_Attempt_Percentage_Correct is layered each 10 percent. From the graph [figure 7], the calculated index is direct proportional to Percentage_Correct. This means that the more First_Attempt_Percentage_Correct is, the less wheelspinning happens. For example, for Percentage_Correct over 70%, the related First_Attempt_Percentage_Correct indexes are all above 70%. 3.2 Result Analysis Firstly, the mean of Percentage_Correct for all activities is calculated, which is 23.39% with 65416 activities. The activities that Percentage_Correct is over 70% are filtered, which have a number of 13511, occupying 20.65% of all activities. Wheelspinning seems to happen frequently around Robotutor by the index of Percentage_Correct. The next step is to split the activity table through six ways. For the evaluations, some principles that relations between the index and wheelspinning are concluded. For example, the frequency of wheelspinning happening is inversely proportional to First_Attempt_Percentage_Correct and direct proportional to TotalProblems. Moreover, wheelspinning is more likely to happen in the field “songs”, the skill “Write a Sentence”, the duration “>=0ms <=100000ms” and activities with “BACKBUTTON” state. Obviously, wheelspinning happens frequently in Robotutor. 4. DISCUSSION 4.1 Limitation The first limitation is the simplistic definition for wheelspinning. This is because wheelspinning suggests no progress in activities despite the efforts are contributed which is a process. However, in this report, the meaning is symbolized by index Percentage_Correct. Percentage_Correct is not well- suited to wheelspinning as Percentage_Correct implies the obstacle encounter in the activity and high Percentage_Correct signifies the extent of success in the activity, which is a result. As a result, more indexes should be mixed with Percentage_Correct in order to illustrate the process, wheelspinning. For example, the index “Attempts over 1” can be mixed with Percentage_Correct over 70%. The reason is that during the analysis of data, many attempts of 1 are figured. If extracting them from the activity table and recalculate the mean of Percentage_Correct, it becomes 93.8%, which is a great process. Secondly, another limitation can be the lack of programming language. Because of the lack, some operations become difficult. For example, all the filtered tables are split by manual copying and pasting, which is time-wasting. On the other hand, the appearance of graphs is ugly and not detailed. If more knowledge of Excel can be mastered, perfect graphs would be painted. Thirdly, some bugs exist in the table. The above table shows many spaces in the formula, which is an obstacle for analyzing and filtering. At last, time is a problem. When doing this project, exams and schoolwork are mixed with the process. This means it is a hurry to finish the evaluation and accomplish the results. 4.2 Future Works In the future, the lack of programming knowledge should be supplemented, which could economize. The next step is to mix various indexes in the table to capture and reveal wheelspinning and the reason for the phenomenon. For example, the focus can be on Subtract Level 1, filtering on Percentage_Correct < 70% and Attempts > 1 and figuring out the similarity between each remaining activity for Subtract Level 1. Also, by the assist of analysis, the reason of wheelspinning should be improved [9]. BCP Social Sciences & Humanities ECSS 2022 Volume 19 (2022) 355 5. Conclusion In conclusion, regarding to the thesis “How frequent wheelspinning happens in Robotutor?”, the first step is to simplify the definition of wheelspinning into Percentage_Correct of activities >70%. In the following steps, the Robotutor activity table is disaggregated through six different ways and analyzed. During the process of analyzing, it is found that wheelspinning happens frequently in Robotutor, with a probability of more than 70% and some relations between one index and Percentage_Correct. Furthermore, most disaggregate tables have less than 70 Percentage_Correct. As a result, the similarity between the tables is figured that most of tables contain short-time activities with 0 Percentage_Correct and “BACKBUTTON” state. This means that the main reason for wheelspinning in Robotutor is no more interest for children to learn subjectively. Moreover, wheelspining happens in the activities that need more language skills, which means the difficulty is also the factor of wheelspinning. Throughout the researching, the method that simplifies the problem is learned. This method can provide logics and thoughts when the problem is abstract and problematic. Take the first step as an example, due to the lack of programming knowledge at the beginning, formal analysis cannot be accomplished. Since the method of simplifying is applied, which separates ten activities at first and research on the relation between Percentage_Correct and other indexes. However, this method has some disadvantages, which mislead the researcher from the true definition a little. 6. APPENDIX Graphs used Figure 1. Completion states for top 1 student Figure 2. Completion states for last 1 student BCP Social Sciences & Humanities ECSS 2022 Volume 19 (2022) 356 Figure 3. Average Percentage_Correct on Matrixname Figure 4. Average Percentage_Correct on activities Figure 5. Average Percentage_Correct on Total problem numbers Figure 6. Average Percentage on Durations 0 10 20 30 40 50 60 70 80 A dd iti on L ev el 2 le tt er s M is si ng … A dd iti on L ev el 1 N um be r. I.D O ra l R ea di ng O rie nt in g to … Q ua nt ify in g Re ad in g… Su bt ra ct L ev el 1 Su bt ra ct L ev el 2 Su bt ra ct L ev el 3 Sy lla bl e So un d W rit e a N um be r W rit e a… W rit in g a W or d W rit in g Le tt er s Activities Percentage_Correct BCP Social Sciences & Humanities ECSS 2022 Volume 19 (2022) 357 Figure 7. Average Percentage_Correct on 1st % Correct Clearer figures are in the “graph” folder. Tables are split and placed in the “table” folder, regarding to each analysis part. References [1] Beck, J.E., Gong, Y. (2013). Wheel-Spinning: Students Who Fail to Master a Skill. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds) Artificial Intelligence in Education. AIED 2013. Lecture Notes in Computer Science (), vol 7926. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39112- 5_44 [2] Matsuda, N., Chandrasekaran, S., & Stamper, J. C. (2016). How quickly can wheel spinning be detected?. In Edm (pp. 607-608). [3] Beck, J., Rodrigo, M., & Mercedes, T. (2014, June). Understanding wheel spinning in the context of affective factors. In International conference on intelligent tutoring systems (pp. 162-167). Springer, Cham. [4] Zhang, C., Huang, Y., Wang, J., Lu, D., Fang, W., Stamper, J., ... & Aleven, V. (2019). Early Detection of Wheel Spinning: Comparison across Tutors, Models, Features, and Operationalizations. International Educational Data Mining Society. [5] Gong, Y., & Beck, J. E. (2015, March). Towards detecting wheel-spinning: Future failure in mastery learning. In Proceedings of the second (2015) ACM conference on learning@ scale (pp. 67-74). [6] Botelho, A. F., Varatharaj, A., Patikorn, T., Doherty, D., Adjei, S. A., & Beck, J. E. (2019). Developing early detectors of student attrition and wheel spinning using deep learning. IEEE Transactions on Learning Technologies, 12(2), 158-170. [7] Owen, V. E., Roy, M. H., Thai, K. P., Burnett, V., Jacobs, D., Keylor, E., & Baker, R. S. (2019). Detecting Wheel-Spinning and Productive Persistence in Educational Games. International educational data mining society. [8] Kai, S., Almeda, M. V., Baker, R. S., Heffernan, C., & Heffernan, N. (2018). Decision tree modeling of wheel-spinning and productive persistence in skill builders. Journal of Educational Data Mining, 10(1), 36-71. [9] Mu, T., Jetten, A., & Brunskill, E. (2020). Towards Suggesting Actionable Interventions for Wheel- Spinning Students. International Educational Data Mining Society. [10] Wan, H., & Beck, J. B. (2015). Considering the Influence of Prerequisite Performance on Wheel Spinning. International Educational Data Mining Society. 1. Introduction 2. RESEARCH QUESTION 3. EXPERIMENT/METHOD 3.1 Measurements 3.2 Result Analysis 4. DISCUSSION 4.1 Limitation 4.2 Future Works 5. Conclusion 6. APPENDIX References