International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol 17 No 06 (2023) Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with Attention Deficit Hyperactivity Disorder https://doi.org/10.3991/ijim.v17i06.35259 Gianella Celis1, Miguel Casas1, David Mauricio1(), José Santisteban2 1 Universidad Peruana de Ciencias Aplicadas (UPC), Lima, Peru 2 Universidad Privada Norbert Wiener, Lima, Peru pcsidmau@upc.edu.pe Abstract—Attention and hyperactivity disorder is one of the most frequent chronic conditions in children, which generates impulsive behaviors that cause them not to be able to concentrate since they do not remember the activities in- dicated to them and, as a result, they are underperforming in school. And it is even more difficult for them since the pandemic because they must pay atten- tion to a laptop for their classes. This study aims to develop and validate if a mobile application (Dilud) allows reinforcing the working memory (WM) of primary school children from 6 to 12 years old through 4 interactive and dy- namic games that use gamification and the training method Tajima Cognitive Test (TCT). To evaluate Dilud, the authors conducted a case study using the “simple random” method, with a 6-year-old child using the WISC-V exam. The case study results show that Dilud boosts working memory (WM) by 19 points, bringing the memory level “Average Normal”. While for the usability evalua- tion, 15 children and their parents rated the usability and satisfaction with a very high score through a survey. Keywords—attention deficit hyperactivity disorder (ADHD), mobile applica- tion, children, cognitive training, WISC-V 1 Introduction Attention deficit hyperactivity disorder (ADHD) is a complex neurodevelopmental disorder characterized by attention, impulsivity, and hyperactivity problems. It is considered one of the most frequent chronic conditions during a person's childhood and one of the most prevalent pathologies [1]. Between 2% and 12% of the pediatric population present this type of disorder [2]. Likewise, it usually affects between 8% and 12% of school-age children and causes them not to be able to perform correctly at school [3][4]. Therefore, the challenge for many parents is to know how to support their children in their school learning stage. Underdeveloped Working Memory (WM) is one of the factors that makes learning difficult in children with ADHD, because WM is responsible for maintaining and manipulating information, allowing various cognitive processes to be carried out [5]. 62 http://www.i-jim.org https://doi.org/10.3991/ijim.v17i06.35259 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… Since it is not well developed in children with ADHD, it makes them unable to re- member certain activities or tasks assigned to them. This problem would result in a memory deficit [6]. The use of rote learning methods in children with ADHD aims to reinforce or im- prove their learning skills. For this reason, various methods have been developed for a long time, which are supported by gamification. For example, the N-Back Task meth- od allows memory, concentration, and intelligence to be measured or reinforced through repetitive cycles [7]. In contrast, the Cog-Fun method reinforces executive function and self-awareness through activities of 10 sessions with games included [8]. But one of the most effective methods is TCT, which strengthens the working memory of children with ADHD and other cognitive areas through mobile application games. Therefore, this is the method that will use in this research. This research uses the Dilud application to reinforce rote learning in children with ADHD. The Dilud application consists of the TCT method and a performance module for parents to assess the progress of their children, something necessary in an educa- tional application [9]. Therefore, this work focuses on determining if an application that uses gamification can contribute to the rote learning of children with ADHD. More specifically, the following research question was identified: RQ1: Can a mobile application contribute to developing and reinforcing the work- ing memory of children with ADHD? This work is organized into 6 sections. Section 2 presents a brief review of the lit- erature on mobile applications to reinforce cognitive areas in children with ADHD. The model of the proposed application is described in section 3. Section 4 describes the software's architecture, technological components, and main modules. In section 5, the entire validation process is described. Finally, the conclusions follow in section 6. 2 Literature review about applications for ADHD 2.1 Mobile applications for learning This research aims to develop a mobile application that reinforces the rote learning of children with ADHD. For this reason, reviews and comparisons were made among several applications developed in this field in order to identify various benefits and limitations, these reviews helped to adjust the solution for the proposed problem. In this sense, the authors of the mHealth application aimed to detect ADHD and improve the learning of these children, especially memory and attention, through games. They concluded that the application works well in detecting children with this deficit. How- ever, regarding improving memory and attention, the authors wanted to focus on adding more options and functionalities for treating children with ADHD [10]. An- other application is PANDAS, which aims to detect children's ADHD quickly through a game. Authors of PANDAS concluded that the model used in their game helped detect ADHD in children with an accuracy of 86.5% and was easy for children to use [11]. iJIM ‒ Vol. 17, No. 06, 2023 63 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… Say-It & Learn application aims to complement children's learning using three modules with interactive games. The first module focused on learning the alphabet, the second on learning mathematics, and the third on learning geometric figures. They concluded that from 5 children who used the application in 6 days, all of them said that they liked it, and the experts indicated that it contributed to improve their learning [12]. Finally, another application is The Monitor Tool, which aims to validate the appli- cation for clinical use in monitoring young people with disabilities by providing vari- ous games. Authors concluded that the application does serve to monitor young peo- ple by 89%. Young people who used the app indicated that the application was attrac- tive because of the games presented in the application [13]. 2.2 Methods for learning The method must adapt to different environments, as this is the case of NEAR, a group program that aims to promote intrinsic motivation and improve deficit cogni- tive areas of adolescent participants. They used the combination of cognitive and interactive exercises and Cognitive-Behavioral Therapy groups with the support of a specialist. Authors concluded the investigation with improvements in memory and attention of adolescent participants with ADHD or ASD (Autism Spectrum Disorder) [14]. While the N-Back Task method, used by [7] [15] and [16] in their research works, focuses on the development of individual exercises to reinforce memory and concen- tration that involve repetitive activities to measure their performance. Such is the case of Breitling [16], which used the method to validate its efficiency with two groups of children with ADHD. Authors concluded that the N-Back Task serves to measure the memory level of a person as well as improve its performance. On the other hand, in the author's research [15], the author sought to develop a method that strengthens children's memory. Therefore, it was based on various existing methods, such as the N-Back Task. They concluded that using many methods simultaneously to reinforce short-term memory can generate a lack of attention in children with ADHD. In addi- tion, research [7] sought to validate the efficacy of N-Back for children with ADHD. Therefore, they used 108 children with ADHD to test the method for 5 weeks. They concluded that the method helped the children to improve their WM and to reduce their inattention and hyperactivity in small measures. The author [8] used another method in his research, the Cog-Fun method, which aims to improve the executive function and self-directed learning of 3 children with ADHD. For this purpose, the children carried out a series of interactive activities with a specialist for 5 weeks. They concluded that children's executive function and self- directed learning improved after using Cog-Fun. In addition, another popular method that tries to improve learning in children is STEM (Science, Technology, Engineer- ing, and Mathematics), which consists of applying different scientific-technological disciplines, knowledge, skills, and practices related to the acronym throughout school learning [17]. An example of this method is the research carried out by [18] and [19], where the authors aimed children can apply this methodology by integrating mobile 64 http://www.i-jim.org Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… technologies to improve their learning— authors concluded that this method does help children's learning. Also, the authors mentioned that this method would have many future benefits for children. Finally, another method is the one provided by the author [3], TCT, which consists of playing several interactive games related to improving and measuring certain cog- nitive areas of children with ADHD, such as attention, memory, and reasoning. In this sense, the author [3] aims to validate the method's efficacy through an application. Therefore, the sample used was a child with ADHD to use the application for eight months, 10 minutes daily. Concluding that the TCT method helped improve the par- ticipant's cognitive areas, and it should emphasize that the author emphasizes that the child's improvement was evident from the first 10 days of using the application. Cor- respondingly, the same author [20] carried out another investigation where she used gamification to improve the learning of children with ADHD. She concluded that it is a very efficient and easy-to-use for improving the knowledge and skills of children learning with special needs. 3 Method 3.1 TCT method and gamification The method applied in this research was the TCT, which consists of daily cognitive therapies to improve the different cognitive areas (attention, memory, calculation, reasoning, and visual-motor coordination, among others), particularly WM [3]. These therapies are developed through various games or activities the child will have to do for 10 minutes per day during the week. Besides, gamification is a learning method aims to use game elements, such as points, rewards, point ranking, and game levels, among others [21], to achieve better results from the participants in terms of absorb- ing new knowledge or strengthening specific skills. Consequently, it can generate motivation in children, as it is a dynamic method used in the classroom or at home; it will make the child play without feeling forced or obligated [22]. As mentioned above, within gamification, there are crucial elements that serve to ensure that it meets its objective. Among these elements is the achievement system, which allows the user to commit and motivate to continue carrying out activities that contribute to their learning since this system comprises challenges and rewards [23]. This inclusion is compatible with the TCT method since gamification seeks to moti- vate children to continue training through a game system, and the inclusion of chal- lenges and rewards will allow this objective to be met. 3.2 Development tools Unity is a commercially designed, cross-platform game engine or software for de- veloping 2D and 3D video games. It is easy to access, easy to use, flexible and effi- cient, which is why it is popular among developers. However, its scope can cover more topics, such as architecture, robotics, and animation [24][25]. This software iJIM ‒ Vol. 17, No. 06, 2023 65 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… offers tools for the rapid iteration and editing of compound development cycles in interactive environments with a high degree of visual fidelity [26]. C# is an object-oriented programming language that does not depend on hardware so that it can migrate to other systems [27]. Therefore, it is the one used by Unity for video game development. Microsoft Azure is a popular cloud platform and infrastructure service provided by Microsoft that enables the development, testing, deploying, and managing of ap- plications through various tools. Likewise, it offers SaaS, PaaS, and IaaS infrastruc- tures with high reliability, scalability, and low-cost infrastructures depending on the business needs [28]. PlayFab is a complete back-end platform developed by Microsoft that allows to build and operate live environments with managed game services, perform data anal- ysis in real time, and improve the user experience. Furthermore, this platform creates and authenticates user accounts in a virtual environment [29]. 3.3 Methodology development Internal design. For the design of Dilud, there are two perspectives, the internal and the external. In case of the internal one, the detailed architecture of the mobile application shows each component carried out during the user's interaction with the application, as shown in Figure 1. Fig. 1. Architecture of the mobile application Front-end. Direct user interaction occurs in the front-end environment, presenting the Dilud application's visible parts, such as interfaces and solution logic, in a C# script. Going into the detail of the interaction, when the user enters the application developed in Unity, Mono is executed. Mono is a framework designed in C#, which allows compiling various applications and systems [30]. After the first interaction, PlayerPrefs is used, a file that stores user information such as key name (encrypted), preferences, and score, among others [31]. It helps store users’ cache, which will determine users' progress in the various modules. Finally, the PlayFab SDK compo- nent allows the connection to the back-end side of the solution, which is in the cloud. Back-end. The services that support the internal interaction generated by the user in the back-end environment are provided by Microsoft Azure, which allows all the information collected in the application to be stored, such as the scores of several 66 http://www.i-jim.org Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… players and their personal information encrypted. In addition, the PlayFab services, which are inside the Azure services, connect with the Dilud application through the Rest API, which is used to connect the front-end side with the back-end of the appli- cation in a two-way communication between them. Finally, the PlayFab services that have direct interaction with the user through the different modules are the following: • "Data Management" is a service played by PlayFab, to review a game's daily and monthly performance with automatic reports available, storing each score or achievement achieved on the web [32]. This service is implemented in the "Moni- toring" component of the SWM model. • "Engagement and Retention" is a service that allows the use of the rules engine and player events to build a powerful achievement system [33]. This service is imple- mented in the "Badge System" component of the SWM model. • "Leaderboard and Statistics" is a service that tracks, compares, and rewards player performance easily [34]. This service is implemented in the "Monitoring" compo- nent of the SWM model. External design. The external design is aimed at usability for children aged 6 to 12 with ADHD and their parents; following the usability principles mentioned in the author's research [35]. In the first place, for children with ADHD, colors such as greens and blues, are used in order to attract their attention and calm them at the same time. Likewise, the friendly figures within the games play an essential role in making their learning experience satisfying and enriching. Secondly, for parents, the usability experience focuses on monitoring their children's progress, so graphics are used to monitor them. The external design is represented in Figure 2 and Figure 3. Fig. 2. Dilud Games: A) Color Balloons; B) Infinite Space; C) Special Cards; D) Face Memory iJIM ‒ Vol. 17, No. 06, 2023 67 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… Fig. 3. Performance analysis interface for the parent 3.4 Usability and satisfaction To determine the level of usability and satisfaction of Dilud application, a survey was carried out in Google Forms, which aims to measure the user's assessment of the application. Sample. The chosen sampling method was “simple random”, which guarantees that all individuals in the population have the same opportunity to be included in the sample [36]. In this sense, the 15 children who participated in the survey were be- tween 6 and 11. They had a predominant level of ADHD of inattention, impulsive behavior, and a combination of both. Furthermore, some of the children were patients of two psychological centers in Lima, Peru. Their parents agreed to participate in the usability survey aimed at parents. Finally, there was the participation of 15 children and 15 parents, details of their condition can be found in Table 1. Table 1. General data of the participating children and parents Id Kid Gender Age Academic Degree ADHD Level Id Parent Gender Age Higher Degree PK01 F 6 First grade Predominant inattention PP01 F 42 Bachelor’s degrees PK02 M 10 Fifth grade Predominant inattention PP02 M 43 Bachelor’s degrees PK03 M 9 Fourth grade Predominant impulsive behavior PP03 F 35 Bachelor’s degrees PK04 M 6 First grade Predominant inattention PP04 F 33 Bachelor’s degrees PK05 M 11 Sixth grade Predominant inattention PP05 F 36 Completed Secondary PK06 F 8 Third grade Combined PP06 M 40 Bachelor’s degrees PK07 F 6 First grade Predominant inattention PP07 F 37 Bachelor’s degrees 68 http://www.i-jim.org Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… PK08 M 7 Second grade Combined PP08 F 39 Completed Secondary PK09 M 8 Third grade Predominant inattention PP09 F 37 Completed Secondary PK10 M 7 Second grade Combined PP10 F 35 Completed Secondary PK11 M 6 First grade Combined PP11 M 44 Completed Secondary PK12 M 6 First grade Predominant impulsive behavior PP12 M 41 Completed Secondary PK13 F 10 Fifth grade Predominant impulsive behavior PP13 F 41 Bachelor’s degrees PK14 F 8 Third grade Predominant inattention PP14 F 39 Completed Secondary PK15 F 11 Sixth grade Combined PP15 M 42 Bachelor’s degrees Evaluation instrument. The usability and satisfaction of children and parents were evaluated through a questionnaire, 5 questions for children and 5 for parents (see Table 2). The questions were based on the User Experience Questionnaire (UEQ), a questionnaire to measure the user experience of interactive products. This question- naire combines measures such as efficiency, attractiveness, stimulation, reliability, and perspicuity in a 26-item survey based on the six abovementioned measures [37][38]. The answers to the questions were measured through the Likert scale (5- Very Good, 4-Good, 3-Fair, 2-Bad, and 1-Very Bad) [39], except for the questions about the most attractive game (QK2, QK3) whose alternatives are games (CB-Color Balloons, SC-Special Cards, IS-Infinitive Space and FM-Face Memory). Table 2. Usability and satisfaction questionnarie Dimension Id Question User Usability QK1 What do you think of the application? Kid QK2 What do you think about the design and color of the application? Kid QP1 What did you think of the app? Parent QP2 What did you think of the "Monitoring” module? Parent QP3 How would you rate the app in terms of ease of use? Parent QP4 What did you think of the child's progress graphs? Parent Satisfaction QK3 What was the game you liked the most? Kid QK4 What was the game you liked the least? Kid QK5 From 1 to 5, how much would you recommend the app? Kid QP5 From 1 to 5, how much would you recommend the app to other parents? Parent This questionnaire was adapted and developed in Google Forms, which allows ask- ing questions and sending a survey to a specific target audience and collecting the resulting information efficiently [40]. Therefore, this tool was used so parents and children could complete the questionnaire at home and, in the end, collect their re- iJIM ‒ Vol. 17, No. 06, 2023 69 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… sponses and analyze them using the same tool. The questionnaires can be seen in Figure 4. Fig. 4. Forms: A) Questions for children; B) Questions for parents 4 Case study This section aims to validate the Dilud application's operation through its use for 30 days. 4.1 Startup stage This stage contains two parts, the steps prior to experimentation and the sample used. Previous steps. In this initial stage, it was requested the help of a specialist in child psychology and cognitive behavioral therapies; in other words, it treats children with ADHD. Also, this specialist works in a renowned psychological center in Lima, Peru. It should be noted that within the study, this specialist will be known by the symbols EE. Sample. The chosen sampling method was “simple random”, as in the usability and satisfaction survey case. Therefore, from the initial sample of 15 children, one was randomly chosen to participate in the case study. This sample is a child that we will denote by the acronym NN. He is six years old, presents ADHD with a predomi- nant level of inattention, and lives in the district of San Isidro in Lima, Peru. NN has been receiving treatment at a child psychology center since he was five. He has trou- ble remembering the homework assigned at school, is easily distracted during classes, and does not listen to his teachers' instructions. 70 http://www.i-jim.org Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… 4.2 Planning stage The planning stage, prior to experimentation, is classified into two parts: the vali- dation metric and the implementation of Dilud to start the experimentation. Validation and analysis metric. The evaluation metric that allowed analyzing the case study results found within the Wechsler Intelligence Scale Evaluation for Chil- dren-V (WISC-V), intended for children between 6 and 16 years of age. Additionally, it focuses on measuring the cognitive level related to academic learning, and it is one of the most used evaluations in several countries [41]. WISC-V contains several tests, such as verbal comprehension, fluent reasoning, and WM, the last one interest this study. For this reason, the "Picture Span" and "Digit Span" subtests use to evaluate the intelligence quotient (IQ) of the WM, whose scores are shown in Table 3. On one hand, the first subtest consists of the child observing a series of drawings in a desig- nated time interval, after which he must select the figures shown to him in the correct order. On the other hand, the second subtest consists of the child having to orally repeat the order of the digits that the specialist in charge has initially dictated. Both subtests are taken with the help of the specialist selected in the initial stage. Table 3. Intelligence Sclae for Children, acoording to Weschler (2014 version) Score Level 130 + Much higher 120 - 129 Superior 110 - 119 Superior Normal 90 - 109 Average Normal 80 - 89 Lower Normal 70-79 Borderline Dilud implementation. Before beginning the experimentation, the psychologist who cares for NN, which we will know as EP, was online trained in the use Dilud by 1 hour. After that, she downloaded the application from the Google Play Store and quickly recognized the software's functionalities, agreeing to use it for NN in the case study. In addition, she explained to NN's mother how both, the mother and her youngest son, should use Dilud. Subsequently, the mother downloaded Dilud on his cell phone and received a user manual. 4.3 Experimentation stage The beginning of this experimentation stage, illustrated in Figure 5, occurs when psychologist EE provides the WISC-V assessment subtests to NN to determine his WM level before using the Dilud application. Subsequently, NN used the application for 30 days, 10 minutes a day, under the supervision of his EP psychologist, where he interacted with the main modules, such as "Games," to play with the two games that she must complete per day, and "Badge System" to review your earned badges. On the other hand, his mother used the "Monitoring" module to verify the progress of her youngest son in the application, as seen in Figure 6. iJIM ‒ Vol. 17, No. 06, 2023 71 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… Fig. 5. Flowchart of the experimentation stage Fig. 6. Usage of Dilud: A) Child using the application; B) Mother teaching how to use the application After 30 days, psychologist EE took the WISC-V assessment for NN for the last time. In addition, she collected the data obtained from the evaluation at the beginning and the end of the experiment. 5 Results and discussions 5.1 About the methodology The TCT method and gamification are evaluated and compared with the other methodologies found in the literature review. Results. With the results obtained from the case study and the usability and satis- faction survey, we want to highlight the good performance and support of the TCT 72 http://www.i-jim.org Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… method and gamification as methods for reinforcing and improving cognitive areas, in this case, WM for children with ADHD. In this sense, the TCT method allowed us to establish the research framework, that is, the frequency of use of the Dilud applica- tion, 10 minutes daily. Likewise, gamification allowed us to develop interactive games based on improving and reinforcing the WM of children with ADHD. In addi- tion, the rewards system, which is part of the gamification, allowed us to establish a more significant commitment on the part of the users, in this case, the children, so that they can constantly develop the assigned activities. Comparison. From the literature, 90% of the articles had in common the use of gamification in developing their research to improve children's memory. Among these, we can highlight the N-Back Task, which measures and reinforces children's visual motor memory, concentration, and intelligence [16]. However, compared with the TCT method, the N-Back Task focuses on just one kind of game, while the TCT allows a wider variety of games to be used depending on the cognitive area they want to improve or reinforce. We can also highlight the NEAR method, which uses games with cognitive behav- ioral therapies to improve children's cognitive functioning. But the limitation of this method is that it depends on a specialist to be applied [14]. In addition, we can high- light the Cog-Fun method as a novel method to consolidate the memory of children with ADHD through a game. However, the N-Back Task method lacks diversity in the part of gamification [8] [15]. Finally, we want to highlight the use of the reward system as an excellent support for developing this research. 5.2 About the case study The use of Dilud application by children with ADHD positively supports them in improving their cognitive abilities, especially WM. For this reason, the purpose of the case study was to validate the hypothesis and demonstrate the effectiveness of the Dilud application in improving the WM of children with ADHD. The results were analyzed by the metrics provided by WISC-V, which has a high level of reliability [41], therefore, it helped us in this investigation to determine the level of improvement that NN had. The results are illustrated in Figure 7, and it is possible to observe both the initial result of the exam before NN uses the Dilud appli- cation and the final result after NN has used the application. The graph presented showed a significant increase in the final rating of WM com- pared to the initial one. In the case of the "Picture Span" subtest, there was an increase from 15 points to 22, which means that NN improved his visual memory [42]. The "Digit Span" subtest, there was an increase from 12 to 20, which means that NN im- proved his immediate memory significantly and, consequently, his WM [43]. Consid- ering both subtests, it can be determined that NN improved his WM from an initial score of 84 to a final score of 103. The results obtained with the table that measures WM, it can be concluded that NN's WM at one start was "Lower Normal"; however, after using the Dilud application, his WM rose to the "Average Normal" level. iJIM ‒ Vol. 17, No. 06, 2023 73 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… Fig. 7. Results of the case study With this evidence, it is possible to answer the general hypothesis raised, and the hypothesis based on games, that an application, in this case, Dilud, can contribute to the development and reinforcement of the WM of children with ADHD supported by the use of interactive games. On the one hand, we have that the Dilud application, compared to the Say-It & Learn application, seen in the literature review, Dilud man- aged to increase the child's WM by 19 points with the help of memory games [12], while the author of the other app focused on math-related games. On the other hand, the Cog-Fun method improved their learning in 5 months [8], while Dilud did it in 30 days. In addition, compared to the mHealth application [10], the Dilud application was possible both at home and with the help of a psychologist since it did not gener- ate more complexity for the child. 5.3 About the usability and satisfaction of the mobile application In order to evaluate the levels of usability and satisfaction of the Dilud application, 15 children and 15 parents took a survey, where they had to use the application for 5 days, 10 minutes a day, and then evaluate their experience. In this sense, Table 4 shows the results obtained in the case of children, where the Dilud application ob- tained a very high rating in usability (4.7), with a very high evaluation for the design and colors of the application (QK2), according to the Likert scale [39]. Lastly, it ob- tained a very high satisfaction rating (4.8). The game the children liked the most was IS. This result can be explained by the spatial theme of the game, while the game they liked the least was SC because it is the least dynamic game compared to the others. Likewise, the children recommended the application with a rating of 4.8 out of 5 (QK5). 74 http://www.i-jim.org Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… Table 4. Results of usability and satisfaction of the participating children Participant code (PK) Questions Usability Satisfaction QK1 QK2 QK3 QK4 QK5 PK1 5 5 FM SC 4 PK2 4 5 IS CB 5 PK3 5 5 IS SC 5 PK4 5 5 CB FM 5 PK5 5 4 FM IS 5 PK6 5 5 IS SC 5 PK7 5 4 IS SC 4 PK8 4 4 CB SC 5 PK9 5 5 IS SC 5 PK10 5 5 SC FM 5 PK11 5 5 IS SC 5 PK12 5 5 IS SC 5 PK13 5 5 FM SC 5 PK14 3 4 IS SC 4 PK15 5 5 FM CB 5 Average 4.7 4.7 IS (53%) SC (67%) 4.8 Additionally, the results obtained from the survey for parents can be seen in Table 5, where the Dilud application obtained a very high rating in all the responses regard- ing usability and satisfaction, reaching an average score of 4.7 out of 5 in both cases. This result can be explained because the application is easy to use, therefore, intuitive (QP3) since it presents graphs that allow visualizing the child's progress (QP4) and its monitoring (QP2), which is very useful for parents. Also, parents mentioned that it is a good application (QP1) that they would recommend (QP5). In addition, the purpose of this survey was to take the information obtained to evaluate the levels of usability and satisfaction and improve future updates regarding the Dilud application based on the comments received from users, which stands out the development of more games to reinforce the WM. Finally, the usability and satisfaction tests were very high according to the answers obtained. The recommendation to add more games in the following updates is high- lighted because when it compares to the Say-It & Learn application, it has more inter- active games focused on improving children's cognitive areas [12]. While the PANDAS application presented an interactive game to measure the WM of children, generating a high value in efficiency and usability [11]. Likewise, in this sense, the badge system, a new module not implemented in the reviewed applications, contribut- ed significantly to children's usability experience and satisfaction, according to the comments received by parents. iJIM ‒ Vol. 17, No. 06, 2023 75 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… Table 5. Results of usability and satisfaction of the participating parents Participant code (PP) Questions Usability Satisfaction QP1 QP2 QP3 QP4 QP5 PP1 5 5 5 5 5 PP2 5 5 5 5 5 PP3 4 4 5 4 5 PP4 4 4 5 4 5 PP5 5 4 5 4 5 PP6 5 4 5 4 5 PP7 5 4 4 4 5 PP8 5 5 5 4 4 PP9 5 5 5 5 4 PP10 5 5 5 5 4 PP11 5 5 5 4 4 PP12 5 5 5 5 4 PP13 4 4 5 5 5 PP14 5 5 4 5 5 PP15 5 5 5 5 5 Average 4.8 4.6 4.9 4.5 4.7 6 Conclusions In conclusion, with the help of the methodologies used, it was possible to develop a mobile application for the improvement and reinforcement of WM, which has had favorable results after the tests, since it helped to increase the level of WM of a child with ADHD who initially presented a "Lower Normal" WM level, and finished the experimentation with "Average Normal" WM level. In addition, the usability and satisfaction tests also showed favorable results, obtaining an average score of 5, "Very Good," both in usability and satisfaction. Additionally, with the help of the methodol- ogies, the framework was proposed and allowed the validation tests in the established time range. Therefore, the proposed objective was met with positive results, demon- strating that children with ADHD can improve and reinforce their WM through inter- active games. Likewise, as future work, developing more interactive games that sup- port the improvement and reinforcement of WM for children with ADHD is proposed. Focusing on other deficient cognitive areas in children with ADHD is not ruled out, such as executive function [44]. Finally, the existing limitations in this research are the number of samples used, which was limited due to the health security measures of the COVID-19 pandemic. It negatively affected the psychological centers that treated children with disorders in person and many people's social life, close relationships, and economic well-being [45]. 76 http://www.i-jim.org Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… 7 Acknowledgment The authors thank the Universidad Peruana de Ciencias Aplicadas (UPC), for fi- nancing of this research. In addition, the authors thank Christian Kian Kanashiro and Nicolas Arakaki Miyoshi for support in the graphic design of Dilud. 8 References [1] F. Rusca and C. Cortez, “Attention Deficit and Hyperactivity Disorder in children and ado- lescents. A clinical review,” Revista de Neuro-Psiquiatria, vol. 83, no. 3, pp. 148-156, 2020. https://doi.org/10.20453/rnp.v83i3.3794 [2] C. Rodriguez, M. Gónzales, M. Arroba, and L. Cabello, “Prevalence of attention deficit hyperactivity disorder in children from an urban area,” Revista Pediatría Aten. Primaria, vol. 19, no. 76, pp. 311-320, Dec. 2017. [Online Serial]. Available: http://scielo.isciii.es/ scielo.php?script=sci_arttext&pid=S1139-76322017000500003&lng=es&nrm=iso [Ac- cessed Oct. 15, 2021]. [3] M. Agustini, “Usage “ADHD Trainer” Game to Improve Cognitive Skill Child with ADHD (The study cases at the child aged 8 years),” In Proc. International Conference on Early Childhood Care Education and Parenting, pp. 51-57, 2020. https://doi.org/10.2991/ assehr.k.201205.085 [4] S. Arora, M. Lawrence, and R. Klein, “The Attention Network Test Database: ADHD and Cross-Cultural Applications,” Frontiers in Psychology, vol. 11, no. 388, 2020. https://doi.org/10.3389/fpsyg.2020.00388 [5] T. Vieites, “Difficulties in attention and memory in Primary Education students with At- tention Deficit Hyperactivity Disorder,” Revista de Psicología y Educación, vol. 14, no. 2, pp. 136-143, 2019. https://doi.org/10.23923/rpye2019.02.178 [6] K. Bul, L. Doove, I. Franken, S. van der Oord, P. Kato, and A. Maras, “A serious game for children with Attention Deficit Hyperactivity Disorder: Who benefits the most?”, PLOS ONE, vol. 13, no. 3, pp. 1-18, 2018. https://doi.org/10.1371/journal.pone.0193681 [7] M. Jones, B. Katz, M. Buschkuehl, S. Jaeggi, and P. Shap, “Exploring N-Back Cognitive Training for Children With ADHD,” Journal of Attention Disorders, vol. 24, no. 5, pp. 704-719, 2020. https://doi.org/10.1177/1087054718779230 [8] M. Kim, H. Park, E. Yoo and J. Kim, “Effects of a Cognitive-Functional Intervention Method on Improving Executive Function and Self-Directed Learning in School-Aged Children with Attention Deficit Hyperactivity Disorder: A Single-Subject Design Study,” Occupational Therapy International, vol. 2020, 2020. https://doi.org/10.1155/2020/1250 801 [9] M. Rodriguez, M. Vizcarra and A. Concha, “Can Rural Children Be Tested with WISC-V? Exploring the Factorial Invariance of Intelligence in Chile,” Iberoamerican journal of di- agnosis and psychological evaluation, vol. 3, no. 60, pp. 117-131, 2021. https://doi.org/10.21865/RIDEP60.3.10 [10] N. Rodríguez, P. Caballero, A. Rivero and J. Toledo, “A secure mHealth application for at- tention deficit and hyperactivity disorder”, Expert Systems, vol. 37, no. 1, pp. 1-14, 2020. https://doi.org/10.1111/exsy.12431 [11] H. Mwamba, P. Fourie and D. Van Den Heever, “PANDAS: Paediatric attention- deficit/hyperactivity disorder application software”, Applied Sciences (Switzerland), vol. 9, no. 8, 2019. https://doi.org/10.3390/app9081645 [12] S. Butt, F. Hannan, M. Rafiq, I. Hussain, C. Faisal and W. Younas, “Say-It & Learn: Inter- active Application for Children with ADHD”, In: Rau, PL. (eds) Cross-Cultural Design. iJIM ‒ Vol. 17, No. 06, 2023 77 https://doi.org/10.20453/rnp.v83i3.3794 http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1139-76322017000500003&lng=es&nrm=iso http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1139-76322017000500003&lng=es&nrm=iso https://doi.org/10.2991/assehr.k.201205.085 https://doi.org/10.2991/assehr.k.201205.085 https://doi.org/10.3389/fpsyg.2020.00388 https://doi.org/10.23923/rpye2019.02.178 https://doi.org/10.1371/journal.pone.0193681 https://doi.org/10.1177/1087054718779230 https://doi.org/10.1155/2020/1250801 https://doi.org/10.1155/2020/1250801 https://doi.org/10.21865/RIDEP60.3.10 https://doi.org/10.1111/exsy.12431 https://doi.org/10.3390/app9081645 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… Applications in Health, Learning, Communication, and Creativity. HCII 2020. Lecture Notes in Computer Science (), vol. 12193, 2020. https://doi.org/10.1007/978-3-030-49913- 6_18 [13] R. Flynn, N. Colón, J. Zhou and J. Bower, “A Game-Based Repeated Assessment for Cog- nitive Monitoring: Initial Usability and Adherence Study in a Summer Camp Setting”, Journal of Autism and Developmental Disorders, vol. 49, no. 5, pp. 2003–2014, 2019. https://doi.org/10.1007/s10803-019-03881-w [14] S. Renou, and C. Doyen, “NEAR (Neuropsychological Educational Approach to Cognitive Remediation) Cognitive Remediation Program in Adolescents with Attention Defi- cit/Hyperactivity Disorder and/or Autism Spectrum Disorder”, Annales Medico- Psychologiques, vol. 177, no. 8, pp. 758–764, 2019. https://doi.org/10.1016/j.amp.2018.07. 012 [15] E. Wells, M. Kofler, E. Soto, H. Schaefer and D. Sarver, “Assessing working memory in children with ADHD: Minor administration and scoring changes may improve digit span backward’s construct validity”, Research in Developmental Disabilities, vol. 72, pp. 166- 178, 2018. https://doi.org/10.1016/j.ridd.2017.10.024 [16] C. Breitling, J. Tegelbeckers, H. Flechtner and K. Krauel, “Economical Assessment of Working Memory and Response Inhibition in ADHD Using a Combined n-back/Nogo Paradigm: An ERP Study”, Frontiers in Human Neuroscience, vol. 14, pp. 1-15, 2020. https://doi.org/10.3389/fnhum.2020.00322 [17] V. López, D. Couso and C. Simarro, “Educación STEM en y para el mundo digital: El pa- pel de las herramientas digitales en el desempeño de prácticas científicas, ingenieriles y matemáticas”, Journal of Distance Education, vol. 20, no. 62, 2020. https://doi.org/ 10.6018/red.410011 [18] M. Kalogiannakis and S. Papadakis, “The use of developmentally mobile applications for preparing pre-service teachers to promote STEM activities in preschool classrooms”, In Mobile Learning Applications in Early Childhood Education, pp. 82-100, 2020. https://doi.org/10.4018/978-1-7998-1486-3.ch005 [19] S. Papadakis, J. Vaiopoulou, E. Sifaki, D. Stamovlasis, M. Kalogiannakis and K. Vass- ilakis, “Factors That Hinder in-Service Teachers from Incorporating Educational Robotics into Their Daily or Future Teaching Practice”, In CSEDU, no. 2, pp. 55-63. 2021. https://doi.org/10.5220/0010413900550063 [20] M. Agustini, Yufiarti and Wuryani, “Development of learning media based on android games for children with attention deficit hyperactivity disorder”. International Journal of Interactive Mobile Technologies, vol. 14, no. 6, pp. 205-213, 2020. https://doi.org/ 10.3991/IJIM.V14I06.13401 [21] R. Ramos and D. Mauricio, “Videogame to Support the Teaching of Reading to Deaf Children using Gamification,” Iberian Journal of Information Systems and Technologies, vol. 2019, no. 23, pp. 145-157, 2019. [22] C. Gonzales, “Gamification in the classroom: gamifying face-to-face teaching-learning spaces and virtual spaces,” 2019. https://doi.org/10.13140/RG.2.2.34658.07364 [23] R. Fabio, M. Bianco, T. Capri, F. Marino, L. Ruta, D. Vagni and G. Pioggia, “Working memory and decision making in children with ADHD: An analysis of delay discounting with the use of the dual-task paradigm,” BMC Psychiatry, vol. 20, no. 1, pp. 1-13, 2020. https://doi.org/10.1186/s12888-020-02677-y [24] F. Hussain, “Unity Game Development Engine: A Technical Survey”, USJICT, vol. 4, no. 2, pp. 73-81, Oct. 2020. [25] R. Jafri, R. Campos, S. Ali and H. Arabnia, “Visual and Infrared Sensor Data-Based Ob- stacle Detection for the Visually Impaired Using the Google Project Tango Tablet Devel- opment Kit and the Unity Engine,” IEEE Access, vol. 6, pp. 443-454, 2018. https://doi.org/10.1109/ACCESS.2017.2766579 78 http://www.i-jim.org https://doi.org/10.1007/978-3-030-49913-6_18 https://doi.org/10.1007/978-3-030-49913-6_18 https://doi.org/10.1007/s10803-019-03881-w https://doi.org/10.1016/j.amp.2018.07.012 https://doi.org/10.1016/j.amp.2018.07.012 https://doi.org/10.1016/j.ridd.2017.10.024 https://doi.org/10.3389/fnhum.2020.00322 https://doi.org/10.6018/red.410011 https://doi.org/10.6018/red.410011 https://doi.org/10.4018/978-1-7998-1486-3.ch005 https://doi.org/10.5220/0010413900550063 https://doi.org/10.3991/IJIM.V14I06.13401 https://doi.org/10.3991/IJIM.V14I06.13401 https://doi.org/10.13140/RG.2.2.34658.07364 https://doi.org/10.1186/s12888-020-02677-y https://doi.org/10.1109/ACCESS.2017.2766579 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… [26] D. Fernández, J. Monroy, J. González, “Improving the Proxemic Behavior of an Autono- mous Robot using Artificial Intelligence Engines Developed for Video Game Platforms”, In Proceedings of the XXXVIII Automatic Conference, pp. 717-723, 2017. https://doi.org/ 10.17979/spudc.9788497497749.0717 [27] L. Chusin and P. Morales, “Development Of A Mobile Application For Android Operating Systems Using Gamification Techniques For Learning Inequalities And Complex Num- bers Aimed At High School Students”, M.S. thesis, College of Syst. Eng., Politécnica Sa- lesiana Univ., Quito, 2022. [Online]. Available: https://dspace.ups.edu.ec/bitstream/ 123456789/22200/1/UPS%20-%20TTS681.pdf [28] R. M. H. Al-Sayyed, W. A. Hijawi, A. M. Bashiti, I. AlJarah, N. Obeid, and O. Y. A. Al- Adwan, “An Investigation of Microsoft Azure and Amazon Web Services from Users’ Perspectives”, Int. J. Emerg. Technol. Learn., vol. 14, no. 10, pp. pp. 217–241, 2019. https://doi.org/10.3991/ijet.v14i10.9902 [29] J. Sudha, N. Nandakumar, S. Raveendran and S. Sandeep, “VR Classroom for Interactive and Immersive Learning with Assessment of Students Comprehension”, Advances in Computing and Network Communications, vol. 735, pp. 133-145, 2021. https://doi.org/ 10.1007/978-981-33-6977-1_11 [30] Mono-Project.com, “About Mono”, 2022. [Online]. Available: https://www.mono- project.com/docs/about-mono/ [Accessed Oct. 21, 2022]. [31] E. Ramos, “PlayerPrefsPro Documentation (V1.0). A hassle-free PlayerPrefs' data manag- er & encryptor”, 2018. [Online]. Available: https://earrgames.github.io/docs/Player– PrefsPro_UserManual.pdf [Accessed Sep. 14, 2021]. [32] Playfab.com, “Data & Analytics. Understand and react to player behavior in real-time with one-stop data analytics, storage, processing and exports,” Azure Playfab, 2021. [Online]. Available: https://playfab.com/analytics/#data-management [Accessed Oct. 2, 2021]. [33] Playfab.com, “LiveOps. Run your game as a service with telemetry, automation, content management, and in-game commerce,” Azure Playfab, 2021. [Online]. Available: https://playfab.com/liveops/ [Accessed Oct. 2, 2021]. [34] Playfab.com, “Multiplayer Services. Grow fearlessly with a set of back-end building blocks for your live game,” Azure Playfab, 2021. [Online]. Available: https://playfab.com/ multiplayer/#stats [Accessed Oct. 2, 2021]. [35] S. Arrieya, N. Saadah, M. Mohamad and M. Zulhilmi, “A Review on Cultural Design El- ements for Mobile Applications User Interface,” International Journal of Interactive Mo- bile Technologies, vol. 16, no. 15, pp. 78-92, 2022. https://doi.org/10.3991/ijim.v16i15. 30851 [36] T. Otzen and C. Manterola, “Sampling Techniques on a Study Population”, Int. J. Mor- phol, vol. 35, no. 1, pp. 227-232, 2017. [37] A. Saleh, H. Abuaddous, I. Alansari and O. Enaizan, “The Evaluation of User Experience on Learning Management Systems Using UEQ”, International Journal of Emerging Tech- nologies in Learning (iJET), vol. 17, no. 7, pp. 145-162, 2022. https://doi.org/10.3991/ ijet.v17i07.29525 [38] I. Díaz, G. López, L. Quesada and Guerrero, “Standardized Questionnaires for User Expe- rience Evaluation: A Systematic Literature Review”, Proceedings, vol. 31, no. 1, pp.14, 2019. https://doi.org/10.3390/proceedings2019031014 [39] A. Matas, “Likert-Type Scale Format Design: State of the Art”, Electronic Journal of Edu- cational Research, vol. 20, no. 1, pp. 38-47, 2018. https://doi.org/10.24320/redie.2018.20. 1.1347 [40] H. Leyva, M. Pérez, and S. Pérez, “Google Forms in the diagnostic evaluation as support- ing the teaching activities. Case Students Bachelor of Tourism”, RIDE, vol. 9, no. 17, pp. 84 - 111, jun. 2018. https://doi.org/10.23913/ride.v9i17.374 [41] M. Rodriguez, M. Vizcarra and A. Concha, “Can Rural Children Be Tested with WISC-V? Exploring the Factorial Invariance of Intelligence in Chile”, Iberoamerican journal of di- iJIM ‒ Vol. 17, No. 06, 2023 79 https://doi.org/10.17979/spudc.9788497497749.0717 https://doi.org/10.17979/spudc.9788497497749.0717 https://dspace.ups.edu.ec/bitstream/123456789/22200/1/UPS%20-%20TTS681.pdf https://dspace.ups.edu.ec/bitstream/123456789/22200/1/UPS%20-%20TTS681.pdf https://doi.org/10.3991/ijet.v14i10.9902 https://doi.org/10.1007/978-981-33-6977-1_11 https://doi.org/10.1007/978-981-33-6977-1_11 http://www.mono-project.com/ https://www.mono-project.com/docs/about-mono/ https://www.mono-project.com/docs/about-mono/ http://playfab.com/ https://playfab.com/analytics/#data-management http://playfab.com/ https://playfab.com/liveops/ http://playfab.com/ https://playfab.com/multiplayer/#stats https://playfab.com/multiplayer/#stats https://doi.org/10.3991/ijim.v16i15.30851 https://doi.org/10.3991/ijim.v16i15.30851 https://doi.org/10.3991/ijet.v17i07.29525 https://doi.org/10.3991/ijet.v17i07.29525 https://doi.org/10.3390/proceedings2019031014 https://doi.org/10.24320/redie.2018.20.1.1347 https://doi.org/10.24320/redie.2018.20.1.1347 https://doi.org/10.23913/ride.v9i17.374 Paper—Dilud: A Mobile Application to Reinforce Rote Learning in Elementary School Children with… agnosis and psychological evaluation, vol. 3, no. 60, pp. 117-131, 2021. https://doi.org/ 10.21865/RIDEP60.3.10 [42] A. Tanabe and N. Osaka, “Picture span test: Measuring visual working memory capacity involved in remembering and comprehension”, Behavior research methods, vol. 41, no. 2, pp. 309-317, 2009. https://doi.org/10.3758/BRM.41.2.309 [43] E. Satorres, J. Meléndez, A. Pitarque, E. Real, M. Abella and J. Escudero, “Enhancing Immediate Memory, Potential Learning, and Working Memory with Transcranial Direct Current Stimulation in Healthy Older Adults”, Int. J. Environ. Res. Public Health, vol. 19, no. 19, pp. 12716. https://doi.org/10.3390/ijerph191912716 [44] A. Gentil, M. Santamaría, L. Mínguez, J. Fernández, J. González, J. González and A. Obregón, “Executive Functions in Children and Adolescents with Autism Spectrum Dis- order in Family and School Environment”, Int. J. Environ. Res. Public Health, vol. 19, no. 13, pp. 7834, 2022. https://doi.org/10.3390/ijerph19137834 [45] T. Karakose, R. Yirci and S. Papadakis, “Examining the associations between COVID-19- related psychological distress, social media addiction, COVID-19-related burnout, and de- pression among school principals and teachers through Structural Equation Modeling”, Int. J. Environ. Res. Public Health, vol. 19, no. 4, pp. 1951, 2022. https://doi.org/10.3390/ ijerph19041951 9 Authors Gianella Celis received the bachelor for Information Systems Engineering, Peru- vian Universidad Peruana de Ciencias Aplicadas, Lima, Peru. She is currently work- ing as a data engineer developer, Lima, Peru. Her research interest is applications of information system technology for education, data science, and mobile programming (email: u201714981@upc.edu.pe). Miguel Casas received the bachelor for Information Systems Engineering, Peruvi- an Universidad Peruana de Ciencias Aplicadas, Lima, Peru. He is currently working as a technology consultant, Bogota, Colombia. His research interest is applications of information system technology for education, artificial intelligence, and mobile pro- gramming (email: u201714074@upc.edu.pe). David Mauricio is a Professor in Universidad Peruana de Ciencias Aplicadas, Faculty of System Engineering and Informatics, Lima, Peru. He has a doctoral degree of Science in Systems Engineering and Computing, and Master of Science in Applied Mathematics from the Federal University of Rio de Janeiro, Brazil. His research in- terest is mathematical programming, artificial intelligence, software Engineering and entreneurship (email: pcsidmau@upc.edu.pe). Jose Santisteban is a Professor in Universidad Privada Norbert Wiener, Lima, Pe- ru. He has a doctoral degree in Systems Engineering. He has more than eighteen years of experience in implementing information systems in different business sectors. His research interest areas are the study of artificial intelligence and technological entre- preneurship (Start-up) (email: jose.santisteban@uwiener.edu.pe). Article submitted 2022-09-10. Resubmitted 2022-12-08. Final acceptance 2022-12-09. Final version published as submitted by the authors. 80 http://www.i-jim.org https://doi.org/10.21865/RIDEP60.3.10 https://doi.org/10.21865/RIDEP60.3.10 https://doi.org/10.3758/BRM.41.2.309 https://doi.org/10.3390/ijerph191912716 https://doi.org/10.3390/ijerph19137834 https://doi.org/10.3390/ijerph19041951 https://doi.org/10.3390/ijerph19041951 mailto:u201714981@upc.edu.pe mailto:u201714074@upc.edu.pe mailto:pcsidmau@upc.edu.pe mailto:jose.santisteban@uwiener.edu.pe