International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 04, 2021 Short Paper—Using Design-Based Research to Develop Innovation in Open University in Thailand Using Design-Based Research to Develop Innovation in Open University in Thailand https://doi.org/10.3991/ijim.v15i04.19007 Gan Chanyawudhiwan, Kemmanat Mingsiritham () Sukhothai Thammathirat Open University, Nonthaburi, Thailand kemmanat.min@stou.ac.th Abstract—This paper discusses Design-Based Research (DBR) to develop an innovation in a distance university. The innovation focuses on self-study through technology. It has resulted in 3 aspects, which are 1) Instructors can as- sess knowledge and prepare the content from both internal and external learning sources following students' needs, 2) Students can analyze their ability in order to plan successful online learning by themselves, and 3) Instructional designers can analyze, plan, design activities, and select learning resources following the content and learning behavior. This paper describes the concept of DBR with the sample of "STOU Smartlearn" innovation design, which is a smart evalua- tion system of Sukhothai Thammathirat Open University (STOU). The innova- tion was designed based on the DBR process, which consists of five steps: Ana- lyze, Design, Develop, Implement, and Evaluation. Testing and revising the system occurred in every step. It has been used with students and could be modified to suit students' needs. The two experiments with higher education students in the academic year 2018 and 2019 found that the effect size increased from .27 to .32. Keywords—Design-Based Research, Innovation, Distance learning, Smart technology, Open University 1 Introduction Sukhothai Thammathirat Open University offers distance learning, which students mainly learn through media by themselves. Therefore, organizing learning according to the learner's context, is essential. Each student has different learning abilities and learning styles. In the past, most instructions were not designed to be tailored to each learner. It is necessary to analyze the different learners. Therefore, the author chose to use Design-Based Research (DBR) to develop and improve teaching and learning in line with the learner's context. DBR is a research methodology that aims to improve instructional management's performance by analyzing, designing, and developing learning activities, curriculum, and media. The design is put into practice. It has interactive, iterative, and flexible features. The evaluation results are used to improve the innovation to be most appro- priate. Also, the work process is flexible during the research process, resulting in 126 http://www.i-jim.org https://doi.org/10.3991/ijim.v15i04.19007 mailto:kemmanat.min@ Short Paper—Using Design-Based Research to Develop Innovation in Open University in Thailand highly flexible research [1], [2], [3], [4]. DBR is a design based on the context of the needs and learning expectations of learners. 2 Design-Based Research: DBR DBR is the development of learning innovations by using research as an innovative design process. The developed innovation has experimented. The research results can benefit the researchers who will learn during the research. The results are innovations and design principles that can be widely applied. A vital feature of the design-based research is the repetition during the research process to improve the innovation to be practical to students. This is different from the practical research in which the research process is conducted by teaching professionals to improve their teaching and learning [4], [5]. Design-based research has 5 important features: [2], [4], [6], [7] 1. Pragmatic: The innovation must be able to apply in a real situation or problem that needs to be solved. 2. Ground: The innovation or design must originate from the actual working condi- tions that the researcher wants to solve. The design must include the intervention. There must be an expansion and adjustment of the design principle and related the- ories. 3. Interactive, iterative, and flexible: The design-based research process must be iterative and use evaluation results to improve the most appropriate design. The work process is flexible during the research process. Therefore, this kind of re- search is highly flexible. 4. Integrative: Another essential feature of design-based research is that there must be the integration of multiple research methods in the study. Both quantitative and qualitative paradigms should be employed to obtain various and useful information depending on each research's needs. The researcher must examine the information to make the findings credible and find evidence to confirm and support the theoret- ical principles under which innovations are created. 5. Contextual: The design of innovations must be based on or consistent with the re- search and user context. Since this type of research focuses on concluding refer- ences, the researcher must pay close attention to data collection throughout the re- search period to ensure that the research results are relevant to the context and the needs of those involved. The research results must propose a guideline to apply in a new context. 2.1 DBR and distance education DBR is an interactive, iterative, and flexible operation relevant to the research and development of innovations for distance education. Distance education emphasizes self-learning. Learners are diverse in age, occupation, work experience, and social diversity. The development of technology media must be flexible for learners with iJIM ‒ Vol. 15, No. 04, 2021 127 Short Paper—Using Design-Based Research to Develop Innovation in Open University in Thailand different characteristics. It must respond to the needs of learners who have to study with self-direction. Learning can occur anywhere without instructors to monitor. The development of innovation for self-learning is a continuous and iterative process to obtain an innovation that responds to different learners. As [8], [9], [10] mentioned that innovation required creative initiatives. It is unnecessary to be new things, but it should contribute to the development and can be applied in real situations. Education- al innovation is also the art of teaching that affects knowledge with new concepts that make learning valuable. It will have a positive impact on student learning and does not attach to a specific person. It must also be able to apply to other contexts with similar characteristics to achieve acceptable practices and improve education quality. 2.2 Developing innovation at STOU by DBR In the distance education system, instructors and students are far apart, and learn- ing does not occur face-to-face. Media is used to transfer information and news and acts as communication channels. The production of media must be a collaboration between instructors and instructional designers. Media design for distance education must consider the quality and efficiency of learning to meet learners' needs with dif- ferent characteristics. Sukhothai Thammathirat Open University is an open university focusing on self- study that provides distance learning. Students find knowledge and learn by them- selves according to their ability. Allowing learners to know their background knowledge is essential to enable them to analyze themselves to achieve learning. They will know their needs for any additional content. Therefore, a pretest is necessary to enable students to know their abilities. STOU has developed a learning evaluation system that can analyze learners’ abilities and recommend learning channels. "STOU Smartlearn" is an evaluation system with an emphasis on analytical capabilities to distinguish students' learning ability. The system introduces earning channels and guides students to learn the content and fulfill the content necessary from internal learning sources and external learning sources. 3 Methodology 3.1 STOU smartlearn STOU Smartlearn is an evaluation system with analytical capabilities to distinguish students’ learning ability. Tools and strategies for measurement and evaluation rele- vant to the way of life, behavior, and realities of learning context have been devel- oped. Students can assess their necessary abilities and needs. The results reflect knowledge, skills, and competencies in which the system can introduce learning channels and guide students to learn and fulfill the experience in the content necessary for them. It focuses on connecting with the learning outcome of users and offers sug- gestions for successful and suitable learning. It provides information and responds on time and offers sources of learning outside the lessons that will help increase 128 http://www.i-jim.org Short Paper—Using Design-Based Research to Develop Innovation in Open University in Thailand knowledge for learners. To develop innovation, the system was assessed by 12 ex- perts. It was found that the quality was at the highest level (Mean = 4.77, SD = 0.48). STOU Smartlearn focuses on developing instructional design, according to DBR (see Figures 1 and 2). Fig. 1. Developing STOU Smartlearn by DBR Fig. 2. DBR process 3.2 The sample and the procedure This research is experimental design research with one group pretest and posttest design. The researchers conducted experiments with higher education students en- rolled in the 52311 First Aid and Basic Treatment course 2th Semester, the academic year 2018 and 2019. iJIM ‒ Vol. 15, No. 04, 2021 129 Short Paper—Using Design-Based Research to Develop Innovation in Open University in Thailand A tool used as a pretest and posttest was a five-choice multiple test withs 35 items. The values for reliability coefficients of the pretest and posttest were 0.87 and 0.83, respectively. 4 Results 4.1 Iteration 1 results The first experiment was conducted with 106 higher education students enrolled in the 52311 First Aid and Basic Treatment course, 2nd semester, the academic year 2018, for 15 weeks. It was found that the average posttest scores were 27.55 and the standard deviations were 4.57, which was higher than the pretest score with average scores at 21.62 and standard deviation at 4.76. There was statistical significance at the level of .05 with the effect size equal to .27 (see Table 1). Table 1. Comparison results of the effect size of academic achievement academic year 2018 2018 n Full scores Mean SD t p-value Effect size (d) pretest 106 35 21.62 4.76 -13.96 .00 .27 posttest 106 35 27.55 4.57 From the interview with instructors and students, it was found that there was no preparation for students before the course in the past. When learning with this support system, learners can prepare themselves before learning. They know their strengths and weaknesses that should be improved. This allows students to plan their studies better. They can answer the instructors’ questions more correctly. The system can give feedback and advice in incomplete sections, making it possible to analyze the test better. Besides, there are additional suggestions as follows. 1. STOU Smartlearn login system was complicated and difficult to access. Therefore, it was combined with the university's online learning system to be a single login system. 2. The test should show the status to let students know which items have been com- pleted or not done yet. It may use colors to show the status of the test. The re- searchers chose to use green color for completed tests and red color for preliminary tests. 3. When finishing the test, there should be a button for students to confirm before submitting. 4. The score report should be available for instructors and students to download. 4.2 Iteration 2 results The second experiment was conducted with 90 higher education students enrolled in the 52311 First Aid and Basic Treatment course, 2nd semester, the academic year 130 http://www.i-jim.org Short Paper—Using Design-Based Research to Develop Innovation in Open University in Thailand 2019. It was found that the average posttest scores were 28.11 and the standard devia- tions were 4.63, which was higher than the pretest score with average scores at 20.07 and standard deviation at 5.24. There was statistical significance at the level of .05 with the effect size equal to .32 (see Table 2). Table 2. Comparison results of the effect size of academic achievement academic year 2019 2019 n Full scores Mean SD t p-value Effect size (d) pretest 90 35 20.07 5.24 -14.16 .00 .32 posttest 90 35 28.11 4.63 From the interview with instructors and students, it was found that STOU Smartlearn can better understand the content. Learning can occur anywhere and any- time. It gives confidence to students because the activities in each practice helps cre- ate more expertise. The media used can also enhance the knowledge gained from books and help to create better understanding. There are additional suggestions as follows. 1. There should be a test bank in which instructors can choose test items by them- selves. Testing should be adaptive. 2. There should be a symbol or flag for the item that the student has not answered yet to go back to complete it. 3. The timing function should be added for each test if instructors want to set the test time. When comparing the effect size classified by 2018 and 2019, it was found that the effect size increased from .27 to .32, respectively (see Figure 3). Fig. 3. Comparison of effect size year 2018 and 2019 0.27 0.32 0.24 0.26 0.28 0.3 0.32 0.34 1 2 Effect Size Comparison iJIM ‒ Vol. 15, No. 04, 2021 131 Short Paper—Using Design-Based Research to Develop Innovation in Open University in Thailand 5 Results and Lessons Learned DBR was introduced to the development of STOU innovation, which has been im- proved and implemented to respond to learners' and instructors' needs. From the ex- periment, it can be seen that the effect size increased. This shows that learners can learn better. STOU Smartlearn meets the needs of students well. The system can ana- lyze learners and make learning suggestions regarding the students' content from internal learning sources and external learning sources. Students can learn by them- selves, make a study plan, and set study goals consistent with the context of distance education. They can study freely and choose and seek knowledge from various chan- nels and media. Therefore, students need to have good learning habits. A good learn- ing plan should be set from the beginning. Smart technology helps to analyze learners from the beginning and enables them to have good planning from the outset. Students can learn and practice the activities during their study with essential guidance. This will help learners prepare and follow the learning process step by step [11]. Technol- ogy contributes to new instruction methods, making learning more comfortable and faster, and creating more interaction between instructors and learners. These are es- sential for self-learning in the distance education system [12], [13]. If the technology is implemented and continuously developed using a systematic combination of both quantitative and qualitative methods, instructors will be able to make comprehensive learning plans that genuinely meet the needs of the online environment [14], [15], [16]. 6 Conclusion The development of technology has made learning in distance education systems convenient, fast, and consistent with self-learning. DBR ongoing research and devel- opment should be done, as [6] stated that this revealed the current gap in DBR re- search studies focused on the iterative design and implementation of digital technolo- gy interventions in undergraduate education. STOU Smartlearn is an innovation that meets the needs of different learners. It can be used in various devices according to the preference of each learner. The system enables learners to study anytime and an- ywhere. As [17] said, new technologies stimulate learners' interest, significantly, the online learning environment, which is easy to use, allowing learners to develop skills and a positive attitude. 7 Acknowledgement This research has supported a budget for the development of distance education media from Research and Development Research Institute, Sukhothai Thammathirat Open University. 132 http://www.i-jim.org Short Paper—Using Design-Based Research to Develop Innovation in Open University in Thailand 8 References [1] Conati, C. (2009). Intelligent Tutoring System: New Challenges and Directions. Proceed- ings of the Twenty-First International Joint Conference on Artificial Intelligence (IJCAI- 09): Pasadena, California, USA. July 11-17, 2009, 2-7. [2] Wang, F., & Hannafin, M. J. (2005). Design-Based Research and Technology Enhanced Learning Environments. Educational Technology Research and Development. 53(4), 5-23. Retrieve from https://link.springer.com/content/pdf/10.1007%2FBF02504682.pdf. https://doi.org/10.1007/bf02504682 [3] Edelson, D. C. (2002). Design Research: What We Learn When We Engage in Design. Re- trieve from https://www.cs.uic.edu/~i523/edelson.pdf https://doi.org/10.1207/s15327809 jls1101_4 [4] Wongwanich, S . (2016). Education Research Methodology: Inspiring New Perspectives. Bangkok :Icon Printing . [5] Lewis, C. (2015). What is improvement science? Do we need it in education? Educational Researcher. 44(1), 54–61. https://doi.org/10.3102/0013189x15570388 [6] Ford, C., McNally, D. & Ford, K. (2017). Using Design-Based Research in Higher Educa- tion Innovation. Online Learning. 21(3), 50-67. https://doi.org/10.24059/olj.v21i3.1232 [7] McKenney, S., & Reeves, T. C. (2014). Educational Design Research. In J. M. Spector, M.D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of Research on Educational Communications and Technology, New York: Springer, 131-140. http://dx.doi.org/10.1007/978-1-4614-3185-5_11 [8] Nicholls, A. (2018). Managing Educational Innovations. Routledge library editions: Edu- cation management. Vol 17, New York. [9] Ukoette, I. U, Etim, P. J & Effiong E. A. (2019). Electronic Learning as Educational Inno- vation and Students' Academic Performance in University of Uyo, Akwa Ibom State. Nige- rian Journal of Business Education. 6(2), 444-453. [10] Serdyukov, P. & Serdyukov, H. (2017). Innovation in Education: What works, What doesn't and What to do about it? Research in innovative teaching and learning. (10)1, 4-33. https://doi.org/10.1108/jrit-10-2016-0007 [11] Sasai. L. (2007). Factors Effecting Completion of Bachelor’s Degree Programs of Su- khothai Thammathirat Open University Students. Master of Education Degree in Educa- tional Measurement, Srinakharinwirot University. [12] Galitskaya, V., & Drigas, A. (2019). ICTs and Geometry. International Journal of Engi- neering Pedagogy (iJEP), 9(5), 103-111. https://doi.org/10.3991/ijep.v9i5.11241 [13] Norasykin, M. Z., Fatima, S. Y., Nurbiha, A. S., Mohd, N. H., Aede, H. M., & Desi, R. (2018). Integration of Peer Instruction in Online Social Network to Enhance Higher Order Thinking Skills. International Journal of Interactive Mobile Technologies (iJIM). 12(8), 30-39. https://doi.org/10.3991/ijim.v12i8.9672 [14] Godfrey, M., Andreas, P., Ghislain, M. N., & Pual, B. M. (2017). Learning Groups in MOOCs: Lessons for Online Learning in Higher Education. International Journal of Engi- neering Pedagogy (iJEP). 7(2), 109-124. https://doi.org/10.3991/ijep.v7i2.6925 [15] Bannan, B. (2013). The Integrative Learning Design Framework: An illustrated example from the domain of instructional technology. In T. Plomp & N. Nieveen (Eds.), Education- al Design Research - Part A: An introduction (114-133). Enschede, The Netherlands: Netherlands Institute for Curriculum Development. Retrieved from http://international.slo.nl/publications/edr/ [16] Kelly, A. E. (2013). When is Design Research Appropriate? In Plomp, T., & Nieveen, N. (Eds.), Educational Design Research - Part A: An introduction (134-151). Enschede, The iJIM ‒ Vol. 15, No. 04, 2021 133 https://link.springer.com/content/pdf/10.1007%2FBF02504682.pdf https://doi.org/10.1007/bf02504682 https://www.cs.uic.edu/~i523/edelson.pdf https://doi.org/10.1207/s15327809jls1101_4 https://doi.org/10.1207/s15327809jls1101_4 https://doi.org/10.3102/0013189x15570388 https://doi.org/10.24059/olj.v21i3.1232 http://dx.doi.org/10.1007/978-1-4614-3185-5_11 https://doi.org/10.1108/jrit-10-2016-0007 https://doi.org/10.3991/ijep.v9i5.11241 https://doi.org/10.3991/ijim.v12i8.9672 https://doi.org/10.3991/ijep.v7i2.6925 http://international.slo.nl/publications/edr/ Short Paper—Using Design-Based Research to Develop Innovation in Open University in Thailand Netherlands: Netherlands Institute for Curriculum Development. Retrieved from http://international.slo.nl/publications/edr/ [17] Papadakis, S., Kalogiannakis, M., Sifaki, E., & Vidakis, N. (2018). Evaluating Moodle use via Smart Mobile Phones. A case study in a Greek University. EAI Endorsed Transactions on Creative Technologies. 5(16), 1-9. https://doi.org/10.4108/eai.10-4-2018.156382 9 Authors Gan Chanyawudhiwan is an Assistant Professor at the Office of Educational Technology, Sukhothai Thammathirat Open University Nonthaburi 11120, Thailand. Research interests include Instructional Design, Learning Design, Universal Design, User Experience (UX) Design, User Interface (U.I.) Design, and Human-computer interaction (HCI). Kemmanat Mingsiritham is an Associate Professor at the Office of Educational Technology, Sukhothai Thammathirat Open University Nonthaburi 11120, Thailand. Research interests include Instructional Design, Virtual Learning Design, and Dis- tance Learning. Email: kemmanat.min@stou.ac.th Article submitted 2020-10-02. Resubmitted 2020-12-31. Final acceptance 2020-12-31. Final version published as submitted by the authors. 134 http://www.i-jim.org http://international.slo.nl/publications/edr/ http://international.slo.nl/publications/edr/ https://doi.org/10.4108/eai.10-4-2018.156382 mailto:kemmanat.min@