RESEARCH PAPER Integration of E-learning into the Physiology Education of Medical Students in their Pre-clinical Curriculum Radha Patel a Kelly J Kovacs a Christopher Prevette a Tian Chen b Coral D Matus a and Bindu Menon a Coresponding author(s): Bindu.Menon@utoledo.edu aDepartment of Medical Education, University of Toledo College of Medicine and Life Sciences, Toledo, Ohio 43614,, and bDepartment of Mathematics and Statistics, University of Toledo, Toledo OH 43614 Purpose: In our institution, we initiated integrated learning in the format of Learning Modules (LM), interactive audio/visual mod- ules developed by the faculty in alignment with session learning objectives, as pre-work for in-class sessions. This pilot study ex- amined students’ perceptions of this new learning method and effectiveness in helping them achieve content mastery. Meth- ods: The instructor provided the LMs in advance, allowing a self- paced introduction of critical concepts that were subsequently discussed in detail during the in-class learning (ICL) session. A cognitive diagnostic assessment was used to analyze the stu- dent’s performance on the exams. This involved identifying six skills, one or more of which were marked to be necessary for answering each exam question correctly. A question-by-skill Q- matrix was constructed, followed by analysis using a Determin- istic Input, Noisy "And" Gate (DINA) model. Results: 70% of the students rated the new approach as "Excellent or Good." 63% of the students attained skills necessary to answer questions that involved integrating information gained separately from LM and ICL sessions. However, only 28% of the students achieved mas- tery in all the six skills. Conclusion: This integrative learning system allowed for time optimization since ICL sessions could focus on more interactive aspects of the content. self-learning | medical physiology | undergraduate medical education | learning modules The implementation of active self-directed learning (SDL) hasbecome an integral part of academic learning at all levels of education. Active SDL is particularly gaining interest and becom- ing a salient feature in many medical schools throughout the United States due to the emphasis on SDL for Liaison Committee on Medi- cal Education (LCME) accreditation. Additionally, when following the Cognitive Flexibility Theory of knowledge acquisition princi- ples, there is a significant positive contribution to the learner’s mas- tery of content (1, 2, 3). The concept of active SDL has been thought of as an "approach where learners are motivated to assume personal responsibility and collaborative control of the cognitive and contextual processes in constructing and confirming meaningful and worthwhile learning outcomes" (4). This form of directed learning requires students to participate in structured activities designed by teachers while indi- vidually engaging with the material (5). Active SDL has given stu- dents the ability to review material at their own pace and allows edu- cators to cover more material through in-person and online learning. Through this approach, students engage with the material on an in- dividual level, encouraging them to go beyond mere memorization and further into comprehension and application of the material (5). E-learning is a form of active SDL that involves the "delivery of education through Information and Communication Technology (ITC)" (5, 6). The employment of E-learning in medical education has been suggested to positively affect learning outcomes and allow students agency over their learning (7). The current study exam- ined the use of E-learning in the format of Learning Modules (LM) [defined internally as interactive audio/visual modules developed by faculty in alignment with course learning objectives] to intro- duce focused SDL for the medical students in their second year. We examined the assessment outcomes to study the effectiveness of the new method. This study shows that E-learning is a learning enhancer, when used as a facilitating feature integrated into the tra- ditional medical education model. Submitted: 06/01/2021, published: 07/02/2021. 12–15 UTJMS 2021 Vol. 9 utdc.utoledo.edu/Translation https://orcid.org/0000-0002-4436-8208 mailto: Bindu.Menon@utoledo.edu Materials and Methods Approach: The new integrated method was introduced to the second-year (M2) class of medical students in their foundational science curriculum (class size, 177) during the gastrointestinal (GI) physiology sessions. The course instructor assigned short e-learning modules (LMs) to the students in preparation for traditional lecture style in-class sessions. Preparation of LMs: The LMs were prepared on the Articulate 360 Rise TM platform. Each LM is expected to take approximately 15-30 minutes for completion. The LMs also had short quizzes in- corporated into them to assess knowledge acquisition. It was clearly communicated to the students that the LMs were required pre-work to be completed before the in-class session. Since the LMs were delivered through the online learning management system, Black- board TM, the instructor could access important analytical data such as the number of times each student viewed the LMs and the time spent on each view. Survey: The researchers conducted a short closed-ended one- question survey to assess student satisfaction with the integrated pedagogies. The survey was administered via the online Poll Ev- erywhere TM software and was open to everyone that attended the in-class session. The overall response rate was 31%. All submis- sions were reported anonymously. Assessment Outcome: We used a cognitive diagnostic assess- ment to analyze students’ performance on the final course exam. The instructor identified six skills, one or more of which were marked to be necessary for answering each exam question correctly (Table 1). A question-by-skill Q-matrix was constructed. For exam- ple, we determined that to answer a question on gastroesophageal reflux disease, students must remember the sphincter muscle’s role at the lower end of the esophagus and needed to possess two skills: S1 and S3 (see Table 1 for skills). Then, a Deterministic Input, Noisy "And" Gate (DINA) model were used to analyze the data (8) of the entire student body (n=177) and estimate the percentage mas- tery for each skill. The student performance data from 12 questions on this content that appeared in the final course exam were analyzed similarly using this method. Table 1. The various skills which the students must possess to correctly answer the assessment items from GI physiology that appeared on the course exam. Student Skills S1 Recall the knowledge of the normal function of the GI tract gained from the in-class sessions (ICL) S2 Recall the knowledge gained from e-learning modules (LM) on the basic structure and function of the GI tract S3 Ability to apply the knowledge of GI physiology learned from the ICL to solve a clinical problem S4 Ability to apply the knowledge of GI physiology learned from the LMs to solve a clinical problem S5 Ability to integrate the knowledge gained separately from the ICLs and LMs and develop a broad understanding of GI physiology S6 Ability to integrate the knowledge of physiology concepts gained separately from the ICLs and LMs to solve a clinical problem Results The survey results showed that the majority of the students ap- preciated the new method; 25% of the responders gave an "excel- lent" (4 out 4) rating, while 55% thought it was "Good" (rating of 3 out 4). Only 5% thought it was below average (1 out of 4). The re- sults of the DINA model analysis are shown in Figure 1. We found that 28% of students acquired complete mastery in GI Physiology content; they attained mastery in all six skills. Mastery in skill 5, which required integration of knowledge from the two separate ped- agogies, was acquired by 63% of the students. Similarly, skill 6, which required the application of informa- tion obtained from the entire course to solve a real-life problem in a clinical scenario, was mastered by 64%. A significant number of students who mastered skills 5 and 6 had also attained mastery in the ICL session content, thus stressing the importance of in-class sessions. It is important to note that none of the students who did not use institutional resources, by not utilizing the LMs and not at- tending the in-class sessions (49%), attained complete mastery. Discussion As our technology advances, so do the methods by which we can process and disseminate information; learning now is not lim- ited to geographic location and has given students more autonomous access to information. With this increased agency over ones’ learn- ing comes the need for adaptive learning styles that encourage stu- dents to become life-long active, self-directed learners. Through the advancement of learning aids, students can acquire and master critical skills beyond in-class learning. SDL is a central concept Patel et al. UTJMS 2021 Vol. 9 13 in adult learning and practice that integrates self-management, self- monitoring, and motivation (4). Our medical school’s integrated curriculum accounts for SDL throughout a students’ progression from the preclinical to clinical years. The application of the LMs is in response to the changes in the LCME standard, redefined as "The faculty of a medical school ensure that the medical curriculum includes self-directed learning experiences and unscheduled time to allow medical students to develop the skills of lifelong learning" (Reference: Functions and Structure of a Medical School; Stan- dards for Accreditation of Medical Education Programs Leading to the M.D. degree). Fig. 1. The cognitive diagnostic assessment. The X-axis shows the various skills necessary to answer the multiple questions on the GI physiology exam successfully. Y-axis gives the percentage of students who achieved mastery in the corresponding skills. SDL involves medical students’ self-assessment of learning needs; independent identification, analysis, and synthesis of rele- vant information; appraisal of the credibility of information sources; and feedback on these skills (Reference: Functions and Structure of a Medical School; Standards for Accreditation of Medical Ed- ucation Programs Leading to the M.D. degree). SDL is based on the principle that learner exercises independence in deciding what is worthwhile to learn and how to approach the task to reach the desired outcomes (4). It is defined as an approach where learn- ers are motivated to assume personal responsibility and collabora- tive control of the cognitive (self-monitoring) and contextual (self- management) processes in constructing and confirming meaningful and worthwhile learning outcomes. However, it is also believed that there is no standard way of implementing a curriculum fo- cused around self-directed learning, but experts have proposed a model that integrates the metacognitive theory of learning (9, 10). In this approach, the learners are to be provided with choices of how they wish to carry out the learning process proactively; "ma- terial resources should be available, approaches suggested, flexi- ble pacing accommodated, and questioning and feedback provided when needed (11). We incorporate this into our model by allow- ing students to exercise focused self-directed learning in preparation for the in-class sessions. In other words, we complement indepen- dent student learning by providing credible material in the form of LMs to aid in the synthesis of information relevant to their identi- 14 utdc.utoledo.edu/Translation Patel et al. fied needs. While completion of LM’s is not tracked or awarded a grade, we strongly recommend them as a learning source. We do not discourage students from using other resources instead of LMs. However, the central framework on which the SDL should be fo- cused is given in the form of learning objectives disseminated at the beginning of the LMs. Our intended outcomes were to assess the efficacy of learning modules and in-class learning on mastery of core concepts. The assessments covered core concepts of the reformed systems-based curriculum at the University. A total of six binary skills were as- sessed when answering the exam questions from the GI physiology curriculum. These skills consisted of recollection of information, applying information, synthesizing information, and integrating in- formation from the in-class learning and e-learning modules. They were categorized to assess a student’s ability to build upon each es- sential skill and ultimately incorporate information from both learn- ing materials to achieve mastery of the content. Our results show that better acquisition of content mastery was found to stem from e- learning modules (81% mastery in skill 2; fig 1) instead of in-class learning sessions (53% mastery in skill 1). However, we caution that faculty guidance is needed to integrate the information from the learning modules with in-class learning for complete mastery of the clinical application of knowledge. However, it is of concern that only 28% of the students achieved complete mastery. This could be attributed to the sessions not be- ing made as mandatory. Several of the students chose not to use the institutional resources (49%). It would be interesting if we could an- alyze similar data from the NBME formatted end-of-course exams and see if there is a correlation to the observed trend in that exam. More importantly, one of the commonly acknowledged shortcom- ings of the pass-fail system is that students can still pass the exam even if they do not acquire complete mastery of the skills. Limitations and Next Steps This pilot study was conducted as a first step in the introduc- tion of SDL in our institution. Two critical components of SDL, as defined by the LCME, are missing from our model; \share the infor- mation with their peers and supervisors" and \receive feedback on their information-seeking skills". Currently, we are in the process of introducing a platform for the students to share with their peers the knowledge acquired from various sources such as LMs. The other missing element, providing feedback to the students, will also be introduced in the next step of the implementation. Another limitation of the study is that it is a preliminary study and uses data from students in the 2018-2019 academic year. We are currently in the process of analyzing data from the students in the more recent classes (2019-2020 and 2020-2021) as well as previous years to examine the new method’s effectiveness in acquiring and retrieving knowledge. Once we have obtained more comprehensive data, it will provide more critical information about the utility of LMs in student learning. Conclusion This pilot study has two major components that contribute sig- nificantly to the field of medical education. We show that providing source material for SDL in the form of LMs can be an efficient learning method to acquire and retain information considering the challenges posed by the ever-expanding wealth of knowledge. Sec- ondly, quantitative analyses, such as the kind used in this study, can actively assess curriculum strengths and weaknesses based on stu- dents’ performances and content mastery. Thus, it allows instructors to refine their teaching approaches and deliver more effective and streamlined teaching material, giving them a more focused clinical knowledge and mastery approach. Conflict of interest Authors declare no conflict of interest. Institutional Review Board approval All analysis and survey involving the students were in accor- dance with the standards of the institutional research committee (University of Toledo Institutional Review Board). The study was granted exempt status by the IRB. Authors’ contributions Conceptualization: C.M, and B.M. Methodology: C.P and K.J.K. Data curation: K.J.K; Analysis: T.C and B.M; Writing: orig- inal draft: R.P; review and editing: K.J.K and B.M. 1. Spiro, RJ, RL Coulson, PJ Feltovich, and D Anderson 1988 Cognitive flexibil- ity theory: Advanced knowledge acquisition in ill-structured domains. Hillsdale, NJ: Erlbaum. 2. Spiro, RJ, PJ Feltovich, MJ Jacobson, and RLIE Coulson 1992 Cognitive flexi- bility, constructivism and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. Hillsdale, NJ: Erlbaum. 3. Spiro, RJ, and JIe Jehng 1990 Cognitive flexibility and hypertext: Theory and technology for the non-linear and multidimensional traversal of complex subject matter. Hillsdale, NJ: Erlbaum. 4. Garrison, DR 1997 Self-Directed Learning: Toward A Comprehensive Model. Adult Education Quarterly 48 18-33. 5. Gleason, BL, MJ Peeters, BH Resman-Targoff, S Karr, S McBane, K Kelley, T Thomas, and TH Denetclaw 2011 An active-learning strategies primer for achiev- ing ability-based educational outcomes. Am J Pharm Educ 75 186. 6. Lawn, S, X Zhi, and A Morello 2017 An integrative review of e-learning in the de- livery of self-management support training for health professionals. BMC Med Educ 17 183. 7. Fransen, F, H Martens, I Nagtzaam, and S Heeneman 2018 Use of e-learning in clinical clerkships: effects on acquisition of dermatological knowledge and learning processes. Int J Med Educ 9 11-17. 8. Bangeranye, C, and YS Lim 2020 How to Use Cognitively Diagnostic Assess- ments of Student Performance as a Method for Monitoring and Managing the Instructional Quality in Undergraduate Medical Education. Acad Med 95 145- 150. 9. Flavell, JH 1976 Metacognitive aspects of problem solving. Hillsdale, NJ: Erl- baum. 10. Flavell, JH 1979 Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34 906 - 911. 11. Hurford, D, and A Read 2011 Do podcasts and screencasts enable or hinder independent learning. Practitioner Research in Higher Education 5 30-38. Patel et al. UTJMS 2021 Vol. 9 15