https://journals.4science.ge/index.php/GS/index 1 Georgian Scientists Vol. 3 Issue 3, 2021 https://doi.org/10.52340/gs.2021.08.01 Improving the Quality of Medical Education: Students’ Achievements and Perceptions on Simulation Based Learning. Irma Manjavidze1; Dali Chitaishvili2; Pirdara Nozadze 1, 2 Department of Clinical Skills and Multidisciplinary Simulation, Tbilisi State Medical University, Tbilisi, Georgia; 1 : https://orcid.org/0000-0002-9298-4187; 2 : https://orcid.org/0000-0002-7631-9785 Abstract Background. Simulation Based Learning (SBL) revolutionized medical education. Today's challenge is to control and improve the quality of the SBL. One way to monitor the quality of the learning process is to conduct regular surveys. Methods.. 4th year medical students were asked to fill out a questionnaire consisting of Likert scale, to assess the training course and SBL in general. Improvement of their knowledge after taking the course was analyzed on the basis of pre- and post-test results. Results. Eighty-two 4th year medical students responses to the 35 questions in the questionnaire were positive and the average score was 4.57 on a 5-point Likert scale. Pre- and post-test analysis proved that the course was really productive. The average points for pretest was 20 points (SD=5,39) and 29 points for pot-test (SD=4,64). Conclusions. “Clinical skills” course has caused a high satisfaction and motivation of the students and improved their knowledge. Knowledge enhancement and skills acquisition took place in a comfortable environment. Keywords: Simulation Based Learning, students’ perception, pre- and post-tests, likert scale, medical education. https://journals.4science.ge/index.php/GS/index 2 Background Georgia is in a process of innovation and reform of the higher education institutions trying to meet international standards .This process is quite complex which implies that we need to realize our role as educators, teaching should become more student-oriented and outcome-based, and medical education programs should be increasingly enriched with modern methods of teaching that will help students not only acquire theoretical knowledge, but also practical skills. Outcome-based education is focused on what the students are expected to know and to do. Nowadays simulation based learning and OSCE format assessment is considered as the best way to acquire and test performance and competence in skills such as communication, clinical examination, medical procedures etc with unnecessary risks to patient. The use of medical simulators revolutionized medical education and has brought a level of expertise and confidence to medical personnel at all levels.The idea of practicing on inanimate objects before contact with real patients backs to antiquity. Earliest simulators in the history of medicine were used for teaching anatomy, surgery, obstetrics etc. (1) When describing the history of simulation based learning Madame Du Cudrey, the “King’s Midwife” must be mentioned. She was midwife in the court of King Louis XV and in the 1700s she created the "machine" i.e. simulators to train midwives in France (2). Her mannequins were very popular as they looked very realistic and could be used to learn how to manage normal childbirth and childbirth complications. Also noteworthy is the German Professor B. Schultze, head of the University Women’s Clinic in Jena, who created bony pelvic phantoms in the 1890s; For the teaching of pelvimetry (2). The beginning of the twentieth century is considered as the "black age" of simulation teaching. And it received special attention back in the late 20th century. The fundamental reforms of medical education in the 20th century were driven by a significant discovery, which was made in 1984 by the Harvard Medical Practice Study 1, which randomly selected 30,000 hospitals in New York State, with histories showing that medical malpractice affected 3.7% of hospital admissions, of which 27.6% were due to negligence. 13.6% of them caused death. (3) A 1999 report by the Institute of Medicine (IOM), Err Is Human, found that medical errors harmed approximately 3% of hospital patients and resulted in an estimated 98,000 deaths in the United States. (4) The modern model of medical simulation training is considered to be taken from aviation, where errors are reduced to zero, because in such industries, which are termed as high reliability organizations there is a high culture of security, which is somewhat conditioned by simulation based training. The need to learn lesson from aviation was driven by the issue of patient safety. And over past decades many centers and laboratories of simulation training in medical schools were opened, and training courses were prepared. https://journals.4science.ge/index.php/GS/index 3 The call to enrich curriculum with modern teaching methods which will ensure to fill the existing gap between reality and classrooms led to the inclusion of Simulation Based Learning (SBL) in the undergraduate curriculum. Today's challenge is to control and improve the quality of the SBL. Tbilisi State Medical University (TSMU) is a member of the European Higher Education Area, so it must meet the quality standards of higher education, which will determine its competitiveness throughout the world. The quality is determined by the content of the curriculum, the qualification of the teachers, the availability of relevant resources. One way to monitor the quality of the learning process is to conduct regular surveys. At TSMU the academic and invited staff as well as training courses are regularly evaluated both by the administration and the student. Appropriate questionnaires are developed by the University Quality Assurance Service and the Departments to assess the course and the teacher. At the Department of Clinical Skills and Multidisciplinary Simulation (CSMS) at the end of each rotation the student anonymously completes a questionnaire, the data of which is analyzed by the staff of Department and then with the Dean of the Faculty. The results and recommendations are discussed at the Department meeting and then at the Faculty Board, and / or, if necessary, individually with particular academic or invited teacher. Based on the evaluation result, success is rewarded with promotion and bonus. CSMS Department offers courses for various faculties of the TSMU. Among them are three compulsory training courses for students of the 2nd, 4th and 6th years of the English-language program of the Faculty of Medicine, where they are given the opportunity to learn various procedures and manipulations using manikins and simulators. The mandatory course for 4th year students (VII or VIII semester) includes the following thematic lessons: 3 days: Obstetrics and Gynecology: ” Insertion of Intrauterine Device”, “Gynecological Examination with Pap Smear” and “Active Management of the Third Stage of Labor.”; 4 days: Pediatrics: “Newborn Physical Examination”, “Newborn Emergency Care”, “Auscultation of Heart in Newborn” and “Auscultation of Newborn’s Lungs”, 1 day: Nursing: “Nasogastric intubation” and “Bladder Catheterization”. 1 day: Communication Scenarios. The aim of our study was 1. to evaluate how our rotation improves knowledge of our students. 2. to learn the perception of 4th year students' regarding our training course, SBL, teachers and learning environment. https://journals.4science.ge/index.php/GS/index 4 Methods: Our Department conducted prospective study in the spring semester from March, 2021 to May, 2021. Inclusion criteria were: VIII or VII semester students of Faculty of Medicine. There were 8 groups who met our inclusion criteria and had rotation at our Department during this period. All students were enrolled (n=82) in our study. A week before the start of our rotation, we send all our students the address of our website, where they have the opportunity to view their training schedule, presentation and video materials on each topic, etc. In the context of the pandemic, students are encouraged to come well prepared for each class in order to devote more time to practice during face to face learning. Study participants were no exception. When students came to our department, before the first class, students were invited to attend a 20- minute presentation to give an explanation of our study: in particular, that as part of this study, they would be asked to write pre- and post-tests that the evaluation of tests would not affect their rating. They were also explained how to fill out the questionnaire. Brief information about our study was provided on the title page of the pre-/post- test and questionnaire. The anonymity of the survey was emphasized. Then the students were then given 30 minutes to write a pre-test consisting of 40 questions that included thematic topics. Each question had 4 possible answers and for each correct answer the student was given 1 point, so the maximum score of the test was 40 points. At the end of the rotation there were asked to complete the same test plus the questionnaire to assess the training course. This time we used a new questionnaire, which is more adopted to simulation based learning. The questionnaire included questions from our department's old questionnaire (which was later replaced by a shorter version), some questions were taken from the literature (6), and also used questions from a questionnaire proposed by TSMU Quality Assurance Service. The questionnaire was divided into three blocks, in the first block the students were asked to evaluate the training course and express their attitudes toward SBL, in the second block they were asked to evaluate teachers and in the third block -learning environment that has a significant impact on students’ academic achievement. Students were asked to express their level of agreement with 35 items using a Likert scale ranging from 1 to 5, from strongly disagree to strongly agree with neutral option in the middle. With 36 question we asked them to rate the course: very poor, poor, moderate, good, excellent. Space was left for additional comments. Results: A total of 82 4th year medical students participated in our study between March 2021 and May 2021. Students ranged in age from 20 to 24 and had a mean age of 21.82 (SD = 0.82). The sample was represented https://journals.4science.ge/index.php/GS/index 5 by seventh- (58.5%, n = 48), and eighth- (41.5%%, n = 34) semester students. There were 45 female (54.9%) and 37 male (45.1%) participants. See Table 1 Table 1: Demographic characteristics of the sample. Sex N % Female 45 54.9 Male 37 45.1 Semester VII 48 58.5 VIII 34 41.5 M SD Range Age 21.82 0.82 20-24 Pre and post-test results were statistical analyses were performed using excel t-test. Alpha was set at 0.05, and p-values of less than 0.05 were considered statistically significant. The average points for pretest was 20 points (mode=25 points; SD=5,39) and 29 points for pot-test (mode=33 points; SD=4,64). Pre- and post-test analysis revealed that students' knowledge was significantly improved. Pre- and post-test more detailed results are presented in Table 2. Table 2. Pre- and post-test analysis. pre-test points post-test points Mean 20,31707317 29,09756098 Variance 29,10809997 21,5459199 Observations 82 82 Pearson Correlation 0,273829978 Hypothesized Mean Difference 0 df 81 t Stat -13,08228918 P(T<=t) one-tail 0,00 t Critical one-tail 1,663883913 P(T<=t) two-tail 0,00 t Critical two-tail 1,989686288 Through our questionnaire we wanted to understand the attitude of students towards our training course and simulation teaching in general. As their objective and honest answers on the questions of the questionnaire will be of the greatest help to evaluate how our department work; Analysis of the answers will aid in improving the quality of our academic activities. https://journals.4science.ge/index.php/GS/index 6 Through the first 16 statements of our questionnaire, we wanted to understand what rating is given to our particular course. Great majority of students think that the course was important, enjoyable, and they learned some manipulations and procedures while working on manikins and that they improved their skills. (See Table 3: statements:-1,2,5,6,8,11). There is no such unanimity among the students regarding the time allotted for each course and for the duration of the course. (Table 3: Statements 3, 13, 14). Majority of students think that the learning objectives were well formulated, the assessment was objective and that the learning material was appropriate (Table 3: Statements 9, 12, 15,16). Statements -4, 7, 10 of questionnaire show that students want to be better prepared for such practical classes and wish that cases were brought from real life. Table 3. Questionnaire (part-1) Statement n SA (%) A (%) N (%) D (%) SD (%) points ST DEV 1. The Course of "Clinical Skills" is important for my future profession. 82 91% (75) 9% (7) 0% 0% 0% 4,91 0,28 2. The experience has improved my skills. 82 95% (78) 5% (4) 0% 0% 0% 4,95 0,22 3. Timing for each simulation case was adequate. 82 39% (32) 30% (25) 26% (21) 5% (4) 0% 4,037 0,92 4. The degree of difficulty of the cases has been adequate to my knowledge. 82 48% (39) 48% (39) 5% (4) 0% 0% 4,43 0,59 5. I have improved my technical skills. 82 68% (56) 24% (20) 7% (6) 0% 0% 4,61 0,62 6. I have learned some clinical cases, procedures and manipulation within this course. 82 68% (56) 24% (20) 7% (6) 0% 0% 4,61 0,62 7. I knew the theoretical side of cases. 82 43% (35) 43% (35) 12% (10) 2% (2) 0% 4,26 0,77 8. I have learned from the mistakes I made during simulation. 82 78% (64) 15% (12) 7% (6) 0% 0% 4,71 0,59 9. Objectives were clear . 82 71% (58) 20% (16) 10% (8) 0% 0% 4,61 0,66 10. Cases recreated real situations. 82 51% (42) 24% (20) 24% (20) 0% 0% 4,27 0,83 11. The course was enjoyable. 82 88% (72) 12% (10) 0% 0% 0% 4,88 0,32 12. The information I received beforehand was relevant 82 66% (54) 29% (24) 5% (4) 0% 0% 4,61 0,58 13. The length of course was appropriate 82 32% (26) 15% (12) 34% (28) 7% (6) 12% (10) 3,46 1,34 14. The course content and delivery pace was appropriate 82 51% (42) 37% (30) 7% (6) 5% (4) 0% 4,34 0,82 15. The course material (ppt and videos )was appropriate for learning 82 72% (59) 18% (15) 7% (6) 2% (2) 0% 4,59 0,73 16. The grading of the course was fair 82 73% (60) 27% (22) 0% 0% 0% 4,73 0,45 https://journals.4science.ge/index.php/GS/index 7 With statements 17-25 we wanted to learn our students view on simulation based learning in general. The vast majority of them strongly agree or agree with the statements that describe the positive aspects of simulation based learning. (See Table 4). Table 4. Questionnaire (part-2) Question n SA(%) A(%) N (%) D (%) SD (%) points ST DEV 17. Simulation is useful to assess the clinical status of a patient. 82 77% (63) 21% (17) 2% (2) 0% 0% 4,74 0,49 18. Simulation practice lets you learn how to avoid making mistakes. 82 83% (68) 12% (10) 5% (4) 0% 0% 4,78 0,52 19. Simulation helps to set priorities for action 82 62% (51) 33% (27) 5% (4) 0% 0% 4,57 0,59 20. Simulation made me think about my next clinical practice. 82 62% (51) 23% (19) 15% (12) 0% 0% 4,48 0,74 21. Simulation improves communication and the ability to work with the team. 82 65% (53) 21% (17) 15% (12) 0% 0% 4,5 0,74 22. Simulation is helpful as it relates theory with practice. 82 76% (62) 20% (16) 5% (4) 0% 0% 4,7 0,55 23. Simulation promotes self-confidence. 82 71% (58) 27% (22) 2% (2) 0% 0% 4,68 0,52 24. This type of practice has increased my commitment. 82 59% (48) 34% (28) 7% (6) 0% 0% 4,51 0,63 25. Interaction with the simulation improves clinical competence. 82 60% (49) 27% (22) 13% (11) 0% 0% 4,46 0,72 With statements 26-32, we tried to get information about what assessment students gave teachers. The analysis of the questionnaire showed that students positively evaluate the work done by teachers. (see Table 5). Table 5. Questionnaire (part-3) Statement n SA (%) A (%) N (%) D (%) SD (%) points ST DEV 26. I felt comfortable and respected during the sessions. 82 88% (72) 10% (8) 2% (2) 0% 0% 4,85 0,42 27. The teacher gave constructive feedback after each simulation 82 80% (66) 20% (16) 0% 0% 0% 4,8 0,39 28. The teachers at the end of my performance helped me reflect on the cases. 82 65% (53) 30% (25) 5% (4) 0% 0% 4,59 0,58 29. The teacher s helped me correct mistakes 82 78% (64) 22% (18) 0% 0% 0% 4,78 0,42 30. The teachers were helpful and supportive 82 85% (70) 12% (10) 2% (2) 0% 0% 4,82 0,44 31. I felt able to ask any questions I had 82 78% (64) 17% (14) 5% (4) 0% 0% 4,73 0,55 32. The proposed scheme of students' work during sessions was acceptable. 82 66% (54) 27% (22) 7% (6) 0% 0% 4,59 0,63 https://journals.4science.ge/index.php/GS/index 8 With statements 33-35 of the questionnaire, we literally tried to find out how students evaluate learning environment and the work of our technical staff and support staff. Table 6. Questionnaire (part-4) Statement n SA (%) A (%) N (%) D (%) SD (%) points ST DEV 33. Facilities and equipment were real. 82 59% (48) 24% (20) 12% (10) 5% (4) 0% 4,37 0,88 34. The webpage of the centre was useful. 82 61% (50) 22% (18) 15% (12) 2% (2) 0% 4,41 0,83 35. The course was well organized. 82 65% (53) 27% (22) 9% (7) 0% 0% 4,56 0,65 With final statement we asked to rate training course in which responders were asked to specify their assessment by marking one of the following words: very poor; poor; moderate; good and excellent. 4 th year student’s level of satisfaction with “clinical skills” course is presented in diagram 1. Majority of the students (79.0%) rated their satisfaction with “clinical skills” rotation experience as excellent or good, while 7% rated as moderate or poor. 4th year student responses to the 35 questions in the questionnaire were positive and the average score was 4.57 on a 5-point Likert scale. Diagram 1. Rate the training course 0 (0%) 1(1%) 6 (7%) 7 (9%) 68 (70%) very poor poor moderate good excellent https://journals.4science.ge/index.php/GS/index 9 Conclusions Feedback from students allows us to monitor the quality of the learning process to some extent. CSMS Department regularly conduct such surveys. Analysis of the answers helps us to improve the quality of our academic activities.We always respond to the trends reflected in the questionnaire answers. There were several cases where it was on the basis of these results that we made changes to the curriculum. At the end of the course, the final evaluation of the students is always done by the Objective Structured Clinical Examination (OSCE), however, before the OSCE our student have to write the MCQ. Within this study we ask our students to write the test before and after the training course thus we tried to reveal how much the knowledge of students improved after taking our course. The best evaluation of the course we think we read in the comments of the questionnaire. "When I was writing the pretest, I wondered how I could answer such difficult questions and I could not imagine that at the end of the course I would be able to answer all the questions and that I would be able to do so many manipulations on my own." https://journals.4science.ge/index.php/GS/index 10 References 1. Gardner R, Raemer DB. Simulation in obstetrics and gynecology. Obstet Gynecol Clin North Am. 2008 Mar;35(1):97-127, ix. doi: 10.1016/j.ogc.2007.12.008. PMID: 18319131. 2. Buck GH. Development of simulators in medical education. Gesnerus. 1991;48 Pt 1:7-28. PMID: 1855669.. 3. Bates DW, Spell N, Cullen DJ, Burdick E, Laird N, Petersen LA, Small SD, Sweitzer BJ, Leape LL. The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA. 1997 Jan 22-29;277(4):307-11. PMID: 9002493. 4. Al-Elq AH. Simulation-based medical teaching and learning. J Family Community Med. 2010 Jan;17(1):35-40. doi: 10.4103/1319-1683.68787. PMID: 22022669; PMCID: PMC3195067. 5. Lateef F. Simulation-based learning: Just like the real thing. J Emerg Trauma Shock. 2010 Oct;3(4):348-52. doi: 10.4103/0974-2700.70743. PMID: 21063557; PMCID: PMC2966567. 6. Alconero-Camarero AR, -Romero AG, Sarabia-Cobo CM, Arce AM. "Clinical simulation as a learning tool in undergraduate nursing: Validation of a questionnaire". Nurse Educ Today. 2016 Apr;39:128-34. doi: 10.1016/j.nedt.2016.01.027. Epub 2016 Feb 4. PMID: 27006044. 7. Codeço A, Dias Coutinho VR, Pereira-Lopes O, Faria-Almeida R, Santos Resende M. Assessing clinical simulation as a learning tool when training motivation skills in Periodontology- Students' perceptions. Eur J Dent Educ. 2020 Nov;24(4):644-649. doi: 10.1111/eje.12544. Epub 2020 Jun 1. PMID: 32396273.