Short Research Report 168 December 2017, Vol. 9, No. 4 AJHPE Chest trauma is one of the most common surgical conditions seen in our teaching hospital at the University of Botswana and in the southern African region.[1] Therefore, intercostal chest drain (ICD) insertion is regarded as a core skill in the medical curriculum. During surgical rotations and assessments we observed varying exposure of students to surgical conditions, despite the high surgical disease burden, and also varying levels of competence in managing chest injuries. Simulation-based training plays a significant role in current surgical training practice,[2] and we hypothesised that chest trauma management, knowledge and skills may be improved by employing low-cost task trainers. Simulation describes a full-range use of physical objects (task trainers) or situations for mimicking real-life scenarios or functions during training. It plays a significant role in current health professions education, and was introduced to address the decreasing exposure to real patients in surgery[2,3] and improve teaching and assessment in medicine.[4] Simulation-based training and its assessment for both cognitive and psychomotor learning domains have been studied extensively.[5] This study employs a pre- and post-test design[6] to assess an educational intervention based on the Kirkpatrick evaluation model for training.[7] The Kirkpatrick model defines four evaluation levels. Levels 1 and 2 evaluate short-term outcomes, such as acceptability and change in level of knowledge and skills, while levels 3 and 4 evaluate behavioural change associated with training and the organisation’s ultimate gain. We hypothesised that chest trauma management, knowledge and skills may be improved by employing simulation in the form of locally made low- cost task trainers. The overall purpose of this pilot study was to improve management of chest trauma. We included simulation in the previously theory-based chest trauma training module and thereafter tested trainees’ knowledge and skills with regard to the effectiveness of the intervention. We also tested acceptance of this new simulation-based module. Methods Research setting The study was conducted at the University of Botswana’s new Faculty of Medicine, where the competence-based curriculum employs a problem- based learning approach and traditional patient encounters. Research design This study employed a quantitative pre- and post-test design. Population and sampling All medical students (n=41) were invited to take part in the study after their last surgical rotation and newer intern doctors (n=20) during their initial rotation – before being deployed to the surgical rotation. Using convenience sampling we included 35 medical students and 14 interns. There were 39 partici- pants in the pre-test assessment and training; 14 completed all three sessions, i.e. the pre-test assessment, training and post-test assessment. Data collection instrument A checklist developed for the module was used for the pre-test and post-test assessments. Thirteen content-based items (total score of 22) evaluated the students’ knowledge relating to the diagnosis and management of a chest injury (cognitive domain) and one item (total score of 3) evaluated their ability to perform the procedure correctly (psychomotor domain). A rating Background. To improve the management of chest trauma at the University of Botswana, Gaborone, Botswana, we incorporated simulation into a theory- based chest trauma module by developing procedural guides, checklists and low-cost simulation. Objectives. To assess the suitability of low-cost simulation-based training and its impact on students’ proficiency, as well as its general acceptability. Methods. A total of 39 medical students who completed their surgical rotation and 20 intern doctors in their first clinical rotation participated. A checklist was used in a pre- and post-test design to assess procedural proficiency, and a rating system categorised scores. Thirteen content-based items assessed the students’ knowledge relating to the diagnosis and management of a chest injury and one item assessed their ability to perform the procedure correctly. A questionnaire was administered after the second assessment to evaluate the acceptability of the training module. Findings were summarised by median, proportion and range, and pre- and post-test outcomes were compared by Student’s paired t-test. Results. Pre- and post-test assessment scores differed significantly (median (range) 11.3 (4.5 - 21.0) and 19.5 (15.5 - 23.0), respectively (p<0.001)). The proportions of participants’ scores categorised as ‘full proficiency’ rose from 7% to 42%, and ‘reasonable proficiency’ from 30% to 60%, while both ‘some proficiency’ and ‘poor proficiency’ decreased from 50% and 20% to 0%. Most (93%) participants ‘strongly agreed’ that the training module was acceptable. Conclusion. Our results demonstrate the suitability of low-cost simulation for training and assessment in resource-constrained settings. Afr J Health Professions Educ 2017;9(4):168-170. DOI:10.7196/AJHPE.2017.v9i4.829 The use of low-cost simulation in a resource-constrained teaching environment M Mwandri,1 MD, MMed (Surgery); M Walsh,1 BSc, MBBS, MS, FRCS (Gen); J Frantz,2 BSc, MSc, PhD; R Delport,3 MSc, MEd, PhD 1 Department of Surgery, School of Medicine, University of Botswana, Gaborone, Botswana 2 Faculty of Community and Health Sciences, University of the Western Cape, Cape Town, South Africa 3 Skills Laboratory, Faculty of Health Sciences, University of Pretoria, South Africa Corresponding author: R Delport (rhena.delport@up.ac.za) This open-access article is distributed under Creative Commons licence CC-BY-NC 4.0. December 2017, Vol. 9, No. 4 AJHPE 169 scale was developed to describe their performance scores, and a survey based on Kirkpatrick’s model was used to assess the trainees’ acceptability of the module by trainees. Data collection and procedure Management of chest trauma was assessed before and after a training intervention. The training module comprised theoretical and practical sections. The theoretical component covered the clinical presentation, chest radiograph interpretation and management of chest trauma. The practical component was taught in a simulated environment using task trainers as low-cost simulation for the insertion of ICDs. The task trainers were built from affordable material and comprised a suture trainer and ICD-insertion trainer, respectively (Fig. 1A and B). Knowledge and understanding (cognitive assessment) and procedural proficiency (psychomotor ability) were evaluated using a checklist that was developed in accordance with the Advanced Trauma Life Support programme founded by the American College of Surgeons. The assessment was designed to evaluate suitability and acceptability of the proposed module – not for summative assessment purposes. The checklist evaluated students’ knowledge and understanding of indications for chest drain insertion, which included the use of diagnostic criteria for chest radiograph interpretations, anatomical considerations in ICD insertion, indications for referral for surgery, maintenance of ICD patency and monitoring for abnormalities regarding drainage. There were multiple possibly correct responses for each item and a mark of 0.5 was allocated for each correct response. The psychomotor section of the checklist assessed appropriate handling of instruments, economy of movements during a procedure, correct forming of knots, and correct suturing. A mark of 0.5 was awarded for each correctly performed step. If students demonstrated smooth forward progression of the procedure (incision and insertion of the drain, suturing to control leakage, anchoring of the drain), they were awarded 1 mark, as it closely demonstrates mastering of psychomotor skills and not merely observing how to perform a task. The pre-test assessment and training were performed on the same day. The post-test assessment employed the same checklist as the pre-test and was conducted 2 weeks after the training session. The scores obtained from the assessments were converted to percentages and a rating system was designed to categorise scores, ranging from ‘poor proficiency’, ‘some proficiency’, ‘reasonable proficiency’ and ‘full proficiency’ for percentage scores of <40, 40 - 59, 60 - 79, and 80 - 100, respectively. After the training session, a course evaluation survey employed Likert scales to assess acceptance of the model as formal training for chest trauma, clarity of the content, and relevance of the content to practise, as perceived by participants who completed pre- and post-training assessments.[8] Data analysis Collated information was analysed using SPSS 16 (SPSS Inc, USA). Mean, median, frequency and proportion were summarised to describe the pre- and post-training test assessment scores. Student’s paired t-test was used to compare pre- and post-test performance. A p˂0.05 value indicated a statistically significant change. Scores were graded using the described rating system. Ethical approval Ethical approval for this study was granted by the University of Botswana (ref. no. X-REF:UBR/ETHI/21). Written informed consent was obtained from each of the participants. Results Thirty-nine participants took the pre-test and underwent training. Fourteen (36%) participants completed all planned sessions, completing the pre-test, training and post-test assessments. The analysis comparing the pre- and post-training tests was performed on the 14 participants who completed all the planned sessions. The median (range) scores for pre- and post- test assessments were 11.3 (4.5 - 21.0) and 19.5 (15.5 - 23.0), respectively (p<0.001). The median scores for the psychomotor assessment (chest drain insertion skill) improved from 1.0 (0 - 3.0) to 2.5 (1.5 - 3.0) of a possible score of 3 in the pre-test and post-test assessments, respectively. Six (42%) participants attained ‘full proficiency’ grades in the post-test assessment compared with 1 (7%) in the pre-test. Participants in the ‘reasonable Short Research Report Fig. 1. (A) Step-wise illustration of creating a suture trainer. (B) Intercostal chest drain-insertion trainer. 170 December 2017, Vol. 9, No. 4 AJHPE profi ciency’ grade rose from 4 (30%) to 8 (60%) in pre-test and post-test assessments, respectively. Participants in the ‘some proficiency’ and ‘poor proficiency’ grades in the pre-test decreased from 6 (50%) and 3 (20%), respectively, to 0% in the post-test. The majority of participants (93%) strongly agreed that they would accept this module as their formal training for chest trauma. Discussion This study evaluated the usefulness and acceptability of low-cost simulation in a resource-limited environment. The process undertaken represents the initial two levels of the Kirkpatrick model: change of knowledge following the training and reaction from the trainees.[7,8] Several previous studies have demonstrated the effectiveness of simulation in assessment.[2-4] At our university and other universities in developing countries, medical training has not fully exploited the use of simulation for training and assessment.[3] Despite the demonstrated advantages of simulation in surgical training, the literature on this topic in sub-Saharan Africa is scarce – probably because of the limited use of simulation.[2,3] Among the possible reasons for these low usages are: high cost of purchasing simulation models and lack of logistical and organisational initiatives.[2,3] Although there is an abundance of patient encounters for trainees and a high disease burden in many developing countries, simulation may still form an important part of training and assessment of critical skills. In comparable situations, simulations have been used for the initial training of novices, e.g. in the military and aviation industries.[2,4] Study limitations The main limitations of this study were: (i) loss to follow-up of participants in the post-test assessment, which may affect the generalisability of our findings; and (ii) ethical issues that restricted the use of control groups in this pilot study, leading to the use of single-group pre- and post-test designs, which are known to have variations in estimations of the effect size of outcomes. Conclusion The results of this study indicated that low-cost simulation can be a useful and readily available aid for training and assessment in a resource- constrained environment. If used in conjunction with the existing surgical curriculum, low-cost simulation appears to contribute to the knowledge and skills of our students and trainees. Low-cost task trainers, as described in this study, are an effective option for training and assessment. We would recommend an extension of this type of model to the rest of the curriculum. We also recommend that further studies should be done to evaluate the long-term impact of low-cost simulation on trainees’ behavioural change and patients’ outcome. Acknowledgements. We acknowledge the contributions of Drs S Kuskov, M Kwati and K Mmalane from Princes Marina Hospital, Gaborone, Botswana, and the 2014 Sub-Saharan Africa-FAIMER Regional Institute (SAFRI) faculty and fellows for their support and contributions. Author contributions. MM: conception, design, analysis, data interpretation and writing of the manuscript; RD: conception, design, data interpretation and critical revision of intellectual content; JF: conception, design, data interpretation and critical revision of intellectual content; and MW: data interpretation and critical revision of intellectual content. All the above-named authors approved publication of this manuscript. Funding. None. Conflicts of interest. None. 1. Clarke DL, Quazi MA, Reddy K, et al. Emergency operation for penetrating thoracic trauma in a metropolitan surgical service in South Africa. J Thorac Cardiovasc Surg 2011;142(3):563-568. https://doi.org/10.1016/j. jtcvs.2011.03.034 2. Raison N, Ahmed K, Dasgupta P. Role of simulation in surgical training. Eur Urol Focus 2016;2(1):63-64. https:// doi.org/10.1038/nrurol.2016.147 3. Taché S, Mbembati N, Marshall N, et al. Addressing gaps in surgical skills training by means of low-cost simulation at Muhimbili University in Tanzania. Hum Resour Health 2009;7:64. https://doi.org/10.1186/1478- 4491-7-64 4. Scalese RJ, Obeso VT, Issenberg SB. Simulation technology for skills training and competency assessment in medical education. J Gen Intern Med 2008;23(1):46-49. https://doi.org/10.1007/s11606-007-0283-4 5. Kardong-Edgren S, Adamson KA, Fitzgerald C. A review of currently published evaluation instruments for human patient simulation. Clin Simul Nurs 2010;6(1):e25-e35. https://doi.org/10.1016/j.ecns.2009.08.004 6. Maier-Riehle B, Zwingmann C. Effect strength variation in the single group pre-post study design: A critical review. Die Rehabilitation 2000;39(4):189-199. https://doi.org/10.1055/s-2000-12042 7. Kirkpatrick D, Kirkpatrick J. Evaluating Training Programs: The Four Levels. 3rd ed. California: Berrett-Koehler, 2006. 8. Allen IE, Seaman CA. Likert scales and data analyses. Qual Progress 2007;40(7):64-65. Accepted 15 June 2017. Short Research Report https://doi.org/10.1016/j.jtcvs.2011.03.034 https://doi.org/10.1016/j.jtcvs.2011.03.034 https://doi.org/10.1038/nrurol.2016.147 https://doi.org/10.1038/nrurol.2016.147 https://doi.org/10.1186/1478-4491-7-64 https://doi.org/10.1186/1478-4491-7-64 https://doi.org/10.1007/s11606-007-0283-4 https://doi.org/10.1016/j.ecns.2009.08.004 https://doi.org/10.1055/s-2000-12042