December 2021, Vol. 13, No. 4 AJHPE 241 Research Medical graduates are rightfully expected to be competent in managing clinical emergencies. South African (SA)  regulations that govern undergraduate medical curricula, however, are fairly quiet about the matter, and in the absence of a national ‘standard’ curriculum, approaches to how  to  convey these capabilities might vary widely between the eight SA medical schools. Internationally, bodies representing emergency medicine and emergency physicians, such as the International Federation for Emergency Medicine and the American College of Emergency Medicine, have published recommendations on the emergency content of undergraduate medical training.[1,2] At the University of the Free State (UFS), the MB  ChB undergraduate medical programme runs over 5  years and is divided into the pre-clinical and clinical phase. At the beginning of their third year, the medical students receive seven 3-hour lecture blocks and two practical sessions (totalling 4  hours per student)  during the ‘clinical skills’ module. These sessions form  part of the students’ introduction into clinical practice in phase 2, which spans semesters 4 and 5. In phase 3 (semesters 6 - 10), the students are exposed to clinical emergency conditions and their management during the rotations in specific medical disciplines. Without a department of emergency medicine at the UFS, there is currently no integrated emergency care curriculum, with only limited co-ordination between the different stakeholders. Nominal group techniques (NGTs)  were initially introduced in the 1960s as a project management tool, allowing a structured approach to the identification of ‘client’ needs and the development of projects in response to such demands.[3] The NGT is regarded as a ‘consensus’ tool, producing prioritised outcomes to guide a project implementation,[4,5] and  has been  gainfully applied in the assessment of a new undergraduate nursing sciences[6] and a redesigned undergraduate medical programme,[7] and in the evaluation of courses within undergraduate medical programmes.[8,9] The NGT has been credited for being creative by containing dominant  group  members and allowing quieter members to express their ideas.[3,10] This study will hopefully contribute to the continuous critical review of current teaching practices to improve the quality of the MB ChB programme and optimise the competencies of graduates. Objective The aim of this study was to explore undergraduate medical students’ perceptions of the strengths and weaknesses of the current emergency care clinical skills module, and to identify, with the help of technical experts, available or desirable solutions to address some of the challenges. The specific objectives were: • to identify the strengths and weaknesses of the current approach as Background. Handling medical emergencies is essential for medical practitioners. Medical students at the University of the Free State have an emergency care block in their third year. Nominal group technique (NGT) has been introduced for programme development, and has been used for the assessment of educational programmes. Objectives. To identify the strengths and weaknesses of the current teaching programme, as experienced by the students, and to obtain advice from lecturers on available resources and additional requirements. Methods. A two-stage NGT was used to identify strengths and weaknesses of the programme from the ‘clients’ (students), and for the ‘experts’ (clinicians and educators)  to suggest possible improvements. Two NGT sessions were conducted with students that had either recently (third-year students)  or 2 years ago (fifth-year students) been exposed to the module. Students were asked to identify positive and negative aspects. Based on these sessions, two further NGTs were conducted with groups of ‘experts’ from the School of Medicine, asking for suggestions for improvement in the current resources, and for additional resources necessary. Results. Students valued the practical skills obtained and some of the format of the teaching, but requested an increase of practical content, as well as additional tools and modes of teaching. Lecturers suggested co-ordinating outcomes to clarify basic concepts and to use additional media, but emphasised the need for human resources, teaching tools and functional clinical equipment. Conclusion. NGT provides a valuable tool to obtain critical suggestions from students and lecturers for improvement of the clinical teaching of emergency care. Afr J Health Professions Educ 2021;13(4):240-245. https://doi.org/10.7196/AJHPE.2021.v13i4.1321 Nominal group technique review of the emergency care content of the clinical skills module in the undergraduate medical programme at the University of the Free State D T Hagemeister, BA, MPH, EMMB, FAfAllgMed, DA, Dip HIV Man (SA), DIRE Department of Family Medicine, Faculty of Health Sciences, University of the Free State, Bloemfontein, South Africa Corresponding author: D Hagemeister (hagemeisterdt@ufs.ac.za) This open-access article is distributed under Creative Commons licence CC-BY-NC 4.0. https://doi.org/10.7196/AJHPE.2021 mailto:hagemeisterdt@ufs.ac.za 242 December 2021, Vol. 13, No. 4 AJHPE Research experienced and expressed by the students who were exposed to the module • to identify existing resources and potential new approaches based on the insights of local technical experts (academic clinicians/educationalists) • to include the refined findings of this study in future teaching practice • to establish a baseline for future empirical research on health professions education in emergency care in the MB ChB programme at UFS. Methods In this study, a prospective, cross-sectional, qualitative approach was applied. As part of ongoing efforts to improve undergraduate medical education at UFS, it was decided to use the NGT to assess the experiences of undergraduate students who had participated in the clinical skills module either recently (current third-year students) or 2 years ago (current fifth-year students). This assessment was complemented by a second round of two NGTs with academic clinicians/educationalists involved in the teaching, who were asked to ‘respond to’ or ‘work with’ the findings of the first round. Approval was obtained from the Health Sciences Research Ethics Committee, UFS (ref. no. HSREC 111/2017), and permission was granted from the authorities at the university. Written informed consent was obtained from all participants. Participation in the research was voluntary, and due to the nature of the method, results or statements are not attributable to an individual, and confidentiality of the participants was maintained throughout the process. Data collection and processing In keeping with the processes described in the literature, four research assistants who had received prior instructions facilitated the NGT sessions. After a brief introduction and explanation of the process, a silent round of idea generation took place. The participants were subsequently asked to present one of their ideas each, which was captured on a flip chart. This was repeated until no additional ideas were presented. The ideas as written down were then discussed for clarification, and where the group felt that two ideas overlapped and should be combined, this was done. This revised list was put up for ‘voting’ when each participant was asked to choose the most important statement (5 points), second-most important (4 points) down to the fifth-most important statement (1 point). Ideas were ranked according to the number of votes each idea received. The NGT sessions followed the structure suggested by Gallagher et  al.[10] An overview of the process and stages is given in Table 1. In the case of a large number of participants, the group was split into two desks, where the process, as described above, took place with a second round of ranking and voting to combine the results from the two desks added. Study population First-round NGT – undergraduate medical students Medical students from two different year groups at UFS who had previously done the emergency care clinical skills module constituted the study population for the first round of the NGT. This included the third-year class, who had done the module earlier in the year, and the fifth-year class, who had completed the module 2 years previously and had been exposed to most of the clinical rotations, thus having a better overview over the full clinical content of the MB ChB programme. With the assistance of class representatives, purposive sampling within the current third- and fifth-year classes was done, with the aim of achieving good gender and ethnic representation in samples from each of the two academic years, ideally totalling 10 - 15 participants per session. Second-round NGT – technical experts Academic clinical staff on the joint staff establishment for the School of Medicine, together with affiliated lecturers in the clinical disciplines and university staff from the School of Medicine (e.g. programme director, departmental teaching and learning co-ordinators and academic staff of the simulation and skills unit)  constituted the total study population for the technical expert group. Based on the concept of purposive sampling,[11] a good representation of clinical disciplines and professional functions was aimed for when recruiting the total of 10  - 15 participants for the second round of the NGT process. Results Four NGT sessions were conducted in September 2017. After conducting the NGT process (idea generation, clarification, vote to rank at the individual desk), the ranked statements, as listed in Tables 2, 3, 4 and 5, were obtained, with the number of votes for each statement. The results are presented separately for the two rounds (undergraduate students and technical experts). For referencing in the discussion section, responses are coded in the tables. The code for the students consists of the question (student question 1/2 = SQ1/SQ2), year group (third-years = 3; fifth-years = 5) and the final ranking. For example, ‘SQ1.3.4’ refers to student question 1, third-year group, fourth-ranked response. The code for the technical experts consists of the question (expert question 1/2 = EQ1/EQ2), NGT session (panel 1 = 1; panel 2 = 2)  and the ranking. For example, ‘EQ2.1.5’ refers to expert question 2, first NGT session (panel 1) and fifth-ranked responses. First-round NGTs (undergraduate students) The two sessions with undergraduate students included 10  third- year medical students (4 men and 6 women, 2 of whom only joined for the second question) and 18 fifth-year medical students (4 men and 14 women). Because of the number of fifth-year students (18), the NGT for this group was run at two desks concurrently, with an additional round of clarification and voting added to consolidate the results from the two desks into one. Table 1. Stages of the nominal group technique* 1 Introduction 2 Silent generation of ideas in writing 3 Listing of ideas on flip chart (round-robin) 4 Discussion of ideas on flip chart 5 Ranking to select the ‘top-10’ ideas 6 Voting on ‘top-10’ ideas 7 Break 8 Discussion of vote 9 Re-ranking and rating revised ‘top-10’ items 10 Conclusion of nominal group (and selection of representatives) *modified from Gallagher et al.[10] December 2021, Vol. 13, No. 4 AJHPE 243 Research Second-round NGTs (technical experts)  The second round of NGT sessions was held with academic clinicians and health educationalists in two separate events with 8 (panel 1)  and 7 participants (panel 2), respectively. Panel 1 comprised 1 female and 7 male participants. These included consultants and medical officers from the departments of trauma, family medicine and surgery as well as a paramedic and an information technology (IT)/technical support staff member. Panel 2 comprised 2 female and 5 male participants. This panel included consultants and medical officers from the departments of family medicine, anaesthesiology and internal medicine as well as from the Clinical Simulation and Skills Unit. Discussion Students’ NGT The students’ responses did not contain any critical comments about individual Table 2. Ranked responses to student question 1 (SQ1): ‘Write down what you valued the most in the emergency care clinical skills module (MCLI3713) in the third year’ (N=28) Third-year medical students (n=10) Fifth-year medical students (n=18) Rank* Responses (score) Rank* Responses (score) SQ1.3.1 Learn to save somebody from dangerous situation/basics of how to save somebody’s life (25) SQ1.5.1 Practical solutions/physically doing on manikins/invited EMS personnel/ exposure to EMS people and ride-along (73) SQ1.3.2 Engaging environment/enthusiasm of lecturers/individual attention (24) SQ1.5.2 Well-structured/practical skills/theory series and practicals/use of scenarios in training and evaluation (64) SQ1.3.3 Immediately know what to do (20) SQ1.5.3 Snake-bites interesting/‘defib’/lectures and choking session (35) SQ1.3.4 Something practical for the first time (15) SQ1.5.4 Interaction with consultants fixing mistakes/during OSCE immediate remediation (32) SQ1.3.5 Content well rounded/comprehensive (12) SQ1.5.5 The timing before our hectic clinical years/helped on other modules (29) SQ1.3.6 Different aspect of medicine from just a normal GP (9) SQ1.5.6 Lecturer knowledge sharing – real-life stories (23) SQ1.5.7 Benchmarked mark of 80% made me feel competent/presence of fifth-year medical students during exam/privacy for examinations (13) EMS = emergency medical services; OSCE = objective structured clinical examination; GP = general practitioner. *E.g. SQ1.3.1 = student question 1 (SQ1), third-year nominal group technique session (3), first-ranked statement (1). Table 3. Ranked responses to student question 2 (SQ2): ‘Write down suggestions on how to improve the emergency care clinical skills module (MCLI3713)’ (N=28) Third-year medical students (n=10) Fifth-year medical students (n=18) Rank* Responses (score) Rank* Responses (score) SQ2.3.1 Start with a practical session/combine class (theory) and practical sessions (31) SQ2.5.1 More exposure to emergency equipment/information on how to perform defibrillation/nebulisation and practical exposure/short instructions on paper/how to use defibrillation/anaesthetic machine/where to put three leads (53) SQ2.3.2 More time to practise the skills/more time and equipment made available to practise (24) SQ2.5.2 Incorporate medical emergencies, e.g. pulmonary oedema/thyroid storm (45) SQ2.3.3 Module guide more structured/module guides must be available from the beginning (organisation better)/ case studies at the end of a session (22) SQ2.5.3 Improve module guide/workbook/more complete notes in module guide/ properly bound module guide/outcomes for theory (37) SQ2.3.4 More practical sessions (19) SQ2.5.4 Lack of continuous evaluation/more assessment during the course/‘mock OSCE’ (29) SQ2.3.5 Videos must be available and students know where to find them/expose to real world and see what they do (Pelonomi trauma) (15) SQ2.5.5 More videos of skills (27) SQ2.3.6 Emergency care should count more towards your module mark (5) SQ2.5.6 Role/responsibilities in an emergency (when you are not on duty/airport/ RTA)/kit – what to keep in car/on you in case of emergency in public setting (24) SQ2.3.7 Get Prof. X to summarise (present) the basic life support lecture (4) SQ2.5.7 Clinical scenarios were lacking: what is taught must be assessed/spacing out stations during exams (21) SQ2.5.8 Boring lectures (17) SQ2.5.9 In the content: knowing what to say to family members while waiting in emergency situation/near drowning/water-orientated sessions/wound management/not specific when say ‘give pain relief ’ or ‘fluids’, etc. (15) OSCE = objective structured clinical examination; RTA = road traffic accident. *E.g. SQ2.3.1 = student question 2 (SQ2), third-year nominal group technique session (3), first-ranked statement (1). 244 December 2021, Vol. 13, No. 4 AJHPE Research lecturers, and only one lecturer was mentioned by name, to the effect that this lecturer should present a specific topic (SQ2.3.7). Students in both years appreciated the course’s practical value in enabling them to address life- threatening situations (SQ1.3.1, SQ1.3.3, SQ1.3.4, SQ1.3.6, SQ1.5.1, SQ1.5.4, SQ1.5.6 and SQ1.5.7), which is in keeping with international literature.[12,13] As the module is taught at the beginning of, and as an introduction to, the clinical years in the MB ChB programme, this is to be expected, but also shows that the module met the students’ expectations in this regard. The way the content was delivered also enjoyed positive feedback in some aspects (SQ1.3.2, SQ1.3.5, SQ1.5.2, SQ1.5.4 and SQ1.5.5). When it came to students’ suggestions for improvements (question  2), the ‘practical theme’ again dominated, with clear requests to increase the practical parts of the learning experience (SQ2.3.1, SQ2.3.2, SQ2.3.4, SQ2.5.1, SQ2.5.4 and SQ2.5.7). In addition to this, the students requested changes to the resources (‘module guide’/‘case studies’ – SQ2.3.3; and SQ2.5.3, ‘videos’/‘real world’ – SQ2.3.5 and SQ2.5.5), hinting at the use of blended learning/flipped classroom techniques.[14,15] Based on their greater clinical exposure, the fifth-year students also requested additions to the content (specific clinical emergencies – SQ2.5.2; specific scenarios  – SQ2.5.6; and dealing with family members and staff – SQ2.5.9), while the third-year students were more concerned with administrative issues (‘mark’ SQ2.3.6)  and the role of a specific esteemed lecturer (‘Prof. X’ – SQ2.3.7). Further comparing the third- and fifth-year students, the latter displayed a more differentiated appreciation of technical aspects (‘manikins’ – SQ1.5.1; ‘scenarios’ – SQ1.5.2; ‘fixing mistakes’/‘immediate remediation’ – SQ1.5.4; ‘real life stories’ – SQ1.5.6), probably owing to their greater exposure to clinical teaching by the time of the NGT session. The fifth-year students also addressed the different professions/ranks in the health system in a more differentiated way (‘EMS personnel’ – SQ1.5.1; ‘consultants’ – SQ1.5.4), again probably owing to their greater exposure to the system. Comments on the timing of the module about the other clinical teaching are only possible in retrospect (SQ1.5.5). Technical experts’ NGTs Clinicians and educationalists put a strong emphasis on the clarification of outcomes and concepts in the module (EQ1.1.1, EQ1.2.1, EQ1.2.3), suggesting a proper ‘scaffolding’ of the content in the Vygotskian sense, rather than a mere accumulation of clinical conditions.[1,16] Table 4. Technical experts question 1 (EQ1): ‘Write down suggestions, given current resources and based on the comments of the students, what you think can be done to improve the module content and delivery’ (N=15) First expert panel (n=8) Second expert panel (n=7) Rank* Responses (score) Rank* Responses (score) EQ1.1.1 Module guide outcomes/condense the content (decide on what is important on that level) (27) EQ1.2.1 Basic principles/clarifying concepts, e.g. what is a cardiac arrest/recognise emergency situations, e.g. low blood sugar/students must understand what is the purpose of their actions/integrate physiology/anatomy, etc./explain clearly/knowing when to stop the resuscitation (26) EQ1.1.2 Integrating practical bits into the theory content (22) EQ1.2.2 Visual aids/props in lectures, e.g. talk about airways – show it/combination of theory and simulation/make use of visual aids/stimulate their interest/ more use of simulation (22) EQ1.1.3 Summary structured lecture: ‘flipped classroom approach’/ more videos, Blackboard media server, (practical station), short videos before class, login on Blackboard/ online resources for blended learning (knowing the equipment) (21) EQ1.2.3 What are the general outcomes of this module (what level)?/start the first day and explain the bigger picture: one piece build onto the other (14) EQ1.1.4 Incorporation of different models (e.g. simulation, skills lab, manikins), more dedicated time in the skills lab (obtain signatures)/small group rotations (18) EQ1.2.4 Continuous evaluation/mock exam/practical sessions/more credits/marks for this module/assess the students/students accept responsibility (11) EQ1.1.5 What the content of the module should be in the lecture (well structured)/integrating with other modules later-on (taking the current level of knowledge into account)/build on the existing knowledge obtained (e.g. pharmacology, etc.) by introducing the practical aspect (basic critical care)/ having simple structured (standard) approaches (PALS/ ATLS etc.) (17) EQ1.2.5 Standardise the teaching material (approach – who are we holding our standards up to)/practise under guidance of professional (10) EQ1.2.6 a) Longitudinal EMC training/start in semester 5 and continue to semester 10 (8) b) Two weeks of emergency care/smaller groups/rotations practicals and tutorials (8) EQ1.2.8 E-learning/Blackboard for content/QuestionMark/more time for practicals/ make use of technology (cellphone) to look at videos etc. on YouTube/for teaching them how to find information (4) EQ1.2.9 Drowning/near drowning should be included – submersion injuries (2) PALS = paediatric advanced life support; ATLS = advanced trauma life support; EMS = emergency medical services. *E.g. EQ1.2.3 = expert question 1 (EQ1), second panel (2), third-ranked statement (3). December 2021, Vol. 13, No. 4 AJHPE 245 Research With regard to the mode of delivery of the content, emphasis was again placed on the importance of practical aspects (EQ1.1.2)  and the use of ‘stimulating’ methods, including simulation exercises (EQ2.1.2, EQ1.1.4) as well as online resources and videos (EQ1.1.3, EQ1.2.8). Continuous evaluation of the students and the inclusion of mock examinations would also aid the learning process (EQ1.2.4), mirroring the emphasis on ‘assessment drives learning’ in the literature.[17,18] Standardisation of the teaching materials and the management approaches was suggested to improve the consistency of the teaching (EQ1.1.5, EQ1.2.5). The content should be integrated with and linked to modules in earlier and later parts of the programme (EQ1.1.5, EQ1.2.6)  and this was mentioned under both questions 1 and 2 (also EQ2.2.5). When considering additional resources, the top priority was the availability of well-trained facilitators, or in other words, human resource management and development issues (EQ2.1.1, EQ2.2.1, EQ2.2.6), including IT training for the ‘older folks’ (EQ2.1.6). The availability of equipment for teaching, simulators and videos was the second major theme (EQ2.1.2, EQ2.2.4), including the availability of appropriate clinical equipment at the actual clinical service delivery sites (EQ2.2.8). Additional ideas that came up were the opportunities offered by the changed language policy of the university, which potentially frees up resources (EQ2.1.5), the utilisation of other emergency care personnel such as the emergency medical services (EMS)  and their training facilities (EQ2.1.3), and the creation of a question bank of multiple- choice questions for formative and summative assessment (EQ2.2.7). Lastly, the general need for additional financial resources was expressed (EQ2.2.3). Implementation of ideas/suggestions In the next run of the module after the NGTs, additional practical CPR training sessions and mock examinations have been included as suggested during the research, while much more work remains to be done regarding a general overhaul of the content and structure of the module, possibly in connection with a major review of the undergraduate curriculum. Weaknesses and possible confounders Firstly, it needs to be acknowledged that the relatively small sample from both student groups might not be regarded as fully representative for the respective years. An additional quantitative, questionnaire-based vote on the finding by a larger sample could have added reliability through ‘triangulation’.[7] Given the 2  years between the exposure to the module of the two year groups, staff attrition and reworking of course content might have introduced differences between the two ‘runs’ of the same module. On the side of the ‘technical experts’, the composition of the group and the absence of some disciplines (due to lack of response from certain departments) might have biased the outcomes. The results of the two sessions in the second round of NGTs, with the technical experts, have not been consolidated into a single ‘rank order’ as suggested by some authors.[19,20] Conclusion Through the use of a two-staged NGT, it was possible to gain valuable feedback from undergraduate students who had experienced the teaching in the module that was under review. Both the positive comments and suggestions for improvement as expressed during this process led to fruitful discussions with the team of academic clinicians and educators who were currently involved in the programme. As also reported in other studies, the use of the NGT provides concise and ‘easy-to-apply’ results on the needs of the ‘users’ (students) and the possible approaches from the ‘experts’. By involving both ‘sides’ in this two-staged NGT, the experts respond directly to the identified challenges. Further evaluation will be necessary to measure the actual impact of this research, including the implementation of the findings. Table 5. Technical expert question 2 (EQ2): ‘Write down suggestions on what additional resources and structures might be needed and how these would influence the delivery of the module content’ (N=15) First expert panel (n=8) Second expert panel (n=7) Rank* Responses (score) Rank* Responses (score) EQ2.1.1 More dedicated facilitators for emergency care, not only in family medicine (27) EQ2.2.1 Trained manpower. Make use of various professions, e.g. nursing, emergency personnel/well-trained facilitators/instructors (25) EQ2.1.2 Somewhere to go to watch videos and access equipment (same space)/standardising the equipment (23) EQ2.2.2 Who takes ownership/where will advanced life support fit in/what level do we teach emergency care? (18) EQ2.1.3 a) Include road-shifts with EMS as clinical practice/more clinical time in the hospital setting (22) b) Alternative training method in instances of no actual resources (table exercises)/using resources at other faculties (expanding your footprint of resources) (22) EQ2.2.3 More funding (12) EQ2.1.4 - EQ2.2.4 More simulators/more equipment (10) EQ2.1.5 Capitalise on one-language policy (8) EQ2.2.5 Vertical integration of emergency care in the program/structure the curriculum (9) EQ2.1.6 IT training for the older folks (3) EQ2.2.6 Possibility of after-hour teaching such as ACLS (extra teaching time/ practicals)/skills training facilities must be available; facilitators also (7) EQ2.2.7 Build a question bank (6) EQ2.2.8 Decent clinical equipment in hospitals (3) EMS = emergency medical services; ACLS = advanced cardiovascular life support; IT = information technology. *E.g. EQ2.2.3 = expert question 2 (EQ2), second panel (2), third-ranked statement (3). 246 December 2021, Vol. 13, No. 4 AJHPE Research Several challenges identified by the students, and some of the suggested solutions, are beyond the scope of the current research, which was focused narrowly on the emergency care teaching occurring in the third year. A more comprehensive approach to review the overall framework of emergency care teaching in the undergraduate programme is currently projected to cover these aspects. Declaration. None. Acknowledgements. The author gratefully acknowledges Prof. Yvonne Botma, Sr Rynsie Hattingh, Mrs Nokuthula Tlalajoe and Dr Joleen Cairncross for their help in conducting the nominal group technique sessions. Ms Theanette Mulder is much appreciated for her support in editing the manuscript. Author contributions. Sole author. Funding. None. Conflicts of interest. None. 1. Hobgood C, Anantharaman V, Bandiera G, et  al. International Federation for Emergency Medicine model curriculum for medical student education in emergency medicine. Emerg Med Australas 2009;21(5):367-372. https://doi.org/10.1111/j.1742-6723.2009.01213.x 2. American College of Emergency Physicians. Guidelines for undergraduate education in emergency medicine. Irving: ACEP, 2015. https://www.acep.org/patient-care/policy-statements/guidelines-for-undergraduate- education-in-emergency-medicine/#sm.00001ogy4kwqeodpat4wmro3ufk77 (accessed 18 September 2018). 3. Delbecq AL, Van de Ven AH. A group process model for problem identification and program planning. J Appl Behav Sci 1971;7(4):466-492. https://doi.org/10.1177/002188637100700404 4. Allen J, Dyas J, Jones M. Building consensus in health care: A guide to using the nominal group technique. Br J Community Nurs 2004;9(3):110-114. https://doi.org/10.12968/bjcn.2004.9.3.12432 5. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ 1995;311(7001):376-380. https://doi.org/10.1136/bmj.311.7001.376 6. Chapple M, Murphy R. The nominal group technique: Extending the evaluation of students’ teaching and learning experiences. Assess Eval High Educ 1996;21(2):147-160. https://doi.org/10.1080/0260293960210204 7. Lloyd-Jones G, Fowell S, Bligh JG. The use of the nominal group technique as an evaluative tool in medical undergraduate education. Med Educ 1999;33(1):8-13. https://doi.org/10.1046/j.1365-2923.1999.00288.x 8. Dobbie A, Rhodes M, Tysinger JW, Freeman J. Using a modified nominal group technique as a curriculum evaluation tool. Fam Med 2004;36(6):402-406. 9. Lancaster T, Hart R, Gardner S. Literature and medicine: Evaluating a special study module using the nominal group technique. Med Educ 2002;36(11):1071-1076. https://doi.org/10.1046/j.1365-2923.2002.01325.x 10. Gallagher M, Hares T, Spencer J, Bradshaw C, Webb I. The nominal group technique: A research tool for general practice? Fam Pract 1993;10(1):76-81. https://doi.org/10.1093/fampra/10.1.76 11. Henning E, Van Rensburg W, Smit B. Finding your Way in Qualitative Research. Pretoria: Van Schaik, 2004. 12. Ten Eyck RP, Tews M, Ballester JM. Improved medical student satisfaction and test performance with a simulation- based emergency medicine curriculum: A randomised controlled trial. Ann Emerg Med 2009;54(5):684-691. https://doi.org/10.1016/j.annemergmed.2009.03.025 13. Morris MC, Conroy P. Development of a simulation-based sub-module in undergraduate medical education. Ir J Med Sci 2019;(epub ahead of print). https://doi.org/10.1007/s11845-019-02050-3 14. Parandavar N, Rezaee R, Mosallanejad L, Mosallanejad Z. Designing a blended training program and its effects on clinical practice and clinical reasoning in midwifery students. J Educ Health Promot 2019;8:131-131. https:// doi.org/10.4103/jehp.jehp_22_18 15. Nakanishi T, Goto T, Kobuchi T, Kimura T, Hayashi H, Tokuda Y. The effects of flipped learning for bystander cardiopulmonary resuscitation on undergraduate medical students. Int J Med Educ 2017;8:430-436. https://doi. org/10.5116/ijme.5a2b.ae56 16. Tews MC, Hamilton GC. Integrating emergency medicine principles and experience throughout the medical school curriculum: Why and how. Acad Emerg Med 2011;18(10):1072-1080. https://doi.org/10.1111/j.1553- 2712.2011.01168.x 17. Phillips AW, Smith SG, Straus CM. Driving deeper learning by assessment: An adaptation of the Revised Bloom’s Taxonomy for medical imaging in gross anatomy. Acad Radiol 2013;20(6):784-789. https://doi.org/10.1016/j. acra.2013.02.001 18. Keeley MG, Gusic ME, Morgan HK, Aagaard EM, Santen SA. Moving toward summative competency assessment to individualise the postclerkship phase. Acad Med 2019;94(12):1858-1864. https://doi.org/10.1097/ ACM.0000000000002830 19. McMillan SS, Kelly F, Sav A, et al. Using the nominal group technique: How to analyse across multiple groups. Health Serv Outcomes Res Methodol 2014;14(3):92-108. https://doi.org/10.1007/s10742-014-0121-1 20. Van Breda AD. Steps to analysing multiple-group NGT data. Soc Work Pract Res 2005;17(1):1-14. Accepted 17 August 2020. https://doi.org/10.1111/j.1742-6723.2009.01213.x https://www.acep.org/patient-care/policy-statements/guidelines-for-undergraduate-education-in-emergency-medicine/#sm.00001ogy4kwqeodpat4wmro3ufk77 https://www.acep.org/patient-care/policy-statements/guidelines-for-undergraduate-education-in-emergency-medicine/#sm.00001ogy4kwqeodpat4wmro3ufk77 https://doi.org/10.1177/002188637100700404 https://doi.org/10.12968/bjcn.2004.9.3.12432 https://doi.org/10.1136/bmj.311.7001.376 https://doi.org/10.1080/0260293960210204 https://doi.org/10.1046/j.1365-2923.1999.00288.x https://doi.org/10.1046/j.1365-2923.2002.01325.x https://doi.org/10.1093/fampra/10.1.76 https://doi.org/10.1016/j.annemergmed.2009.03.025 https://doi.org/10.1007/s11845-019-02050-3 https://doi.org/10.4103/jehp.jehp_22_18 https://doi.org/10.4103/jehp.jehp_22_18 https://doi.org/10.5116/ijme.5a2b.ae56 https://doi.org/10.5116/ijme.5a2b.ae56 https://doi.org/10.1111/j.1553-2712.2011.01168.x https://doi.org/10.1111/j.1553-2712.2011.01168.x https://doi.org/10.1016/j.acra.2013.02.001 https://doi.org/10.1016/j.acra.2013.02.001 https://doi.org/10.1097/ACM.0000000000002830 https://doi.org/10.1097/ACM.0000000000002830 https://doi.org/10.1007/s10742-014-0121-1