September 2018, Vol. 10, No. 3 AJHPE 183 Research The emergency care profession in South Africa (SA) is regulated by the Health Professions Council of SA: Professional Board for Emergency Care (HPCSA: PBEC).[1] One of the mandates of the PBEC is to provide quality assurance of emergency care education programmes. The PBEC advocates integrated clinical simulation, using a full-body mannequin, as a summative assessment instrument for all registered emergency care qualifications.[1] Concurrently, the PBEC is subject to compliance with assessment guidelines outlined by the SA Qualifications Authority (SAQA). These guidelines clarify assessment criteria and integrated assessment and promote the evaluation of applied competence using work-relevant activities (Box 1).[2-4] SAQA identifies simulation and role-play in its list of assessment instruments.[3,4] Role-play in clinical simulation occurs when students assume the roles of qualified practitioners in the context of ‘a situation, a problem or an incident, to which they have to respond’.[3] The conditions suggested as suitable for using simulation include: ‘where demonstrations and observation will provide reliable and valid results, but where, for a number of reasons, it is difficult or not practicable to assess under actual conditions’.[3] The assessment principles of transparency, fairness, validity and reliability, outlined by SAQA, are assumed to apply to integrated clinical simulation for summative assessment.[1,3,4] SAQA states that the validity of assessment involves ‘setting authentic or applied tasks in the learning programme that closely simulate real world contexts’.[4] By applying the principle of validity to the simulation assessment, the following assumptions can be made: (i) actions by students in simulation are assumed to replicate what they would do in a similar clinical case in practice; and (ii) competence in simulation infers competence in practice. The PBEC requires assessments that lead to an advanced life support (ALS) qualification to be moderated (Box 1).[1] The PBEC provides guidelines for moderators to report on the simulation assessments with regard to:[5] • design, structure and facilitation of the simulation • authenticity and realism of the event • adherence to the principles of fairness, validity, reliability and practicality. Despite these reporting requirements for moderators, the education regulators provide neither assessment criteria nor guidelines for emergency care educators to fulfil such requirements. In the absence of such guidelines, how clinical simulations are assessed is left to the discretion of assessors Background. The Professional Board for Emergency Care (PBEC), the statutory body regulating the quality of emergency care education programmes in South Africa, has mandated these programmes to use integrated clinical simulation as an instrument for authentic assessment. In support of the validity of this instrument, actions by students during simulation are assumed to replicate what they would do in similar circumstances in practice. Objectives. To present and discuss perspectives of advanced life support (ALS) paramedics on the use of integrated clinical simulation as a summative assessment instrument, offer a critique of assumptions regarding the use of this assessment instrument, and recommend improvements for its use. Methods. A qualitative, single, embedded case study design was used to address assessment criteria and case types for integrated clinical simulation as a summative assessment instrument. Qualitative data were collected by means of focus group interviews. Perspectives of ALS paramedics emerged from the results of this study. Results. Participants agreed that integrated clinical simulation was an appropriate assessment instrument if assessment principles were adhered to. Accurate replication of the contextual elements of emergency care practice was perceived as central for eliciting authentic responses associated with ALS paramedic practice. The conditions, context and range of life-threatening conditions across medical disciplines challenged the idea that a single, once-off assessment event could be a valid reflection of competence. Conclusion. To elicit authentic responses, the design of integrated clinical simulation events for summative assessment should include relevant clinical, environmental and social-professional elements of ALS paramedic practice. More than one assessment should be done, and should address the range and complexity of medical and trauma emergencies, thereby assessing the true competence of ALS paramedic students. Assessors should acquire the requisite skills to assess simulation effectively. Afr J Health Professions Educ 2018;10(3):183-188. DOI:10.7196/AJHPE.2018.v10i3.962 Perspectives of advanced life support paramedics on clinical simulation for summative assessment in South Africa: Is it time for change? R G Campbell,1 BTech (EMC), MHPE; M J Labuschagne,2 MB ChB, MMed (Ophth), PhD (HPE); J Bezuidenhout,1 BA Ed, HDipEd, PGDipHPE, BA Hons Psych, MEd (Psych of Ed), DTech (Ed) 1 Division of Health Sciences Education, Office of the Dean: Health Sciences, Faculty of Health Sciences, University of the Free State, Bloemfontein, South Africa 2 Clinical Simulation and Skills Unit, Support School of Medicine, Faculty of Health Sciences, University of the Free State, Bloemfontein, South Africa Corresponding author: R G Campbell (phdhpe@gmail.com) This open-access article is distributed under Creative Commons licence CC-BY-NC 4.0. mailto:phdhpe@gmail.com 184 September 2018, Vol. 10, No. 3 AJHPE Research and moderators, who may rely on traditional practices rather than current best practice in simulation. A qualitative study was conducted by the primary author in fulfilment of a Master’s in Health Professions Education to identify assessment criteria for integrated clinical simulation for emergency care education programmes in SA. Perspectives of ALS paramedics on the use of this instrument for summative assessment were captured in Theme 1 of the Results, as set out below. The objective of this article was to present and discuss these perspectives, offer a critique of assumptions regarding the use of this assessment instrument, and recommend improvements for its use. Methods A qualitative, single, embedded case study design was used. The main unit of analysis was integrated clinical simulation as a summative assessment instrument, and the two subunits of analysis were assessment criteria and case types. The experiences and perceptions of ALS paramedics as students and their post-qualification encounters with life-threatening emergencies in clinical practice were identified as valuable sources of data for populating the units of analysis. These qualitative data were collected by means of four focus group interviews. This type of interview was chosen as the method of data collection, based on the expectation that information would be enhanced through group interaction.[6] A non-random sampling method, using both convenience and purposive sampling, was employed. The survey population included all ALS paramedics in the Bloemfontein area in the Free State, SA, who had at least 2 years’ clinical experience at ALS level and had obtained their qualification in SA. A total of 42 ALS paramedics were identified and invited to participate in the study, of whom 28 volunteered to participate in the focus group interviews. Before the interviews commenced, participants signed consent forms and were informed that the interview session would be audio-taped. Confidentiality was ensured by concealing the identities of the participants to people outside the group. Participant response references were omitted in this article to achieve anonymity. Ethical approval (ref. no. ECUFS 204/2013) for the study was obtained from the Ethics Committee, Faculty of Health Sciences, University of the Free State, Bloemfontein. An independent, experienced facilitator conducted all four focus group interviews. The number of participants in each focus group was 10, 7, 5 and 6, respectively. The interview guide was standardised for all four groups to enable comparison of responses across the different groups. Questions focused on the following areas: • the behaviours and attitudes required of ALS paramedics to effectively manage a critical patient in the out-of-hospital context • competencies that should be assessed when using integrated clinical simulation • case types or scenarios that present the best opportunities to determine the ability of an ALS paramedic to deal with a life-threatening condition effectively, and which are essential for simulations • factors that should be included in the design of integrated clinical simulations for summative assessment. The audio-taped interviews were transcribed by the primary author and verified by the focus group facilitator and group members. Open-coding analysis of transcriptions was used to link participant perspectives to elements of assessment criteria in the relevant categories. Inductive and deductive processes of interpreting results were used in the context of the literature review, experience of the primary author and voices of participants to form thick descriptions.[7] The themes and categories into which descriptive data were placed were derived from the elements of assessment criteria clarified by SAQA (Box 1). The co-authors verified the coding process and thematic analysis. As this is a descriptive study, the results may support analytical generalisation, where the logic of results and discussion can be applied to other similar situations.[8] Results and discussion Of the focus group participants, 4 (14%) had the minimum of 2 years’ clinical experience, 8 (29%) had 3 - 4 years’ clinical experience, 13 (46%) had 5 - 10 years’ clinical experience and 3 (11%) had >10 years’ clinical experience. Box 1. Clarification of terms ALS paramedic: someone registered with the HPCSA and qualified to render ALS in a specified scope of practice. ALS qualifications include Critical Care Assistant (CCA), Emergency Care Technician (ECT), National Diploma: Emergency Medical Care (NDip: EMC), Bachelor of Technology: Emergency Medical Care (BTech: EMC) and the Bachelor degree: Emergency Medical Care (professional degree). Integrated clinical simulation: the holistic process of assessing and managing a simulated patient in a realistic clinical setting using a full-body mannequin; during this process, appropriate medical procedures can be performed and appropriate physiological responses to management can be portrayed. This is also known as ‘full-scale, scenario-based simulation’ or ‘full-mission simulation’.[5] Assessment criteria: defined by SAQA as ‘the standards used to guide learning and assess learner achievement and/or evaluate and certify competence’, and include ‘statements that describe the standard to which learners must perform the actions, roles, knowledge, understanding, skills, values and attitudes stated in the outcomes’.[2] They are a clear and transparent expression of requirements against which successful (or unsuccessful) performance is assessed.[3] The assessment criteria should stipulate: • ‘The knowledge, understanding, action(s), roles, skills, values and attitudes that a learner has to display in order to provide evidence that outcomes and competence have been achieved • The level of complexity and quality of these • The context of and conditions under which demonstrations should occur.’[3] Summative assessment: ‘assessment conducted at the end of sections of learning or at the end of a whole learning programme, to evaluate learning achievements related to a particular qualification, part-qualification, or professional designation’.[2] Integrated assessment: a ‘form of assessment which permits the learner to demonstrate applied competence and which uses a range of formative and summative assessment methods’.[4] Applied competence: defined by SAQA as a ‘learner’s ability to integrate concepts, ideas and actions in authentic, real-life contexts’.[3] HPCSA = Health Professions Council of South Africa; ALS = advanced life support; SAQA = South African Qualifications Authority. September 2018, Vol. 10, No. 3 AJHPE 185 Research Some participants had obtained more than one ALS qualification. The ALS qualifications were represented as follows: 7 (25%) Critical Care Assistant, 24 (85%) National Diploma: Emergency Medical Care (NDip: EMC) and 13 (46%) Bachelor of Technology: Emergency Medical Care (BTech: EMC). There were no emergency care technicians who met the inclusion criteria. The critical care assistant training institutions included Netcare 911 School of Emergency and Critical Care, and the provincial Colleges of Emergency Care situated in Durban, Pietermaritzburg and Cape Town. The NDip: EMC training institution included the former Technikon Natal, the current Durban University of Technology, Cape Peninsula University of Technology and Central University of Technology, Bloemfontein. The BTech: EMC training institution included the Durban and Cape Peninsula universities of technology. Participants also had experience of teaching the various levels of qualifications in emergency care. Of the 28 participants, 5 (18%) had experience teaching at Basic Life Support level, 9 (32%) at Intermediate Life Support level, and 17 (61%) at ALS level. Some had taught more than one level. Regarding simulation, 18 (64%) had some experience with designing simulations, 21 (75%) with presenting simulations and 21 (75%) with assessing simulations. From an analysis of the participant group, the range of experience in clinical practice and involvement in simulation together with representation across the ALS qualifications suggest that this group was well suited to render valuable perspectives on the research questions. Theme 1 addressed perspectives of participants on clinical simulation as an integrated summative assessment instrument (Table 1). Focus group participants were in favour of assessing student performance in authentic situations (Table 2). The authentic situation was identified as the real patient, real-time interaction and real conditions.[1,4] Assessment in the clinical practice setting, although seen as ideal, was identified as impractical and unreliable owing to inconsistent case presentation (type and severity) and unpredictable occurrence of cases suitable for assessment. Focus group participants highlighted the need for patient simulations to replicate the clinical setting as realistically as possible (Table 2). Objective aspects of a real clinical case in out-of-hospital emergency care includes the context, environment, clinical expression, response to interventions, use of technology and realism with the tasks and procedures that should be performed. This prerequisite for realism underpins a belief that student performance in clinical simulation should correlate with student performance in clinical practice. In contradiction to the support given by participants to the ideals of using clinical simulation for assessment, the experience-based perspectives of participants expressed the opposite, i.e. that the integrated clinical simulation lacks the validity, reliability and generalisability subsumed in its practice. The following reasons were identified for this paradox in performance between the simulation setting and clinical practice (Table 3): • unrealistic or inadequate representation of ALS paramedic practice in simulation • focusing simulation use on summative assessment. Unrealistic aspects of the clinical simulation experience include limitations of the simulator, insufficient replication of environmental conditions, poor translation of clinical cases through simulation that causes confusion and misinterpretation by students, and inadequate assessment across the range and complexity of clinical emergencies. The unrealistic and limited features of full-body mannequins for replicating human anatomy and physiology were perceived to undermine true performance in clinical simulation for assessment. How physical and pathophysiological features of the patient’s condition are represented and obtained in simulation may conflict with how such information is gathered Table 1. Summary of theme 1 Theme 1 Category Clinical simulation as an integrated summative assessment instrument 1. Assessment in the authentic situation 2. Fictional features of clinical simulation confounding true student performance 3. Influence of assessment on true student performance in clinical simulation 4. Assessment principles and the integrated clinical simulation Table 2. Qualitative responses reflecting perspectives of using integrated clinical simulation for summative assessment in emergency care education in South Africa[11] 2.1 Support for assessment in the authentic situation ‘I mean if you really want to assess somebody on patient management and identification of underlying illnesses it should actually be on real patients with real conditions.’ ‘Taking them out and assessing them on the road, the fairness of it I don’t see because you cannot be consistent in what you are assessing because each student is going to be getting a different patient every time.’ ‘But if we can get that student to believe that is the real thing, then you are going to get the demonstration hopefully as close as possible to what they would do in real life.’ 2.2 Concerns about the validity, reliability and generalisability subsumed in the historical use of the integrated clinical simulation ‘In my experience we have had cases where there’s been a clear difference between the way the student paramedic performs in a simulated environment as compared to the way they perform in real life … and we’ve had confident students who excel at the roadside under real circumstances and then fall to pieces in a simulation laboratory kind of situation, and then quite the opposite where others have excelled in the laboratory and yet when it comes to the real thing they are not able to deal with it.’ ‘People find it difficult to take something that’s fiction and you have to treat it as if it’s reality.’ 186 September 2018, Vol. 10, No. 3 AJHPE Research in clinical practice settings. Verbally transmitting information, which would normally be obtained through observation, inspection, auscultation and palpation, leads to misinterpretation and may confound information gathering. Reliance on memory and imagination may present a barrier to engaging realistically in a stressful situation. Additional factors confounding realistic experience in simulation include: • fast-tracking participant actions, such as being told the intravenous line is sited, when it has not been physically performed • fast-tracking patient progress (improvement or deterioration) and response to intervention into the time limit given for the assessment. Environmental and contextual factors in paramedic practice were seen to influence performance. Poor lighting and noise may hinder obtaining clinical information through observation, inspection and auscultation. Interference or distractions by family members or bystanders adds to the challenges of patient assessment and treatment. The quality of interprofessional interaction at the scene and at the medical facility contributes to patient care efforts. When these variables are excluded from the integrated clinical simulation, assessing applied competence is confounded. The range of possible life-threatening emergencies in clinical practice spans medical disciplines, age groups and unique patient categories often complicated by environmental factors. Given that the integrated clinical Table 3. Qualitative responses reflecting reasons for the paradox of performance between the simulation setting and that expected in clinical practice 3.1 Unrealistic or inadequate representation of ALS paramedic practice in simulation 3.1.1 Limitations of patient simulators ‘You know the dolls that we are getting … they are making it close to reality, it is still not reality … it is not a human being.’ ‘The differences between a simulation and reality is that reality is a person who can either talk to you, you can see and you can visualise what is actually wrong with the patient.’ 3.1.2 Inadequate representation of the clinical case ‘If you are told the patient has a femur fracture when you walk in there and you don’t even see any fracture … you can even miss the fracture and treat everything else and even forget that.’ ‘There is always somebody that interprets everything for you – so there’s always somebody, you are asking somebody a question; is he pale, is he fat?’ ‘The time that is spent in a simulation is not the same as the time it would take in real life, but you are forever compressing certain aspects to say, ok, we consider that as done although it is not yet done … it’s unrealistic.’ 3.1.3 Inadequate representation of the conditions and context of ALS paramedic practice ‘Being in a simulation environment, you walk in – it’s quiet – you’ve got the simulator, you basically do what you see there. Once you get on the road there’s a difference there, because there’s traffic, there’s noise, it’s raining – the students start reacting differently – their thought process is different – what they might have passed here – they forget the stuff they have to do on the road – there are dogs barking, there’s family fighting – all those kind of things that you don’t find when doing a simulation.’ 3.1.4 Insufficient representation of range and complexity of clinical emergencies ‘The problem is this shot in the dark with one or two simulations that then represent what that person can do for all aspects and all categories – a little bit hit and miss – you might have got them on a good day or on a bad day … if there are no limitations then the ideal situation would be to test in each category – in other words, a paediatric simulation, a maternity simulation, a peri-arrest cardiac patient, a trauma patient.’ 3.2 Simulation use for summative assessment 3.2.1 The problem with the once-off nature of the simulation assessment ‘Not by putting the poor fellow or lady at the end of the year into a situation where this is your be-all and end-all … you’ve wasted your whole year if you do not pass today’s assessment!’ 3.2.2 Stress of once-off assessment confounds student performance ‘If he walks in the room and he is all sweaty and trembling … so there’s a psychological aspect, how he presents himself and carries himself inside the plastic environment.’ ‘Your whole CCA depends on one sim and you get in that room and there’s 7 or 8 doctors, professors and paramedics, so it really intimidates you and even some of the best students failed the sim.’ 3.2.3 Factors influencing objectivity and quality of assessment ‘With the evaluator, his knowledge, skills and experience will also determine the way he’s going to evaluate the specific student.’ ‘In some cases I think it is more the fault of the assessors who are not properly assessing people – letting them through the system just to fulfil the number of people to go through the system instead of quality assessment on those students.’ ‘But how harsh is that to the evaluator or assessor because now you are sitting with 5 or 10 or 20 or whatsoever assessments during that specific day – you also get exhausted, you are also getting tired.’ 3.2.4 Extent of student adaptation to simulation for assessment ‘Ya, they’ve got to imagine it, but if the simulation isn’t close to reality and the student fails to pick up the guy is not breathing because it’s not set as it would be in reality, then I mean there’s no fairness towards the student.’ ‘You are looking at the narrator … looking at your facial expressions, I mean it’s not natural because you are trying to fish for things there; clues or voice tones and stuff like that which are actually not giving you that realistic environment … some individuals … they are not actors and you have to have a sense of acting or role-play in a simulation.’ ALS = advanced life support; CCA = critical care assistant. September 2018, Vol. 10, No. 3 AJHPE 187 Research simulation summative assessment has traditionally been conducted as a once-off evaluation, means there is an under-representation in assessment of the range of medical and trauma emergencies. When replication of the conditions, context and scope of clinical practice is inadequate, then true performance is confounded and the authentic situation for summative assessment is questionable. Within the context of this complex array of factors affecting student performance, participants also expressed concern about the unfair risk to student success by using the integrated clinical simulation as a once-off assessment. Specifically, the stress associated with this manner of assessment was reflected as a factor confounding authentic student performance. Factors cited as exacerbating the stress of this assessment were the presence and number of assessors, perceived bias and unpreparedness of assessors to judge performance in simulation, and assessor fatigue during lengthy assessments. Student success in simulation assessment may result from students adapting to this artificial context by developing skills to cope with stress and navigate fictional aspects of the simulation event. Students who cannot cope or make this adaptation, risk underperforming in the assessment. The authors suggest that this coping and adaptive behaviour renders true performance in assessment difficult or impossible and prevents translation to clinical practice. This is prevalent where: (i) rules that convert a clinical case into a clinical simulation are not explicit;[9] (ii) rules that address how inconsistencies between the clinical simulation and the real clinical case will be managed are hidden or change constantly; (iii) there is dissonance in constructive alignment, with differing standards for formative and summative simulation events with regard to design, facilitation and assessment practices;[10] and (iv) the assessment criteria and performance standards are unclear. Furthermore, the focus of a formative simulation experience may be on how many simulation events can be offered in preparation for assessment, without attention to quality feedback, debriefing and translation to clinical practice. A signature component of ALS paramedic practice is the unique context of infinite variations and combinations of environments, socioeconomic conditions and patient profiles.[11,12] Bland et al.[13] argue that realistic representation of real-life situations by simulation is central to engaging learners in authentic responses. For example, Mills et al.[14] demonstrated that inadequate environmental realism in clinical simulation impeded paramedic student performance, with increased levels of anxiety. Bland et al.[13] further contend that, although objective reality is essential to simulation, the students’ subjective interpretation of the simulation experience influences how realistically they engage in simulation. Together with engineered realism, authentic clinical experiences in simulation are an integrated function of participant perceptions of reality and willingness to engage realistically.[13] Rudolph et al.[15] argue for a psychologically safe environment for learners to engage realistically in a simulation event. Psychological safety is a state where learners feel safe enough to accept the discomfort that accompanies performance in front of lecturers, examiners and/or peers. Clarifying expectations, instituting a ‘fiction contract’ and consistently applying simulation rules and rituals (governing how clinical cases are converted to hypothetical simulated cases, how fictional elements are addressed and how simulation events are facilitated) are necessary elements of a psychologically safe environment.[9,10,13] Preparing learners for participation in simulation for learning and assessment is therefore crucial to ensure fairness and validity of simulation as an authentic assessment. The threat associated with summative assessment can be mitigated through effective formative development and trust in a robust and consistently applied simulation process. Barr et al.,[10] for example, demonstrated improved benefits of clinical simulation to paramedic students by: (i) shifting the focus from summative assessment to learning through simulation; (ii) making assessment criteria explicit; and (iii) applying these criteria to both formative and summative simulation assessments. McKenna et al.[12] suggest that the three essential ingredients for effective employment of simulation in emergency care education are simulation facilities, training of educators in the use of simulation, and resources to integrate simulation in the curriculum. In clarifying the reliability of assessment, SAQA states that ‘assessors should be trained and competent in administering assessments’.[3] Topping et al.[16] argue that educators using simulation ‘should have a minimum skill set [to] integrate simulation appropriately into program delivery’. A recent consensus survey of simulation use in accredited paramedic programmes across the USA revealed under- utilisation of simulation.[12] The two main barriers to using simulation effectively were lack of staff training in facilitating simulation, and the writing of scenarios. Lack of training in debriefing was sixth on the list. In their updated policy on assessment, SAQA states that ‘assessment is used to facilitate learning. This kind of assessment – assessment with instruction, engagement and feedback – is adopted wherever feasible and appropriate.’[2] Simulation is a complex educational methodology that blends educational and clinical practice elements. Medical simulators are becoming more technologically complex, and educators are expected to use simulators to engage students in authentic learning and assessment. McKenna et al.[12] highlight the complex array of factors involved in using simulation, and suggest that ‘effective instruction using technology requires that educators have adequate knowledge about content, pedagogy, when to select a particular method to teach specific content, and the technology being used’. Recommendations for change The following recommendations from findings are offered to employ clinical simulation more effectively in the context of emergency care education in SA: • Simulation design, facilitation and feedback strategies used for teaching, learning and assessment should be evidence based, replicate essential features of the clinical practice environment, demand real-world actions and responses from students and support a longitudinal approach to measuring performance across the range of emergencies and medical disciplines. • Students need to be prepared effectively to learn and be assessed through simulation, where such learning and assessment are focused on clinical practice realities and achievement of qualification outcomes. Such preparation includes clarity on the rules of simulation that address the discrepancies between the simulation and real clinical context. • Simulation should first be employed to facilitate learning prior to its use for summative assessment. The formative use of simulation should apply the same assessment principles and criteria employed in the summative assessment process to ensure alignment between assessment for learning and assessment of learning. • Since simulation-based learning and assessment require a specific skill set, facilitator and examiner training is required to use this specific instrument effectively. 188 September 2018, Vol. 10, No. 3 AJHPE Research • The PBEC as the regulatory body for emergency care education, in conjunction with emergency care education providers, should develop and promote best-practice guidelines and assessment criteria for using integrated clinical simulation as an assessment instrument. Conclusion Achievement of qualification outcomes and appropriate preparation of newly qualified ALS paramedics for clinical practice are the prerogative of emergency care education programmes. To accomplish this, simulation has been promulgated as a valuable instrument for assessing applied competence. The ideals of simulation address the range of medical and trauma emergencies, with transfer of knowledge and skills to clinical practice. By means of this study, however, perspectives of participating ALS paramedics on their experience with simulation for summative assessment shed doubt on whether these ideals are being achieved. The issues raised challenge the credibility and assumptions of simulation as a summative assessment instrument, yet offer education regulators and emergency care education providers an opportunity to develop evidence-based guidelines and practices for employing simulation for teaching, learning and assessment. Having such guidelines, together with trained facilitators and assessors, is likely to promote strategies for authentic student engagement, design and facilitation of simulation that foster realism and immersion, and constructive alignment of simulation for formative and summative assessment across the scope of ALS paramedic practice. With developments in simulation technology, the challenge remains for emergency care educators to remain up to date in the selection and use of the most appropriate simulators that enable learning, achievement and assessment of qualification outcomes in conjunction with other facilitation and assessment modalities. Innovative approaches to accurately replicate pertinent clinical, environmental and social-professional elements of ALS paramedic practice in simulation are required. Important elements for eliciting true performance and influencing performance, and methods of reliably assessing the spectrum of simulation in emergency care, are subjects for further research. This study did not compare simulation with other integrated assessment instruments or other platforms, such as workplace assessment. Using simula- tion as a learning-centred instructional methodology was not addressed in this study. The use of other types of simulators by emergency care education programmes in SA was not discussed and requires additional research. The design of a simulation-based programme for emergency care education, incorporating the range of simulation activities and best use of simulators, needs further investigation for optimising programme delivery. The results of this study can be applied in similar settings in southern Africa. Acknowledgements. We acknowledge the focus group participants for their valuable contribution to the study. Author contributions. RGC: conducted the study in fulfilment of a Master’s degree in Health Professions Education from the University of the Free State, Bloemfontein, and subsequently prepared the manuscript. MJL: study leader, and gave guidance and expertise in the field of simulation. JB: co-study leader, and provided input on technical detail and research aspects. Both co-authors assisted with verifying the coding of data, with development of themes and categories, and provided a critical review of the content and technical aspects of the manuscript. Funding. The Health and Welfare Sector Education and Training Authority (HWSETA) provided funding for the study. Conflicts of interest. At the time of preparing the manuscript, the primary author was in the employ of a private company promoting and conducting healthcare simulation training. 1. Health Professions Council of South Africa. Professional Board for Emergency Care. Guidelines for the Completion of the Portfolio for Institutions Wishing to Offer the Emergency Care Assistant (ECA), Emergency Care Technician (ECT) and Emergency Care Practitioner (ECP) Programmes. Pretoria: HPCSA, 2015. 2. South African Qualifications Authority. National Policy and Criteria for Designing and Implementing Assessment for NQF Qualifications and Part-Qualifications and Professional Designations in South Africa. Pretoria: SAQA, 2015. 3. South African Qualifications Authority. Criteria and Guidelines for the Assessment of NQF Registered Unit Standards and Qualifications. Pretoria: SAQA, 2001. 4. South African Qualifications Authority. Guidelines for Integrated Assessment. Pretoria: SAQA, 2005. 5. Health Professions Council of South Africa. Professional Board for Emergency Care. Template and Guidelines for the Completion of the External Moderator’s Report. Pretoria: HPCSA, 2014. 6. Bender DE, Ewbank D. The focus group as a tool for health research: Issues in design and analysis. Health Transit Rev 1994;4(1):63-80. 7. Ponterotto JG. Brief note on the origins, evolution, and meaning of the qualitative research concept thick description. Qual Rep 2006;11(3):538-549. 8. Yin RK. Applications of Case Study Research. 3rd ed. Los Angeles: SAGE Publications, 2012:18-19. 9. Dieckmann P, Gaba D, Rall M. Deepening the theoretical foundations of patient simulation as social practice. Sim Healthcare 2007;2(3):183-193. https://doi.org/10.1097/SIH.0b013e3180f637f5 10. Barr N, Readman K, Dunn P. Simulation-based clinical assessment: Redesigning a signature assessment into a teaching strategy. Australas J Paramed 2014;11(6). 11. Smith A, Andersen P. Proven effective: Simulation-based assessment facilitates learning and enhances clinical judgment. J Emerg Med Serv 2014:3-8. 12. McKenna KD, Carhart E, Bercher D, Spain A, Freel J. Simulation in EMS education: Charting the future. Association of EMS Educators (NAEMSE). 2015. https://c.ymcdn.com/sites/naemse.site-ym.com/resource/ resmgr/Docs/SimPressRelease15.pdf (accessed 13 July 2018). 13. Bland AJ, Topping A, Tobbell J. Time to unravel the conceptual confusion of authenticity and fidelity and their contribution to learning within simulation-based nurse education. Nurse Educ Today 2014;34(7):1112-1118. https://doi.org/10.1016/j.nedt.2014.03.009 14. Mills BW, Carter OB-J, Rudd CJ, Claxton LA, Ross NP, Strobel NA. Effects of low- versus high-fidelity simulations on the cognitive burden and performance of entry-level paramedicine students: A mixed-methods comparison trial using eye-tracking, continuous heart rate, difficulty rating scales, video observation and interviews. Sim Healthcare 2016;11(1):10-18. https://doi.org/10.1097/SIH.0000000000000119 15. Rudolph JW, Raemer DB, Simon R. Establishing a safe container for learning in simulation: The role of the presimulation briefing. Sim Healthcare 2014;9(6):339-349. https://doi.org/10.1097/SIH.0000000000000047 16. Topping A, Bøje RB, Rekola L, et al. Towards identifying nurse educator competencies required for simulation- based learning: A systemised rapid review and synthesis. Nurse Educ Today 2015;35(11):1108-1113. https://doi. org/10.1016/j.nedt.2015.06.003 Accepted 22 January 2018. https://c.ymcdn.com/sites/naemse.site-ym.com/resource/resmgr/Docs/SimPressRelease15.pdf https://c.ymcdn.com/sites/naemse.site-ym.com/resource/resmgr/Docs/SimPressRelease15.pdf https://doi.org/10.1016/j.nedt.2014.03.009 https://doi.org/10.1016/j.nedt.2015.06.003 https://doi.org/10.1016/j.nedt.2015.06.003