Australasian Journal of Educational Technology, 2020, 36(3). 56 Simulation acceptance scale (SAS): A validity and reliability study Baris Sezer, Gulsen Tasdelen Teker, Tufan Asli Sezer*, Melih Elcin Hacettepe University, *Ankara University A review of literature reveals serious problems in the validity and reliability of the measurement tools used in simulation technology acceptance studies conducted in the context of various technology acceptance models. To address this gap in literature, this study proposes a measurement tool that will allow a valid and reliable measurement of students’ acceptance levels. The study was conducted in the 2017–2018 academic year and involved a group of 409 health sciences students. Exploratory factor analysis (EFA) was conducted to examine the construct validity of the conclusions based on gathered measurements. At the end of the EFA, a construct with a single factor and 24 items which explained 54.87 percent of the total variance was obtained. Based on the findings of the research, it was concluded that simulation acceptance scale produced from the EFA could be used for valid and reliable measurements regarding the general acceptance of simulation technologies by the health sciences students. Implications for practice or policy: • The SAS can be used to produce valid and reliable measurements regarding the general acceptance of simulation technologies by students studying health sciences. • Educators may need to consider students’ simulation acceptance levels before a simulation course. Keywords: technology acceptance, medical education, simulation, health sciences students, skill training, scale development Introduction The development of skills and behaviours in medical education is regarded as important as the acquisition of knowledge, and it is achieved by working on real patients in a significant part of the education. Teaching and learning in medicine began as an apprenticeship model but has evolved into a more structured model, reflecting of the developments and innovations in medicine and educational sciences. Medical education in the twenty-first century is a pedagogy based on qualifications, prioritising practical training where miscellaneous information and education technologies can be used (Elçin, 2010). Combined with developments in education and the innovations in the delivery of healthcare services, medical students are facing new challenges in their learning environments: the ageing population, the increase in chronic diseases, the decrease in diagnosis and treatment durations, the increasing number of students, and the decreasing number of trainers. Hence, the training of health professionals during the delivery of healthcare services should be reorganised. One of the most important maxims in medical science has been “first, do no harm”, and over the past 25 years, patient safety has become one of the most prominent issues in the field. “To err is human”, a report published in the United States in 1999, revealed that many people were losing their lives every year as a result of medical errors. This is higher than the number of lives lost to traffic accidents in the United States (Institute of Medicine, 1999). Similarly, more than 10 to 16% of inpatients in the United Kingdom and Australia respectively, suffer from the same fate (Institute of Medicine, 1999). Such figures demanded measures be taken not only in-service delivery models but also in medical education. Within the scope of health sciences training, it is not possible to go beyond a certain level of qualification without working on real patients. Even though the theoretical approach is to “treat the patient, not the disease”, witnessing the physical, psychological and social differences, and diversity in individuals, and being able to make an analysis based on a critical approach is a significant benefit. The basic goal in such training is to be able to monitor and assess each patient with regard to their unique condition. Simulation is a technique which can be employed to overcome the above challenges in medical education. Australasian Journal of Educational Technology, 2020, 36(3). 57 Simulation Simulation is defined as a technique that changes or develops real experiences through guided experiences, using a totally participatory style in a naturality that is created by repeating or imitating the existing aspects of the real world (Gaba, 2004). The first simulations in medicine involved simple anatomical models that date back long before discovery of modern plastics and computers (Kunkler, 2006). Over the course of time, and gaining rapid popularity at the end of twentieth century, human-patient simulations emerged. This was a significant step in the development of health sciences education (Rosen, 2008). Fidelity, the preferred term used to define the verisimilitude and technical capacity of the simulation, is generally addressed in three categories: low, medium, and high (Elçin & Odabaşı, 2016). (1) Low fidelity simulation: These simulators have no feedback mechanism and include part-task trainers, fresh-frozen cadavers, and some screen-based simulations. (2) Medium fidelity simulation: This group of simulators include screen-based simulators, full body task trainers, and haptic simulators. They are able to provide feedback. (3) High fidelity simulation: This group includes sophisticated mannikins that can provide instant feedback in the form of reactions to any interventions. Simulation practices offer advantages when it is difficult, risky, or expensive to provide a real practical environment (Motola, Devine, Chung, Sullivan, & Issenberg, 2013). From the viewpoint of health education, simulation-based training is highly effective in the achievement of knowledge, skills, and behaviours among health professionals by providing a safe learning environment both for patients and learners (Boyle et al., 2016; Datta, Upadhyay, & Jaideep, 2012; Motola et al., 2013). Figure 1 shows how the skills categories are matched to the simulation types: shown in corresponding colours. The simulations used in health sciences practices can be divided into three basic categories: physical, virtual, and biological (Cobett & Snelgrove-Clarke, 2016; Luctkar-Flude, Wilson-Keates, & Larocque, 2012). Figure 1. The skills triangle and the simulation triangle in healthcare (Lampotang et al., 2013) A physical simulator is primarily for learning psychomotor skills and has no virtual components. Part task trainers suitable for IV injection are examples of this type of simulator (Lampotang et al., 2013). A virtual simulator such as the virtual anesthesia machine (VAM) (Fischler, Kaschub, Lizdas, & Lampotang, 2008) is a web-enabled, screen-based simulator suitable for learning cognitive skills. Virtual simulators do not contain any physical or tangible components. A biologic simulator (standardised/simulated patients/participants) provides learners with high fidelity simulations of interpersonal interactions (Lampotang et al., 2013) and also provides an innovative approach to emphasise the critical role of affective skills (MacLean, Kelly, Geddes, & Della, 2017). These three types of simulators can also be utilised in Australasian Journal of Educational Technology, 2020, 36(3). 58 healthcare education in a combined manner. For instance, a central venous access (CVA) part task trainer combining a physical mannikin and its virtual counterpart (mixed simulation) can be used together for learning psychomotor and cognitive skills. Another combined simulation type is hybrid simulation (Kneebone, 2010) which combines standardised patients with part task trainers for learning affective and psychomotor skills at the same time such as breast or prostate examination (Kotranza & Lok, 2008; Lampotang et al., 2013). Yet another combined simulation type is mixed reality human (MRH), a new type of embodied agent/avatar that affords touch-driven communication and behaves like a real human (Cordar, Wendling, White, Lampotang, & Lok, 2017; Kotranza, Lok, Deladisma, Pugh, & Lind, 2009). Regardless of the modality of simulation, four basic benefits of using simulations in medical education have been reported (Kunkler, 2006): (1) developing (enriching) educational experience, (2) increasing patient safety, (3) cost-benefit effectiveness, and (4) opportunities for continuous professional development. All of these benefits depend on the proper use of simulation methodologies (Issenberg, McGaghie, Petrusa, Lee Gordon, & Scalese, 2005), which should define clear and measurable targets and outputs for the training, integrate simulation into the curriculum, provide effective feedback, ensure repetitive practice, gradually complicate the practice, control the training environment, and finally take individual characteristics into consideration (Lammers et al., 2008). To make the best use of these technologies in educational environments, the participants need to develop positive attitudes, beliefs, and intentions toward their use. Technology acceptance The technology acceptance structure consists of cognitive and psychological elements related to the use of technology, and several models (e.g., technology acceptance model, technology acceptance model 2, and innovation diffusion theory) have been developed, with an aim to reveal the factors affecting the effective and productive use of technology. Believing that a single model would be insufficient to explain technology use, and that this topic needs to be examined in a multidimensional manner, Venkatesh, Morris, Davis, and Davis (2003) developed the unified theory of acceptance and use of technology (UTAUT) model (Figure 2). Figure 2. UTAUT (Venkatesh et al., 2003, p.447) This model, which combines elements similar to those included in eight different models (social cognitive theory, innovation diffusion theory, technology acceptance model, theory of planned behaviour, combined Australasian Journal of Educational Technology, 2020, 36(3). 59 technology acceptance model and theory of planned behaviour, motivational model, model of PC utilisation, and theory of reasoned action), identifies performance expectancy (PE), effort expectancy (EE), social influence (SI), and facilitating conditions (FC) as the four basic elements determining a behavioural use intention. PE refers to the belief that performance will improve through the use of technology, EE refers to the belief that the technology will be easy to use, SI refers to the belief held by significant people in one’s social environment that the technology in question should be used, and FC refer to the belief that various elements exist that support the use of technology. The UTAUT model developed by Venkatesh et al. (2003) includes intermediary variables that indirectly affect intention or use (such as gender, age, experience, and volunteering) while also determining the factors with direct effects, as mentioned above. A review of the literature on UTAUT uncovers some studies that include intermediary variables in addition to determining variables (Magsamen-Conrad, Upahdyaya, Joa, & Dowd, 2015), whereas others (Carlsson, Carlsson, Hyvonen, Puhakainen, & Walden, 2006; Escobar-Rodriguez, Carvajal-Trujillo, & Monge-Lozano, 2014; Nicholas, Azeta, Chiazor, & Omoregbe, 2017; Oktal, 2013; Sedana & Wijaya, 2010; Thomas, Singh, & Gaffar, 2013; Zhou, Lu, & Wang, 2010) do not. Venkatesh et al. (2003) determined that the model covering the determining variables explained 70% of the intention. Accordingly, the present study does not include the mentioned intermediary variables, focusing rather on the determining factors that affect the acceptance of the system by users. The UTAUT model is one of the most frequently used models cited in the literature, and has been used to examine many different technologies such as social media (Escobar-Rodríguez et al., 2014), homecare technologies (Kutlay, 2015), medical devices (Kurtuluş, 2015), mobile learning (Carlsson et al., 2006; Thomas et al., 2013), online family dispute resolution systems (Casey & Wilson-Evered, 2012), tablet devices (Magsamen-Conrad et al., 2015), information systems (Oktal, 2013), and learning management systems (Jong, 2009; Maina & Nzuki, 2015; Nicholas et al., 2017; Raman, Don, Khalid, & Rizuan, 2014; Sedana & Wijaya, 2010; Sezer & Yilmaz, 2019). Among these technologies, simulation tools/practices (screen-based simulation, part task trainers, mannikins, haptics etc.) hold a prominent place as a frequently used educational material/resource. Previous studies have determined the more users accept such simulations may increase their performance (performance expectancy), the more likely they are to use these technologies (Boyle et al., 2016; Rasimah, Zaman, & Ahmad, 2011; Zhu, Hadgar, Massielle, & Zary, 2014). These technologies need to be accepted by users in both daily life and in the educational context in order to provide the maximum possible benefits. Purpose of the study Literature indicates that use of a technology is based on the belief, attitude, and intentions of the potential user, and the same also applies to simulation (Casey & Wilson-Evered, 2012; Carlsson et al., 2006; Escobar- Rodríguez et al., 2014; Maina & Nzuki, 2015; Nicholas et al., 2017; Sedana & Wijaya, 2010; Thomas et al., 2013). The acceptance of simulation practices by the users is the first requirement for the development of knowledge, skills, or attitudes (Botezatu, Hult, Tessma, & Fors, 2010; Cant & Cooper, 2010; Deladisma et al., 2007; Johnsen, Fossum, Vivekananda-Schmidt, Fruhling, & Slettebø, 2018; Lin, Travlos, Wadelin, & Vlasses, 2011; Oh, Jeon, & Koh, 2015; Stroup, 2014). The acceptance of simulations among students increased in accordance with the visual reality (Kotranza et al., 2009; Robb, Cordar et al., 2015) such as animation quality, the coherence between mouth movements and voice, the image quality in screen-based simulations, and with the behavioural reality (Robb, Lok et al., 2015) such as audio quality, eye contact, convenience of behaviours, and non-verbal behaviours. Studies of the acceptance of technology have examined the level of acceptance of miscellaneous technologies at a college level using various models, some of which make use of the UTAUT model. However, despite their frequent use, literature is lacking any valid and reliable measurement tool that focuses on the acceptance of simulation tools and practices in the field of health sciences. Serious validity and reliability problems have been observed in studies (Johnsen et al., 2018; Kurenov et al., 2017; Rasimah et al, 2011) measuring the level of acceptance of simulation technologies among users. It has been further noted that new scales have been developed for various technology acceptance models that can be applied to simulation. However, limited information is available regarding their validity. Furthermore, the Cronbach's alpha value was calculated in only one of these studies (Rasimah et al., 2011) with the aim of providing proof of the reliability of the measurements. The aim of our study is to develop a simulation acceptance scale (SAS) measuring the level of general acceptance of simulation technologies based on the UTAUT model, and to provide valid and reliable results. Australasian Journal of Educational Technology, 2020, 36(3). 60 Method Study group At Hacettepe University over the course of 3 years students of the faculty of medicine engage in various simulation practices within the scope of the Good Medical Practices program, involving standard patient interviews, practice of occupational skills on part task trainers/mannikins, and the application of medical skills in simulated environments. By the time the students reach the third year, they have gained adequate experience and knowledge in simulation. Accordingly, for the purpose of this study, data was collected from students studying in their third year in the Faculty of Medicine. The number of students who fitted the profile of the scale was 415, although 6 of the students were removed from the data set after giving the same answer to all items on the measurement tool. Accordingly, the final study group comprised 409 out of the 531 students studying in their third year at the Hacettepe University Faculty of Medicine in the 2017– 2018 academic year. A convenient sampling method was used in the determination of the study group, of which 51.3% were female, and the remaining 48.7% male. The mean age of the students was 21.3 years, and the numbers of students studying medicine in Turkish and in English were 240 (58.7%) and 169 (41.2%), respectively. Procedure First, a content validity analysis was performed in the study according to the validity procedures. While developing the SAS, items related to the four basic variables defined by Venkatesh et al. (2003) as determinants in the acceptance of technology by individuals, that is PE, EE, SI, and FC, were created. The opinions of four experts were obtained while preparing the items of the scale, with the aim of creating an item pool that reflected precisely the structure to be measured. All of the experts were post-doctoral, with one working in the field of medical education, and the other three working in computer and instructional technologies education. The SAS item pool comprised a total of 36 items representing the four sub-dimensions was used to developing the scale. A 5-point Likert scale (strongly agree [5], agree [4], neither agree nor disagree [3], disagree [2] and strongly disagree [1]) was used for all the statements in the scale. In line with the expert feedback, three items in the scale were removed and the remaining items were amended as required. Before proceeding with the implementation, the opinions of one assessment and evaluation expert and one Turkish language expert were garnered, and upon making the required corrections to the format and wording, the scale was ready for a preliminary study. The primary objective of a preliminary study is to determine the comprehensibility of the scale items among the target group and make it possible to obtain feedback regarding the time needed to complete the scale. Accordingly, the preliminary study was conducted with a total of eight students who, upon completion of the study, were interviewed to gain insight into the comprehensibility of the scale items. As a result, two items were clarified in line with the feedback from the students, and the time given to complete the scale was determined based on the average time spent by the students during the preliminary study. The data collected during the preliminary study, which was conducted solely to obtain feedback, was not included in the data set used in the analyses. Data analysis Within the scope of the validity studies, an exploratory factor analysis (EFA) was conducted to determine the validity of the structure, with the aim being to discover the limited number of conceptually significant variables by gathering together a large number of interrelated variables (Çokluk, Şekercioğlu & Büyüköztürk, 2012; Stevens, 2009; Tabachnick & Fidell, 2014). EFA is a method used to turn a large set of variables into a smaller set of factors or components (Pallant, 2016). One of the most significant points required to be taken into consideration while determining whether the data set is suitable for the factor analysis is the sample size. Even though the literature contains different arguments regarding sample size, it is generally accepted that the larger the sample, the better the results (MacCalllum et al., 1999; Pallant, 2016). Kline (1994) reported that a sample of 200 persons would be adequate to obtain reliable results from the factor analysis, while Field (2018) and Tabachnick and Fidell (2014) reported that a sample of at least Australasian Journal of Educational Technology, 2020, 36(3). 61 300 persons was required for the factor analysis, but that a smaller sample of 150 persons may also be adequate in cases where the factor loads are sufficiently high. Another approach taken into consideration in the determination of sample size is to base it on the ratio between the number of participants and the number of items. Tabachnick and Fidell (2014) argued that the number of participants should be five times of the number of items, while according to and Hair, Black, Babin, Anderson, and Tatham (2006) and Nunnally (1978) it should be 10- and 20-fold, respectively. Based on the previous literature, the sample size of 409 students in the study was found to be appropriate, since it met the majority of criteria related to the factor analysis sample size. Another means of checking whether a data set is appropriate for the factor analysis is to examine the Kaiser- Mayer-Olkin (KMO) value and to carry out a Bartlett’s test. In the literature, for a set of data to be appropriate for the factor analysis, the KMO coefficient should be greater than 0.60, and Bartlett's sphericity test should provide statistically significant results (Büyüköztürk, 2010; Pallant, 2016). In this study, a parallel analysis method was also used to determine the number of dimensions. The Cronbach's alpha coefficient was calculated to estimate the reliability of the SAS data, and the adjusted item-total correlation was calculated to determine the items’ discrimination levels. For the purposes of this study, the SPSS 22.0 package was used for EFA, Cronbach's alpha, and item analyses; while a parallel analysis was conducted using the Monte Carlo PCA for parallel analysis software developed by Watkins (2000). Findings Validity studies Construct validity EFA was conducted to determine the construct validity of the SAS. The KMO value was analysed and Bartlett test was conducted to define whether the data was suitable for the factor analysis. The KMO value (0.968) and Bartlett test result (p = 0.000) obtained from the analysis suggests that the data could be used in the factor analysis. As a result of the first factor analysis, items 4, 15, 19, 24, and 32 were removed from the scale, as they appeared under two factors, while the remaining items were subjected to another factor analysis, and the items with very low factor loads, items 6, 10, 22, and 29, were also removed from the scale. As a result of the final factor analysis applied to the remaining 24 items, it was determined that the items loaded onto a single factor with very high factor loads of between 0.574 and 0.854, and that the scale was uni-dimensional. Table 1 presents the factor loads of the items and the explained variance value. Table 1 SAS’ factor loads Item number Item Factor load 1 Attending simulation practices speeds up my learning process. 0.780 2 I learn easily how to conduct simulation practices. 0.634 3 Trainers think that it is beneficial for me to use simulations during training. 0.596 5 Attending simulation practices increases my performance. 0.793 7 Those whose ideas are important to me recommend that I use simulation in my training. 0.670 8 I have trainers to help me with any problems I may encounter during simulation practices. 0.596 9 Using simulation increases my efficiency in the practicing of occupational skills. 0.825 11 I think having simulation practices in our faculty is a significant indicator of good status. 0.574 12 The simulation environment helps me to see my mistakes. 0.719 13 Attending simulation practices facilitates my learning process. 0.837 14 I think learning with simulation is easy. 0.735 16 I think the repetitive nature of simulation practice supports my learning. 0.730 17 I believe simulations will make a positive contribution to my future career. 0.735 18 I think simulation practices are comprehensible. 0.611 20 Training prior to simulation facilitates practice. 0.726 Australasian Journal of Educational Technology, 2020, 36(3). 62 21 I find it beneficial to use simulation in training. 0.817 23 I would like to benefit from simulation practices at least to the extent that my friends do. 0.742 25 Lectures that make use of simulations are more effective. 0.836 26 Learning through simulation practices fits my learning approach. 0.840 27 The use of simulations ensures effective learning. 0.854 28 Training process can be finalised earlier through the use of simulations. 0.737 30 Simulation practices are highly useful in skill training. 0.779 31 I can carry out my tasks more easily using simulation. 0.834 33 Thanks to the trainers, I am able to reach adequate amount of information before using the simulation. 0.645 Total variance explained 54.807% To define the number of factors, the scree plot in Figure 3 was analysed. Büyüköztürk (2010) argues that a sudden decrease after the first factor, followed by the horizontal change along a line chart indicating eigenvalues of all other factors can be put forward as proof of uni-dimensionality. From this position it is clear in Figure 3 that a sharp break occurs between the first and second factors, and that the first factor explains a greater amount of variance than the other factors. As a result, Cattell’s (1978) scree test was used, and single factor was decided to be maintained in the following analyses. Figure 3. Scree plot on SAS factors These results support those obtained from the parallel analysis. In the parallel analysis method developed by Horn (1965), eigenvalue averages are calculated using a random correlation matrix covering the same number of variables and participants as the real data (Yavuz & Doğan, 2015). While determining the number of factors, the number of columns in which the eigenvalues obtained from the real data are greater than the eigenvalue estimated based on random data, are taken into consideration (O’Connor, 2000). As shown in Table 2, the first eigenvalue obtained from the real data is greater than the first eigenvalue estimated based on random data; while for the second eigenvalues, the value estimated based on random data is observed to be greater. Therefor uni-dimensionality was checked using the parallel analysis method. Australasian Journal of Educational Technology, 2020, 36(3). 63 Table 2 Eigenvalues obtained from the parallel analysis Item number Real eigenvalue Eigenvalue estimated based on random data 1 13.154 1.4621 2 1.244 1.3902 Item analysis Another proof of the validity of the SAS structure was obtained from the item analysis. Table 3 illustrates the findings of the item-total correlations, estimated to determine the discrimination levels of the SAS items. Accordingly, based on the findings shown in Table 3, it is apparent that the values related to the item-total correlation vary between 0.547 and 0.827. Table 3 SAS item analysis results Item number Cronbach’s alpha if item deleted Adjusted item-total correlation Mean SD 1 0.960 0.751 4.23 0.896 2 0.961 0.610 3.92 0.927 3 0.962 0.575 4.12 0.928 5 0.960 0.765 4.12 0.898 7 0.961 0.643 3.82 0.942 8 0.962 0.578 3.94 1.057 9 0.960 0.805 4.19 0.897 11 0.963 0.547 3.86 1.196 12 0.960 0.701 4.10 0.900 13 0.959 0.810 4.13 0.873 14 0.960 0.703 4.03 0.868 16 0.961 0.699 4.17 0.932 17 0.960 0.706 4.02 0.945 18 0.962 0.582 3.94 0.851 20 0.961 0.698 4.14 0.919 21 0.960 0.786 4.14 0.914 23 0.960 0.715 4.13 0.947 25 0.959 0.812 4.06 0.938 26 0.959 0.813 4.11 0.922 27 0.959 0.827 4.23 0.853 28 0.960 0.700 4.00 0.936 30 0.960 0.751 4.06 0.899 31 0.960 0.805 4.00 0.862 33 0.961 0.622 3.96 0.969 Reliability The internal consistency reliability of the SAS measurements was estimated using Cronbach's alpha coefficient, and the uni-dimensional scale’s Cronbach’s alpha reliability coefficient was determined to be 0.962. SAS has 24 items collected in a single dimension. A 5-point Likert scale (strongly agree [5] to strongly disagree [1]) is used for the items in the scale. The scale has no item that requires the points to be reversed. The range of possible scores from the scale is 24 – 120, and the higher the total points, the greater the students’ acceptance of simulation practices. Discussion The objective of this study was to develop a measurement tool for the valid and reliable measurement of the level of acceptance among students’ simulation practices. The construct validity of the scale was analysed using EFA, and a single-factor structure with 24 items that explained 54.81% of the total variance was developed. This single-factor structure was supported by the findings of the parallel analysis. While developing the SAS, UTAUT was used to explain user behaviour and technology use, being a strong model Australasian Journal of Educational Technology, 2020, 36(3). 64 with four main dimensions: PE, EE, FC, and SI. The literature review indicated that four dimensional structures are obtained from measurement tools created and developed based on UTAUT for the acceptance of certain technologies (Carlsson et al., 2006; Casey & Wilson-Evered, 2012; Escobar-Rodríguez et al., 2014; Jong, 2009; Kurtuluş, 2015; Kutlay, 2015; Magsamen-Conrad et al., 2015; Maina & Nzuki, 2015; Nicholas et al., 2017; Oktal, 2013; Raman et al., 2014; Sedana & Wijaya, 2010; Thomas et al., 2013). In this study, a uni-dimensional structure was created representing these four structures, and the developed scale has an adequate number of items representing each sub-dimension, with the number of items for the sub-dimensions of PE (items 1, 5, 9, 13, 17, 21, 25, 27, 30), EE (items 2, 14, 18, 28, 31), SI (items 3, 7, 11, 23), and FC (items 8, 12, 16, 20, 26, 33):that is 9, 5, 4, and 6 items respectively. Ideally, the Cronbach's alpha coefficient of a scale should be at least 0.70 (DeVellis, 2012; Nunnaly & Bernstein, 1994; Tezbaşaran, 1997). The Cronbach's alpha coefficient (0.962) used to determine reliability of SAS measurements was estimated to be considerably higher than the value defined in literature. This can be interpreted as proof of the reliability of the SAS measurements. Finally, the item analyses (Table 3) were examined to define the discrimination levels of the items in the scale and items with an item-total correlation value of at least 0.30 were accepted as capable of distinguishing the characteristic to be measured (Erkuş, 2012). In this case, it was interpreted that all items on the scale showed good discrimination levels. Conclusion Although international studies reviewed in the literature offer up a variety of tools for the measurement of the level of acceptance of simulation practices among students, there are number of problems with regards to the validity and reliability of the results obtained from these tools (Johnsen et al., 2018; Kurenov et al., 2017; Rasimah et al., 2011). This study can be considered significant given the extent to which the concept of simulation has been studied in literature in recent years, and its creation of a tool to estimate the acceptance of simulation practices among students. Based on the findings of this research, it was concluded that the SAS developed could be used to produce valid and reliable measurements regarding the general acceptance of simulation technologies by students studying health sciences at the beginning of a simulation training. Students with low acceptance levels can be informed about the importance of using simulation in medical education before participating in a simulation course/lesson/application. Repetitive practices with instant feedbacks about performance can be provided by educators to increase their perception levels before, during and following practice. Limitations and recommendations Besides the strengths mentioned above, the study has some limitations that need to be addressed in future studies. As a consequence of simultaneous application of a measurement tool that measures a different structure which may be associated to the simulation acceptance perception and SAS to study group, then the correlation between the results to be obtained can be compared in analysing concurrent validity. It may also be necessary to repeat the validity and reliability analyses of the data collected from different segments of the study group. Future studies conducted using the SAS will have significance in terms of their contribution to the measurement capability of the scale. A scale measuring the level of acceptance of simulations among students was developed based on this study, but the educators’ perception of acceptance is also very important in the successful integration of simulations into higher education institutions. Accordingly, future studies may seek to develop measurement tools for the measurement of the level of acceptance among educators. Future research using intermediary variables to determine whether personal differences, such as gender, education level, age, and familiarity with technology have an effect on the acceptance and usage of simulations would be helpful. References Botezatu, M., Hult, H., Tessma, M. K., & Fors, U. (2010). Virtual patient simulation: Knowledge gain or knowledge loss? Medical Teacher, 32(7), 562-568. https://doi.org/10.3109/01421590903514630 Boyle, E. A., Hainey, T., Connolly, T. M., Gray, G., Earp, J., Ott, M., ... & Pereira, J. (2016). An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Computers & Education, 94, 178-192. Retrieved from https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1579556 https://doi.org/10.3109/01421590903514630 https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1579556 Australasian Journal of Educational Technology, 2020, 36(3). 65 Büyüköztürk, Ş. (2010). Sosyal bilimler için veri analizi el kitabı (The data analysis handbook for social sciences). Ankara: Pegem Akademi Yayınları. Cant, R. P., & Cooper S. J. (2010). Simulation-based learning in nurse education: Systematic review. Journal of Advanced Nursing, 66(1), 3–15. https://doi.org/10.1111/j.1365-2648.2009.05240.x Carlsson, C., Carlsson, J., Hyvonen, K., Puhakainen, J., & Walden, P. (2006). Adoption of mobile devices/services searching for answers with the UTAUT. Proceedings of the 39th Annual Hawaii International Conference on System Sciences (Vol. 6, pp. 1–10). https://doi.org/10.1109/HICSS.2006.38 Casey, T., & Wilson-Evered, E. (2012). Predicting uptake of technology innovations in online family dispute resolution services: An application and extension of the UTAUT. Computers in Human Behavior, 28(6), 2034–2045. https://doi.org/10.1016/j.chb.2012.05.022 Cattell, R. B. (1978). The scientific use of factor analysis. New York, NY: Plenum Press. Cobbett, S., & Snelgrove-Clarke, E. (2016). Virtual versus face-to-face clinical simulation in relation to student knowledge, anxiety, and self-confidence in maternal-newborn nursing: A randomized controlled trial. Nurse Education Today, 45, 179-184. https://doi.org/10.1016/j.nedt.2016.08.004 Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2012). Sosyal bilimler için çok değişkenli istatistik: SPSS ve LISREL uygulamaları (Multivariate statistics for the social sciences: SPSS and LISREL applications). Ankara: Pegem Akademi Yayıncılık. Cordar, A., Wendling, A., White, C., Lampotang, S., & Lok, B. (2017, March). Repeat after me: Using mixed reality humans to influence best communication practices. Paper presented at the IEEE Virtual Reality (VR) Conference, Los Angeles, CA. Retrieved from https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7892242 Datta, R., Upadhyay, K. K., & Jaideep, C. N. (2012). Simulation and its role in medical education. Medical Journal of Armed Forces India, 68(2), 167-172. https://doi.org/10.1016/S0377- 1237(12)60040-9 Deladisma, A. M., Cohen, M., Stevens, A., Wagner, P., Lok, B., Bernard, T., … & Lind, S. (2007). Do medical students respond empathetically to a virtual patient? The American Journal of Surgery, 193, 756-760. https://doi.org/10.1016/j.amjsurg.2007.01.021 DeVellis, R.F. (2012). Scale development: Theory and applications (3rd ed.). Thousand Oaks, CA: Sage. Elçin, M. (2010). Tıp eğitiminin tarihçesi. (History of medical education) Hacettepe Tıp Dergisi, 41, 195- 202. Elçin, M. & Odabaşı, O. (2016). Beceri eğitimi. In İ. Sayek, (Ed.), Tıp eğiticisi el kitabı içinde (Medical educator), (179-193). Ankara: Güneş Tıp Kitapevleri. Erkuş, A. (2012). Psikolojide ölçme ve ölçek geliştirme. (Measurement and scale development in psychology). Ankara: Pegem Akademi Yayınları. Escobar-Rodríguez, T., Carvajal-Trujillo, E., & Monge-Lozano, P. (2014). Factors that influence the perceived advantages and relevance of Facebook as a learning tool: An extension of the UTAUT. Australasian Journal of Educational Technology, 30(2), 136–151. https://doi.org/10.14742/ajet.585 Field, A. (2018). Discovering statistics using IMB SPSS Statistics: And sex and drugs and rock’n’roll (5th ed.). London: SAGE Publications Ltd. Fischler, I. S., Kaschub, C. E., Lizdas, D. E., & Lampotang, S. (2008). Understanding of anesthesia machine function is enhanced with a transparent reality simulation. Simulation in Healthcare, 3(1), 26-32. https://doi.org/10.1097/SIH.0b013e31816366d3 Gaba, D. (2004). The future vision of simulation in health care. Quality Safety Health Care, 2(2), 126- 135. Retrieved from https://qualitysafety.bmj.com/content/qhc/13/suppl_1/i2.full.pdf Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., Tatham, R. L., 2006). Multivariate data analysis (6th ed.). Upper Saddle River, NJ: Prentice-Hall Inc. Horn, J. L. (1965). A rationale for the number of factors in factor analysis. Psychometrica, 30, 179– 185. Institute of Medicine (1999). To err is human: Building a safer health system. Retrieved from http://www.nationalacademies.org/hmd/~/media/Files/Report%20Files/1999/To-Err-is- Human/To%20Err%20is%20Human%201999%20%20report%20brief.pdf Issenberg, S., Mcgaghie, W. C., Petrusa, E. R., Lee Gordon, D., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27(1), 10-28. https://doi.org/10.1080/01421590500046924 Johnsen, H. M., Fossum, M., Vivekananda-Schmidt, P., Fruhling, A., & Slettebø, Å. (2018). Nursing students' perceptions of a video-based serious game's educational value: A pilot study. Nurse EducationTtoday, 62, 62-68. https://doi.org/10.1016/j.nedt.2017.12.022 https://doi.org/10.1111/j.1365-2648.2009.05240.x https://doi.org/10.1109/HICSS.2006.38 https://doi.org/10.1016/j.chb.2012.05.022 https://doi.org/10.1016/j.nedt.2016.08.004 https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7892242 https://doi.org/10.1016/S0377-1237(12)60040-9 https://doi.org/10.1016/S0377-1237(12)60040-9 https://doi.org/10.1016/j.amjsurg.2007.01.021 https://doi.org/10.14742/ajet.585 https://doi.org/10.1097/SIH.0b013e31816366d3 https://qualitysafety.bmj.com/content/qhc/13/suppl_1/i2.full.pdf http://www.nationalacademies.org/hmd/%7E/media/Files/Report%20Files/1999/To-Err-is-Human/To%20Err%20is%20Human%201999%20%20report%20brief.pdf http://www.nationalacademies.org/hmd/%7E/media/Files/Report%20Files/1999/To-Err-is-Human/To%20Err%20is%20Human%201999%20%20report%20brief.pdf https://doi.org/10.1080/01421590500046924 https://doi.org/10.1016/j.nedt.2017.12.022 Australasian Journal of Educational Technology, 2020, 36(3). 66 Jong, D. (2009, December). The acceptance and use of the learning management system. Paper presented at the Innovative Computing, Information and Control (ICICIC). 2009 Fourth International Conference IEEE (pp. 34–37), Kaohsiung, Taiwan. https://doi.org/10.1109/ICICIC.2009.347 Kline, R. B. (1994). An easy guide to factor analysis. New York, NY: Routledge. Kneebone, R. (2010). Simulation, safety and surgery. British Medical Journal Quality & Safety, 19(3), 47-52. https://doi.org/10.1136/qshc.2010.042424 Kotranza, A., & Lok, B. (2008, March). Virtual human+ tangible interface = mixed reality human an initial exploration with a virtual breast exam patient. Paper presented at the Virtual Reality Conference, 2008. (pp. 99-106), Reno, Nevada. Kotranza, A., Lok, B., Deladisma, A., Pugh, C. M., & Lind, D. S. (2009). Mixed reality humans: Evaluating behavior, usability, and acceptability. IEEE Transactions on Visualization and Computer Graphics, 15(3), 369-382. https://doi.org/10.1109/TVCG.2008.195 Kunkler, K. (2006). The role of medical simulation: An overview. The International Journal of Medical Robotics and Computer Assisted Surgery, 2, 203-210. https://doi.org/10.1002/rcs.101 Kurenov, S., Cendan, J., Dindar, S., Attwood, K., Hassett, J., Nawotniak, R., ... & Peters, J. (2017). Surgeon-authored virtual laparoscopic adrenalectomy module is judged effective and preferred over traditional teaching tools. Surgical innovation, 24(1), 72-81. https://doi.org/10.1177/1553350616672971 Kurtuluş, C. (2015). A user research for a medical device based on unified theory of acceptance and use of technology (Master’s thesis). Istanbul University, Turkey. Kutlay, A. (2015). Analyses of factors affecting acceptance of homecare technologies by patients with chronic diseases (Master’s thesis). Middle East Technical University, Ankara, Turkey. Lammers, R. L., Davenport, M., Korley, F., Griswold‐Theodorson, S., Fitch, M. T., Narang, A. T., ... & Hamann, C. J. (2008). Teaching and assessing procedural skills using simulation: Metrics and methodology. Academic Emergency Medicine, 15(11), 1079-1087. https://doi.org/10.1111/j.1553- 2712.2008.00233.x Lampotang, S., Lizdas, D., Rajon, D., Luria, I., Gravenstein, N., Bisht, Y., … Robinson, A. (2013). Mixed simulators: Augmented physical simulators with virtual underlays. Proceedings of the IEEE Virtual Reality 2013 Meeting, Orlando, Florida, 7-10. Retrieved from https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6549348 Lin, K. P, Travlos, D. V. P., Wadelin, J. W. P., & Vlasses, P. H. P. (2011). Simulation and introductory pharmacy practice experiences. American Journal of Pharmaceutical Education, 75(10), 1-9. Retrieved from http://eds.b.ebscohost.com/eds/pdfviewer/pdfviewer?vid=0&sid=8609b88a-2554- 4f98-8583-3063bb280959%40pdc-v-sessmgr01 Luctkar-Flude, M., Wilson-Keates, B., & Larocque, M. (2012). Evaluating high-fidelity human simulators and standardized patients in an undergraduate nursing health assessment course. Nurse Education Today, 32(4), 448–452. https://doi.org/10.1016/j.nedt.2011.04.011 MacLean, S., Kelly, M., Geddes, F., & Della, P. (2017). Use of simulated patients to develop communication skills in nursing education: An integrative review. Nurse Education Today, 48, 90-98. https://doi.org/10.1016/j.nedt.2016.09.018 Magsamen-Conrad, K., Upahdyaya, S., Joa, C. Y., & Dowd, J. (2015). Bridging the divide: Using UTAUT to predict multigenerational tablet adoption practices. Computers in Human Behavior, 50, 186–196. https://doi.org/10.1016/j.chb.2015.03.032 Maina, M. K., & Nzuki, D. M. (2015). Adoption determinants of e-learning management system in institutions of higher learning in Kenya: A case of selected universities in Nairobi metropolitan. International Journal of Business and Social Science, 6(2), 233–248. Retrieved from http://business.ku.ac.ke/images/stories/docs/adoption_determinants_of_elearning.pdf Motola, I., Devine, L. A., Chung, H. S., Sullivan, J. E., & Issenberg, S. B. (2013). Simulation in healthcare education: A best evidence practical guide. Association for Medical Education in Europe Guide No. 82. Medical Teacher, 35(10), 1511-1530. https://doi.org/10.3109/0142159X.2013.818632 Nicholas, O. O. S., Azeta, A. A., Chiazor, I. A., & Omoregbe, N. (2017). Predicting the adoption of e- learning management system: A case of selected private universities in Nigeria. Turkish Online Journal of Distance Education, 18(2), 106–121.https://doi.org/10.17718/tojde.306563 Nunnally, J. (1978). Psychometric Theory. New York, NY: McGraw- Hill Nunnally, J. & Bernstein, I. (1994). Psychometric Theory. New York, NY: McGraw-Hill. O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32(3), 396-402. Retrieved from https://link.springer.com/content/pdf/10.3758/BF03200807.pdf https://doi.org/10.1109/ICICIC.2009.347 https://doi.org/10.1136/qshc.2010.042424 https://doi.org/10.1109/TVCG.2008.195 https://doi.org/10.1002/rcs.101 https://doi.org/10.1177/1553350616672971 https://doi.org/10.1111/j.1553-2712.2008.00233.x https://doi.org/10.1111/j.1553-2712.2008.00233.x https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6549348 http://eds.b.ebscohost.com/eds/pdfviewer/pdfviewer?vid=0&sid=8609b88a-2554-4f98-8583-3063bb280959%40pdc-v-sessmgr01 http://eds.b.ebscohost.com/eds/pdfviewer/pdfviewer?vid=0&sid=8609b88a-2554-4f98-8583-3063bb280959%40pdc-v-sessmgr01 https://doi.org/10.1016/j.nedt.2011.04.011 https://doi.org/10.1016/j.nedt.2016.09.018 https://doi.org/10.1016/j.chb.2015.03.032 http://business.ku.ac.ke/images/stories/docs/adoption_determinants_of_elearning.pdf https://doi.org/10.3109/0142159X.2013.818632 https://doi.org/10.17718/tojde.306563 https://link.springer.com/content/pdf/10.3758/BF03200807.pdf Australasian Journal of Educational Technology, 2020, 36(3). 67 Oh, P. J., Jeon, K. D., & Koh, M. S. (2015). The effects of simulation-based learning using standardized patients in nursing students: A meta-analysis. Nurse Education Today, 35(5), 6-15. https://doi.org/10.1016/j.nedt.2015.01.019 Oktal, Ö. (2013). Analysing the factors affecting the user acceptance of information systems from the UTAUT perspective. H.Ü. İktisadi ve İdari Bilimler Fakültesi Dergisi, 31(1), 153–170. https://doi.org/10.17065/huniibf.103660 Pallant, J. (2016). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (6th ed.). London: McGraw-Hill Education. Raman, A., Don, Y., Khalid, R., & Rizuan, M. (2014). Usage of learning management system (Moodle) among postgraduate students: UTAUT model. Asian Social Science 10(14), 186–192. https://doi.org/10.5539/ass.v10n14p186 Rasimah, C. M. Y., Ahmad, A., & Zaman, H. B. (2011). Evaluation of user acceptance of mixed reality technology. Australasian Journal of Educational Technology, 27(8), 1369- 1387. https://doi.org/10.14742/ajet.899 Robb, A., Cordar, A., Lampotang, S., White, C., Wendling, A., & Lok, B. (2015). Teaming up with virtual humans: How other people change our perceptions of and behavior with virtual teammates. IEEE Transactions on Visualization and Computer Graphics, 21(4), 511-519. Retrieved from https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7014272 Robb, A., White, C., Cordar, A., Wendling, A., Lampotang, S., & Lok, B. (2015). A comparison of speaking up behavior during conflict with real and virtual humans. Computers in Human Behavior, 52, 12-21. https://doi.org/10.1016/j.chb.2015.05.043 Rosen, K. R. (2008). The history of medical simulation. Journal of Critical Care, 23(2), 157-166. https://doi.org/10.1016/j.jcrc.2007.12.004 Sedana, I. G. N., & Wijaya, S. W. (2010). UTAUT model for understanding learning management system. Internetworking Internet Journal, 2(2), 27–32. Sezer, B., & Yilmaz, R. (2019). Learning management system acceptance scale (LMSAS): A validity and reliability study. Australasian Journal of Educational Technology, 35(3), 15-30. https://doi.org/10.14742/ajet.3959 Stevens, J. P. (2009). Applied multivariate statistics for the social sciences. New York, NY: Taylor and Francis. Stroup, C. (2014). Simulation usage in nursing fundamentals: Integrative literature review. Clinical Simulation in Nursing, 10(3), 155-164. https://doi.org/10.1016/j.ecns.2013.10.004 Tabachnick, B. G., & Fidell, L. S. (2014). Using multivariate statistics (6th ed.). Essex: Pearson Education Limited. Tezbaşaran, A. (1997). Likert tipi ölçek hazırlama kılavuzu (Likert type scale preparation guide). Ankara: Türk Psikologlar Derneği. Thomas, T. D., Singh, L., & Gaffar, K. (2013). The utility of the UTAUT model in explaining mobile learning adoption in higher education in Guyana. International Journal of Education and Development using Information and Communication Technology, 9(3), 71–85. Retrieved from https://www.learntechlib.org/p/130274/article_130274.pdf Turkish Medical Association (2010). Hasta Güvenliği: Türkiye ve Dünya (Patient safety: Turkey and the World). Ankara: Türk Tabipleri Birliği Yayınları. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425–478. Retrieved from https://www.jstor.org/stable/pdf/30036540 Watkins, M. W. (2000). Monte Carlo PCA for parallel analysis. State College, PA: Ed & Psych Associates. Yavuz, G., & Doğan, N. (2015). Boyut sayısı belirlemede Velicer’in map testi ve Horn’un paralel analizinin kullanılması (Using Velicer’s map test and Horn’s parallel analysis for determining component number). Hacettepe University Journal of Education, 30(3), 176-188. Retrieved from http://www.efdergi.hacettepe.edu.tr/yonetim/icerik/makaleler/674-published.pdf Zhou, T., Lu, Y., & Wang, B. (2010). Integrating TTF and UTAUT to explain mobile banking user adoption. Computers in Human Behavior, 26(4), 760–767. https://doi.org/10.1016/j.chb.2010.01.013 Zhu, E., Hadadgar, A., Masiello, I., & Zary, N. (2014). Augmented reality in healthcare education: An integrative review. PeerJ, 2, 1-17. https://doi.org/10.7717/peerj.469 https://doi.org/10.1016/j.nedt.2015.01.019 https://doi.org/10.17065/huniibf.103660 https://doi.org/10.5539/ass.v10n14p186 https://doi.org/10.14742/ajet.899 https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7014272%20 https://doi.org/10.1016/j.chb.2015.05.043 https://doi.org/10.1016/j.jcrc.2007.12.004 https://doi.org/10.14742/ajet.3959 https://doi.org/10.1016/j.ecns.2013.10.004 https://www.learntechlib.org/p/130274/article_130274.pdf https://www.jstor.org/stable/pdf/30036540 http://www.efdergi.hacettepe.edu.tr/yonetim/icerik/makaleler/674-published.pdf https://doi.org/10.1016/j.chb.2010.01.013 https://doi.org/10.7717/peerj.469 Australasian Journal of Educational Technology, 2020, 36(3). 68 Corresponding author: Baris Sezer, barissezer13@hotmail.com Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC- ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY- NC-ND 4.0. Please cite as: Sezer, B., Teker, G. T., Sezer, T. A., & Elcin, M. (2020). Simulation acceptance scale (SAS): A validity and reliability study. Australasian Journal of Educational Technology, 36(3), 56-68. https://doi.org/10.14742/ajet.4950 https://creativecommons.org/licenses/by-nc-nd/4.0/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.4950