Australasian Journal of Educational Technology, 2018, 34(5). 13 Making it real: The development of a web-based simulated learning resource for occupational therapy students Kelli Nicola-Richmond and Valerie Watchorn Deakin University Equipping students with graduate attributes that ensure they are ready to work in complex health environments is highly important. Recently there has been increased use of simulation in health education to assist in the acquisition of graduate attributes, and to provide meaningful and authentic learning opportunities. The Deakin University Occupational Therapy program developed a unique web-based Occupational Therapy Simulations for Learning resource. This resource comprised 18 filmed scenarios that were previously non- existent or only available to students in written format. The scenarios depict people with a range of health conditions, mock graduate interviews, and occupational therapists conducting simulated clinical assessments and interventions. This study explored the effectiveness of the resource in the development of empathy, active listening, and information gathering skills. It also compared experiences of students and academics who had previously used written case studies. Survey results demonstrated statistically significant differences in relation to the realness of case studies, understanding how the client was feeling, development of empathy, clinical assessment skills, and the usefulness of the resource overall. This study demonstrates that the use of simulated case studies offers authentic learning for occupational therapy students and promotes development of key graduate attributes. Introduction The provision of authentic learning experiences for students is a challenge that requires academics to be both innovative and resourceful. An authentic learning activity is defined as: [T]hat which reflects, for the students, a combination of personal meaning and purposefulness within an appropriate social and disciplinary framework. The learning experience is authentic for the learner while simultaneously being authentic to a community of practice. (Stein, Isaacs, & Andrews, 2004, p. 241) These learning activities provide real-world opportunities whilst protecting students from harmful or irrelevant situations (Sutherland & Markauskaite, 2012). Learning experiences require several features to prove authentic. They should be somewhat ill-defined so that they mimic real-world situations and allow students to develop their own solutions (Herrington, Oliver, & Reeves, 2003); they should mimic some of the routines and practices of the profession so that students begin to acquire skills and values associated with real-world practice (Sutherland, Scanlon, & Sperring, 2005; Rule, 2006); they should involve the use of thinking and enquiry skills by students and must include discourse undertaken by a community of students (Rule, 2006); students must perceive the task as authentic (Sutherland & Markauskaite, 2012); and experiences must assist students to link the knowledge and skills taught in their lectures and tutorials to real-world practice (Sutherland & Markauskaite, 2012). Authentic learning experiences are considered an important part of health education as they are thought to prepare students for the complex environments that they will face in clinical practice (Gibbs & Dietrich, 2017), and to encourage experiential learning (learning through doing) which is motivating to students (Yardley, Teunissen, & Dornan, 2012) and valued by academics (Lombardi & Oblinger, 2007). Experiential learning has been shown to be effective in the development of behaviours and skills required for clinical practice (Knecht-Sabres & Kovic, 2013). Authentic learning experiences in health professional education commonly include problem-based or project-based learning, clinical placement, and simulated learning. Simulated learning One method of providing authentic learning for students is via the use of simulation. Simulated learning activities have been defined as “an educational technique that allows interactive, and at times immersive, Australasian Journal of Educational Technology, 2018, 34(5). 14 activity by recreating all or part of a clinical experience without exposing patients to associated risks” (Maran & Glavin, 2003, p. 22). Simulated learning activities can be delivered in a range of formats. They may involve role-play where students or academics act as patients, make use of simulated or standardised patients (using actors to role-play a patient), or use computerised or non-computerised mannequins. Immersive virtual learning environments, virtual patients, and avatars may also be used (Butina, Brooks, Dominguez, & Mahon, 2013). Simulated learning may also make use of written and video-recorded case studies, and interactive computer based programs (Bradley, Whittington, & Mottram, 2013; King et al., 2016). These processes may be used in combination, and the scenario can range from a simple discrete task to a highly complex and changeable environment (Bradley, et al., 2013). There are many exemplars of simulated learning activities in the literature. However, as King et al. (2016) have described, the approach taken in health-related simulation development has been largely atheoretical and lacking an evidence based approach. This study sought to contribute to this area of knowledge. Advantages of simulated learning activities for students include an opportunity for guaranteed exposure of all students to a set range of skills that may be considered core to a profession, practice of skills within low risk environments, and opportunity for repeated practice (Rodger, Bennett, Fitzgerald, & Neads, 2010). There is increasing use of simulated learning activities within health care education (Bradley, 2006) and this type of learning experience has been shown to be popular with students. Several studies have demonstrated that following participation in simulated learning activities, students report enhanced perceptions of their knowledge, motivation, and skill set (Rodger, et al., 2010). There is however, limited research relating to the use of filmed case studies or web-based simulations for health professional students, and whether these provide authentic, effective learning experiences remains unclear. Pulman, Scammell, and Martin (2008) evaluated a simulated web-based community learning resource called Wessex Bay that aimed to enhance learning of nursing students. The simulation consisted of a map on which students could select a house containing character profiles for each resident. Teaching staff felt that the resource was helpful as a one-stop case study environment with which students could interact but felt that it was uni-dimensional and not sufficiently interactive. The researchers concluded that use of this technology enhanced learning but that the design needed to be further developed. The value of simulation as a learning tool for qualified clinicians has also been explored. King et al. (2016) developed a series of six video simulations that focussed on the development of clinical listening skills in clinicians working in paediatric rehabilitation. The authors adopted an evidence-based approach to the development of simulations and included input from a wide range of stakeholders to establish content validity and authenticity. An evaluation was also undertaken to establish the complexity of each simulation so that the series of simulated learning activities could be presented with a gradual increase in skills required. Whilst the literature described here provides some evidence of the value of filmed case-studies and web-based simulation resources in the health professions, there remains limited literature that explores whether these provide authentic learning experiences and whether web-based simulation resources facilitate greater learning than traditional paper-based resources. Simulated learning for students of occupational therapy Simulated learning experiences using a range of modalities have been adopted in occupational therapy education (Bennett, Rodger, Fitzgerald, & Gibson, 2017). Written case studies have commonly been used but have been criticised for being uni-dimensional and lacking the richness of authentic real-life experiences (Bennett et al., 2017). There are also published instances of videos of real or simulated scenarios being used in occupational therapy education. These have either been presented in the format of DVD material (Brown & Williams, 2009) or have focused on isolated areas of skill development such as fabricating a hand orthosis (Hagemann, Williams, McKee, Stefanovich, & Carnahan, 2014). There is no published literature exploring the use of an interactive online repository specifically designed for use by occupational therapy students or the effectiveness and usefulness of such a resource. The Occupational Therapy Simulations for Learning resource In 2013, the Deakin University Occupational Therapy program received Health Workforce Australia funding to develop an online simulated learning resource for students. The project, led by the authors, involved the development of 18 filmed scenarios and related content. Production methodology aimed to Australasian Journal of Educational Technology, 2018, 34(5). 15 create scenes that were as realistic as possible. Filming took place in simulated environments, such as hospital wards, that allowed natural movement and activity, or in a green-screen recording studio. Lighting was a mix of available and artificial light. Sound was recorded with shotgun or wired neck microphones. Footage was recorded using digital video and DSLR cameras, sometimes using multiple cameras from different angles that were matched and edited accordingly. Actors were given clear written character descriptions and were briefed on health conditions by health professionals. Depending on the role and background of the actor, some scenes were scripted by academics while others were unscripted allowing actors to improvise for natural dialogue. Post-production employed the use of Adobe Premiere Pro Version 7.1 software. Each scene was reviewed and edited by the production team, which included academics. A bespoke web-based platform was designed to house the scenarios. Hosted within the university’s domain, the mobile friendly website design was built using HTML 5, Javascript, and visually defined per CSS 3 specifications (W3Schools, 2017). Accessibility of design was considered and Web Content Accessibility Guidelines (WCAG) 2.0 AA accessibility standards (World Wide Web Consortium [W3C], 2017) were adhered to. At the time of data collection, the Occupational Therapy Simulations for Learning resource included eight case studies that had been redeveloped from paper-based format to filmed format. These case studies depicted eight individuals experiencing a range of physical and mental health conditions, such as cerebral palsy, multiple sclerosis, depression, and spinal cord injury. Some scenarios included family members and/or an occupational therapist. Additionally, the resource included 10 newly developed video-based simulation resources that reflected a range of assessments and interventions undertaken by occupational therapists in practice. These included three scenarios of an occupational therapist working with a person experiencing mental illness, two graduate occupational therapy workplace interviews with panel discussions, one home visit assessment, one workplace assessment, one upper limb assessment, a paediatric occupational therapy session, and a shower assessment. All video clips were supported by supplementary material, such as referral forms and handover notes in the form of pdf documents and audio recordings of phone calls with family members. This supplementary material was accessible to all users and offered students additional information to that contained within the video recordings. Occupational Therapy Simulations for Learning was used as a teaching and learning resource in a range of units across all 4 years of the undergraduate occupational therapy program in 2014. This study aimed to evaluate its effectiveness. More specifically this study sought to: • compare the experiences of students and academics who have previously used paper-based case studies to those who have used the new simulation resource; • explore the accessibility of the resources for students and academics; and • explore student and academic experiences of using the resource. It was hypothesised that by viewing the filmed scenarios of simulated occupational therapy clients (rather than reading paper-based case studies) students would be challenged to use active listening and information gathering skills that are fundamental to being an effective health practitioner. It was also expected that viewing and listening to the stories of real people rather than reading paper-based descriptions of people’s lives would enhance the development of empathy in students. In its systematic application and mapping of graduate learning outcomes throughout all courses, Deakin University emphasises the need for its graduates to demonstrate both strong knowledge of their discipline of study and the achievement of learning outcomes relevant to all graduates in a competitive employment market. These graduate learning outcomes are: communication; digital literacy; critical thinking; problem- solving; self-management; teamwork; and global citizenship (Deakin University, 2017). Information gathering, active listening and empathy are important graduate attributes that occupational therapy students must acquire to be work-ready on graduation. Scanlon (2006) has defined graduate attributes as "the generic skills, knowledge, dispositions and attitudes undergraduates develop during their university studies" (p. 125). As an integral communication skill active listening has been defined as involving three key elements: (1) the listener indicates their involvement via providing unconditional attention; (2) the listener para- phrases both the content and the feelings that they are hearing the speaker express to demonstrate their understanding; and (3) the listener asks questions to seek additional information from the speaker (Thistle & McNaughton, 2015). Active listening is considered a vital component of developing person-centred and collaborative relationships with clients (Bruder & Dunst, 2005). It has also been associated with increased Australasian Journal of Educational Technology, 2018, 34(5). 16 patient satisfaction (Wanzer, Booth-Butterfield, & Gruber, 2004). Information gathering is considered a core occupational therapy graduate attribute as indicated by its inclusion as one of the eight major areas of competence that is assessed during occupational therapy clinical placement (Allison & Turpin, 2004). As described in the Occupational Therapy Australia (2010a) Australian Minimum Competency Standards for New Graduate Occupational Therapists, information gathering involves completion of comprehensive and relevant assessments of occupational performance that are: client-centred; collaborative; and consider the client’s context, environment, and personal characteristics, and the interpretation of this information (Occupational Therapy Australia, 2010b). Empathy has also been shown to be a pivotal characteristic of effective health professionals (Williams, Brown, Boyle, & Dousek, 2013). Empathy consists of two distinct aspects, an ability to understand the world from another's perspective (cognitive empathy) and the ability to feel a connection with another person’s feelings or experience (Davis, 1994). It has been suggested that the development of empathy in health students is complex and may require exposure beyond clinical experiences that are usually included in undergraduate programs (Sheffler, 1995). Whilst empathy, active listening, and information gathering are considered important in healthcare, research describing the mechanisms used to teach them is limited. The literature suggests that empathy, active listening, and information gathering are typically taught theoretically during subjects that address professional behaviour (Williams, Boyle, & Howard, 2016) and clinical skills (Brown & Woods, 2011). Despite the presence of some evidence that teaching empathy in conjunction with role play and simulation is effective (Williams et al., 2016) there is limited report of simulation being used to teach these skills. It is likely that academics also expect students to be exposed to and learn about these skills during clinical placement. Given the value and importance of these three graduate attributes and the potential complexity in assisting students to acquire them, the evaluation undertaken in this study sought to explore these concepts as well as addressing the research questions outlined above. Method This quantitative research study sought to measure the effectiveness of the simulation resource from the perspective of academics and students. A cross sectional research design was employed to gather data from the populations of interest via a survey method. Cross sectional studies are used to determine prevalence of a variable of interest at one point of time (Mann, 2003). This methodology is easy and cost-effective to administer and serves to provide descriptive information on one or more variables of interest and any associations that may exist between variables (Cottrell & McKenzie, 2011). The study was approved by the Deakin University Human Research Ethics Committee. Participants All 308 (1st year n = 95, 2nd year n = 86, 3rd year n = 64 and 4th year n = 63) students enrolled in the Deakin University occupational therapy program were invited to participate in this study. More than one third (n = 109) of students elected to participate in 2014. Four out of six invited academic participants who used the web-based resource in their teaching also took part in the study either in 2014 or 2015. More than half (57%) of the 109 student participants were 4th year undergraduate occupational therapy students who had used the pre-existing paper-based case studies in two 2013 3rd year subjects. The remaining 52 students were using the Occupational Therapy Simulations for Learning resource in the same two 3rd year units in 2014 (n = 19), or in a 1st year unit (n = 15), a 2nd year unit (n = 10) or in a 4th year unit (n = 8). Procedure To gather quantitative data, all participants were invited to complete a survey that included Likert scale and open-ended questions. Participants were invited to take part via email and face to face introduction. Deakin University operates a trimester system whereby the academic year consists of three equal periods of time. At the beginning of 2014, in trimester 1, students who were in their 4th year were invited to participate in the study during a class. Those who agreed (n = 57) were asked a series of questions about the use of paper- based case studies during their 3rd year of occupational therapy, using a hard copy version of the survey. The survey was disseminated in hard copy so that students could be simultaneously provided with a copy of the case study that they had used the year before to trigger their memory of it. Australasian Journal of Educational Technology, 2018, 34(5). 17 After the Occupational Therapy Simulations for Learning resource had been used in academic units, the study was introduced during classes to all 1st, 2nd, and 3rd year students in trimester 2 of 2014. Fourth year students who used the web-based resource were also invited to respond, this time in relation to the filmed scenarios. A message about the study was also posted on the online learning management system for each year level. All 1st, 2nd, 3rd, and 4th year students were sent an email that provided a link to the online version of the survey and a plain language statement. Two reminder emails were sent to each participant group and lecturers were asked to remind students about the study during class time. Instruments Surveys using between 9 and 11 Likert scale questions (1 = strongly agree, 2 = agree, 3= neither agree or disagree, 4= disagree 5 = strongly disagree) and 2 to 3 open ended questions were developed for each participant group. Electronic surveys were designed using SurveyMonkey® software. Online surveys offer the advantage to participants of the flexibility to participate at their convenience (Evans & Mathur, 2005). They also offer researchers a controlled way to target a broad audience and the flexibility to use a range of question formats to best yield data for the purposes of the study (Evans & Mathur, 2005). The surveys were piloted by both authors and a third experienced researcher. The survey was then edited to improve clarity of questions and to ensure that each question only explored a single element regarding use of the resource. Tables 1 and 2 present the questions used for the student and academic participant groups. The questions sought to explore whether the resource was useful, accessible, authentic, and meaningful. Questions also addressed the development of empathy, assessment and information gathering skills. Table 1 Survey questions for students Likert scale questions Open-ended questions It felt like the people depicted in the OT Simulations for Learning website were real people. What are your overall perceptions of the usefulness or otherwise of the OT Simulations for Learning website in regard to your learning? Do you have any other information you would like to give us in regard to the OT Simulations for Learning website? It felt like the situations /scenarios depicted in the OT Simulations for Learning website were real. It felt like I could understand how the person/people depicted in the OT Simulations in Learning website were feeling. I developed empathy for the person/people depicted in the OT Simulations for Learning website. The videos depicted in the OT Simulations for Learning website provided me with all of the information I needed to undertake the learning / assessment tasks in my unit. It felt like using the videos depicted in the OT Simulations for Learning website allowed me to develop my OT assessment and information gathering skills. The OT Simulations for Learning website is easy to navigate. The OT Simulations for Learning website looks appealing. The OT Simulations for Learning website is a useful resource for students. OT = Occupational Therapy Australasian Journal of Educational Technology, 2018, 34(5). 18 Table 2 Survey questions for academics Likert scale questions Open-ended questions The OT Simulations for Learning website is easy to navigate. If you have previously used paper-based versions of the website content how would you compare these to the new website content? What are your overall perceptions of the usefulness or otherwise of the OT Simulations for Learning resource in regard to the teaching and learning of OT students? Do you have any other information you would like to give us in regard to the OT Simulations in Learning resource? The OT Simulations for Learning website looks appealing to students. The OT Simulations for Learning website looks appealing to me. The videos used on the OT Simulations for Learning website are easy to understand. The written resources used on the OT Simulations for Learning website are easy to understand. The resources contained in the OT Simulations for Learning website assist students to develop empathy for the people depicted. The resources contained in the OT Simulations for Learning website provide students with authentic learning experiences. The resources contained in the OT Simulations for Learning website require students to engage in deeper learning when compared to written case studies. The resources contained in the OT Simulations for Learning website assist students to develop occupational therapy assessment and information gathering skills. The OT Simulations for Learning website provides students with the information needed to undertake related assessment tasks for the unit I teach. Students responded positively to the OT Simulations for Learning website content. Data analysis Quantitative data were analysed using descriptive statistics with mean, median, and mode calculated. Data were compared between those students that had used the paper-based case studies (57) in 2014 and 3rd year students (19) in 2014 who had used the Occupational Therapy Simulations for Learning resource using Mann-Whitney U tests and T tests Wilcoxon’s W tests in SPSS Version 22. Data comparing the paper- based case studies and all other student responses relating to the filmed scenarios were also compared so that an overall understanding of the differences between the two forms of case study could be explored. Responses to the open-ended questions were reviewed by the two authors and were used to augment and add richness to the quantitative findings. Results Participant responses were primarily very positive for use of both the web-based and the paper-based case studies. High numbers of participants from across all year levels strongly agreed or agreed that the Occupational Therapy Simulations for Learning resource was a valuable tool. Table 3, demonstrates that all of the survey items scored positively for the web-based case studies with mean responses of less than 2, Australasian Journal of Educational Technology, 2018, 34(5). 19 aside from the question, “The videos depicted in the OT Simulations in Learning website provided me with all of the information I needed to undertake the learning/assessment tasks in my unit”, which had a mean of 2.43. Responses from 4th year students regarding the paper-based case studies were also consistently positive with means all falling below 2.75 on the 5-point scale (Table 3). Table 3 Participants’ perceptions of written case studies and web-based case studies Statement Source N M SD It felt like the people depicted were real people. written case study 57 2.1228 .78080 website case study 52 1.8654 .74172 It felt like the situations/scenarios depicted were real. written case study 49 2.1224 .80707 website case study 52 1.6538 .68269 It felt like I could understand how the person/people depicted were feeling. written case study 57 2.4035 .77597 website case study 51 1.9020 .70014 I developed empathy for the person/people depicted. written case study 57 2.5789 .77799 website case study 51 1.7843 .67272 The videos/case study provided me with all of the information I needed to undertake the learning/assessment tasks in my unit. written case study 57 2.2281 .82413 website case study 51 2.4314 .85452 It felt like using the videos/case study allowed me to develop my OT assessment and information gathering skills. written case study 57 2.0526 .54841 website case study 51 1.8039 .66392 The website/case study is easy to navigate. written case study 57 1.8421 .41359 website case study 51 1.8039 .87223 The website/case study looks appealing. written case study 56 2.7321 .72591 website case study 51 1.9216 .86817 The website/case studies is a useful resource for students. written case study 57 1.9474 .69233 website case study 50 1.5000 .58029 When the responses of students who had used the paper-based case studies were compared with data of all students who had used the web-based case studies there were statistically significant differences (p = .005) evident for the questions, “It felt like the situations/scenarios depicted were real”; “It felt like I could understand how the person/people depicted were feeling”; “I developed empathy for the person/people depicted”; “It allowed me to develop my OT assessment and information gathering skills”; “It looks appealing”; and, “It is a useful resource for students”. These results (Table 4), show that students more strongly agreed with the statements in respect to the filmed case studies in comparison with the paper-based case studies. Table 4 Comparisons of students who had used written case studies with students who had used web-based case studies Statement Mann- Whitney U Wilcoxon W Z p value It felt like the people depicted were real people. 1215.500 2593.500 -1.818 .069 It felt like the situations / scenarios depicted were real. 854.000 2232.000 -3.191 .001 It felt like I could understand how the person/people depicted were feeling. 977.000 2303.000 -3.316 .001 I developed empathy for the person/people depicted. 695.500 2021.500 -4.979 .000 I was provided with all of the information I needed to undertake the learning / assessment tasks in my unit. 1213.500 2866.500 -1.651 .099 It allowed me to develop my OT assessment and information gathering skills. 1161.000 2487.000 -2.127 .033 It was easy to navigate. 1280.000 2606.000 -1.237 .216 It looks appealing. 664.500 1990.500 -5.064 .000 It is a useful resource for students. 919.000 2194.000 -3.569 .000 Australasian Journal of Educational Technology, 2018, 34(5). 20 When the responses from students who used the simulated learning web-based case studies for the 3rd year units of study in 2014 (n = 19) were compared to those of students who had used the paper-based case studies in those same units in 2013 (n = 57) it was again evident for all statements that students more strongly agreed with the statements in relation to the web-based case studies. Other than the statement, “The case study provided me with all of the information I needed to undertake the related learning/assessment tasks in my unit”, all of the mean responses were below 1.7 on the 5-point scale. This finding demonstrates that students using the same case studies in the same units but with two different modes of delivery (paper-based and web-based) found the web-based case studies to be more effective on a number of levels. The means and standard deviations for this comparison are described in Table 5. For eight of the nine questions statistically significant differences were evident, as described in Table 6. The simulated case studies appear to have felt more real for students and engaged them in such a way as to develop a greater sense of empathy and feeling for the client. Students also felt that it enabled them to develop their occupational therapy assessment and information gathering skills to a greater degree. Table 5 Mean scores of students comparing written and website case studies Statement Source N M SD It felt like the person in my case study was a real person. written case study 57 2.1228 .78080 website case study 19 1.5789 .60698 It felt like the situations/scenarios depicted were real. written case study 49 2.1224 .80707 website case study 19 1.4211 .50726 It felt like I could understand how the person in my case study was feeling. written case study 57 2.4035 .77597 website case study 19 1.6842 .58239 I developed empathy for the person in my case study. written case study 57 2.5789 .77799 website case study 19 1.6842 .67104 The case study provided me with all of the information I needed to undertake the related learning/assessment tasks in my unit. written case study 57 2.2281 .82413 website case study 19 2.2105 .71328 It felt like the case study allowed me to develop my OT assessment and information gathering skills. written case study 57 2.0526 .54841 website case study 19 1.6842 .58239 The information contained within the case studies was easy to locate and navigate. written case study 57 1.8421 .41359 website case study 19 1.4737 .77233 The case studies looked appealing. written case study 56 2.7321 .72591 website case study 19 1.5263 .61178 The case studies are a useful resource for students. written case study 57 1.9474 .69233 website case study 19 1.4211 .60698 Table 6 Comparison of results from students who used paper-based case studies with those who used web-based case studies Mann- Whitney U Wilcoxon W Z p value It felt like the person in my case study was a real person. 336.500 526.500 -2.748 .006 It felt like the situations/scenarios depicted were real. 232.000 422.000 -3.594 .000 It felt like I could understand how the person in my case study was feeling. 280.000 470.000 -3.574 .000 I developed empathy for the person in my case study. 228.000 418.000 -4.002 .000 The case study provided me with all of the information I needed to undertake the related learning / assessment tasks in my unit. 525.500 2178.500 -.230 .818 It felt like the case study allowed me to develop my OT assessment and information gathering skills. 376.000 566.000 -2.491 .013 The information contained within the case studies was easy to locate and navigate. 315.000 505.000 -3.356 .001 The case studies looked appealing. 128.500 318.500 -5.238 .000 The case studies are a useful resource for students. 311.000 501.000 -3.143 .002 Australasian Journal of Educational Technology, 2018, 34(5). 21 Although the number of academics using the Occupational Therapy Simulations for Learning web resource, and consequently the number of responses, was small (n = 4), responses to the survey were overwhelmingly positive. All respondents strongly agreed or agreed that the web-based resource was easy to navigate and understand, looked appealing, assisted in the development of empathy for students and was authentic. All participants strongly agreed or agreed that the resource helped to develop students’ assessment and information gathering skills and encouraged deeper learning when compared to paper-based case studies. Means and standard deviations for academic responses are listed in Table 7. Table 7 Responses from academics in relation to their use of the Occupational Therapy Simulations for Learning resource Statement N M SD The OT Simulations in Learning website is easy to navigate. 4 1.5 .57735 The OT Simulations in Learning website looks appealing to students. 4 1 0 The OT Simulations in Learning website looks appealing to me. 4 1.25 .5 The videos used on the OT Simulations in Learning website are easy to understand. 4 1.25 .5 The written resources used on the OT Simulations in Learning website are easy to understand. 4 1.25 .5 The resources contained in the OT Simulations in Learning website assist students to develop empathy for the people depicted. 4 1.75 .5 The resources contained in the OT Simulations in Learning website provide students with authentic learning experiences. 4 1.5 .57735 The resources contained in the OT Simulations in Learning website require students to engage in deeper learning when compared to written case studies. 4 1 0 The resources contained in the OT Simulations in Learning website assist students to develop occupational therapy assessment and information gathering skills. 4 1.25 .5 The OT Simulations in Learning website provides students with the information needed to undertake related assessment tasks for the unit I teach. 4 1.5 .57735 Students responded positively to the OT Simulations in Learning website content. 4 2.25 .5 In summary, the use of paper-based case studies as a learning tool was positively viewed by students. However, findings demonstrate that the web-based case studies provided participants with experiences that were more real, evoked greater empathy and allowed participants to develop greater self-perceived information gathering and assessment skills. The web resource was also considered easy to navigate and a useful resource. When the responses of students to the open-ended questions (Table 1) were reviewed, overwhelmingly students reported that using case studies, in both paper-based and filmed case-study formats, to practice their occupational therapy skills was beneficial. For instance, one participant who used the web resource reported: “This has been a great resource to have. It has provide[d] a variety of ‘real life’ examples to use in learning and analysis as we would see out in the workforce”, while another that had used the paper-based scenarios stated: “I think they were very useful, they are practical and interesting to work with”. Those students who used the paper-based case studies found that not being able to see the client meant that it was difficult to imagine the person, their physical traits and assess areas such as the person’s movement, stating: “I chose to look at the individual as a real person although at times it was difficult as we just had written word to conjure an image of the person”, and “Although the case study related to areas around here and painted a strong picture, without seeing the person they did not feel real.” Australasian Journal of Educational Technology, 2018, 34(5). 22 One of the potential values of the web-based resource, as perceived by the researchers, is that students are required to use information gathering skills to sift through verbal and written information to draw out the salient points, as is required of graduate occupational therapists, rather than the key information being concisely documented for them. However, the increased challenge of this concerned a small number of students. For instance one stated: I think the videos are a great asset but feel that they should be used in conjunction with the written to ensure all the information is provided and students can distinguish between what something looks like on paper as well as in person. Students who had used the paper-based case studies generally felt that the case studies provided relevant information, although only some students found the paper-based case studies sufficient to develop empathy for the client. The majority found that they could not develop as much empathy using the paper-based case studies when compared to the web resource. This is supported by the quantitative data, which found that there was a statistically significant difference in perceived empathy development in students who had used the filmed case studies, with students commenting: “Much easier [to develop empathy] when you can picture the person, their situation, and see their emotions than just reading a case study” and “Researching the condition and spending a lot of time on the assignments, and having a real person's face to put as the reason behind my research certainly put a lot more meaning to the assignments”. With regard to empathy, for the paper-based case studies, one student reported “I felt it was harder to develop empathy as without seeing the person I found it harder to see the case study as a real person”. Although findings were very positive overall, students did provide evidence that nothing can take the place of real clinical experiences throughout the responses: “There were a lot more questions I [would] have liked to ask the client” and “The client was good, I found the OT in the video for Wendy to be a little hard to listen to- you could really tell she was acting. It was still much more real than any other assignments I've done”. A small number of students provided feedback about how they felt the web resource could be improved, stating: “More information that simulates a real life situation would have been better e.g., wheelchair measurements, formal assessment results”. Several students also wanted additional information. For instance, “I feel like the information was too brief and that the phone conversation wasn't enough to get a grasp on Wendy's full occupational performance capabilities”. Discussion This study provides new findings in relation to the use of web-based simulation for students studying in health disciplines. Findings demonstrate that the use of web-based simulations develops statistically significant higher levels of self-perceived empathy for clients than the use of paper-based case studies. The development of empathy in health professionals is important in the delivery of effective health services that provide superior patient outcomes as has been described by Williams et al. (2013) and Hojat et al. (2011). Students also reported statistically significant results relating to the realness of the clients depicted. This directly addresses the authenticity of the filmed case studies in the web-based resource. As supported by Brown & Williams (2009) and findings from this study, simulated learning activities, such as those depicted in the Occupational Therapy Simulations for Learning resource, can never fully take the place of placement experiences, and working with a real person. However, improving the authenticity of learning activities within the classroom environment may assist in improving student preparation for placement and for practice. Prior to the development of the Occupational Therapy Simulations for Learning resource, one of the anticipated advantages perceived by academics was that students would have to locate and extract relevant information rather than having all relevant information documented clearly as in paper-based case studies. This was potentially a way to further challenge and enhance students' information gathering skills. However, it caused discomfort for students with some requesting greater written information. As is evident in the results, responses were consistently less positive to the question relating to students having all of the Australasian Journal of Educational Technology, 2018, 34(5). 23 information that they required. Whilst providing more information may have made assessment tasks associated with the filmed case studies easier for students, the deliberate choice was made not to include greater levels of written information. Despite wanting more information, students did demonstrate statistically significant improvement in their information gathering and assessment skills when using the web-based resource as compared to paper-based case studies. Gathering information and assessment of clients are skills that are fundamental to the health disciplines. In the authors' experience, it is not uncommon for final year students to still be struggling to attain these skills, and as described previously this is one of the eight key domains assessed during clinical placement. Thus activities and resources that promote these skills are of value in undergraduate programs to support the development of graduates who are ready for practice. Feedback about the way the web-based resource looked and its ease of use was almost universally positive. Both students and academics reported positively in regard to the fact that the resource was easy to navigate, appealing to look at, and overall that it was a useful resource. This supported previous findings of Brown and Williams (2009), and Williams, Brown, Scholes, French, and Archer (2010) who found video-based descriptions of clients were perceived as useful learning tools. The web-based resource offers some key features that were perceived as advantageous by students. Several students reported that the ability to go back to the filmed clip on repeated occasions was an advantage as they could extract information from multiple views over a period of time. This is an advantage that web- based resources have over, the often more commonly used, role play or simulated patients where students may only have one exposure to the information. Whilst the opportunity to view a client on repeated occasions during clinical practice may not always be possible, the authors consider this repeated opportunity to view the web-based footage as beneficial for students as they are still developing skills in areas such as information gathering. Additionally, although initial costs of the development of web-based resources may be higher, this type of simulation resource may be more cost effective in the long term than the on-going costs associated with using actors or simulated patients. The Occupational Therapy Simulations for Learning resource offers a flexible digital medium through which students can engage with simulated learning material. In line with rapid advancement of educational technologies it provides opportunity for occupational therapy learning resources to be delivered in an innovative manner that better meets the technological needs and expectations of students, academics, and universities (Russell, Malfroy, Gosper & McKenzie, 2014). Additionally, the use of an open web-based platform allows for revision and updating of simulated scenarios and the addition of new material over time. The simplicity of its design does not require specialised skill development of academics or students in order to use the resource but a team of skilled technicians are required to provide maintenance and updates as needed. Research Limitations Whilst this study has provided useful findings relating to the use of web-based simulations in health education, there are several limitations. Surveys were administered using self-assessment measures so improvements that may have occurred as a result of using the web resource were a measure of perceptions of students and academics rather than a measure of actual skill development. The surveys were also designed specifically for this study so their reliability and validity has not previously been demonstrated. However this was necessary as no validated survey was available that related to the research questions for the study and steps were taken to trial and refine the survey prior to use. Future research in this area could explore the use of measures, such as The Jefferson Scale of Physician Empathy which has previously been adapted for use with paramedic students (Williams et al., 2013) so that empathy development could be explored in greater depth. Studies that measure whether the use of the web- based simulation in learning activities improves preparation for placement or readiness for the work place would also be of significant value. Additionally, there is scope for investigating whether this tool is effective in the professional development of qualified occupational therapy clinicians. Australasian Journal of Educational Technology, 2018, 34(5). 24 Conclusion This study sought to explore the effectiveness of a web-based simulation tool using a sample of occupational therapy students and academics. The accessibility of the resource and whether it offered enhanced learning opportunities in relation to the graduate attributes of empathy, information gathering and active listening skills were also explored. A survey using Likert scale and open-response questions was administered. Students found The Occupational Therapy Simulations in Learning web resource was realistic, appealing to look at, and easy to navigate. They perceived that the resource supported their development of empathy for the client. Information gathering and assessment skills were also enhanced. Academics using the resource in their teaching reported that the resource was authentic and assisted in the development of empathy in their students. Advancements in educational technology such as The Occupational Therapy Simulations in Learning web resource offer an engaging and authentic means of teaching key graduate attributes that are integral for preparation for practice in the complex healthcare environments. References Allison, H., & Turpin, M. (2004). Development of the student placement evaluation form: A tool for assessing student fieldwork performance. Australian Occupational Therapy Journal, 51(3), 125– 132. https://doi.org/10.1111/j.1440-1630.2004.00414.x Bennett, S., Rodger, S., Fitzgerald, C., & Gibson, L. (2017). Simulation in occupational therapy curricula: A literature review. Australian Occupational Therapy Journal, 64(4), 314-327. http://dx.doi.org/10.1111/1440-1630.12372 Bradley, G., Whittington, S., & Mottram, P. (2013). Enhancing occupational therapy education through Simulation. British Journal of Occupational Therapy, 76(1), 43-46. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.905.7238&rep=rep1&type=pdf Bradley, P. (2006). The history of simulation in medical education and possible future directions. Medical Education, 40(3), 254-262. https://doi.org/10.1111/j.1365-2929.2006.02394.x Brown, J. A., & Woods, J. J. (2011). Performance feedback to support instruction with speech-language pathology students on a family-centred interview process. Infants and Young Children 24(1), 42-55. https://doi.org/10.1097/IYC.0b013e3182001bf4 Brown, T., & Williams, B. (2009). The use of DVD simulation as an interprofessional education tool with undergraduate occupational therapy students. British Journal of Occupational Therapy, 72(6), 266- 274. https://doi.org/10.1177/030802260907200607 Bruder, M. B., & Dunst, C. J. (2005). Personnel preparation in recommended early intervention practices: Degree of emphasis across disciplines. Topics in Early Childhood Special Education, 25 (1), 25–33. Retrieved from http://journals.sagepub.com/doi/abs/10.1177/02711214050250010301 Butina, M., Brooks, D., Dominguez, P. J., & Mahon, G. M. (2013). Utilization of virtual learning environments in the allied health professions. Journal of Allied Health 42(1), e7-e10. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/23471291 Cottrell, R. R., & McKenzie, J. F. (2011). Health promotion and education research methods using the five chapter thesis/dissertation model (2nd ed.). Sudbury, MA: Jones and Bartlett Publishers. Davis, M. H. (1994). Empathy: A social psychological approach. Madison, WI: Brown and Benchmark. Deakin University. (2017). Deakin graduate learning outcomes. Retrieved from http://www.deakin.edu.au/about-deakin/teaching-and-learning/deakin-graduate-learning-outcomes Evans, J., & Mathur, A. (2005). The value of online surveys. Internet Research, 15(2), 195–219. https://doi.org/10.1108/10662240510590360 Gibbs, D., & Dietrich, M. (2017). Using high-fidelity simulation to impact occupational therapy student knowledge, comfort and confidence in acute care. The Open Journal of Occupational Therapy, 5(1), Article 10. https://doi.org/10.15453/2168-6408.1225 Hagemann, E., Williams, C. K., McKee, P., Stefanovich, A., & Carnahan, H. (2014). Using model hands for learning orthotic fabrication. American Journal of Occupational Therapy, 68(1), 86-94. https://doi.org/10.5014/ajot.2014.009001 https://doi.org/10.1111/j.1440-1630.2004.00414.x http://dx.doi.org/10.1111/1440-1630.12372 http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.905.7238&rep=rep1&type=pdf https://doi.org/10.1111/j.1365-2929.2006.02394.x https://doi.org/10.1097/IYC.0b013e3182001bf4 https://doi.org/10.1177/030802260907200607 http://journals.sagepub.com/doi/abs/10.1177/02711214050250010301 https://www.ncbi.nlm.nih.gov/pubmed/23471291 http://www.deakin.edu.au/about-deakin/teaching-and-learning/deakin-graduate-learning-outcomes https://doi.org/10.1108/10662240510590360 https://doi.org/10.15453/2168-6408.1225 https://doi.org/10.5014/ajot.2014.009001 Australasian Journal of Educational Technology, 2018, 34(5). 25 Herrington, J., Oliver, R., & Reeves, T. (2003). Patterns of engagement in authentic online learning environments. Australian Journal of Educational Technology, 19(1), 59-71. Hojat, M., Louis, D., Markham, F., Wender, L., Rabinowitz, C., & Gomella, J. (2011). Physicians' empathy and clinical outcomes for diabetic patients. Academic Medicine, 86(3), 359-364. https://doi.org/10.1097/ACM.0b013e3182086fe1 King, G., Shepherd, T.,A., Servais, M., Willoughby, C., Bolack, L., Strachan, D., … McNaughton, N. (2016). Developing authentic clinical simulations for effective listening and communication in pediatric rehabilitation service delivery. Developmental Neurorehabilitation, 19(5), 284-294. Retrieved from http://www.tandfonline.com/doi/pdf/10.3109/17518423.2014.989461 Knecht-Sabres, L.J., & Kovic, M. (2013). Preparing occupational therapy students for the complexities of clinical practice. The Open Journal of Occupational Therapy, 1(3), Article 4. https://doi.org/10.15453/2168-6408.1047 Lombardi, M., & Oblinger, D. (2007). Authentic learning for the 21st century: An overview. Educause Learning Initiative. Retrieved from https://library.educause.edu/resources/2007/1/authentic-learning- for-the-21st-century-an-overview Mann, C.J. (2003). Observational research methods. Research design II: Cohort, cross sectional and case- control studies. Emergency Medical Journal. 20, 54-60. https://doi.org/10.1136/emj.20.1.54 Maran, N.J, & Galvin, R. J. (2003) Low- to high-fidelity simulation - a continuum of medical education? Medical Education. 37(1), 22-8. https://doi.org/10.1046/j.1365-2923.37.s1.9.x Occupational Therapy Australia (2010a). Australian minimum competency standards for new graduate occupational therapists (ACSOT) 2010. Retrieved from https://www.otaus.com.au/sitebuilder/onlinestore/files/37/australiancompetencystandardsentrylevelele ccopy.pdf Occupational Therapy Australia (2010b). Performance record for the Australian competency standards for occupational therapists (PracSoT). Retrieved from http://www.otaus.com.au/sitebuilder/aboutus/knowledge/asset/files/16/pracsot_pdf.pdf Pulman, A., Scammell, J., & Martin, M. (2008). Enabling interprofessional education: The role of technology to enhance learning, Nurse Education Today, 29(2), 232-239. https://doi.org/10.1016/j.nedt.2008.08.012 Rodger, S., Bennett, S., Fitzgerald, C., & Neads, P. (2010). Use of simulated learning activities in occupational therapy curriculum. The University of Queensland on behalf of Health Workforce Australia. Retrieved from http://www.hwa.gov.au/sites/default/files/simulated-learning-in- occupational-therapy-curricula-201108.pdf Rule, A. C. (2006). Editorial. The components of authentic learning. Journal of Authentic Learning, 3(1), 1-10. Retrieved from http://hdl.handle.net/1951/35263 Russell, C., Malfroy, J., Gosper, M., & McKenzie, J. (2014). Using research to inform learning technology practice and policy: A qualitative analysis of student perspectives. Australian Journal of Educational Technology. 30(1), 1-15. https://doi.org/10.14742/ajet.629 Scanlon, L. (2006). Graduate attributes and the transition to higher education. In P. Hager, & S. Holland (Eds.), Graduate attributes, learning and employability (pp.125-148). Dordrecht: Springer. Sheffler, S. J. (1995). Do clinical experiences affect nursing students' attitudes toward the elderly? Journal of Nursing Education, 34(7), 312-316. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/8576739 Stein, S., Isaacs, G., & Andrews, T. (2004). Incorporating authentic learning experiences within a university course. Studies in Higher Education, 29(2), 239-258. https://doi.org/10.1080/0307507042000190813 Sutherland, L., & Markauskaite, L. (2012). Examining the role of authenticity in supporting the development of professional identity: An example from teacher education. Higher Education, 64(6), 747-766. https://doi.org/10.1007/s10734-012-9522-7 Sutherland, L., Scanlon, L., & Sperring, A. (2005). New directions in preparing professionals: Examining issues in engaging students in communities of practice through a school-university partnership. Teaching and Teacher Education, 21(1), 79-92. https://doi.org/10.1016/j.tate.2004.11.007 Thistle, J., & McNaughton, D. (2015). Teaching active listening skills to pre-service speech-language pathologists: A first step in supporting collaboration with parents of young children who require AAC. Language, Speech, and Hearing Services in Schools, 46, 44-55. https://doi.org/10.1044/2014_LSHSS- 14-0001 https://doi.org/10.1097/ACM.0b013e3182086fe1 http://www.tandfonline.com/doi/pdf/10.3109/17518423.2014.989461 https://doi.org/10.15453/2168-6408.1047 https://library.educause.edu/resources/2007/1/authentic-learning-for-the-21st-century-an-overview https://library.educause.edu/resources/2007/1/authentic-learning-for-the-21st-century-an-overview https://doi.org/10.1136/emj.20.1.54 https://doi.org/10.1046/j.1365-2923.37.s1.9.x https://www.otaus.com.au/sitebuilder/onlinestore/files/37/australiancompetencystandardsentryleveleleccopy.pdf https://www.otaus.com.au/sitebuilder/onlinestore/files/37/australiancompetencystandardsentryleveleleccopy.pdf http://www.otaus.com.au/sitebuilder/aboutus/knowledge/asset/files/16/pracsot_pdf.pdf http://dx.doi.org/10.1016/j.nedt.2008.08.012 http://www.hwa.gov.au/sites/default/files/simulated-learning-in-occupational-therapy-curricula-201108.pdf http://www.hwa.gov.au/sites/default/files/simulated-learning-in-occupational-therapy-curricula-201108.pdf http://hdl.handle.net/1951/35263 https://doi.org/10.14742/ajet.629 https://www.ncbi.nlm.nih.gov/pubmed/8576739 https://www.ncbi.nlm.nih.gov/pubmed/8576739 https://doi.org/10.1080/0307507042000190813 https://doi.org/10.1007/s10734-012-9522-7 http://dx.doi.org/10.1016/j.tate.2004.11.007 https://doi.org/10.1044/2014_LSHSS-14-0001 https://doi.org/10.1044/2014_LSHSS-14-0001 Australasian Journal of Educational Technology, 2018, 34(5). 26 W3Schools (2017). CSS3 Introduction. Retrieved from https://www.w3schools.com/css/css3_intro.asp Wanzer, M.B., Booth-Butterfield, M., Gruber, K. (2004). Perceptions of health care providers’ communication: relationships between patient-centered communication and satisfaction. Health Communication, 16(3), 363–83. https://doi.org/10.1207/S15327027HC1603_6 Williams, B., Boyle, M., & Howard, S. (2016). Empathy levels in undergraduate paramedic students: A three-year longitudinal study. Nurse Education in Practice, 16(1), 86-90. https://doi.org/10.1016/j.nepr.2015.09.002 Williams, B., Brown, T., Boyle, M., & Dousek, S. (2013). Psychometric testing of the Jefferson Scale of Empathy Health Profession students' version with Australian paramedic students. Nursing and Health Sciences, 15(1), 45-50. https://doi.org/10.1111/j.1442-2018.2012.00719.x Williams, B., Brown, T., Scholes, R., French, J., & Archer, F. (2010). Can interdisciplinary clinical DVD simulations transform clinical fieldwork education for paramedic, occupational therapy, physiotherapy, and nursing students? Journal of Allied Health, 39(1), 3-10. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/20217001 World Wide Web Consortium (2017). Web content accessibility guidelines overview. Retrieved from https://www.w3.org/WAI/intro/wcag Yardley, S., Teunissen, P.W., & Dornan, T. (2012). Experiential learning: Transforming theory into practice. Medical Teacher, 34(2), 161-164. https://doi.org/10.3109/0142159X.2012.643264 Corresponding author: Kelli Nicola-Richmond, k.nicolarichmond@deakin.edu.au Australasian Journal of Educational Technology © 2018. Please cite as: Nicola-Richmond, K., & Watchorn, V. (2018). Making it real: The development of a web- based simulated learning resource for occupational therapy students. Australasian Journal of Educational Technology, 34(5), 13-26. https://doi.org/10.14742/ajet.3196 https://doi.org/10.1207/S15327027HC1603_6 https://doi.org/10.1016/j.nepr.2015.09.002 https://doi.org/10.1111/j.1442-2018.2012.00719.x https://www.ncbi.nlm.nih.gov/pubmed/20217001 https://www.w3.org/WAI/intro/wcag https://doi.org/10.3109/0142159X.2012.643264 mailto:k.nicolarichmond@deakin.edu.au https://doi.org/10.14742/ajet.3196 Kelli Nicola-Richmond and Valerie Watchorn Deakin University Introduction Simulated learning Simulated learning for students of occupational therapy The Occupational Therapy Simulations for Learning resource Method Procedure Instruments Table 1 Table 2 Results Table 3 Table 4 Table 5 Table 6 Table 7 Discussion Research Limitations Conclusion References Williams, B., Brown, T., Boyle, M., & Dousek, S. (2013). Psychometric testing of the Jefferson Scale of Empathy Health Profession students' version with Australian paramedic students. Nursing and Health Sciences, 15(1), 45-50. https://doi.org/10.1111/... World Wide Web Consortium (2017). Web content accessibility guidelines overview. Retrieved from https://www.w3.org/WAI/intro/wcag Yardley, S., Teunissen, P.W., & Dornan, T. (2012). Experiential learning: Transforming theory into practice. Medical Teacher, 34(2), 161-164. https://doi.org/10.3109/0142159X.2012.643264