Template PME28 Emerging Perspectives ep.journalhosting.ucalgary.ca Vocational Training for Adolescents and Adults with Autism Spectrum Disorder: Effectiveness of the Practical Assessment Exploration System (PAES) Amanda Smith-Demers, Adam W. McCrimmon, Katrina S. Shaw University of Calgary There are few specific, validated vocational training programs for individuals with Autism Spectrum Disorder (ASD). The Practical Assessment Exploration System (PAES) is a functional vocational skills training tool that teaches and measures vocational potential in individuals with developmental disabilities. This project utilized a mixed methods approach to better understand the effectiveness of PAES in enhancing the vocational abilities of twelve adolescents and young adults with ASD. Results indicated a statistically significant improvement in vocational skill and behaviour after completing PAES. Additional qualitative interviews with parents and attendees highlighted four primary themes: strengths, benefits derived, limitations, and program suggestions. Implications of these findings are discussed. Keywords: Autism Spectrum Disorder, vocation, intervention, adolescents, young adults Smith-Demers, A., McCrimmon, A. W., & Shaw, K. S. (2017). Vocational training for adolescents and adults with autism spectrum disorder: Effectiveness of the Practical Assessment Exploration System (PAES). Emerging Perspectives, 1, (1), 1-19. Individuals with Autism Spectrum Disorder (ASD) experience varying degrees of impairment in socio-communicative functioning and restricted, repetitive patterns of behaviours, activities, or interests (American Psychiatric Association [APA], 2013). Due to the variety and severity of demonstrated impairments, each individual with ASD can present with unique challenges that change throughout development, resulting in significant challenges with providing targeted intervention (Cimera & Cowan, 2009). The focus of research and intervention efforts for ASD has largely been dedicated to early identification and intervention in infancy and childhood, with little emphasis on adolescents and young adults (Taylor et al., 2012). This restricted focus has limited our understanding of the specific and varying outcomes for adolescents and adults with ASD (Bailey, 2012; Taylor et al., 2012). This issue is particularly concerning when considering these individuals’ transition to adulthood and the developmentally appropriate interventions that are often necessary during this period to facilitate independent daily living, communication, social, and vocational functioning to promote their independence throughout adolescence and adulthood (Seltzer, Shattuck, Abbeduto, & Greenberg, 2004). The purpose of this article is to report the findings of an empirical investigation of a unique vocational intervention, the Practical Assessment Exploration System (PAES), designed to enhance the job-related skills of adolescents and adults with a developmental disability and its specific effectiveness with individuals with ASD, including program strengths, limitations, and potential Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 2 areas for improvement. We begin with an overview of vocational skills and their importance to developmental outcome in adulthood. We then review the challenges with vocational skills in adults with ASD. PAES is introduced as a possible intervention program for adolescents and adults with ASD. We describe the mixed-methodology employed in the present study and the results in support of the program for this population. Finally, implications of the findings are discussed. Vocational Skills Two broad categories of skills essential to positive employment outcomes are work or vocational skills and social skills (Chadsey, 2007). Vocational skills involve a variety of daily living abilities and capacities to enhance independence in the home, work, and community environments (Canella-Malone et al., 2006). Core skills include literacy, numeracy, paperwork, computer use, problem solving, decision making, critical thinking, task planning and organization, memory, oral communication, interpersonal skills, and continuous learning, among others (Human Resources and Skills Development Canada, 2011). In addition, there is a wide range of task- specific skills that may require visual-motor coordination, processing speed, and fine and/or gross motor coordination. Previous research has suggested that there are vocational skills that can be generalized across job sites, such as following instructions, but that these skills must be socially validated in what is expected of individuals for each job placement (Hendricks, 2010). For example, social validation of following instructions would involve detailing a specific set of instructions for their workplace and why that set would be appropriate for the environment/context of the employment. Regardless of the workplace-specific skills, the skills taught to each employee will be contingent on the needs of the employer and the employee’s vocational interests and capabilities (Chadsey, 2007). Acquisition of skills may also depend on the interests, abilities, and cognitive capacities of the trainees; however, there is the potential that as more direct and intensive supports are required, additional resources will also be necessary to accommodate these individuals and help them adapt in an employment setting. Vocational Skills in ASD Adults with ASD struggle to obtain and maintain employment, likely due to under- developed vocational skills (Howlin, 2000; Hurlburt & Chalmers, 2004; Müller, Schuler, Burton, & Yates, 2003; Nesbitt, 2000) in addition to their characteristic socio-communicative and behavioural impairments (Cameto, Marder, Wagner, & Cardoso, 2003; Dew & Alan, 2007). Indeed, competitive, independent employment for these individuals was once considered unlikely (Mawhood & Howlin, 1999; Nesbitt, 2000) and it has been estimated that 50 to 75% of adults with ASD are unemployed (Shattuck et al., 2012; Wehman et al., 2012). Notably, this unemployment rate is the highest of any disability category studied (Shattuck et al., 2012; Shattuck, Wagner, Narendorf, Sterzing, & Hensley, 2011). Regarding employment locations, recent research indicates that 56% of young adults with ASD sampled were employed in sheltered workshops or day activity centers, rather than independent competitive community employment (Taylor & Seltzer, 2011). Both sheltered workshops and day activity centers are alternatives to community employment, whereby individuals usually attend a facility-based program and engage in “relatively simple work activities” (p. 1) that match the individual’s skillset (Migliore, 2010). Subsequent research indicates that only 7.4% of adults with ASD sampled were employed in a community-based setting without supports and 10.2% were employed in the community with supports (Taylor & Seltzer, 2012). In Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 3 comparison, Taylor and Seltzer (2012) indicated that 34.7% of individuals were engaged in a sheltered vocational setting for 10 hours per week or less. Furthermore, young adults with ASD and intellectual disability were three times more likely to engage in sheltered or supported daytime vocational activities than individuals without intellectual disability, who typically do not participate in sheltered or supported settings (Taylor & Seltzer, 2011). Even among higher functioning individuals with ASD, who have cognitive ability in the average or above ranges, researchers have indicated that the employment rate rarely surpasses 30 percent and that these employees are underpaid and underskilled (Howlin, 2000; Howlin, Alcock, & Burkin, 2005). Despite these findings, community-based supported employment is becoming more common for these individuals (Howlin & Mawhood, 1996; Müller, Schuler, Burton & Yates, 2003) and researchers have shown that individuals with ASD are capable of working in a variety of occupations and organizations with this additional support (O’Brien & Daggett, 2006). Current perspectives emphasize the implementation of technology and distinctive and specialized vocational support to assist adolescents and adults with ASD in developing vocational skills and obtaining competitive employment (Gentry, Wallace, Kvarfordt, & Lynch, 2010; Schaller & Yang, 2005) as they do not often benefit from general vocational training (Müller et al., 2003). This prior research indicates that vocational support is not meeting the needs of the ASD population and emphasizes the importance of specific vocational support directed at individuals with ASD. Indeed, despite the identified challenges with vocation and employment in adults with ASD, vocational programs for this population are virtually non-existent, resulting in low levels of independence and employment in adulthood (Howlin et al., 2005; Müller et al., 2003). In sum, it is apparent that the vocational needs of this population are not being adequately met and research findings indicate that adolescents and young adults with ASD would benefit from support for obtaining and maintaining employment (Gentry et al., 2010; Schaller & Yang, 2005). Indeed, individuals with ASD should have the same entitlement to work as is afforded to the rest of society (Hendricks, 2010). Employment can provide an opportunity to enhance personal dignity and has been shown to improve cognitive performance for individuals with ASD (Garcia-Villamisar & Hughes, 2007; García-Villamisar, Wehman, & Navarro, 2002; Persson, 2000). Enhancing the vocational abilities of adolescents and young adults with ASD has been identified as an important intervention goal (Mavranezouli et al., 2014; Taylor et al., 2012). As individuals with ASD have under-developed vocational skills and present with unique behavioural and cognitive symptoms, this population often does not benefit from typical vocational intervention as generalized supportive services fail to meet their unique vocational needs (Müller et al., 2003). Hence, a distinct and specialized intervention approach to support vocational skill development for individuals with ASD is sorely needed. As mentioned, there are limited specific, validated vocational training programs for individuals with ASD (Wehman, Targett, Schall, & Carr, 2014). However, PAES is a functional vocational skills program that teaches and measures vocational potential in individuals with disabilities. As such, its intended purpose may address the unique vocational needs of adolescents and young adults with ASD. The Practical Assessment Exploration System The Practical Assessment Exploration System (PAES; Swisher, Green, & Tollefson, 1999) teaches a variety of vocational tasks associated with business, home economics, and the industrial arts to participants with various developmental disabilities. These tasks include filing, collating papers, food preparation, sewing by hand, using linear measurement tools, and using hand tools. Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 4 PAES is a group-based intervention that runs in 12-week increments and attendees are divided into groups based on age: adolescents are aged 13 to 17 and adults are aged eighteen or older. Each group contains 8 to 12 participants, and both adolescent and adult groups are run simultaneously at different, preset weekly times. Within these groups, trainees are able to sign up and repeat the PAES program as individuals acquire skills at different rates. Attendees in the PAES program typically demonstrate cognitive impairment and the programming in PAES is tailored to meet their needs. Vocational skills are systematically taught to participants, including appropriate workplace behaviours such as arriving on time, clocking in for a shift, storing belongings upon arrival, and break time. PAES also introduces the concept of supervision. Although PAES is designed to assist the educational and vocational decision-making processes for students with developmental disabilities in general (Swisher et al., 1999), its effectiveness in the ASD population has yet to be investigated. Thus, a gap in the research literature pertaining to the use of PAES with this population exists. Research Purpose This project aimed to provide a comprehensive understanding of improvements in vocational skills and overall enjoyment associated with participation in PAES. Specifically, we evaluated the effectiveness of PAES as a vocational intervention for adolescents and adults with ASD, including identification of the strengths and limitations of PAES for this population. Research Questions 1. Will PAES effectively enhance the vocational skills of adolescents and adults with ASD? 2. What are the strengths and limitations of PAES for adolescents and adults with ASD? 3. What suggestions can be made in regards to programming to increase the effectiveness of PAES for this population? We anticipated that PAES would effectively enhance the vocational and related skills of adolescents and adults with ASD. Additionally, we expected that qualitative data from attendees and their parent(s) would highlight program strengths, limitations, and potential areas for improvement. Methods Participants Twelve adolescents and young adults with ASD participated in this study. Attendees were required to have a diagnosis of ASD made by an appropriately licensed professional prior to participating. All attendees were registered to attend PAES and subsequently were recruited to participate in this research study upon giving consent. This research sample represents two adolescent cycles and two adult cycles of PAES with data collected over a six-month time period. The mean age of the sample was 17.7 years, with a range of 14 to 22 years. Nine attendees were male, equating to an expected gender ratio given the distribution of the population diagnosed with ASD (APA, 2013; Baio, 2012; Newschaffer et al., 2007). Attendees completed a measure of cognitive ability to determine their current level of development in this domain, and their ability to understand and answer interview questions. Those who were verbal, i.e., demonstrated spontaneous, communicative speech, completed the Wechsler Abbreviated Scale of Intelligence – Second Edition (WASI-II; Wechsler, 2012) whereas those that presented with language impairment (i.e., Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 5 minimally or non-verbal) completed the Comprehensive Test of Nonverbal Intelligence (CTONI; Hammill, Pearson, & Weiderholt, 2009). In addition, the attendees’ parents completed the Adaptive Behaviour Assessment System – Second Edition (ABAS-II; Harrison & Oakland, 2006) to evaluate their child’s current level of adaptive functioning. Lastly, the attendees’ parents and program facilitators completed a semi-structured interview upon completion of PAES. Attendee demographics and performance on inclusionary measures are presented in Table 1. Table 1. Demographic and Clinical Information. Mean SD Range Age (years) 17.7 3.06 14.1-22.7 Gender (% male) 75% (n=9) n/a n/a FSIQ WASI-II (n=4) 83.0 16.09 65-102 CTONI-2 (n=8) 72.9 19.57 47-105 ABAS-II (n=12) 54.7 14.99 42-91 Note. Age is reported in decimalized format (e.g., 19 years, 6 months is 19.5 years). The Wechsler Abbreviated Scale of Intelligence, 2nd Edition (WASI-II) is from Wechsler, 2012; and the Comprehensive Test of Nonverbal Intelligence, 2nd Edition (CTONI-2) is from Hammill, Pearson, & Weiderholt, 2009. FSIQ refers to Full Scale Intelligence Quotient. The Adaptive Behaviour Assessment Scale – Second Edition (ABAS-II) is from Harrison & Oakland, 2006. Mean and standard deviation performance for each of these measures is reported in standard score units. Measures As there were no standardized assessment tools designed to evaluate improvement in vocational ability, a mixed-methods approach was utilized to gather information on vocational skills in the sampled adolescents and young adults with ASD. Specifically, both quantitative and qualitative approaches were utilized to provide converging evidence in support of PAES for use with adolescents and young adults with ASD. TEACCH Transition Assessment Profile. The Treatment and Education of Autistic and related Communication Handicapped Children (TEACCH) Transition Assessment Profile (TTAP; Mesibov, Thomas, Chapman, & Schopler, 2007) is a non-standardized measure of vocational and related skills in adolescents and adults with ASD. The TTAP was created using evidence from successful transition programs such as the Adolescent and Adult Psycho Educational Profile (AAPEP) as well as education laws and current research (Mesibov & Shea, 2010; Mesibov et al., 2007). Direct observation, parent, and school/work scales provide information on six domains across different settings. Specifically, this measure evaluates vocational skills, vocational behaviour, independent functioning, leisure skills, functional communication, and interpersonal behaviour. For the purposes of this project, the vocational skills and vocational behaviour subscales were used from a direct observation standpoint to measure change in the amount of vocational skill and behaviour items that were passed, both pre- and post-intervention. These two subscales were administered following the standardized procedure outlined in the examiner’s manual. Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 6 The TTAP was created using a scoring system on which each item is scored as a pass when the task is completed successfully, emerging when the task is partially completed and an initial understanding is displayed, or a fail when an individual is unwilling or unable to complete the task. According to the Individuals with Disabilities Education Act (IDEA, 1997), assessments should aid in identifying life-skill domains requiring accommodation; therefore, the TTAP utilizes structured teaching to identify how much support an individual requires before they are able to complete a task independently (Mesibov et al., 2007). Qualitative. Semi-structured interviews were conducted with program attendees who were verbally and cognitively able, a parent for each attendee, and seven out of the eight facilitators working in the program. The goal of these interviews was to tap into domains that may not have been assessed adequately by quantitative measures. Specifically, interviewees were asked about perceived strengths and limitations of the program, perceived changes in routines, self-esteem, social behaviour, motivation, any positive or negative side effects they associated with program participation, and any suggestions they may have had for program improvement. Interviews were semi-structured so as to allow participants to expand on topics that were not otherwise covered within the interview protocol. The interview questions are presented in Appendices A, B and C. Research Design A convergent mixed-methods approach was used as both quantitative and qualitative data were collected, analyzed separately, and then compared (Creswell, 2014). This approach holds the key assumption that both types of data provide different sources of information and together should provide similar results (Creswell, 2014). Regarding the quantitative aspect of analysis, a quasi- preexperimental design was used as it involved a pre-post no-control group design without randomization of participants (i.e., all participants completed PAES at the same time). Procedure The intake process for PAES involved four phases: 1) parents of attendees contacting the program expressing interest, 2) telephone screening, 3) written application for programming, and 4) an intake meeting with each potential attendee and his/her family/guardian. At the time of the intake, each attendee and his/her family was asked for their consent to participate in this study and was provided a consent form. Upon consent, the ABAS-II and WASI-II/CTONI-2 were administered prior to the start of the intervention. The TTAP was administered both before and directly after the intervention. After the program was complete, semi-structured interviews were conducted with attendees and their parent(s), as well as program facilitators. Participants were informed of their right to withdraw consent and were assured of confidentiality. Results Quantitative A Wilcoxon Signed-Rank Test was used to compare participant performance on the Vocational Skills and Vocational Behaviours domains of the TTAP before (T1) and after (T2) intervention. The Wilcoxon Signed-Rank Test is a non-parametric alternative to the standard t-test when comparing two related samples or repeated measurements from a single sample. This statistical method was selected as the most appropriate due to the small sample size (n = 12). Results of this analysis indicated that both Vocational Skills, T1 (Mdn = 9.0) and T2 (Mdn = 11.0), Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 7 z = -2.69, p = .007, and Vocational Behaviours, T1 (Mdn = 8.5) and T2 (Mdn = 11.0), z = -2.37, p = .018, significantly increased following completion of PAES. Qualitative Thematic analysis was selected for evaluating and illustrating themes within the qualitative data as the primary researcher sought a deeper understanding of parental/guardian and attendee experiences surrounding participation in PAES. Thematic analysis consists of five steps and interested readers are directed to the work of Braun and Clark (2006) to familiarize themselves with the process. Transcription of the interviews occurred subsequent to their completion and hard copies were provided to the first author. Interviews were then read and notes were made as this process occurred. At this point, open coding occurred line by line and notes were made when possible codes or ideas occurred to the primary researcher. Once open coding was completed, grouping into preliminary codes and themes began. After all of the interviews had been coded, segments of interviews were retrieved and written with codes to further develop code groupings. Further grouping and code re-establishment aided in the development of themes, and the resulting themes were then named and defined to answer each research question. Evaluating qualitative research. In general, thematic analysis is not tied to any particular theoretical framework but instead can be used in tandem with many theories. This characteristic makes thematic analysis particularly well suited to a mixed methods approach as it avoids philosophical paradigm clashes. For the purposes of this study, an essentialist or realist perspective was adopted, which entailed reporting experiences/meaning and reality of participants (Braun & Clarke, 2006). This perspective allowed for the theorizing of experiences, motivations, and meaning in a direct way while assuming a unidirectional relation between experiences/meaning and the language used in expression (Braun & Clarke, 2006). It is also essential that qualitative research is methodologically rigorous (Creswell, 2014) and thus steps were taken to ensure trustworthiness of the qualitative data. Throughout the research process, the primary researcher engaged in a process of reflection, identifying any biases she held, to ensure as much objectivity as possible. During the initial coding process, re-coding of the data occurred as necessary to add or remove themes. It was also during this step that an objective individual re-coded the data to ensure intercoder reliability and consistency of theming. Any discrepancies were discussed between the two coders and resolved to ensure agreement between both coders before moving forward. Finally, a process of triangulation also occurred, as the researcher examined the findings both qualitatively and quantitatively as a further means to check the accuracy of the data (Creswell, 2014). Findings The qualitative questions explored the strengths, limitations, and suggestions for improvement as discussed by verbal program attendees, their parents, and program facilitators using thematic analysis. Through the use of thematic analysis, a fourth theme was highlighted, as all interviewees spoke to the benefits achieved by program attendees post programming. Of the 12 program attendees, only six were cognitively and verbally able to complete an interview with the primary researcher. A total of 11 parents completed a final interview; the remaining guardian was unable to complete an interview due to time constraints and the transition of his/her child attendee into adult services. Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 8 Qualitative analysis indicated four primary themes with several subthemes within each identified by interview respondents: 1) Program Strengths (i.e., program enjoyment and structure); 2) Benefits Derived (i.e., personal growth, engagement, and reduction of anxiety); 3) Program Limitations, such as length of program too short, program overly complex; and 4) Program Suggestions, such as increased length of program, inclusion of visual aids. A description of each theme and sample quotes are provided below and presented in Table 2. Table 2. Summary of themes for the PAES program as identified by attendees, parents, and facilitators. Theme Description Example quote from interviews Program Strengths Qualities of PAES that are working well, qualities of the program that benefit program attendees, and specific assets of the program for individuals with ASD “He enjoys coming every week.” “It’s really hands on and tangible, where they came and did actual learning skills that will benefit them.” “One of the strengths is the amount of activities and tasks that there are. So it gives a lot of the participants the opportunity to see what they may be interested in and that they didn’t even know existed” “The structured setting is really a strength because it can help us identify the needs that they require or what strengths they have or what areas need an adaptation so they can go out in the community.” Benefits Derived Benefits of the program as observed in behavioural and/or skill change in the attendees “I met a new friend” “He wants to be more sociable. He will now go up and talk to people.” “I definitely have seen social skills or being able to communicate with staff change. Being more comfortable in a work place setting you see a lot of their skills increase.” “It does give them the sense of accomplishment, which is something that I don’t know if they get that in a ton of other stuff that they do.” Program Limitations Qualities of the program that currently are not working well, that restrict program attendees, and specific parts of the program that are not designed well for individuals with ASD “Wish it was longer or more times per week” “Part of the problem with PAES is the instructions that the individuals are given. I think the instruction cards and how they are worded, and sometimes they seem very rigid. I think if those were to change then it would serve a greater range.” “Autism is such a spectrum. To find one task that meets everyone’s needs directly. Like for some it may be too easy whereas other clients it may be too hard… So it’s hard because they are all different” Program Suggestions Ideas or recommendations for program improvement “Just awesome the way it is” “Lengthening the program span” “It would be helpful if the facilitators broke down to families (what they have been working on in the program) or whoever is supporting them, so you can see changes are happening outside the program”. “Simplify instructions, and reduce the amount of steps” “Re-doing all of the instruction cards, trying to make everything consistent.” Theme one: Program strength. This theme included the qualities of PAES that are working well and/or benefit program attendees, and specific assets of the program for individuals with ASD. Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 9 In general, interviewees were very positive about PAES. The most common program strength identified by 10 parents was overall program enjoyment. Parents spoke to the quality of programming in that attendees could come to learn new skills and still experience enjoyment and a willingness to participate. As has been identified within previous literature, it is important to consider the overall enjoyment experienced by individuals to further improve independence and overall quality of life (Hendricks, 2010). Parents also indicated diversity as a strength of the program in that the activities used in the intervention were varied and afforded attendees the opportunity to engage in different types of vocational tasks to both develop new skills and to explore novel employment activities. Overall, program enjoyment and diversity of activities contribute to the attendees’ overall enjoyment of and success with the program. Parallel to this notion of overall program enjoyment, participants emphasized learning as a program strength. Concurrent with learning, parents also described specificity of skills as a subtheme by speaking about skills that involved “actual learning.” Specifically, two parents stated, respectively, “it’s really hands on and tangible, where they came and did actual learning skills that will benefit them” and “the tasks they do actually have a purpose.” The subthemes of learning and specificity of skills are consistent with quantitative data pertaining to the TTAP as there was a significant improvement in vocational skill and behaviour displayed by attendees. The most common strength identified by program facilitators was the structure that PAES provides for program attendees. Structure has typically been emphasized within the literature as beneficial for individuals with ASD because, much like transitioning, structure provides adequate preparation and predictability and reduces confusion (Dettmer et al., 2000). Both facilitators and parents described structure as an overarching quality of the program that supported the needs of an ASD population. This was noteworthy as PAES was designed for students with various disabilities (Swisher et al., 1999) and thus this strength further signifies some preliminary support for benefits of using PAES with an ASD population. Theme two: Benefits derived. This theme is described as benefits perceived by the interviewees after they attended PAES. Within this theme, parents and facilitators highlighted the attendees’ personal growth, belonging/engagement, and reduced anxiety after completing PAES. Furthermore, parents and facilitators described positive changes in social communication and overall social skills. Specifically, one parent/guardian attributed her son’s increase in social communication as a direct reflection of participation in PAES as he was not participating in any other programming or learning environments. Facilitators also highlighted this positive change in social behaviour by describing the attendees’ improvement in communication with staff during programming. Social communication is one of the core deficits for individuals with ASD (APA, 2013) and research continues to emphasize the need to enhance this domain of functioning (Centelles, Assaiante, Etchegoyhen, Bouvard, & Schmitz, 2012). Theme three: Program limitations. The third theme, program limitations, is described as qualities of the program that are not currently working well or that restrict program attendees and specific parts of the program that are not designed well for individuals with ASD. The most common limitation highlighted by parents and facilitators was the length or overall duration of PAES. Several parents stated that they wished that the program was recurrent or longer, as learning is continual and does not stop once individuals with ASD reach adulthood. Indeed, many parents mentioned that there are few supportive programs for individuals with ASD once they reach adolescence or adulthood. It was also stressed that learning can be a longer process for many individuals with ASD than for individuals with other developmental disabilities and therefore Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 10 parents felt that their children would have acquired more skills if PAES was longer. It is then not surprising that parents recommended an increase in program duration as a program suggestion, highlighting the importance for continual learning and need for repeated teaching in the acquisition of new skills for individuals with ASD. Facilitators also emphasized program materials as a limitation and suggestion for program improvement, primarily due to an “unnecessary degree of complexity” within program instructions. This topic was highlighted by all seven facilitators and thus should be considered significant when considering modifications to this program for individuals with ASD. Specifically, facilitators suggested re-doing instruction cards and including visuals as primary suggestions for improving PAES for use with those with ASD. This suggestion is consistent with much of the literature as individuals with ASD present with variable communicative abilities (APA, 2013; Human Resources and Skills Development Canada, 2011) and this variability must be addressed for more individuals to achieve success within a vocational environment Theme four: Program suggestions. The fourth theme, program suggestions, is described as ideas or recommendations for program improvement. In alignment with topics from Theme Three, parents indicated an increase in the duration of the program as a potential improvement to the program. Additionally, they stated that more direct communication to parents of the program goals from the program facilitators may have enhanced their understanding of the program and its influence on behaviors outside of the intervention context. Attendees varied in their responses, which included the belief that the program should be lengthened, the program should be made much more difficult, and that the program was effective as designed in the manual. Facilitators noted that simplification of instructions and modification to the visual materials may increase the program’s effectiveness with the ASD population, as they observed that the attendees often required additional support around task instructions, etc. Discussion This is the first study to report on the efficacy and implementation of a vocational intervention for adolescents and young adults with ASD. Integrating a qualitative approach produces a different kind of understanding of the experience of vocational training and perceptions pertaining to programming. The qualitative data provides a significant contribution to the overall experience of vocational training in the ASD community, as the data provides insight into firsthand experiences of the effectiveness of PAES with the ASD population. Results confirmed our primary hypothesis. In general, both the quantitative and qualitative results of this study provide convergent preliminary support for PAES in enhancing the vocational and related skills of adolescents and adults with ASD. Results from the TTAP indicated significant improvement on the vocational skill and vocational behaviour domains. The development of the four themes is significant as attendees, parents, and program facilitators reported on benefits of the program, including personal growth, belonging/engagement, and anxiety reduction achieved by the attendees because of their participation in PAES. The subthemes within the benefits derived theme further emphasize the benefit of PAES as an intervention for the ASD population and enhance our understanding of the experience of vocational training and perceptions pertaining to PAES. A major strength of the PAES specialized programming appears to be the experience it provides to attendees and their families. The most common program strength, identified by eight Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 11 parents, was overall program enjoyment experienced by individuals attending PAES. This is an important finding as it is important to consider the overall enjoyment experienced by individuals (Hendricks, 2010) to further improve independence and overall quality of life. Parallel to this notion of overall program enjoyment, attendees emphasized learning as a strength of the program. Concurrent with learning, parents also described specificity of skills as a subtheme and spoke about skills that involved “actual learning.” The subthemes of learning and specificity of skills are consistent with quantitative data from the TTAP as attendees’ vocational skill and behaviour significantly improved. Parents emphasized diversity as a strength of the program, which not only spoke to the variety of different tasks that attendees were learning but also the different facilitators that attendees had supporting and supervising them each session. Parents indicated that the variety of and change in tasks each week helped to facilitate transitions for their child, which can often be challenging for individuals with ASD (Lorimer, Simpson, Miles, & Ganz, 2000). Furthermore, parents described the program structure as a quality of the program that supported the unique needs of the ASD population; in turn, this finding provides preliminary support for the use of PAES with adolescents and adults with ASD. While many significant strengths were identified, there were also program limitations that are important to recognize. The most common limitation subtheme highlighted by parents was the length, or overall duration of PAES. Parents stated that they wished the program was recurrent or longer as learning is continual and does not stop once individuals with ASD reach adulthood. Length and participation in programming was emphasized and was seen as significant because, as many parents mentioned, there are not many programs for individuals with ASD once they reach adolescence or adulthood. Limitations Several limitations are evident in the current study. The most prominent limitation is the small sample size. Due to the specificity of the programming and diagnosis of the assessed population, as well as the intensive nature of the data collection, obtaining a larger sample size was not possible. Additionally, this study did not use a control group and did not control for participation in other programming or additional skill development that might have impacted vocational or related skills as doing so would have further limited the study’s sample size. The representativeness of the sample to the general ASD population may also have been limited by a variety of elements. Specifically, participation in this study required a substantial commitment of time, travel, and energy for both the individuals who attended the program and their parents. Moreover, it was limited to individuals who resided in the city where the program was offered. There is the possibility that a subset of the ASD population may be more likely to participate in or benefit greatly from PAES. Due to a vast range in capabilities it was difficult to provide adequate programming for some program attendees that were substantially lower or higher functioning than the rest of the group and some facilitators thought the program would be better suited for a more specific subset of the ASD population. It is likely that all of these factors influenced characteristics of the participant sample and, therefore, the generalisability of the findings. Finally, because of the nature of the study and the use of convenience sampling, the qualitative results are context bound. It is important that these results are considered in context as they are intended to provide a greater understanding of the experience of these particular families. Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 12 Future Directions and Implications Future research exploring vocational and independence training for individuals with ASD should expand on the present study by collecting data from a larger sample and utilizing a waitlist control group matched for developmental level. Additionally, future research could screen for other services being provided to participants to control for the influence of these other programs on findings related to PAES. Furthermore, future research could employ a longitudinal design to capture the maintenance of vocational skills learned within PAES and to follow individuals exploring the outcome of participation in this programming. This research can be considered an encouraging step towards future research in the area of vocational ability for individuals with ASD. Prior research only employing quantitative research has been considered problematic (De Haan, Hawley, & Deal, 2002) as the overall experience of participants is not adequately captured. The current findings have implications for vocational intervention for individuals with ASD. Employment for individuals with ASD increases financial self-reliance and reduces dependence on both government and independently funded programming (Hendricks, 2010). When programs like these are validated, individuals with ASD can be provided with specific programming unique to this complex population, thereby increasing the employment success of individuals with ASD. The current project highlights the necessity for continued research in the area of vocational and independence training as well as the need to focus on adolescent and young adults with ASD. This research and focus may provide families with a more positive outlook and hope for the future as individuals with ASD face many challenges throughout their lives and have different needs. Conclusion The results provide convergent preliminary support for PAES in enhancing the vocational skills of adolescents and adults with ASD as well as in some areas of potential programmatic improvement. These findings are particularly important as there are few, if any, validated vocational training tools for this population. When programs such as PAES are validated, individuals with ASD can be provided with specific programming to increase their employment success. Vocational research in the context of ASD is limited. The preliminary evidence from the current study indicates that participants’ vocational ability and behaviours improved subsequent to completion of PAES. Additional qualitative inquiry yielded information on programmatic strengths and weaknesses that can be used to improve the program. The study also highlights the necessity for continued research in the area of vocational and independence training for adolescents and young adults with ASD. Additionally, the study emphasizes the need to focus on this period in development to provide families with a more positive outlook and hope for the future of their loved ones with ASD. Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 13 References American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders, 5th edition. Author: Washington, D.C. doi:10.1176/appi.books.9780890425596.744053 Bailey, A. (2012). Autism in adults. Autism Research, 5, 1-2. doi:10.1002/aur.233. Baio, J. (Ed.). (2012). Prevalence of autism spectrum disorders – Autism and developmental disabilities monitoring network, United States, 2008, MMWR;61(No. 3). Atlanta, GA: Centers for Disease Control and Prevention. Retrieved from: https://www.cdc.gov/mmwr/preview/mmwrhtml/ss6103a1.htm Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. doi:10.1191/1478088706qp063oa Cameto, R., Marder, C., Wagner, M., & Cardoso, D. (2003). NLTS2 Data Brief: Youth Employment. A Report from the National Longitudinal Transition Study-2. Menlo Park, CA: SRI International. Canella-Malone, H., Sigafoos, J., O'Reilly, M., de la Cruz, B., Edrisinha, C., & Lancioni, G. E. (2006). Comparing video prompting to video modeling for teaching daily living skills to six adults with developmental disabilities, Education and Training in Developmental Disabilities, 41(4), 344-356. Chadsey, J. G. (2007). Vocational Skills and Performance. In J.W. Jacobson, J. A. Mulick, & J. Rojahn (Eds.), Handbook of Intellectual and Developmental Disabilities (pp. 619-634). New York, NY: Springer. Cimera, R.E., & Cowan, R.J. (2009). The costs of services and employment outcomes achieved by adults with autism in the US. Autism, 13, 285-302. doi:10.1177/1362361309103791. Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches. Los Angeles: Sage. De Haan L. D., Hawley D. R. & Deal J. E. (2002). Operationalizing family resilience: A methodological strategy. American Journal of Family Therapy, 30, 275-291. doi:10.1080/01926180290033439 Dew, D. W., & Alan, G. M. (Eds.). (2007). Rehabilitation of individuals with autism spectrum disorders (Monograph no. 32). Washington, DC: George Washington University, Center for Rehabilitation Counseling Research and Education, Institute on Rehabilitation Issues. Garcia-Villamisar, G. & Hughes,C. (2007). Supported employment improves cognitive performance in adults with autism. Journal of Intellectual Disability Research, 51(2), 142– 150. doi:10.1111/j.1365-2788.2006.00854.x. García-Villamisar, D., Wehman, P. & Navarro, M.D. (2002). Changes in the quality of autistic people’s life that work in supported and sheltered employment. A 5-year follow-up study. Journal of Vocational Rehabilitation, 17(4), 309–312. Gentry, T., Wallace, J., Kvarfordt, C., & Lynch, K. B. (2010). Personal digital assistants as cognitive aids for students with autism: Results of a community-based trial. Journal of Vocational Rehabilitation, 32(2), 101-107. doi:10.3233/JVR-2010-0499. Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 14 Hammill, D. D., Pearson, N. A., & Weiderholt, J. L. (2009). Comprehensive Test of Nonverbal Intelligence—Second Edition (CTONI-2). Austin, TX: Pro-ed. Hendricks, D. (2010). Employment and adults with autism spectrum disorders: Challenges and strategies for success. Journal of Vocational Rehabilitation, 32, 125-134. doi:10.3233/JVR-2010-0502. Howlin, P. (2000). Outcomes in adult life for more able individuals with autism or Asperger syndrome. Autism, 4(1), 63-83. doi:10.1177/1362361300004001005. Howlin, P., & Mawhood, L. (1996). An evaluation of a pilot two-year supported employment service for people with autism. London: National Autistic Society. Howlin, P., Alcock, J., & Burkin, C. (2005). An 8-year follow up of a specialist supported employment service for high-ability adults with autism or Asperger syndrome. Autism, 9(5), 533-549. doi:10.1177/1362361305057871. Human Resources and Skills Development Canada (2011). Literacy and essential skills: Definitions. Retrieved from http://www.hrsdc.gc.ca/eng/workplaceskills/LES/definitions/definitions.shtml Hurlburt, K. & Chalmers, L. (2004). Employment and adults with Asperger syndrome. Focus on Autism and Other Developmental Disabilities, 19, 215-222. doi:10.1177/10883576040190040301 Individuals with Disabilities Education Act of 1997, P.L. 105-17, 20 U.S.C. § 1400 et seq. Kaufman, A. S., & Kaufman, N. L. (1993). Kaufman adolescent and adult intelligence test. Circle Pines, MN: American Guidance Service. Lorimer, P. A., Simpson, R. L., Myles, B. S., & Ganz, J. B. (2002). The use of social stories as a preventative behavioral intervention in a home setting with a child with autism. Journal of Positive Behavior Interventions, 4(1), 53-60. doi:10.1177/109830070200400109 Mawhood, L., & Howlin, P. (1999). The outcome of a supported employment scheme for high- functioning adults with autism or Asperger syndrome. Autism, 3(3), 229-254. doi:10.1177/1362361399003003003 Mavranezouli, I., Megnin-Viggars, O., Cheema, N., Howlin, P., Baron-Cohen, S., & Pilling, S. (2014). The cost-effectiveness of supported employment for adults with autism in the United Kingdom. Autism, 18(8), 975-984. doi:10.1177/1362361313505720 Mesibov, G. B., & Shea, V. (2010). The TEACCH program in the era of evidence-based practice. Journal of Autism and Developmental Disorders, 40(5), 570-579. doi:10.1007/s10803-009- 0901-6 Mesibov, G. B., Thomas, J. B., Chapman, S. M., & Schopler, E. (2007). TTAP: TEACCH transition assessment profile (2nd ed.). Austin, TX: Pro-ed. Migliore, A. (2010). Sheltered workshops. In J. H. Stone & M. Blouin (Eds.) International Encyclopedia of Rehabilitation Online. Retrieved from http://cirrie.buffalo.edu/encyclopedia/en/article/136/ Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 15 Müller, E., Schuler, A., Burton, B., & Yates, G. B. (2003). Meeting the vocational support needs of individuals with Asperger syndrome and other autism spectrum disabilities. Journal of Vocational Rehabilitation, 18(3), 163-175. Nesbitt, S. (2000). Why and why not? Factors influencing employment for individuals with Asperger syndrome. Autism, 4, 357-370. doi:10.1177/1362361300004004002 Newschaffer, C. J., Croen, L. A., Daniels, J., Giarelli, E., Grether, J. K., Levy, S. F., Windham, G. C. (2007). The epidemiology of autism spectrum disorders. Annual Review of Public Health, 28, 235-258. doi:10.1146/annurev.publhealth.28.021406.144007 O’Brien, M., & Daggett, J. A. (2006). Beyond the autism diagnosis: A professional’s guide to helping families. Baltimore, MD: Brookes Publishing Company. Persson, B. (2000). Brief report: A longitudinal study of quality of life and independence among adult men with autism, Journal of Autism and Developmental Disorders, 30(1), 61–66. doi:10.1023/A:1005464128544 Rossen, E. A., Shearer, D. K., Penfield, R. D., & Kranzler, J. H. (2005). Validity of the comprehensive test of nonverbal intelligence (CTONI). Journal of Psychoeducational Assessment, 23, 161-172. doi:10.1177/073428290502300205 Schaller, J., & Yang, N. K. (2005). Competitive employment for people with autism: Correlates of successful closure in competitive and supported employment. Rehabilitation Counselling Bulletin, 49(1), 4-16. doi:10.1177/00343552050490010201 Seltzer, M. M., Shattuck, P., Abbeduto, L., & Greenberg, J. S. (2004). Trajectory of development in adolescents and adults with autism. Mental Retardation and Developmental Disabilities Research Reviews, 10(4), 234-247. doi:10.1002/mrdd.20038 Shattuck, P. T., Narendorf, S. C., Cooper, B., Sterzing, P. R., Wagner, M., & Taylor, J. L. (2012). Postsecondary education and employment among youth with an autism spectrum disorder. Pediatrics, 129(6), 1042-1049. doi:10.1542/peds.2011-2864 Shattuck, P. T., Wagner, M., Narendorf, S., Sterzing, P., & Hensley, M. (2011). Post–high school service use among young adults with an autism spectrum disorder. Archives of Pediatrics and Adolescent Medicine, 165(2), 141-146. doi:10.1001/archpediatrics.2010.279 Sparrow, S. S., Balla, D. A., & Cicchetti, D. V. (1985). Vineland adaptive behaviour scales- Classroom edition manual. Circle Pines, MN: American Guidance Service. Swisher, J. D., Green, S. B., & Tollefson, N. (1999). Using curriculum-embedded assessment for making educational decisions: An empirical study with implications for including students with disabilities in accountability. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal, QC. Taylor, J. L., & Seltzer, M. M. (2012). Developing a vocational index for adults with autism spectrum disorders. Journal of Autism and Developmental Disorders, 42(12), 2669-2679. doi:10.1007/s10803-012-1524-x Taylor, J. L., McPheeters, M. L., Sathe, N. A., Dove, D., Veenstra-VanderWeele, J., & Warren, Z. (2012). A systematic review of vocational interventions for young adults with autism spectrum disorders. Pediatrics, 130(3), 531-538. doi:10.1542/peds.2012-0682 Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 16 Taylor, J. L., & Seltzer, M. M. (2011). Employment and post-secondary educational activities for young adults with autism spectrum disorders during the transition to adulthood. Journal of Autism and Developmental Disorders, 41(5), 566-574. doi:10.1007/s10803-010-1070-3 Wechsler, D. (2012). Wechsler Abbreviated Scale of Intelligence – Second Edition. San Antonio, TX: The Psychological Corporation. Wechsler, D. (2008). Wechsler adult intelligence scale–Fourth Edition (WAIS–IV). San Antonio, TX: NCS Pearson. Wechsler, D. (2003). Wechsler intelligence scale for children–Fourth Edition (WISC-IV). San Antonio, TX: The Psychological Corporation. Wehman, P., Targett, P. S., Schall, C., & Carr, S. (2014). Meaningful employment for individuals with autism spectrum disorders. In M. Tincani & A. Bondy (Eds.)., Autism Spectrum Disorders in Adolescents and Adults. New York, NY; Guilford. Wehman, P., Lau, S., Molinelli, A., Brookie, V., Thompson, K., Moore, C., & West, M. (2012). Supported employment for young adults with autism spectrum disorders: Preliminary data. Research and Practice for Persons with Severe Disabilities, 37(3), 160-169. doi:10.2511/027494812804153606 Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 17 Appendix A: Attendee Interview Guide Thank you for agreeing to participate in this research project and to talk with me today. I would like to talk with you about your experience of having been in the PAES program. Although I would like this to be more like a conversation than an interview, I do have a few questions written down to make sure I cover everything. If you feel uncomfortable with any of the questions, you can choose not to answer that question and you do have the right to stop the conversation at any point. 1. What was it like for you to participate in the PAES program? Prompts if needed: What did you like? Not like? 2. What did you learn in the PAES program? Prompt if needed: Any new skills? 3. Has the way you get along with people changed as a result of the program? Prompt if needed: How so? 4. Has anything changed in your daily routine since you attended PAES? 5. Has the way you think about yourself changed this coming to the program? 6. What are the strengths of the program? 7. What are the limitations? 8. Do you have any suggestions on the way the program could be improved? Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 18 Appendix B: Parent Interview Guide Thank you for agreeing to participate in this research project and to talk with me today. I would like to talk with you about your experience of having your son (daughter) participate in the PAES program. Although I would like this to be more like a conversation than an interview, I do have a few questions written down to make sure I cover everything. If you feel uncomfortable with any of the questions, you can choose not to answer that question and you do have the right to stop the conversation at any point. 1. What was it like for you to have your son (daughter) participate in the PAES program? Prompts if needed: What did you like? Not like? 2. What did your son (daughter) learn in the PAES program? Prompt if needed: Any new skills? Did you notice any change in level of his (her) motivation? Changes in level of independence? 3. Has the way your son (daughter) gets along with people changed as a result of the program? Prompt if needed: How so? 4. Has anything changed in your son’s (daughter’s) daily routine since he (she) attended PAES? 5. Has the way your son (daughter) thinks about himself (herself) changed since attending the program? Prompt if needed: Self-esteem? 6. From your perspective what are the strengths of the program? 7. What are the limitations? 8. Do you have any suggestions on the way the program could be improved? Smith-Demers, McCrimmon, Shaw/ Emerging Perspectives (2017) 1-19 19 Appendix C: Facilitator Interview Guide Thank you for agreeing to participate in this research project and to talk with me today. I would like to talk with you about your experience of working in the PAES program. Although I would like this to be more like a conversation than an interview, I do have a few questions written down to make sure I cover everything. If you feel uncomfortable with any of the questions, you can choose not to answer that question and you do have the right to stop the conversation at any point. 1. How well do you think the PAES program works with the ASD population? 2. What changes have you noticed in your participants as they go through the program? Prompts if required? Self-esteem? Social behaviour? Communication? Workplace behaviour? Motivation? Problem solving? Independence? Repetitive behaviours? Anxiety? Behaviour problems? 3. From your perspective what are the strengths of the program? 4. What are the limitations? 5. Do you have any suggestions on the way the program could be improved?