*Corresponding author - rebecca.wilsonmah@royalroads.ca Wilson-Mah, R. (2019). A study of authentic assessment in an internship course. Papers on Postsecondary Learning and Teaching: Proceedings of the University of Calgary Conference on Learning and Teaching, 3, 65-73. A STUDY OF AUTHENTIC ASSESSMENT IN AN INTERNSHIP COURSE Rebecca Wilson-Mah* Royal Roads University Internship courses are an active and applied opportunity to integrate educational learning programs and courses within practice settings. Individual learners in internship courses have different work internship placements and they start a unique professional and practice journey. How can this authentic learning environment be supported with authentic assessment? What assessments are suitable for internship courses? This study explored authentic assessment concepts and the practice of internship assessment in a graduate internship course in international and intercultural communications. Students’ impressions of their internship assessments are explored with a particular focus on students’ impressions of authentic assessment, and the integration of their education and their real-world experience. To link this research to course improvement, this study also includes the instructor’s critical reflection on the course and her plans for integrating the study into her assessment practice. The purpose of this study was to answer the question: Are the design of course assessments in an internship course authentic? This question was examined by inquiring into the learners’ perspectives on the authenticity of their internship assessments. Institutions are increasingly recognizing the value of internships for their students (NACE, 2014). In internships, learning is primarily situated within a workplace. Learning is typically supported by an online internship course, learning outcomes, readings, and assessment. The learner, the internship faculty advisor, and the workplace supervisor/mentor cooperate to support a productive learning outcome (Benavides, Dicke & Holt, 2013). An internship offers an individualized context for learning. The internship organization is chosen by the learner, typically through a competitive selection process. The internship reflects each learner’s interests, capabilities and experience and the employer’s particular needs, operating environment and culture. The tasks, social context and physical context of learning are authentic as they take place in the real-world work environment (Gulikers, Bastiaens & Kirschner, 2004). Related to this authentic learning space are the graded assessments that are required for course completion. Given the individualized nature of an internship, and the opportunity to learn in a real-world setting, the internship course presents an interesting space to investigate learners’ perceptions of authentic assessment. After over a decade supporting students in internships the researcher sought to examine principles of authentic assessment, and to reflect on the design of her internship course assessments. Internships offer an immersive experiential learning context, and a particularly suitable opportunity for authentic assessment. From a practice perspective the researcher sought to complete a reflective ‘check in’ on the appropriateness of her assessment processes and practices. This researcher’s reflective process of intentionally scrutinizing assessment design, Wilson-Mah (2019) 66 practice and process is complemented by a qualitative study of graduate students’ experience with assessment in an internship course. AUTHENTIC ASSESSMENT Any discussion on assessment has to start by reflecting on our perspective and practices (Boud, 2007). What is the purpose of assessment? What are we currently doing with assessment? How does our thought process around the role of education and assessment affect how we discuss, design, apply and evaluate the assessments we use? These reflective questions were the impetus for this study, encouraging me to think deeply about my internship assessments, feedback to learners and general approach to supporting learning through assessment. There are multiple definitions of authentic assessment; authenticity is subjective, and learners and instructors often perceived authenticity differently (Gulikers, Kester, Kirschner & Bastiaens, 2008). The definition of authentic assessment applied for this study is an approach to assessment that focuses on assessing the application of knowledge and skills to perform real- world tasks (Mueller, 2005). The alignment of knowledge and skills in a course with activities and tasks that occur in a real-world setting (Boud, 2007) offers a knowledge to practice learning focus. Authentic assessment is typically embedded in authentic instruction and learning, where there is a focus on, for example, real-world problems, practices, planning, self-awareness and regulation, judgement (Biggs, 1996, Boud, 2007) decision making case studies and learning communities (Lombardi, 2007). An alignment between instruction, learning, and assessment (Biggs, 1996) is both pragmatic and sensible, yet developing a cohesive instructional approach within a course necessitates care, thought, reflection, attention and time. An incompatibility in instructional approach (instruction, learning and assessment) can negatively impact student learning and lead to confusion and distress (Gulikers et al., 2004). As noted, there are multiple approaches to authentic assessment. On review of different studies on authentic assessment the following common distinguishing features of authentic assessment held some congruence to the approach I had developed for the internship course. 1. Aligns. Authentic assessment aligns with approaches to instruction and learning; the instructor is intentional about their choices in these areas (Lindstrom, Taylor, & Weleschuck, 2017). The assessments in the internship course were developed to align with the real-world context of the internship. For example, the course is online and asynchronous, so learners can proceed to accommodate their schedule and work commitments. 2. Integrates. Authentic assessment incorporates summative and formative assessment activities (Lombardi, 2007) with a focus is on supporting student capacity to receive and integrate feedback, adapt, and continuously improve after the course ends (Boud, 2000). There were two forms of formative and summative assessment in the internship course. The learner and the learner’s supervisor completed a parallel evaluation on learner progress in the internship at the mid-way point in the internship, and on the conclusion of the internship. 3. Supports the needs of learners. The foundation of authentic assessment is an engagement in learner centred approaches (Boud, 2007). These approaches may include encouraging active learning, inviting input from students on assessments and rubrics, and the careful consideration of how the assessment links to the attainment and demonstration of course learning outcomes. Engagement in learner centred approaches in internship included requiring learners to have a progress discussion mid-way through the internship Wilson-Mah (2019) 67 with both their supervisor and their faculty advisor. This conversation was typically completed using web-based conferencing. 4. Considers context. There are multiple ways to develop authentic assessments that are situated appropriately to the discipline, course, knowledge level of learners, professional practices and other variables (Brown, 2005). Advocating for expanding our view of the context in which learning occurs, Boud (2007) challenged the boundaries of assessment traditionally inside a course, to expand assessment outside the course into the real world of practice. Boud proposed that assessment should be re-framed to be a conversation about the “formation of a capable person who can engage in professional work and contribute to society as an informed citizen” (2007, p. 30). The learners completed their internship course in a real world, professional context. Given a purposeful approach to create authentic assessment for the internship course, I sought the opportunity to hear directly from the students and to deeply appreciate the relationship between the assessments and their internship context. Authentic Assessment in Internships The graduate learners in this study were completing an MA in Intercultural and International Communications (MAIIC). The six-credit internship course was situated in actual workplaces related to the field and discipline of communications, and each learner had their own internship host. Examples of internships hosts included Shaw TV, Diabetes Canada and Vancouver Island South Film and Media Commission. The course was designed to support learners to apply theoretical knowledge and professional skills in a supervised, professional, domestic or international context relevant to their program of study. Learners were required to complete a minimum of 420 contact hours with their host organization, normally completed over a period of 3 to 5 months and each assessment was graded. The broad aim of the course was to “bring classroom learning, theoretical concepts and academic literature together with practical learning, insights from the field of practice and application of skills” (Royal Roads University, 2017). The internship course assessment included six different assessments as follows: • Asssessment One: Cultural mapping exercise (300 words) and an internship plan. The internship plan required the student to make their own decision on assignment due dates. • Assessment Two: Spotlight on a key challenge in practice. This assessment included participation in a preliminary team forum discussion on challenges in practice and an individual summary analysis of a key challenge. • Assessment Three: Mid-internship assessment of progress. This was completed by the learner and also by the learner’s supervisor. The faculty advisor facilitated a meeting between the learner and the supervisor to review progress. • Assessment Four: An analysis of experience learning. This was the most significant assessment representing 40% of the course grade. The report was submitted either as an academic paper, or as a digital presentation or visual story board with accompanying narrative. This assessment focused on exploring connections the learner made between skills and knowledge developed in the program and real-world experience in internship. • Assessment Five: Professional development plan and personal reflection. – The learner was asked to share a reflective account of 1) what they learned during the internship and 2) future professional plans. • Assessment Six: Final supervisor evaluation of student progress and performance. Wilson-Mah (2019) 68 Returning to Mueller’s (2005) definition of authentic assessment as assessment of the application of knowledge and skills to perform real-world tasks, an internship accommodates authentic assessment offing a real site where learners experience a professional workplace over an extended length of time (Maxwell, 2012). In their internships, learners extend their knowledge through action, develop their situated practical knowledge, and experience the opportunity to explore their own agency as members of an organization (Mueller, 2005). Furthermore, learners build professional relationships with their supervisor, clients and colleagues. The challenge is to ensure that assessments reinforce and support the creation of linkages between what has been learned in an academic program and professional work practices and experiences (Gulikers et al., 2008). RESEARCH DESIGN The purpose of this qualitative study was to explore students’ perceptions of authentic assessment in their MAIIC graduate internship course. This research proceeded after the researcher received ethical approval through her university. The focus of the inquiry was to encourage the students to share their experience in the internship course and placement and to reflect on the authenticity of the assessments that they completed. In addition to an analysis of learner perspectives on authentic assessment, the inquiry also included the researcher’s critical reflection on the content of her assessments, and her experience facilitating the internship course. The research questions were: 1) When you consider an authentic assessment of your real- world experience in internship what do you think about? 2) To improve authenticity in assessment, what do you recommend? Data and Sample A purposeful sampling strategy was used to invite students to participate in the study. The sample for this qualitative study were graduate students who had recently completed their six- credit internship course. The students were in a Master of Arts in Intercultural and International Communication (MAIIC). The program was 36 credits in total and the internship course was the penultimate course in the program. Method of Data Collection and Analysis One focus group was conducted in March 2018 on campus. The learners in the small focus group (4 students) were on-campus completing their MAIIC studies. The internship course had finished, and all marks had been posted. The internship course enrollment was 10 participants in total; the sample size represented 40%. A focus group inquiry was conducted to gain insight into the learners’ perceptions and to encourage research participants to share and reflect with one another on each of the research questions (Breen, 2006). The focus group was approximately 60 minutes and the discussions were recorded and transcribed. The focus group schedule included the following stages: welcome, overview of authentic assessment, sharing a definition of authentic assessment, and discussion of the ground rules for the focus group including confidentiality. For research question one, the group prompt was: If you are completing an authentic assessment in an internship course what are you assessing? Responses were captured on a white board. To inquire Wilson-Mah (2019) 69 into research question two, the participants reviewed the assessment outlines. They were asked to share their perception of the relative authenticity of each assessment and to make recommendations for improvement. This was a small study with four participants in one focus group. An analysis of the focus group transcript was conducted to complete an analysis of 1) individual contribution and 2) the contribution of all group members. It was noticed that when responding to the questions, there was a high level of a agreement within the group. All specific or intensive input (Breen, 2006) from the participants was noted as significant and this data is reflected in the findings and discussion. FINDINGS AND DISCUSSION Real-world Experience In an opening discussion on the content of authentic assessment for internship the four participants all emphasized the relationship between the assessment and real-world experience “Something that relates to the real-world, not something strictly academic”. When asked to elaborate with examples of real-world experience that was particularly poignant to them in internship, the students discussed: adapting to their work culture and environment, for example, moulding oneself to adjust to other colleagues, staying calm, adapting to the pressures of work, and, adjusting to cultural expectations. There was some discussion about ‘fine tuning’ oneself to the work culture and the effect of work culture on you; one participant commented, “I know what culture is from an academic point of view, in the work scenario work culture is different, you have to fine tune to your work culture.” The word adapt and adapting to work, and “rolling with the punches” were other skills that the participants felt were particularly relevant to real world experience. The participants also agreed that soft skills were particularly necessary in the workplace “No matter how much knowledge you have, soft skills like adaptability are important. Soft skills are so important.” The research participants had all completed internships in Canada, and they commented that an intern in Canada has considerable agency over how they do their work “People are trusting here – so you have to roll with it and adapt.” These comments align with Beckett and Hager’s argument that central to workplace learning is the capability to discern and make different kinds of judgements (2000). Relating these experiences back to assessment, the participants agreed that their most significant challenge was with assessments that required the learner to relate intercultural or communication theories to their real-world work-place challenges. They agreed that theories and knowledge are helpful foundations, “The fact you have learned about communication perhaps helps you explore it more differently in your workplace” and “Just knowing a few things before- hand can help”. However, there was a clear sense of struggle or frustration with the assessment(s) that required the participants to apply a theory or practice to an internship related context, “Realistically, I’m not sure application of theory really happens much”. Authentic Assessment The participants reviewed the knowledge domains, learning outcomes and assessment criteria established for the course. The participants agreed that the language used for the assessment criteria was not sufficiently practice based. One participant suggested that the assessment criteria was too impersonal and technical, creating a mis-alignment with the subjective and highly interpersonal nature of communication studies. The participants returned to Wilson-Mah (2019) 70 the focus in the assessments on theoretical linkages between coursework and the internship, “I think our academic courses were helpful and our internship course was helpful yet narrowing and connecting between the two to theoretical linkages is not easy.” On completion of a review of all the assignments the participants gave specific suggestions for changes to improve authenticity as follows: 1. Continue to give us the opportunity to create our own timeline and make it more flexible by creating it in a Wiki so we can adjust our timelines. This is more authentic, and true to life as a professional is constantly adjusting their plans and deliverables to accommodate changing priorities and expectations. One participant suggested a “time container” would be a helpful approach, offering a timeline window for each assignment. 2. Continue to include a class forum, as class-wide discussions are valuable. The class forum discussion in advance of an assignment allows a learner to gain insights and support from fellow learners and this focused communication and encouraged connection. 3. The mid-internship and end of internship progress evaluations from supervisors are very helpful, “I feel like my employer cared enough to complete the evaluation and sit down to discuss it with me and my faculty advisor”. 4. For the highest weighted assessment, an analysis of experience learning, the participants commented that they would like to see less emphasis on theory and more emphasis on sharing, analysing and reflecting on something achieved during the internship, “I would like to include an artifact I made, something I have done”. One participant suggested that the analysis of theoretical linkages to internship required in this assessment could be an option. One participant recalled being approved to include a work project in the assessment submission and noted that sharing something she completed in internship was vital to her, “it made everything feel complete”. 5. The participants agreed that the concluding assignment, a professional development plan and personal reflection tied the course together. One participant noted that “I felt this assignment was for my personal benefit, which I appreciated”. This is a reminder of the significance of timing an assignment appropriately to meet the needs of the learner, and to encourage and support next steps beyond the course (Boud, 2000). INSTRUCTOR’S REFLECTION ON ASSESSMENT IN AN INTERNSHIP COURSE To reflect on assessment in internships I chose to apply Brown’s (2005) ‘fit for purpose’ diagnostic for assessment. Brown suggested that a productive way to think about assessment is to reflect on the “what, why, who and when for assessment content and design” (p.82). This diagnostic is also a practice-based approach to assessing the distinguishing features of assessment including distinguishing features of authentic assessment previously noted: aligns, integrates, supports the needs of learners and considers context. What am I Assessing? The calendar description for this course emphasised the integration of classroom learning, theoretical concepts and academic literature together with practical learning, insights from the field of practice and application of skills. Each learner was required to complete a minimum of 420 contact hours at an approved host organization over a period of 3 to 5 months. This description is relatively succinct. The course assessment outline included, five learning domains, Wilson-Mah (2019) 71 eight learning outcomes and ten assessment criteria. Next time I assess learning in this course… I will reduce the number of learning domains and learning outcomes to focus specifically on internship as a practice-based learning environment. I will write the assessment criteria in clearer, more practice-based terms. In the first week of the course I will also review the assessment criteria with the learners and ask for their impressions on the meaning and content of the course assessment outlines. I will use learner feedback to improve the clarity of the assessment instructions and associated rubrics. How am I Assessing? As noted previously, in the section on authentic assessment in internships, there were six assessments in the internship course. The participants noted that the assessments generally related well to their real-world experience. The main change that the participants sought was an adjustment to assessment four. This assessment was included to ensure the course provided learners the opportunity to relate classroom learning, theory and academic literature with practice learning and insights developed in their internship setting. The participants noted that they struggled to connect theory to practice. Next time I assess learning in this course… For assessment four, I will explore the option of including an assessment that asks learners to submit a substantial work-related project or a selection of smaller projects with a reflective narrative on purpose, process and outcomes. Practice learning will be the core focus of the assessment and linkages to classroom learning, theory and academic literature will be a secondary focus. I am interested to see if a re-positioning of the focus of the assessment will make a difference to the learners’ perceptions of how this particular assessment relates to their professional knowledge and practice. Choice was provided to learners on how to format their response to assessments one, two, four and five. It was interesting to note that over 70% of the learners submitted traditional academic papers. In internships situated in communications, media, marketing and public relations, communications output takes many forms such as info graphics, proposals, communication plans, briefing notes, data analysis, social media, photographs, video and so on. To improve applied practice, I will not give students the option of submitting an academic paper, rather, they will be invited to select a method of communication used in their workplace and adapt that method to complete the assessment. Who is Assessing? The assessments of progress included two supervisor evaluations of student progress in internship, one mid-way through the internship, and another at the end. In most cases, these evaluations were completed carefully by the learner’s supervisor. In addition, in the mid- internship assessment of progress completed by a supervisor, the learners also rate their own progress. The internship advisor facilitated a reflective discussion with the learner to compare the two assessments of progress. Wilson-Mah (2019) 72 Next time I assess learning in this course… I plan to discuss the mid-internship and end of internship evaluation criteria and questions with some supervisors and managers to get their insights on how appropriately the criteria relate to their view of key progress indicators for a learner on an internship. I’m interested to understand what expectations they have of the learners, and in particular, their expectations of learners’ readiness to learn and adapt in the workplace. Furthermore, I plan to invite learners to involve their supervisor in determining a suitable work-related project for assignment four, thus increasing collaboration with the host supervisor (Benavides et al., 2013). When am I Assessing? The learners were given choice to create their own course progression timeline, including self- assignment of assessment due dates. Next time I assess learning in this course… For this course, I would like to create a balance between flexibility and structure. This will be accomplished by providing learners with a ‘window’ of time for each assessment. Within this window, the learner will assign their own assessment due date. I will implement the research participants’ suggestions to include a Wiki in the course, so the learners can easily adjust and update timelines. RECOMMENDATIONS: APPROACHES TO SUPPORT AUTHENTIC ASSESSMENT IN INTERNSHIP COURSES Based on my experience, focus group inquiry, and reflection there are several general recommendations for those wishing to develop authentic assessment in internship courses: ▪ Connect your course assessment to the learning outcomes assigned to the course and focus on no more than 4-5 learning outcomes. Select learning outcomes that relate to self-assessment, self-awareness, reflection, application, work-related practice and output, adaptability, work-related decisions and judgement and/or work-related process improvements. ▪ Align assessment to real-world work. Participants shared some apprehension regarding the feasibility of applying academic theories they had learned in their courses to the internship context. The participants noted that they had struggled to draw on theory in internships, preferring a more practice-based approach to learning, including working in teams, reflective analysis and internship-related projects. ▪ Give learners the opportunity to profile their work and accomplishments during the internship period. Learners appreciate the feedback and recognition that is associated with profiling what they have done and achieved. ▪ Provide flexibility. A course and an internship can be challenging for a learner. There is significant adjustment in a new organization and time is needed to settle in to the role, culture, tasks and relationships. Give the learner some choice around their assessment due dates so they can balance their commitments. ▪ Check in with learners regularly. The participants in this study expressed their gratitude to me for spending time to listen to their perspectives and feedback. The process enabled me to validate some of my assessments and to also understand the gaps and opportunities available for improvement. My reflection on practice, and the study data Wilson-Mah (2019) 73 both contributed to a deeper understanding of authentic assessment culminating in a practice-based plan for making improvements. REFERENCES Beckett, D., & Hager, P. (2000). Making judgments as the basis for workplace learning: Towards an epistemology of practice. International Journal of Lifelong Education, 19, 300-311. Benavides, A. D., Dicke, L. A., & Holt, A. C. (2013). Internships adrift? Anchoring internship programs in collaboration. Journal of Public Affairs Education, 19, 325-353. doi.org/10.1080/15236803.2013.12001736 Biggs, J. (1996). Assessing learning quality: Reconciling institutional, staff and educational demands. Assessment and Evaluation in Higher Education, 21(1), 5-16. Boud, D. (2007). Reframing assessment as if learning were important. In D. Boud & N. Falhikov (Eds.) Rethinking assessment in higher education: Learning for the longer term (pp. 14- 26). New York, NY: Routledge. Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22, 151-167. Breen, R. L. (2006). A practical guide to focus-group research. Journal of Geography in Higher Education, 30, 463-475. Brown, S. (2005). Assessment for learning. Learning and Teaching in Higher Education, (1), 81- 89. Gulikers, J. T., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52(3), 67-86. Gulikers, J. T., Kester, L., Kirschner, P. A., & Bastiaens, T. J. (2008). The effect of practical experience on perceptions of assessment authenticity, study approach, and learning outcomes. Learning and Instruction, 18(2), 172-186. https://doi.org/10.1016/j.learninstruc.2007.02.012 Lindstrom, G., Taylor, L., & Weleschuk, A. (2017). Guiding Principles for Assessment of Students’ Learning. Calgary, AB: Taylor Institute for Teaching and Learning, University of Calgary. Retrieved from: http://studentassessment.ucalgaryblogs.ca/files/2017/06/Guiding-Principles-for- Assessment-of-Student-Learning-FINAL.pdf Lombardi, M. M. (2007). Authentic learning for the 21st century: An overview. Educause Learning Initiative, 1,1-12. Maxwell, T. W. (2012). Assessment in higher education in the professions: Action research as an authentic assessment task. Teaching in Higher Education, 17, 686-696. https://www.tandfonline.com/doi/ref/10.1080/13562517.2012.725220?scroll=top Mueller, J. (2005). The authentic assessment toolbox: enhancing student learning through online faculty development. Journal of Online Learning and Teaching, 1(1), 1-7. National Association of Colleges and Employers (NACE). (2014). The class of 2014 student survey report. Bethlehem, PA: Author. Royal Roads University. (2017). IICS680: Internship (course outline). Available from moodle.royalroads.ca