Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 1 Bridging the gap between markers’ tacit knowledge and students’ assessment literacy Dr Gemma Mansi University of Greenwich, UK Abstract There has been an increasing emphasis within higher education on the need for explicit articulation of assessment standards and requirements and for these to be communicated effectively to students (O’Donovan et al., 2004; Sadler, 2010; Bloxham et al., 2011; McGrath et al., 2019). However, in practice, this can be difficult to achieve, depending, firstly, on markers’ abilities to effectively articulate their tacit knowledge and expectations and, secondly, on the knowledge and understanding of assessments that students bring with them into their degree. This student ability often depends on their previous varied educational experiences. Both students and staff may therefore find this a matter of challenge and frustration. This case study is based on discussions with students on the Childhood and Youth Studies (Extended) programme at the University of Greenwich and with staff across the University, as well as on the researcher’s personal reflections on the findings. It highlights: how important it is for staff to work in partnership with students early on in their studies, so as to develop their understanding of assessment language and the standards expected; how staff consider the experiences students have had with assessment and academic writing; and how they draw implications for assessment practices in their teaching context. Furthermore, the study aims to provide innovative recommendations for how markers can develop a continuing, meaningful dialogue with students, to enable them to build an understanding of their markers’ tacit knowledge about assessment and feedback in their discipline. Keywords: tacit knowledge, student feedback, criteria, dialogue, assessment writing Introduction There has been significant development in assessment materials across programmes at the University of Greenwich, particularly since the introduction of a new ‘Feedback and Assessment Policy’ in 2019. Assessment-specific grading criteria, rubrics, assignment briefs and a standardised format for student feedback have all been introduced to deliver a more detailed and specific understanding of what is expected from student assignments. However, there remains contention among markers and students over how the markers communicate to their students the tacit knowledge they hold about how to write their assessments. Nonaka et al. (1996) define ‘tacit knowledge’ as ‘personal knowledge’ – often derived from individual beliefs, values, hunches, intuition and experiences – which is therefore difficult to articulate and capture in language in order to transmit it to others. Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 2 This problem tends to be most apparent amongst first-year undergraduate students. The contention manifests itself as a strong underlying frustration that tacit knowledge is a significant barrier to student satisfaction – with consequent harm to NSS scores relating to assessment and feedback (Elton, 2010), not to mention hampering student achievement itself. Many authors have discussed how assessment expectations and standards may be communicated to students. Polanyi (1962) used the metaphor of apprenticeship to suggest how tacit knowledge may be delivered from person to person and suggests that this happens through the learner’s emulating the examples of the teacher. Subsequently ‘the apprentice’ unconsciously begins to pick up on the ‘rules of the trade’. Polanyi also suggested that the ‘hidden rules’ of assessment language vary according to the discipline. Elton (op.cit.), concurs with this point, that, for assessment language to have meaning, it must be embedded within a disciplinary context. Similarly, McGrath et al. (2019) state that assessment literacy and knowledge of subject matter must be developed together. They argue that academic skills support offered outside the discipline may well not assist within it the acquisition of the tacit knowledge needed for success. Carless and Boud (2018) recommend that students are provided with opportunities within taught sessions to engage actively with marking criteria before they attempt assessments. They argue that this approach is more likely to be effective in developing students’ assessment literacy than would be the provision of, say, more extensive feedback, because students may also lack full understanding of feedback language. There is therefore a real risk that feedback will be ineffectual because students may fail to grasp the implications of statements made within it (Sadler, 2010). The lack of time currently dedicated to supporting the development of academic literacy poses a challenge when module teaching remains primarily focused on the delivery of subject knowledge. Subject specialists may also lack the expertise required to deliver assessment literacy effectively. The aim of delivering subject-specific teaching and academic skills teaching together may thus be difficult to achieve (McGrath et al., op.cit.). Most recently, Wollscheid et al. (2021) discuss the perspective of staff in relation to student preparedness for higher education (HE). They claim that staff tend to focus on developing the student from the perspective of student failure rather than on building students’ understanding gradually across the modules, which would be more empowering for the student. It is argued that this ‘deficit’ model of assessment literacy may follow from staff assumptions that students already know and understand academic expectations in HE and, if not, should themselves take responsibility for learning about it. However, where there is tacit assessment knowledge that is inaccessible to students, the barrier remains. Sadler (2005) adds to this debate by arguing that, despite the use of marking criteria, the final judgements that markers make about the quality of student work can remain subjective and largely hidden from the students. Furthermore, there is evidence to suggest that many markers do not necessarily apply the grading criteria in the way intended and instead use them post hoc to justify a holistic decision and refine written feedback (Bloxham et al., 2011). In addition, O’Donovan et al (2004) points out that there is potential for inconsistencies in Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 3 tacit knowledge held within and between marking teams. Again, these points suggest that markers’ failure to make explicit their tacit knowledge may mean that there is a gap between their understanding and that of their students. The aim of this case study was to explore the informal narratives of students and teaching staff about the language used in assessments and to discuss pilot interventions designed to help bridge the gap between how staff articulate their expectations and instructions and how these are understood by students, particularly if they have recently joined university. Rationale for the case study Since 2016, I have been programme leader, module leader and personal tutor on the BA Childhood and Youth Studies (Extended) (CYS) at the University of Greenwich. During that time, I have become increasingly aware of a sense of anxiety and confusion among students while discussing assessments and standards with them, particularly in their first year of university. During group tutorials, I deliberately aired what I felt I was observing in the class and the students seemed relieved that a member of staff had noticed and understood what was becoming a real frustration for them. As a result, I wanted to explore, through informal student group discussions, the issues that were causing these feelings about assessment. I also arranged to carry out informal meetings with staff across the University who worked on extended programmes, the better to understand their experiences of working on assessments with their students. The entry route into university for the majority of the CYS (Extended) cohort is BTEC in Health and Social Care. The BTEC national is one of the most widely recognised qualifications for admission to HE in the United Kingdom (UK) and is one of the top three qualifications used by applicants to enter the University of Greenwich. BTEC is a vocational qualification and is focused on work-related learning. Assessments normally focus on research projects and case studies, giving students an opportunity to apply theory, concepts and skills to real situations (Pearson, 2019). The programme specification emphasises the development of assessment skills as part of preparation for university (Pearson, 2019). It certainly appears that the BTEC specification at level three addresses the transferable skills necessary for a student to thrive at university. However, there remains some form of assessment challenge for students when it comes to assessment. This research was therefore aimed at exploring student and staff narratives about what they felt the issues were for students making their academic transition to university and what we could do as teaching staff to address some of those issues. The findings of this case study were also intended to help identify strategies for supporting students across the University who have also come from BTEC backgrounds. Method Student participants Participants in the informal student discussion groups were drawn from the student cohort on the CYS (Extended) programme. This cohort is predominately female, aged 18-24, and the majority are from a black, Asian and minority ethnic (BAME) background. All students from year zero (level three) and year one (level four) of the CYS (Extended) programme were invited to participate in the study and all twenty-one agreed to participate. Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 4 The data collection took place via informal group discussions for each year group, so allowing students to share their feelings and opinions about understanding assessments at university. To facilitate the discussion and enable students to identify specific examples of guidance that they did not understand, module assessment briefs and subject-specific grading criteria were provided at the meetings. Data collection and findings were based on my own reflective note-taking and my own personal narrative of student discussions. The personal narrative was based on key points highlighted by students across the CYS (Extended) programme. Staff participants Participants of the informal staff discussion group were four module leaders from across the University who had worked with extended degree students. I met with colleagues separately and data collection was based on personal reflective note-taking on those meetings. Results Student responses It became evident relatively early in the discussions that assessment materials alone, particularly the grading criteria, can be relatively meaningless to students. The students identified that numerical grading criteria were new to them. For example, students who had undertaken a BTEC said that the criteria they were used to took the form of pass, merit and distinction; a numerical scale broken down into sub-sections for each criterion was not something they were accustomed to and they felt it to be too complex for complete understanding. In support of these findings, Bloxham et al. (op.cit.) noted that, regardless of whether criteria are explicitly stated, their accessibility is restricted when there is little help to understand and contextualise the language used. Nevertheless, students reported that they particularly liked the visual presentation of the criteria on the rubric which identified strengths and areas for improvement, providing a clearer picture of feed-forward. In addition, students mentioned that some of the language used in the criteria was unclear to them. Examples included terms such as ‘analysis’, ‘argument’ and ‘evaluation’. Students noted that teaching staff regularly mentioned these terms in their assessment preparation sessions but felt there was minimal explanation of what they meant and a lack of concrete examples to illustrate how they would be displayed in the assessment. This is consistent with the comment by Wollscheid et al. (op.cit.), that academic staff may see assessment preparedness as primarily the responsibility of students. Subsequently, it became yet more frustrating for students when such terms were used in written feedback as points needing further development, when, at all stages of discussing the assessment, the students felt they were no clearer about what the terms actually meant. These responses are consistent with O’Donovan et al. (op.cit.) who concluded that assessment feedback regularly highlights what the assessment doesn’t do and provides little in the way of explicit examples revealing how assessment terms could be demonstrated within the context of the specific assessment. This can be a constant frustration for students, given they have quite a limited time to construct and adapt various interpretations of what is required for different assessments and for different assessors. Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 5 However, regardless of the types of material or their detail, what was becoming increasingly evident from the students was that the degree of effectiveness of the communication of the marker’s tacit knowledge about key assessment terms had the greatest bearing on their understanding of the assignment. Staff responses The staff were concerned that too many assessment materials and too much support offered about the assignment led to students’ becoming increasingly literal and prescriptive in their approach to their assignments. There was concern that student engagement with the content could be lost to focusing on the requirements only of the assessment. A more holistic approach to judging assessments was therefore considered preferable, as enabling more student autonomy over the work. Typically, grading criteria take either a holistic or analytic approach. A holistic rubric requires teachers to score the overall assessment intuitively, while an analytic rubric requires markers to score separate, individual parts of the assessment, before summing the individual scores to obtain a total (Ragupathi and Lee, 2019). The typical form of assessment for first-year extended students within the areas of business and engineering was project-based learning. Staff felt that one of the main reasons behind using projects was that it enabled the students to be assessed in a way already familiar to them from their BTEC studies. Staff were very mindful of students’ previous experiences of assessment and of the need to support them in making a successful transition and giving them early opportunities to succeed. They felt that research projects were very helpful in doing that. Similarly, presentations were regularly used in the assessment diet of first-year extended students, because it also was a form of assessment that students had experienced during their previous studies. Myhill (2020) agrees with this assessment strategy, identifying that universities need to ensure a greater diversity of assessment and reduce the more traditional forms of assessment, such as written essays and exams, to avoid disadvantaging students whose further education assessments were more varied in kind. Staff emphasised that both projects and presentations tended to produce good results and students reported that they particularly enjoyed this type of assessment. Staff also noted that they did not perceive significant differences in the extended student’s ability to pick up on knowledge and understanding of the content itself; in fact, they felt it was very similar to that of students who entered the programme at level four. Similarly, neither level 3 or level 4 entry students necessarily have previous experience or knowledge of the discipline. However, staff tended to find differences in ability between levels three and four in relation to academic skills. Programme teams working with extended students therefore had specific discipline-based interventions to address the development of academic skills, which, in accordance with the literature, is the most effective way to support students in developing their assessment skills in line with their subject (Elton, op.cit.). Interestingly, in engineering, at the end of the extended degree programme, there were positive upward trends in terms of assessment outcomes for extended students compared to those studying on the three-year undergraduate programme who had not received the foundation year focused specifically on developing academic skills. Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 6 Discussion Based on the findings, there is a clear argument that moving students from tacit to explicit knowledge of assessment expectations within a specific discipline goes beyond just telling them what to do. The process requires a shift towards thinking about the assessment type – in terms of how it supports students’ learning, particularly at key transition points – and building in opportunities for students to practise and develop their assessment literacy, enabling them to develop the ability themselves to make complex appraisals of different types of assessment (Sadler, 2010). Following the knowledge conversion model of Nonaka et al. (1996), tacit knowledge can be passed from one person to another through ‘socialisation’. In the context of assessment, this would involve open dialogue between markers and students. ‘Externalisation’ is the process of making tacit knowledge explicit and this requires markers to articulate their standards and expectations. Through `internalisation’, which is experiential and requires active engagement, the learner understands and absorbs the explicit knowledge and can now act on it. Finally, ‘combination’ commences where explicit knowledge can be articulated in different formats. Successful interventions from this case study are consistent with this approach to thinking about sharing tacit knowledge. They include directed class time to discuss the assessment materials and opportunities for students to discuss and practise marking previous student exemplars, so enabling markers and students to share beliefs and learn how better to articulate their thinking through practice, instant feedback and simultaneous exchange of ideas. The findings also point to the need to maintain dialogue about written feedback, as this was another area where students needed opportunities to access the marker’s tacit knowledge. One possible approach to this is to ask students to complete a form evaluating how well they feel they have achieved in each of their assessments; the marker then comments on the same form how well s/he feels the student has met a particular objective or skill. Both sides then discuss these in a tutorial session. While the findings in this study provide an important picture of staff and student views in relation to tacit knowledge throughout the assessment and marking process, a degree of caution is inevitably needed, given the small sample size, deriving as it does from one undergraduate degree programme at one university. A recommendation for the future would be a more extensive and systematic investigation into the topic, to investigate whether the findings are generalisable. Nevertheless, the findings provided here largely reflect wider concerns identified within the literature. There has been significant development of assessment resources at the University of Greenwich. However, what this case study illustrates is that such resources in isolation can hold little value for students. Engaging students in a meaningful way, with activities that support the development of assessment literacy within the discipline, is needed to bridge the gap between staff expectations and students’ understanding of what exactly is required of them in their assessments. Delivering a clear and simple explanation of tacit knowledge is by no means easy and having academics work alongside central academic skills services and ensuring standardisation amongst the marking team, to enable a dialogue and Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 7 consensus about criteria, are essential. Also, providing opportunities within modules where students can engage actively in the marking process not only develops their awareness and understanding of tacit knowledge but also broadens the assessment, from assessment of learning to assessment for learning (O’Donovan et al., op.cit.). This case study suggests that staff need to reflect upon students’ previous educational experiences and the significance of the academic transition for many students. It is evident that many staff are mindful of the types of assessment used in students’ previous study and do include a variety of assessments to support the transition to HE assessment. However, focusing only on the type of assessment and students’ knowledge of subject content is not sufficient. Rather, their development of assessment literacy and what that means within the context of their discipline are key. The findings of this case study are consistent with the literature: that academics have ‘presumed knowledge’ of their students when they enter university (Wollscheid et al., op.cit; McGraph et al., op.cit.) This may explain why there may be a much smaller amount of contact time dedicated to developing academic skills within the discipline. Consequently, staff may be unintentionally missing vital opportunities to work with their students on essential skills, which will help in bridging tacit knowledge between staff and students. Conclusion Based on the findings of this case study, what teaching staff need to consider in the delivery of their modules is a balance between delivering content and developing students’ assessment literacy. Providing tasks which enable students to be active in the marking process and familiarise them with the differences between grade bandings is more productive in its mission for students to be successful than extensive amounts of written feedback to students. Open conversations between markers and their peers continue the dialogue about what expectations are and, through experience, transform tacit knowledge into explicit knowledge which can be acted upon. Reference list Bloxham, S., Boyd, P. and Orr, S. (2011) ‘Mark my words: the role of assessment criteria in UK higher education grading practices.’ Studies in Higher Education, 36(6), 655-670. Available at: https://doi.org/10.1080/03075071003777716 (Accessed: 22 February 2021). Carless, D. and Boud, D. (2018) ‘The development of student feedback literacy: enabling uptake of feedback.’ Assessment & Evaluation in Higher Education, 43(8), 1315-1325. Available at: https://doi.org/10.1080/02602938.2018.1463354 (Accessed: 22 February 2021). Elton, L. (2010) ‘Academic writing and tacit knowledge.’ Teaching in Higher Education,15(2), 151-160. Available at: https://doi.org/10.1080/13562511003619979 (Accessed: 12 March 2021). McGrath, L., Negretti, R. and Nicholls, K. (2019) ‘Hidden expectations: scaffolding subject specialists’ genre knowledge of the assignments they set.’ Higher Education, 78(5), 835- 853. Available at: https://doi.org/10.1007/s10734-019-00373-9 (Accessed: 12 March 2021). https://doi.org/10.1080/03075071003777716 https://doi.org/10.1080/02602938.2018.1463354 https://doi.org/10.1080/13562511003619979 https://doi.org/10.1007/s10734-019-00373-9 Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 8 Myhill, D. (2020) Transforming transitions, Addressing Barriers to Student Success: Final report to the Office for Students, Office for Students. Available at: ABSS Project Final Report (officeforstudents.org.uk) (Accessed: 17 March 2021). Nonaka, L., Takeuchi, H. and Umemoto, K. (1996) ‘A theory of organizational knowledge creation.’ International Journal of Technology Management, 11(7-8), 833-845. Available at: https://doi.org/10.1177/0170840606066312 (Accessed: 23 February 2021). O'Donovan, B., Price, M. and Rust, C. (2004) ‘Know what I mean? Enhancing student understanding of assessment standards and criteria.’ Teaching in Higher education, 9(3), 325-335. Available at: https://doi.org/10.1080/1356251042000216642 (Accessed: 26 March 2021). Pearson (2019) BTEC Level 3 National Certificate in Health and Social Care, Specification, Issue 6. London: Pearson Education Limited. ISBN: 978-1-446-95089-0 Polanyi, M. (1962) ‘Tacit knowing: Its bearing on some problems of philosophy.’ Reviews of modern physics, 34(4), 601. Available at: https://doi.org/10.1103/RevModPhys.34.601 (Accessed: 26 March 2021). Ragupathi, K. and Lee, A. (2019) ‘Beyond Fairness and Consistency in Grading: The Role of Rubrics in Higher Education.’ In: Sanger, C.S. and Gleason, N.W. (eds.) Diversity and Inclusion in Global Higher Education, lessons from across Asia. Singapore: Palgrave Macmillan. ISBN: 978-981-15-1628-3 Sadler, D.R. (2010) ‘Beyond feedback: Developing student capability in complex appraisal.’ Assessment & Evaluation in Higher Education, 35(5), 535-550. Available at: https://doi.org/10.1080/02602930903541015 (Accessed: 14 March 2021). Sadler, D.R. (2005) ‘Interpretations of criteria‐based assessment and grading in higher education.’ Assessment & evaluation in higher education, 30(2), 175-194. Available at: https://doi.org/10.1080/0260293042000264262 (Accessed: 14 March 2021). University of Greenwich (2019) Assessment and Feedback Policy, Available at: Assessment-and-Feedback-Policy-Feb-2020-with-contents-table.pdf (gre.ac.uk) (Accessed: 12 March 2021). Wollscheid, S., Lødding, B. and Aamodt, P.O. (2021) ‘Prepared for higher education? Staff and student perceptions of academic literacy dimensions across disciplines.’ Quality in Higher Education, 27(1), 1-20. Available at: https://doi.org/10.1080/13538322/2021.1830534 (Accessed: 19 April 2021). https://www.officeforstudents.org.uk/media/f65a2a58-da6b-4b37-8130-62b161f17638/abss-transforming-transitions-exeter-report.pdf https://www.officeforstudents.org.uk/media/f65a2a58-da6b-4b37-8130-62b161f17638/abss-transforming-transitions-exeter-report.pdf https://doi.org/10.1177%2F0170840606066312 https://doi.org/10.1080/1356251042000216642 https://doi.org/10.1103/RevModPhys.34.601 https://doi.org/10.1080/02602930903541015 https://doi.org/10.1080/0260293042000264262 https://docs.gre.ac.uk/__data/assets/pdf_file/0028/137827/Assessment-and-Feedback-Policy-Feb-2020-with-contents-table.pdf https://doi.org/10.1080/13538322/2021.1830534