TEF Special Edition Compass: Journal of Learning and Teaching, Vol 10, No 2, 2017 The Emperor has no clothes! Chris Rust Abstract This piece argues that an opportunity has been wasted and the proposed metrics and methodology for the TEF will not validly assess teaching excellence. Some years ago now, mid-afternoon in a workshop I was running for new academic staff in a research-intensive university, a reluctant participant said to me, “Well I can see why this would be important if you wanted to improve your teaching.” He, however, was one of many recently appointed for his research, hopefully to improve the institution’s RAE (Research Assessment Exercise) scores. He resented the fact that his contract required him to teach, let alone attend a whole-day workshop. Many of us who have worked in universities around the world, trying to improve the quality of teaching and learning, are all too familiar with the complaint from faculty that, despite the rhetoric of Vice-Chancellors, it’s research that’s important – not teaching. Research gets the promotions and the kudos: there is no parity of esteem. And this has been reinforced by the way the two have been treated. For research, there has been the carrot, with rewards (largely financial) for success; for teaching, there has been the stick – no money for doing well and negative publicity for doing badly. So, when the TEF was first mooted, whilst having no delusions about the potential difficulties, I was enthusiastic about engaging with the idea and seeing how it might work and I urged others to do the same. Fulfilment of the intention - to create a similar, parallel framework to the REF, publicly to recognise centres of excellence, to encourage the pursuit of teaching excellence and to give teaching parity of esteem with research - is long overdue. However, what is now being implemented will do none of those things. The White Paper (Department for Business, Innovation and Skills, 2016) actually includes quite an enlightened definition of teaching excellence: We take a broad view of teaching excellence, including the teaching itself, the learning environments in which it takes place, and the outcomes it delivers. We expect higher education to deliver well designed courses, robust standards, support for students, TEF Special Edition Compass: Journal of Learning and Teaching, Vol 10, No 2, 2017 career readiness and an environment that develops the ‘soft skills’ that employers consistently say they need. These include capacity for critical thinking, analysis and teamwork, along with the vital development of a student’s ability to learn. (Para 6, p. 43) So how can anybody involved in writing that paragraph, or subscribing to what it says, end up with a framework fundamentally based on the three metrics of retention, employment (after six months) and student satisfaction. Let’s consider the problems with these three in turn. Whilst retention may well be improved by excellent teaching, if students are already high- achieving and motivated, they will almost certainly not drop out, even if the teaching is mediocre, and especially not, if attending an institution deemed to be prestigious. Repeated studies, such as Yorke and Langden (2004), have identified that the most common reason for dropping out is not, in fact, the teaching, but the course not having been what the student expected, and therefore considered as ‘not right for me’. There are similar problems with employment as a measure. Though excellent teaching may well contribute to a student’s employment chances, there are numerous other factors which probably have even more influence (Blasko et al, 2002) – the student’s cultural capital, the reputation of the particular university attended, the subject studied, the unemployment rate in the geographical location of the university (as many students like to stay in the area where they have studied) and the state of the general labour market at the time. Additionally, the rather bizarre six-month time limit takes no account of such situations as entry to accountancy and law, for which students have to engage in further professional/vocational courses. We should also note that not all students have the same ambitions and that some have aspirations other than finding traditional, so-called ‘graduate jobs’. Out of the three metrics, the NSS has probably the greatest claim to a link to the quality of teaching. There have been many criticisms of its focus on ‘satisfaction’ with, as a consequence, some very sensible recent moves to take ‘student engagement’ as a better indicator, but, in my experience, where a course has had a low NSS score, there has always been an issue that needed addressing – but not always the teaching and sometimes something beyond the course itself, a wider institutional problem. However, by far the greatest flaw in what is being proposed is that it is going to operate at the level of the institution and, even though a move to assessments at subject/discipline level is TEF Special Edition Compass: Journal of Learning and Teaching, Vol 10, No 2, 2017 intended, these will still be aggregated for the whole institution. This makes absolutely no sense at all. We know that, with NSS scores, there is a wider range of difference between different courses in the same institution than there is across different institutions (Surridge, 2009). So even if the metrics to be used were much better than those proposed and could claim to be able to assess validly the excellence of teaching, aggregating them for the whole institution would render them meaningless. Then, to reduce that to the designations of gold, silver or bronze is just crude and crass. Given the arguments above, the claim in the White Paper to be applying “sector-wide rigour to the assessment of teaching excellence” (Para. 10, p. 44) can surely not be taken seriously? We should certainly benefit from a system that rigorously assessed teaching excellence – but this is not it! The emperor has no clothes and the stark reality must be made clear: the sector must have the courage to stand up, voice its concerns and demand a re-think, before more time and money is wasted. At the moment, the proposal is both embarrassing and foolish. Reference list Blasko, Z., Brennan, John., Little, B. and Shah, T. (2002) Access to what: analysis of factors determining graduate employability. London: HEFCE. Department for Business, Innovation and Skills (2016) Success as a Knowledge Economy: Teaching Excellence, Social Mobility & Student Choice. London: HMSO. Surridge, P. (2009) The National Student Survey three years on: What have we learned? York: Higher Education Academy. Yorke, M. and Langden, B. (2004) Retention and student success in Higher Education. Maidenhead: SRHE/OUP. http://www.voced.edu.au/search/site/all_creators%3A%22Brennan%2C%20John%22