Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 Peer Observation of Teaching in E-learning Environments in UK Universities Zeynep Kacmaz Ifs University College Abstract Peer observation of teaching (POT) is presented as a valuable mechanism allowing for the dissemination of ‘best practice’ within university teaching. With the rapid rise of online learning courses, it might be expected that POT would be extended to the online environment. In practice, however, it appears that this development is inconsistent. Likewise, there has been little research into the experiences of those who deliver online teaching, with research focused more on the technological aspects than on teaching methodology. Using a document analysis of a sample of UK universities and interviews with academics to assess a cross section of current institutional practice, this paper explores the intersection between POT and the online delivery of teaching and learning. The findings indicate that POT has not yet become a standard evaluative and/or developmental process in e-learning environments, although some universities do implement it within their CPD (Continuous Professional Development) or teacher-training programmes. Introduction Peer observation of teaching (POT) in academic environments is a process undertaken for a variety of purposes, including staff development, quality monitoring, and staff promotion decisions (Gosling, 2005:5). The POT literature generally views it as a process that can benefit the professional development of lecturers and tutors (McKinnon, 2001; Gosling, 2002; Hammersley-Fletcher and Orsmond, 2005). Gosling (2005:13) identifies three broad models of POT:  An evaluation model based on a summative approach, aiming to assist managerial and personnel decisions or to comply with quality standards and to prepare for audits;  A developmental model based on a formative approach, helping academic staff to improve their teaching practice and to qualify for fellowship or membership of professional bodies;  A collaborative model based on a formative approach, aspiring to develop a community of practice for the improvement of teaching. It is reported that a majority of universities use a combination of models (Gosling and O’Connor, 2009), with POT often deployed, following the recruitment of new lecturers, as a tool for evaluation during their probationary period. Many universities implement POT in cycles of one or two years for developmental purposes for all staff, using a collaborative model (Gosling, 2005). Others adopt a university-wide process, especially in the case of probationary lecturers, although others are organised on a departmental level. Some concerns have been raised about the implementation of POT and the potential risks arising from the process. Gosling and O’Connor (2009:7) claim that “many staff are ill- equipped, without further training to evaluate and provide feedback on the effectiveness of Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 others’ teaching”. Shortland (2004) has suggested that staff often engage in POT in order to comply with institutional policies rather than through any real desire to improve their teaching practice. Peel (2005) questions whether making POT compulsory would reduce its potential for supporting individuality and empowerment. This is supported by other studies which argue that POT works best as a voluntary practice (Atkinson and Bolt, 2010). The demands of delivering flexible online teaching to an international body of students has resulted in the recruitment of a workforce of largely hourly-paid ‘associate’ lecturers, who are willing and able to work from home, with unpredictable and often unsocial hours (Fazackerley, 2013). This effective ‘outsourcing’ of teaching, with its geographically- distributed and ‘virtual’ nature, means that both the monitoring of performance and ensuring of high standards of delivery are at the same time difficult to implement and easy to overlook. This author’s experience in teaching distance learning (DL) programmes has indicated that peer observation of teaching is not undertaken regularly for lecturers and tutors involved in e-learning environments. This research sought to investigate how peer observation of teaching is currently implemented in e-learning environments in UK universities, what criteria are appropriate to POT in such environments, whether digital/virtual environments constrain the POT process and what kind of guidelines or processes might be desirable and practicable for the implementation of POT in e-learning environments. As the research questions were focused upon the theories of understanding rather than explaining, and building rather than testing, an interpretative approach was adopted. Although Swenson and Taylor (2014) affirm the teacher’s role in delivery, they also highlight the fact that teachers are moving away from their traditional role as the sole source of information. Clearly, as online learning environments become more sophisticated over time, teachers need to adapt to new environments and to align their knowledge with the requisites of Learning Management Systems (LMS) (Hinson and LaPrairie, 2006; Connolly, Jones and Jones, 2007; Swenson and Taylor, 2014). E-learning environments in HE (Higher Education) and POT According to a survey on Virtual Learning Environment (VLE) use by the Universities and Colleges Information Systems Association (UCISA), 81% of universities in the UK used VLE.s in 2001, increasing to 95% by 2005. Since 2005, the increased variety of online technologies has been reflected in a broadening of the scope of the UCISA survey to include “any online facility or system that directly supports learning and teaching” (UCISA, 2014). This includes VLE.s, eLearning tools and Web 2.0 tools. Various definitions of e-learning have tended to focus on the use of the internet for teaching and learning practice, although broader definitions include audio-videotapes, satellite broadcast, interactive TV and CD-ROM (Mason and Rennie, 2006). Within this learning context, the following modes can be identified:  Online learning, requiring that the teaching takes place via the internet.  Blended learning, combining face-to-face teaching with online teaching and learning methods.  Distance learning, where students are not required to attend some or all on-site teaching or learning activities. Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016  Open learning, where the learner defines independently what, when and how to study without formal and compulsory classroom participation. A 2013 paper from the Changing the Learning Landscape (CLL) partnership highlights major cultural issues that can act as barriers in the implementation of Technology-Enhanced Learning (TEL) strategies; it also points out that development of the digital literacy of stakeholders and institutional in-house collaboration are both important issues for a more uniform and consistent provision of e-learning tools (ALT, 2013). The rapid integration of digital technologies into university teaching has created both debate about and research into the role of teachers in e-learning. Goodyear et al (2001) define eight roles for teachers involved in online teaching, highlighting the role of ‘technologist’ as a new element. In addition, existing roles can be seen to change as a result of the online influence on the whole teaching environment. For example, the role of facilitator assumes a new dimension, with the internet producing an enormous amount of content and thus requiring the guidance of a teacher to enable learners to access the most relevant sources. Baran et al (2011) reviewed eleven key articles published since 2000 and examined online teacher roles and competencies. This study also emphasises the role of technologist. Rogers (2013:231) also highlights another significant development, the ‘reprofessionalization’ of teachers working in the e-learning environment. This remark is pertinent in terms of the continuous professional development (CPD) of university teachers, as the extensive use of technology requires them to upgrade themselves continuously through appropriate training (Armellini and Jones, 2008). In the context of the emergence of blended learning, Gosling (2014:18) argues that “POT has to be radically reconsidered to accommodate this form of delivery”, suggesting a need to move to a model of peer review of teaching that is more flexible and more inclusive of the whole range of activities involved in designing, delivering and assessing teaching. However, in practice, the development of POT to accommodate the online environment appears to be patchy at best; for example, The Quality Assurance Agency for Higher Education (QAA) has never required POT to be used in the context of e-learning (Swinglehurst et al, 2008), which perhaps helps also to explain a relative lack of research in this area. McKenzie and Parker (2011) highlight this, whilst noting that many subjects in universities in English-speaking countries are now delivered in blended mode or online environments. Kraglund-Gauthier et al (2010) also point out that there has been little research into the experiences of teachers as they shift from face-to-face to online teaching, more research having been done on the technologies, methods and tools used in the online environment than on the delivery of teaching. Bennett and Santy (2009:406) approach what they refer to as online peer observation (OLPO), from an action learning approach, “aiming to exploit the flexibility of the internet and offer individuals a powerful source of support for their professional development.” They argue that “OLPO can be a window across boundaries of discipline, e-learning environment and mode of delivery...in an interdisciplinary, cross-technology, multi-cultural and global context.” Swinglehurst et al (2008:385) maintain that “the role of the e-tutor extends beyond that of a designer of learning tasks and that in fact much (although not all) of an online teacher’s behaviour is observable, although the limitations of such a focus stand.” They created a Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 ‘Peer-to-peer Reflection on Pedagogical Practice’ (PROP) model to implement in a web- based master’s degree programme, focused upon six main themes, some of which are also common to face-to-face POT: anxieties provoked by POT; teaching reviewed as ‘performance evaluation’ leading to little learning; tensions inherent in the practice of POT; defining ‘good’ online teaching; redefining POT as collective ‘sense-making’; the opportunity for innovation and collaboration. Their research went on to argue that effective implementation of POT in the online environment required: ensuring clarity of purpose; ensuring that the process remains ‘true’ to its purpose; moving away from conceptualising teaching as performance; fostering a team culture valuing critical discourse about teaching; ensuring protected time; exploiting the unique features of the online environment (e.g. the permanency of the record, the opportunity to expand POT beyond what is observed within the classroom, the opportunity to reflect on new conceptualisations of learning); fostering a climate of trust, mutual support, and ownership (Swinglehurst et al, 2008:390-91). Bennett and Barp (2008) have investigated the implementation of POT in the online learning environment, and argue that “many aspects of peer observation do not simply ‘translate’ directly online”, maintaining that “… the flexibility of the online medium raises questions and forces decisions that do not have to be faced when working face-to-face, such as challenges concerning what is observed online, how the observation process is managed and structured and how to help participants maximise the new opportunities as they adapt to a context in which many lack the perspective as online learners” (Bennett and Barp, 2008:568). Bennett and Barp (2008:568) note that one issue particular to the nature of online learning concerns the difficulty of selecting and isolating a ‘chunk’ of learning and teaching activity to be the focus of online peer observation. Classroom-based peer observation typically involves an observer’s attending a distinct ‘lesson’, whereas online courses are not necessarily organised in terms of ‘lessons’ and structuring observation may not be as straightforward as suggested by Tonkin and Baker (2003), who talk of replacing a 45-60 minute classroom observation by logging into the VLE a number of times over a week, on the basis that “one week in Blackboard is equivalent to a week’s worth of face-to-face class time” (Tonkin and Baker, 2003:3). Bennett and Barp (2008:568) argue that the participants would work in an environment without the usual ‘natural time boundaries’ and this, in turn, would create a pressure on ‘staff time’. Bennett and Santy (2009:404) emphasise that “Peer observation of the online learning/teaching process must not merely replicate the traditional use of peer observation of class-room teaching but capitalise on the additional opportunities it offers.” Beckton and Penney (2011:12) argue that the POT process could be an effective way for educational developers to reach more staff for development purposes. As Bennett and Barp (2008:559) point out, there is a wide range of issues that require further research in the context of online POT, such as teacher/learner rights within online observation, the role of the emerging e-pedagogy and patterns of student learning in the online context. Indeed, they question the very appropriateness of peer observation as a tool for CPD within the online context. However, they conclude that “as the use of online and blended learning continues to expand and as, almost inevitably, online peer observation gains greater prominence within the HE sector, it will be important to expand such investigation in order to explore the strategies which can be employed to overcome such Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 challenges and/or exploit new opportunities, to identify key aspects that influence the effectiveness of peer observation online and to complement good practice already established for classroom-based peer observation” Bennett and Barp (2008:568-9). Methodology for the analysis of university documents In order to ascertain the extent and nature of POT implementation, with particular reference to the online environment, a review of documents and webpages of UK universities was undertaken. Documents on POT were assessed against a number of criteria, including: whether they were available online, whether they referred to POT in e-learning environments and whether POT procedures had been developed by the university itself or adapted from other universities. Via the Higher Education Statistics Agency (HESA) and Universities UK websites, a total of 130 universities were identified. Of these, 103 university websites were consulted in more detail, in two stages. In the first stage, three key phrases were used to identify relevant webpages and documents: 1. Peer Observation of Teaching 2. Academic Development 3. Learning and Teaching Strategy In the second stage, when POT-related documents were gathered, a search within the documents was made using the phrase ‘Technology Enhanced Learning’, in order to gain more insight into the HEI’s relevant strategies. It was decided to undertake further analysis on twenty per cent (twenty) of the universities, selected randomly, on the basis that this would constitute a significant sample of the whole. Analysis of university documents The main objective of the analysis was to determine the extent to which POT was implemented within university environments, with subsidiary criteria aimed at identifying whether POT was included in the university’s Teaching and Learning Strategy documents, how the e-learning environment was promoted at the university, and whether the online environment was included as part of the POT process. One criterion addressed whether the Higher Education Academy (HEA) fellowship was cited, on the basis that this would be an important measure in understanding how committed the university was to the UK Professional Standards Framework (UKPSF). Also, as postgraduate programmes in academic development, which aim for academic staff to obtain the HEA fellowship, incorporate the process of POT in their syllabus, it was felt to be important to identify whether the Postgraduate Certificate in Higher Education programmes were voluntary or compulsory in these institutions as well. The analysis of the sample list of universities indicates that seventeen out of twenty had policies for POT that were available via their webpages. However, only half of the universities had the relevant POT forms available publicly on their website, with some explicitly indicating that they were limited to staff access on that institution’s intranet. Teaching and Learning or Teaching, Learning and Assessment Strategies were publicly available for eleven out of twenty universities. However, none of these documents, where Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 they were available, included POT explicitly, although some mentioned peer review or peer mentoring. For example, the University of Wolverhampton noted its aim to “Develop academic and professional communities of practice for example, through supportive and developmental peer review and reflection in order to increase the systematic dissemination and embedding of good practice in learning and teaching and related subject and pedagogical research” (University of Wolverhampton, 2013). However, many such documents appeared to be aimed more at current and prospective students than staff. Twelve out of twenty universities indicated a centralised department that oversaw the implementation of POT and seven of the universities in the sample claimed that they had recently (2010-2013) reviewed their POT processes. Where stated, most universities (twelve out of fourteen) intended POT to cover all academic staff, whilst two focused on new staff or those in their probationary period. All of the universities that mentioned frequency of POT implementation (twelve out of twenty) implemented POT annually or bi-annually, therefore indicating that it was implemented as a cyclical, continuous procedure. Most universities appeared to apply POT as a university-wide policy; however, at the same time, it was clear that in many cases schools and departments could adapt university-wide guidelines to their individual requirements. For example, the University of Bedfordshire advised that “The Peer Support of teaching scheme…operates in all academic departments. The precise nature of its implementation will be determined by your department but it must consist of two instances of activity for full time staff and one instance of activity for part time staff” (University of Bedfordshire, 2014). Most universities generally cited the HEA as a source of literature and further information on POT, with a couple referring to other UK or foreign universities. For example, the University of Hull stated that “Some of the forms used in this site are based on those produced by the Universities of Birmingham and St Andrews and we wish to acknowledge their cooperation” (University of Hull, 2014). The University of Leicester directed staff to a link to ‘How to do Peer Review of Teaching’, from the University of Macquarie in Australia, as well as to the Teaching and Learning Portal at University College, London (University of Leicester, 2014). Only five universities out of twenty made direct reference to online activities as part of their POT process. Two of these had POT forms specifically designed for e-learning/teaching, one had a dedicated page describing an e-tutoring scenario for peer review and the other two mentioned virtual learning environments in their POT guide. The University of Bedfordshire guidelines noted that “For full time staff, one peer supported activity must involve a typical classroom-based session with consideration of associated BREO unit site materials/activities. The second peer supported activity is negotiable and could emanate from a range of possibilities e.g. unit design, classroom or online learning activity, assessment design or activity” (University of Bedfordshire, 2012). The University of Leicester noted that its POT scheme “operates for all teaching staff (both on campus and distance learning)” and it included a separate e-learning form (University of Leicester, 2014). Four of the universities in the sample were seen to have adopted wider Peer Review/Support schemes with the aim of covering a broader range of academic support and development issues, although the schemes varied in their focus and aims. For example, the University of Bath’s Peer Review of Teaching scheme “aims to enhance the student learning experience and enable lecturers to teach and facilitate learning even more effectively…Peer review differs from teaching observations which are more narrowly focused on 'observable Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 teaching events'. Instead, it aims to give feedback that: moves beyond teaching style and delivery, is more specific and meaningful, offers greater challenge or provides new insights, is less linked to evaluative observations of probationary staff” (Exley, 2014). Similarly, at the University of Cardiff, “Peer Review of Learning and Teaching (PRLT) is a pan-University activity for all staff whose role directly impacts on the student learning experience. It is an opportunity for staff to take part in an annual, scholarly, peer-assisted reflection on the breadth of their teaching/learning support practice. PRLT considers both professional development and commitment to the enhancement of the student experience, and goes beyond classroom observation to cover all aspects of learning and teaching practice” (University of Cardiff, 2013). All the universities that implemented POT confirmed that the procedure was part of their academic development/CPD, and were keen to emphasise the developmental nature of the process, with the School of Oriental and African Studies (SOAS) Guidelines, for example, emphasising that “the introduction of POT at SOAS is not for judgemental or assessment purposes, but will follow a Continuing Professional Development (CPD) approach whereby constructive mutual feedback from trusted colleagues supports the sharing of good practice” (SOAS, date unavailable). None of the universities in the sample had any Technology Enhanced Learning strategy in their websites. However, several universities had dedicated e-learning/online learning pages. One, (the University of Wolverhampton) had two guides referring to TEL and Blended e-learning. All but three universities endorsed HEA fellowship on their website. All except one university cited a Postgraduate Certificate in Higher Education programme. Most universities offered these certificate courses on voluntary basis to all staff, but seven made it mandatory for new lecturers or probationary lecturers. The analysis of university webpages and documents points to a number of tentative conclusions. Firstly, the implementation of POT processes appears to be fairly ubiquitous in UK universities, with most institutions implementing it as a formal procedure on a university- wide basis. However, they tend to be at pains to emphasise that it is a developmental rather than a judgemental process and, where it is mandatory, this tends to be only for newer staff and those on probation. Moreover, the actual implementation of POT tends to allow departments and individual academics a good deal of freedom to interpret the guidelines as they see fit, again avoiding claims of heavy-handedness, with flexibility around the frequency and nature of observation, and also the nature of the activities being observed. This flexibility tends to lead to vagueness in the guidelines concerning the nature of activities to be observed, with classroom observation seen as the key area for observation, but little clarity on what else does and does not count as a teaching/learning activity. Some guidelines mention e-learning environments as areas that could be covered, but few go as far as the University of Leicester in providing specific forms for the e-learning environment. The analysis shows that the Postgraduate Certificate in Higher Education has become the norm amongst UK universities, especially for new lecturers, and it is clear that this is a route through which participants are exposed to POT in various environments, including e- learning. Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 Finally, in a few universities it can be seen that POT is becoming a part of wider Peer Review or Support schemes. Such schemes are keen to demonstrate that they are not about expanding central observation, but rather an extension of the developmental nature of peer assessment, and a further move away from the box ticking procedures of the past (Shortland, 2004: 224). Methodology for interviews In order to explore the approach and opinions of a variety of stakeholders involved in the POT process in UK universities, a selected group of academic staff was interviewed, using a semi-structured approach. In order to obtain participants for the interviews, an initial request was sent via an exploratory e-mail to individuals selected from the current Staff and Educational Development Association (SEDA) Jisc mail list, to participants of a HEA workshop which the researcher attended in 2013 and to colleagues from various institutions for which the researcher had carried out work. A total of fifty-nine individuals were sent invitations, which resulted in nine interviews conducted in total. The majority of the interviewees were working or had worked in academic development or related roles. All had teaching experience in the UK and only one of them had not been exposed fully to a VLE environment as he had retired before VLE became mainstream. However, this participant had worked for the distance learning programmes of a UK HEI, so his input was considered to be relevant. Analysis of interviews The questions were grouped in order to cater to different categories of academic staff; they were also focused on two core topics: continuous professional development (CPD) and peer observation of teaching (POT). This focus aimed to explore how POT fits into current CPD schemes available in universities. The interview process revealed that each participant had her/his own unique experience of the topic. However, there were some recurrent themes. One key theme emerging from the interviews was the use of technology and the rapid rise of e-learning environments. With regard to POT, subjects with considerable relevant experience (twenty years and over) mentioned the QAA Subject Review and the compulsory evaluative implementation of POT, whilst those with less than twenty years’ experience focused on the developmental POT model. Almost all participants had experienced POT as observer or observee, indicating that it is a standard procedure in universities. The one participant who had never experienced POT noted that, as her job was largely administrative, it fell outside academic practice. All participants agreed that e-learning had developed rapidly and that it has now become mainstream; as one participant remarked, it is regarded as “the way forward” (PRT1). The academic developers within the interview participants noted that e-learning serves to meet increased demand for higher education, but that the pressing challenge now is “mainstreaming the uses of technology into skills training” (PRT3). Regarding the incorporation of e-learning within CPD, participants generally noted that their institutions provide training on e-learning through postgraduate certificate programmes. One participant confirmed that they were designing a specific programme for distance and blended learning, as their institution’s share of distance learning programmes was growing rapidly (PRT1). Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 However, another participant revealed that their institution had wound up its first certificate programme in e-learning a few years previously (PRT4). All the institutions mentioned by the participants provided some e-learning training within their CPD schemes, whether as postgraduate certificate programmes or specific training to implement new technologies, such as a new VLE. One participant believed that institutional vision could encourage lecturers to be more innovative in the use of technology as part of the Technology Enhanced Learning strategy (PRT3). This consideration was also highlighted by the 2013 paper published as part of the CLL partnership (ALT, 2013). Individual experience of POT The responses about peer observation of teaching revealed that almost all of the respondents (eight out of nine) had undertaken it, with all of those having participated both as observer and observee. One participant regarded himself as a “peer observation enthusiast”, asserting that he learned more by observing others than “from articles and research” (PRT1), however, another respondent admitted to being "...not too keen on peer observation of teaching" (PRT2). The one respondent with no exposure to POT managed a professional development programme for practitioners and believed that her “mentoring” of distance learning tutors amounted partially to a POT process (PRT5). A common characteristic of respondents was that the POT schemes implemented were developmental in nature. However, a participant who had experienced POT first through the QAA Subject Review noted that it had created an environment of mistrust and partiality, owing to institutional, departmental or even political rivalries (PRT2), as also confirmed in the literature by Gosling and O’Connor (2009), Gosling, (2005), Allen (2002) and Cosh (1998). From the point of view of the observee, one of the participants claimed that the POT process allows teachers to become “less inhibited and freer to teach in their own personal style” (PRT1). Another emphasised the benefits from both pedagogical and practical points of view: “the benefits are that when we did the observations we would have somebody who'd be an academic but we'd also use somebody who would be an e-learning technologist…There's so many things that having peer assessment helps with both from an academic and a technology perspective so that's why we use the two” (PRT9) An interesting common comment concerned the extent to which the observers themselves benefited from the process. Respondent PRT6 maintained that “the benefits are stronger for being observer”, and PRT1 claimed that “watch[ing] the class interactions gives you a three dimensional view of educational experiences.” These comments accord with literature which emphasises the beneficial aspects of POT schemes (Gosling, 2014; Bell and Cooper, 2013). In terms of the potential disadvantages of the POT experience, participants cited concerns for its use as an evaluative or HR tool: "I think as long as you don't get threatened by the process if it is not graded if again it is just discussion and then you can get a lot from it" (PRT7). There was also a concern about feeling judged: “it becomes much more personal observation of personal style and personality so it becomes quite challenging and that and you know people can be quite defensive”. (PRT5) One participant felt that the value of POT was limited: “the intention is quality control but…it is a formal procedure only. After certain years it does not benefit as one develops its own best teaching way…once the quality of Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 lecturing or teaching reaches to an acceptable level the extra effort could be very marginal” (PRT6). Institutional POT environment Whilst most participants confirmed that the objective of the POT processes that they had undertaken was developmental, one participant noted that, in the Further Education (FE) environment, it is still regarded as a managerial procedure and is compulsory. However, she also claimed that, in practice, “you pretend to do it”, therefore reducing POT to a box-ticking exercise (PRT7), as affirmed by Shortland (2004), asserting that staff take part in POT exercises at least in part to comply with institutional policies. Four participants confirmed that POT was established on an institutional level in their workplace (PRT3, PRT8, PRT7, PRT5). Two respondents stated that their institution had not adopted a formal POT procedure at the institutional level (PRT4, PRT1), so the experience of these participants was based on their own departmental frameworks or on the postgraduate programmes or teacher training programmes their institutions offered. One participant noted that in their institution there was “stiff resistance to POT”, owing both to a conservative staff approach and also to the fear that it might develop into an evaluative scheme (PRT1). The second participant, however, did note that their university had recently adopted formal POT guidelines and that the academic development team was currently developing a new developmental scheme for various departments (PRT4). The management of the POT process has been covered by Hammersley-Fletcher and Orsmond (2004), Lomas and Kinchin (2006), and McMahon et al (2007). These authors suggest that balancing anxieties and concerns while emphasising the importance of the process may actually reverse negative attitudes towards the POT process. In terms of the application of POT to the online environment, only two participants had any experience of this, and then rather in the context of the recording of live teaching than monitoring of online teaching as such, although one participant noted that some of their students would be delivering distance learning exclusively, so that observation would need to be undertaken on that basis (PRT8). Both of the academic developers who ran teacher- training programmes as distance learning courses had undertaken such monitoring with their international partner institutions for quality assurance purposes. Their focus was on logistical and technical issues, and a concern to adopt a systematic approach to assessment in a range of circumstances, rather than any belief that POT in the online environment required a fundamentally different approach (PRT8, PRT4). However, one participant had recently attended a training session at Edge Hill University, where “they have developed a very interesting instrument for collecting pre-observation information, and they had a systematic model of peer observation of teaching. Interestingly it was not very different from a general phenomenographic approach to peer observation so there was a lot of alignment with our practice but we have not put in a box for online use yet” (PRT8). In terms of potential issues of online POT, one participant highlighted technical problems, such as lines being down, but felt that the fundamental problem with adoption was lack of internal support: “It is a nice model but it could well need a lot of support for doing that, that is why we did not go with it” (PRT4). McKenzie and McCallie, in Baker et al (2006), emphasise the need for such support. Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 The implementation of POT in e-learning environments appears to be a very immature phenomenon, even for academic developers. One participant concluded that POT "should be a natural part of teaching…like reflective practice", but predicted that it would take “five to ten years” for POT become an embedded part of the academic world as a developmental process (PRT1), a view shared by Bennett and Barp (2008). Conclusions In terms of the application of POT as a general process, institutional policies and centralised frameworks/guidelines are prevalent and these are assumed to cover all learning environments, including e-learning. Whilst there are guidelines and forms for e-learning modes in some cases, the pedagogical or the phenomenographic foundations tend to be assumed to be the same, as confirmed by interviewees. This directly contradicts the recommendations in the literature - for example, Bennett and Barp (2008). The objective of implementing POT in general is mainly the evaluation of teaching and personal development, and this is also valid for applications in e-learning environments. However, as stated by the interviewees, the application of POT in e-learning environments is sometimes limited to teacher training or PG Cert. programmes. Non-implementation of POT in certain universities appears in some cases to stem from previous evaluative applications, which created a resistance to and mistrust in the process, or alternatively from the university’s focus on research rather than teaching, as mentioned. These negative aspects have been noted by various authors in the literature (Allen, 2002; Chism, 2007; Gosling and O’Connor, 2009; Shortland, 2010). However, it would be difficult to apply these statements generally across the whole of the UK HE sector. The application of POT in the e-learning environment can be seen to cover all kinds of online synchronous and asynchronous activities in distance learning, online learning and blended learning programmes. Feedback on online POT, according to the analyses of university documents, tends to be given in written form, as there are institutional forms to fill in, but they also include an oral component, relating to either pre- or post-observation discussion. The existing limited literature on the subject seems to bear this out (Gosling, 2014; Swinglehurst, 2008). POT is generally considered in a positive light, according to both the relevant literature (Bennett and Barp, 2008; Beckton and Penney, 2011; Harper and Nicolson, 2012) and the interviews and analysis of university documents. POT in e-learning environments is also deemed to make a positive contribution. Some of the more negative comments made by the interviewees have helped to address the secondary research question, which focused on the limiting effects of digital/virtual environments. Technical problems were mentioned by the interviewees, as well as bureaucratic aspects. These barriers have been discussed by Bennett and Barp (2008) and also by Tonkin and Baker (2003). It was not possible to identify specific guidelines or processes for e-learning environments. However, the information obtained through these data sources strongly suggests that POT in general terms in the UK context is acceptable by staff only if applied for developmental purposes. Moreover, as confirmed in the literature, (Gosling, 2005; Hammersley-Fletcher and Orsmond, 2005; Allen, 2012), an institutional drive for implementation is required, together with institutional understanding that the process needs to avoid being a burden on staff, especially in the context of distance learning teaching undertaken by part-time or Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 atypical staff. Additionally, in terms of requirements for quality control, there is a need for maintenance of evidence in the e-learning environments. Overall, the study indicated that the application of POT in e-learning environments is not yet fully developed. Whilst the sample cohort included in the study cannot be seen to be representative of the UK HE sector as a whole, it does provide an insight into current practice. Although implementation of POT in e-learning environments is developing slowly, academic developers and academic development centres in the universities tend to believe that the pedagogy underlying it should be the same as for classroom-based POT. However, they also acknowledge that implementation needs to accommodate the characteristics of the digital technologies employed in delivery. Future research on this subject should examine various technologies and their impact on the POT process. E-learning environments currently create technical barriers and also make many academics nervous, as they constitute a novel vehicle for teaching. Moreover, they are also a source of concern for administration, as maintenance of evidence is seen as problematic. In the light of the recently proposed Teaching Excellence Framework (TEF), QAA’s response to the Green Paper affirms that the coverage of such frameworks should go “beyond teaching and look into broader aspects of learning environment and assessment processes”, thus broadening the scope to include e-learning environments (QAA,2016:6). In this context, it is important to note that some academics view the POT as a snapshot in time rather than as a long-term evaluation tool, as one reader of Derfel Owen’s blog on the Times Higher Education website remarked (Owen, 2015). However, the Vice-Chancellor of the University of Hertfordshire, Quintin McKellar, supports “the class observation as the way for the academics who would seek promotion on the basis of their teaching” (Grove and King, 2015). Therefore the proposed TEF seems to have started a fresh debate about POT in terms of measuring teaching quality and the skills offered by academics. Further research is needed on this subject both at macro and micro level. At the macro level, a more comprehensive survey of policy and implementation in UK HEI.s would be valuable. At the micro level, in-depth case studies of implementations of POT e-learning environments could be carried out at university level, or comparative studies undertaken of the application of POT in e-learning environments in selected UK universities. Such studies could attempt to build up empirical evidence as to whether POT really can deliver better teaching in the online environment, and also assemble detailed guidelines for its implementation, taking into consideration the pedagogical implications of the differences between online and face-to- face teaching, as suggested in the literature. Finally, owing to the ‘reprofessionalization’ of teachers engaging in e-learning environments, as suggested by Rogers (2013:231), it would also be worthwhile to research POT processes in terms of promoting ‘best practice’ among colleagues. This would also look into teaching quality, as the proposed TEF seeks to embrace a wider learning environment than on-campus teaching. Since existing literature does not focus on how POT contributes to the development of skills in virtual environments, it would be valuable to focus on, for example, communication skills or management of intercultural contexts in virtual environments, and on how POT can identify development opportunities in order to enhance the learning experience of a diverse and global student cohort. Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 Reference list Allen, L. (2002) “Consenting Adults in Private” Union and Management Perspectives on Peer Observation of Teaching. Generic Centre: Learning and Teaching Support Network, February 2004. ALT (2013) Changing the learning landscape. Occasional Paper, CLL Consortium. Available at: http://repository.alt.ac.uk/2310/1/ALT_Occasional_paper_CLL_Project_2013.pdf (Accessed: 02 August 2014). Armellini, A and Jones, S (2008) ‘Carpe Diem: seizing each day to foster change in e- learning design.’ Reflecting Education, 4(1),17-29. Available at: http://www2.le.ac.uk/departments/beyond-distance-research- alliance/learning%20inn/media/Carpe%20Diem%20seizing%20each%20day%20to%20foste r%20change%20in%20e-learning%20design.pdf (Accessed: 24 August 2014). Atkinson D.J., and Bolt, S. (2010) ‘Using teaching observations to reflect upon and improve teaching practice in higher education.’ Journal of the Scholarship of Teaching and Learning, 10(3), 1-19. Available at: http://josotl.indiana.edu/article/viewFile/1798/1795 (Accessed: 24 August 2014). Baker, J.D., Redfield, K.D. and Tonkin, S. (2006) ‘Collaborative Coaching and Networking for Online Instructors.’ Online Journal of Distance Learning Administration, 9(4). Available at: http://www.westga.edu/~distance/ojdla/winter94/baker94.htm (Accessed: 25 August 2014). Baran, E., Correia, A.P. and Thompson, A. (2011) ‘Transforming online teaching practice: critical analysis of the literature on the roles and competencies of online teachers.’ Distance Education, 32(3), 421-439. Available at: http://dx.doi.org/10.1080/01587919.2011.610293 (Accessed: 04 July 2014). Beckton, J. and Penney, E. (2011) Peer observation of on-line teaching in a Distance learning environment. Blackboard Users Group Conference, 6th-7th January 2011, Durham. Available at: http://eprints.lincoln.ac.uk/3953/ (Accessed: 17 June 2013). Bell, M. and Cooper, P. (2013) ‘Peer observation of teaching in university departments: a framework for implementation.’ International Journal for Academic Development, 18(1), 60- 73. Available at: http://dx.doi.org/10.1080/1360144X.2011.633753 (Accessed: 15 June 2013). Bennet, S, and Barp, D. (2008) ‘Peer Observation - a case for doing it online.’ Teaching in Higher Education, 13(5), 559-570. Available at: http://dx.doi.org/10.1080/13562510802334871 (Accessed: 15 June 2013). Bennet, S. and Santy J. (2009) ‘A window on our teaching practice: Enhancing individual online teaching quality though online peer observation and support. A UK case study.’ Nurse Education in Practice 9, 403-406. Available at: http://repository.alt.ac.uk/2310/1/ALT_Occasional_paper_CLL_Project_2013.pdf http://www2.le.ac.uk/departments/beyond-distance-research-alliance/learning%20inn/media/Carpe%20Diem%20seizing%20each%20day%20to%20foster%20change%20in%20e-learning%20design.pdf http://www2.le.ac.uk/departments/beyond-distance-research-alliance/learning%20inn/media/Carpe%20Diem%20seizing%20each%20day%20to%20foster%20change%20in%20e-learning%20design.pdf http://www2.le.ac.uk/departments/beyond-distance-research-alliance/learning%20inn/media/Carpe%20Diem%20seizing%20each%20day%20to%20foster%20change%20in%20e-learning%20design.pdf http://josotl.indiana.edu/article/viewFile/1798/1795 http://www.westga.edu/~distance/ojdla/winter94/baker94.htm http://dx.doi.org/10.1080/01587919.2011.610293 http://eprints.lincoln.ac.uk/3953/ http://dx.doi.org/10.1080/1360144X.2011.633753 http://dx.doi.org/10.1080/13562510802334871 Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 http://www.nurseeducationinpractice.com/article/S1471-5953(09)00020-1/abstract (Accessed:15 June 2013). Chism, N. V. N. (2007) Peer Review of Teaching: A Sourcebook, 2nd edition. San Francisco: Anker & Jossey-Bass. Connolly, M., Jones, C. and Jones, N. (2007) ‘New approaches, new vision: capturing teacher experiences in a brave new online world.’ Open Learning: The Journal of Open, Distance and e-Learning, 22(1), 43-56. Available at: http://dx.doi.org/10.1080/02680510601100150 (Accessed: 04 July 2014). Cosh, J. (1999) ‘Peer observation: a reflective model.’ ELT Journal, 53(1), 22-27. Available at: http://eltj.oxfordjournals.org (Accessed: 15 June 2013). Exley, K. (2014) Peer Review of Teaching. Available at: http://www.bath.ac.uk/learningandteaching/advance-your-teaching/peer-review.html (Accessed: 12 August 2014). Fazackerley, A. (2013) ‘Why are many academics on short-term contracts for years?’ Available at: http://www.theguardian.com/education/2013/feb/04/academic-casual-contracts- higher-education (Accessed: 21 August 2014). Goodyear, P., Salmon G., Specter J.M., Steeples C. and Tickner, S. (2001) ‘Competences for Online Teaching: A Special Report.’ ETR&D, 49(1), 65-72. Available at: http://link.springer.com/article/10.1007%2FBF02504508 (Accessed: 24 August 2014). Gosling, D. (2002) Models of peer observation of teaching. Generic Centre: Learning and Teaching Support Network. Gosling, D. (2005) Peer Observation of Teaching, SEDA Paper 118. London: Staff and Educational Development Association. Gosling D. and O’Connor, K.M. (2009) Beyond the Peer Observation of Teaching, SEDA Paper 124, London: Staff and Educational Development Association. Gosling D. (2014) ‘Collaborative Peer-Supported Review of Teaching.’ Chapter 2 in: Sachs, J. and Parsell, M. (eds.) Peer Review of Learning and Teaching in Higher Education: International Perspectives. Dordrecht: Springer. Grove J. and King, J. ( 2015) ‘How might a teaching excellence framework be built?’ Times Higher Education, 23 July 2015. Available at: https://www.timeshighereducation.com/features/teaching-excellence-framework-tef-how- might-it-be-built. (Accessed: 18 January 2016). Hammersely-Fletcher, L. and Orsmond, P. (2004) ‘Evaluating our peers: is peer observation a meaningful process?’ Studies in Higher Education, 29(4), 489-503. Available at: http://dx.doi.org/10.1080/0307507042000236380 (Accessed: 15 June 2013). http://www.nurseeducationinpractice.com/article/S1471-5953(09)00020-1/abstract http://dx.doi.org/10.1080/02680510601100150 http://eltj.oxfordjournals.org/ http://www.bath.ac.uk/learningandteaching/advance-your-teaching/peer-review.html http://www.theguardian.com/education/2013/feb/04/academic-casual-contracts-higher-education http://www.theguardian.com/education/2013/feb/04/academic-casual-contracts-higher-education http://link.springer.com/article/10.1007%2FBF02504508 https://www.timeshighereducation.com/features/teaching-excellence-framework-tef-how-might-it-be-built https://www.timeshighereducation.com/features/teaching-excellence-framework-tef-how-might-it-be-built http://dx.doi.org/10.1080/0307507042000236380 Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 Hammersley-Fletcher, L. and Orsmond, P. (2005) ‘Reflecting on reflective practices within peer observation.’ Studies in Higher Education, 30(2), 213-224. Available at: http://dx.doi.org/10.1080/03075070500043358 (Accessed: 15 June 2013). Harper, F. and Nicolson, M. (2012) ‘Online peer observation: its value in teacher professional development, support and well-being.’ International Journal for Academic Development: 1-12. Available at: http://dx.doi.org/10.1080/1360144X.2012.682159 (Accessed: 15 June 2013). Hinson J.M. and LaPrairie, K.N. (2005) ‘Learning to Teach Online: Promoting Success through Professional Development.’ Community College Journal of Research and Practice, 29(6), 483-493. Available at: http://dx.doi.org/10.1080/10668920590934198 (Accessed: 04 July 2014). Kraglund-Gaulthier, W. L., Chareka, O., Orr, A.M. and Forran, A. (2010) ‘Teacher Education in Online Classrooms: An Inquiry into Instructors’ Lived Experiences.’ The Canadian Journal for the Scholarship of Teaching and Learning, 1(2) 1-11. Available at: http://ir.lib.uwo.ca/cjsotl_rcacea/vol1/iss2/4/ (Accessed: 15 June 2013). Lomas,L. and Kinchin, I. (2006) 'Developing a Peer Observation Program with University Teachers.' Teaching and Learning in Higher Education, 18(3), 204-214. Available at: http://www.isetl.org/ijtlhe/pdf/IJTLHE118.pdf (Accessed: 15 June 2013). McMahon, T., Barrett, T. and O'Neill, G. (2007) 'Using observation of teaching to improve quality: finding your way through competing conceptions, confusion of practice and mutually exclusive intentions.' Teaching in Higher Education, 12(4), 499-511. Available at: http://dx.doi.org/10.1080/13562510701415607 (Accessed: 15 June 2013). Mason, R. and Rennie, F. (2006) ELearning: the Key Concepts. Oxford: Routledge. McKenzie, J. and Parker, N. (2011) Peer Review in Online and Blended Learning Environments: Final Report, Australian Learning and Teaching Council, South Australia. Available at: http://www.uts.edu.au/sites/default/files/final-report.pdf (Accessed: 15 June 2013). Owen, D. (2015) ‘How to build the teaching excellence framework.’ Times Higher Education, 10 July 2015. Available at: https://www.timeshighereducation.com/blog/how-build-teaching- excellence-framework (Accessed: 18 January 2016). Rogers, S. (2013) Teaching in Virtual Classroom in Gornall, L. Cook, C. and L. Daunton(eds) Academic Working Lives: Experience, Practice and Change, 2013, London: Bloomsbury Academic Shortland, S. (2004) 'Peer Observation: a tool for staff development or compliance?' Journal of Further and Higher Education, 29(2), 219-28. Available at: http://dx.doi.org/10.1080/0309877042000206778 (Accessed: 10 May 2012). http://dx.doi.org/10.1080/03075070500043358 http://dx.doi.org/10.1080/1360144X.2012.682159 http://dx.doi.org/10.1080/10668920590934198 http://ir.lib.uwo.ca/cjsotl_rcacea/vol1/iss2/4/ http://www.isetl.org/ijtlhe/pdf/IJTLHE118.pdf http://dx.doi.org/10.1080/13562510701415607 http://www.uts.edu.au/sites/default/files/final-report.pdf https://www.timeshighereducation.com/blog/how-build-teaching-excellence-framework https://www.timeshighereducation.com/blog/how-build-teaching-excellence-framework http://dx.doi.org/10.1080/0309877042000206778 Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 Shortland, S. (2010) ‘Feedback within peer observation: continuing professional development and unexpected consequences.’ Innovations in Education and Teaching International, 47(3), 295-304. Available at: http://dx.doi.org/10.1080/14703297.2010.498181 (Accessed: 15 June 2013). SOAS (date unavailable) Guidelines for the Peer Observation of Teaching. Available at: www.soas.ac.uk/staffdevelopment/documents/file41580.doc (Accessed: 14 August 2014). Swenson, P. and Taylor N.A. (2012) Online Teaching in the Digital Age. Los Angeles: Sage. Swinglehurst, D., Russell, J. and Greenhalgh, T. (2008) ‘Peer observation of teaching in the online environment: an action research approach.’ Journal of Computer Assisted Learning, 24(5), 383-393. Available at: http://dx.doi.org/10.1111/j.1365-2729.2007.00274.x (Accessed: 15 June 2013). QAA (2016) Fulfilling our Potential: Teaching Excellence, Social Mobility and Student Choice, QAA's Response. Available at: http://www.qaa.ac.uk/en/Publications/Documents/QAA-Green-Paper-Response-Jan-16.pdf (Accessed: 18 January 2016). Tonkin S. and Baker J.D. (2005) ‘Peer Coaching for Online Instruction: An Emerging Model for Faculty Development.’ 19th Annual Conference on Distance Teaching and Learning. Available at: http://www.uwex.edu/disted/conference/Resource_library/proceedings/03_31.pdf (Accessed: 25 August 2014). UCISA (2014) 2014 Survey of Technology Enhanced Learning. Available at: http://www.ucisa.ac.uk/tel (Accessed: 01 August 2014). University of Bedfordshire (2014) Peer Support of Teaching Scheme. Available at: http://www.beds.ac.uk/learning/professional-development/peer-support (Accessed: 08 July 2014). University of Bedfordshire (2012) Guide No 15 Peer Support of Teaching Scheme (2nd edition). Available at: http://www.beds.ac.uk/__data/assets/pdf_file/0010/274159/CLE- Guide-15-Peer-Support-of-Teaching-Scheme.pdf (Accessed: 12 August 2014). University of Cardiff (2013) Peer Review of Learning and Teaching Available at: http://learning.cf.ac.uk/peerreview (Accessed: 14 August 2014). University of Hull (2014) Peer Support for the Enhancement of Learning and Teaching. Available at: http://www2.hull.ac.uk/administration/leap/learning_teaching/pselt.aspx (Accessed: 09 August 2104). University of Leicester (2014) Peer Observation of Teaching. Available at: (http://www2.le.ac.uk/offices/academic-practice/learning-and-teaching/observation-of- teaching (Accessed: 09 August 2104). http://dx.doi.org/10.1080/14703297.2010.498181 http://www.soas.ac.uk/staffdevelopment/documents/file41580.doc http://dx.doi.org/10.1111/j.1365-2729.2007.00274.x http://www.qaa.ac.uk/en/Publications/Documents/QAA-Green-Paper-Response-Jan-16.pdf http://www.uwex.edu/disted/conference/Resource_library/proceedings/03_31.pdf http://www.ucisa.ac.uk/tel http://www.beds.ac.uk/learning/professional-development/peer-support http://www.beds.ac.uk/__data/assets/pdf_file/0010/274159/CLE-Guide-15-Peer-Support-of-Teaching-Scheme.pdf http://www.beds.ac.uk/__data/assets/pdf_file/0010/274159/CLE-Guide-15-Peer-Support-of-Teaching-Scheme.pdf http://learning.cf.ac.uk/peerreview http://www2.hull.ac.uk/administration/leap/learning_teaching/pselt.aspx http://www2.le.ac.uk/offices/academic-practice/learning-and-teaching/observation-of-teaching http://www2.le.ac.uk/offices/academic-practice/learning-and-teaching/observation-of-teaching Articles Compass: Journal of Learning and Teaching, Vol 8, No 12, 2016 University of Wolverhampton (2013) Learning, Teaching and Assessment Sub-Strategy (2012-2017). Available at: http://www.wlv.ac.uk/staff/PDF/strategy-unit-Learning-Teaching- and-Assessment-Strategy-210312.pdf (Accessed: 10 July 2014). http://www.wlv.ac.uk/staff/PDF/strategy-unit-Learning-Teaching-and-Assessment-Strategy-210312.pdf http://www.wlv.ac.uk/staff/PDF/strategy-unit-Learning-Teaching-and-Assessment-Strategy-210312.pdf