Microsoft Word - 4. Layout Abdurrazaq 18779-58493-1-RV.docx Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 233 Teacher Appraisers' Perceptions of Teacher Appraisal Teacher Appraisers' Perceptions of Teacher Appraisal Ahmed Hamid Abdulrazzaq1 1English Department, Institutes of Nursing, United Arab Emirates DOI: 10.23917/ijolae.v4i3.18779 Received: June 21st, 2022. Revised: September 2nd, 2022. Accepted: September 3rd, 2022 Available Online: September 5th, 2022. Published Regularly: September 1st, 2022 Abstract This study investigates teacher appraisers' perceptions of teacher appraisal in terms of its purpose, methods, and criteria. Fourteen teacher-appraisers were administered semi-structured interviews to gain an in-depth understanding of their perceptions of appraisal. Analysis of the responses revealed that participants seemed to advocate the use of teacher appraisal for both professional improvement and decision-making purposes. Par- ticipants also seemed to agree about the value of classroom observation as an objective method of evaluation. Still, they were divided regarding the use of student evaluation, student achievement, and other indirect ob- servation methods, especially for decision-making purposes. As for evaluation criteria, the results revealed that while teacher-appraisers paid attention to all aspects of the teaching process in their evaluation, their focus remained mainly on those skills exhibited during the actual delivery of teaching. Keywords: appraisers' perceptions, teacher appraisal, teacher evaluation, leadership and management Corresponding Author: Ahmed Hamid Abdulrazzaq, English Department, Institutes of Nursing, United Arab Emirates Email: a.h.razzaq@gmail.com 1. Introduction In a bid to introduce a new system of teacher appraisal for the English Department at the Institutes of Nursing (ION), a study was conducted by the researcher (Abdulrazzaq. in press) to investigate English teachers' percep- tions of teacher appraisal. The results of that study revealed that teachers' perceptions, though positive towards the process of teacher appraisal in general, disagreed with the cur- rent system in terms of the purposes it should serve, the methods used, as well as some of the criteria. In order to work out the best way of im- plementing changes to our appraisal system, it is necessary to investigate appraisers' per- ceptions of teacher appraisal and see if those would agree with appraisees' opinions. Over- all, 15 people are currently practicing the role of teacher-appraiser at the three branches of the ION: 14 appraising nursing teachers, and one, the researcher, appraising English teach- ers. All 15 appraisers use more or less the same appraisal process in terms of its purpose, methods, and (at least for lecture-based courses) criteria. These same appraisers also constitute the curriculum committee of the ION and usually take the decisions regarding appraisal as well as other policies. For all these reasons and based on the re- searcher's agreement with Bailey's (1996) be- lief that decisions on teacher evaluation should be made in conjunction with all stake- holders, the current study is the second step in the longer process of revising the ION teacher evaluation system. Therefore, this paper aims to investigate ION appraisers' perceptions of teacher Indonesian Journal on Learning and Advanced Education http://journals.ums.ac.id/index.php/ijolae Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 234 Teacher Appraisers' Perceptions of Teacher Appraisal appraisal in terms of its purpose, methods, and criteria. To address the purpose of the study, three research questions were posited (a) what are ION teacher appraisers' perceptions of the purpose of the teacher appraisal process; (b) what do ION teacher appraisers perceive as the best methods for appraising teachers; and (c) what do ION teacher appraisers perceive as the best criteria for appraising teachers. The study will shed light on how teacher- appraisers view the appraisal process at the ION and thus help the ION administration im- prove our teacher evaluation process and build a better appraisal tool for assessing our teachers. It will also provide some insight into all similar tertiary institutions in the UAE. Evaluating, controlling, or "verifying whether everything occurs in conformity with the plan adopted, the instructions issued, and principles established" (Fayol, 1949) is the fourth element of management according to Fayol's classical definition of management – the other three being planning, organizing, and leading. It is this particular function of management that ensures that there is effec- tive and efficient utilization of organizational resources in order to achieve the planned goals, i.e., that actual performance meets ob- jectives. One of the major components of this im- portant function in educational institutions is performance appraisal, where appraisers try to determine how well teachers carry out the duties of their assigned job in a bid to help teachers know the level of their job perfor- mance and as well as the expectations that the educational institution has of them. According to the Missouri Department of Elementary and Secondary Education, teacher appraisal is “the process of collecting data and making professional judgments about perfor- mance for the purpose of decision-making” (1999). In TESOL settings, a similar definition, but with a more general purpose, is offered by Genesee (2001), who defines teacher evaluation as “a process of collecting, analyzing and interpreting information about teaching and learning in order to make in- formed decisions that enhance student achievement and the success of educational programmes.” The concept appraisal has also been la- beled evaluation and assessment by different writers. In line with Braskamp and Ory's (1994) confirmation that the terms assessment and evaluation “have now become inter- changeable in higher education”, for the pur- poses of this study, all three words will be used interchangeably and will all refer to the process of collecting data and making profes- sional judgments about teachers' perfor- mance. There seems to be a variety of opinions about what constitutes the major purposes of teacher appraisal. Some scholars talk about two purposes, while others talk about three, four, and even more, as shown below. In their seminal work on the different ap- proaches to teacher evaluation in TESOL, Pennington and Young (1989) identify two types of teacher evaluation in terms of the pur- poses, or motivations, behind them. The first is formative evaluation, whose purpose is “to help teachers improve their performance by providing them with information, judgments, and suggestions on what and how to teach,”; and the second type is summative evaluation which aims at “providing information for de- cision-making with respect to hiring, firing, tenure, promotion, assignments, and salary.” Similarly, such authors as Danielson and McGreal (2000), Casey et al. (1997), Ur (2012), Hutchinson (1995), and Braskamp and Ory (1994), all talk about two purposes of evaluation, naming the first summative, qual- ity assurance, institutional accountability, or hiring-and-firing: and naming the second Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 235 Teacher Appraisers' Perceptions of Teacher Appraisal formative, professional development, staff development, or individual improvement. While adopting the formative and sum- mative purposes of evaluation, Bailey (2006) agrees with Daresh (2001) in adding yet a third primary purpose for evaluation, the di- agnostic purpose, where appraisal is con- ducted in order to obtain “baseline data about the normal state of affairs prior to any inter- vention or treatment.” On the other hand, Darling-Hammond, Wise, and Pease (1983) talk about four pur- poses for teacher evaluation: “Individual staff development, School improvement, Individ- ual personnel decisions - School status deci- sions”. Thus, they clearly distinguish pur- poses on two levels. First is the stakeholder level, i.e., individual/institution. Second is the end product level, i.e., improvement/decision- making. The literature on methods of teacher ap- praisal seems to talk about a small list of such methods. In terms of the professional devel- opment purpose of teacher appraisal, Ur (2012) lists the following three methods of evaluation: personal reflection, collaborative discussion with colleagues, and student feed- back. On the other hand, Pennington and Young (1989) provide a longer list of seven methods of evaluation that cover both the pro- fessional development and the hiring/firing aspects. These methods are: “teacher inter- views, competency tests, student evaluations, student achievement, classroom observation, peer review, and faculty self-evaluation”. Danielson and McGreal (2000) provide a similar list but add such new methods as structured reflection (which is presented as a different method than self-assessment), as- sembling of a professional portfolio, and questionnaires given to parents. Most scholars in the field of education agree with the importance of having a set of standards or criteria against which teachers' work can be compared (Daresh, 2001; Dan- ielson & McGreal, 2000; Lally & Myhill, 1994) and maintain that "a clear, visible, and appropriate set of evaluation criteria" (McGreal, 1988) is required for a teacher evaluation system to be successful. Nevertheless, many of these educators recognize the extreme difficulty of “determin- ing the standards” (Bailey, 2006) due to the complexity of the educational environment, to the extent that some of them state that “it is virtually impossible to identify universal cri- teria for language teacher evaluation” (Bailey, 2006). Despite the complexity of evaluating teachers' performance, most educators seem to agree that teacher appraisal is an indispen- sable tool for enhancing the quality of educa- tion at any institution. Thus, Acheson and Gall (1997) list nine teacher performance standards, or “characteristics of successful teachers.” The characteristics are clarity, vari- ety of materials and methods, enthusiasm, task-oriented approach, avoidance of harsh criticism, indirect teaching style, emphasizing content covered in tests, providing an over- view, and using questions at many levels. Goodwin and Stevens (1993) offer a sim- ilar list of generally accepted characteristics of good teachers that shares three elements with the previous list: enthusiasm, clarity, and avoidance of harsh criticism. The list, how- ever, adds the following new criteria: knowledge of the subject area, stimulation of interest in the subject area, organization, con- cern and caring for students, use of higher cognitive levels in discussions and examina- tions, use of visual aids, encouragement of ac- tive learning and student discussion, and pro- vision of feedback. Brown and Lee (2015) present a list of 30 characteristics of good language teaching, which he then classifies into four major Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 236 Teacher Appraisers' Perceptions of Teacher Appraisal groups: background knowledge, pedagogical skills, interpersonal skills, and personal qual- ities. Several studies have been conducted on appraisers' perceptions of teacher appraisal. Although they do not all necessarily focus on the purpose, methods, and criteria of teacher appraisal, some of the findings offer valuable insights for the current study. Smith (1995) examined staff appraisal in higher education by conducting a survey of the opinions of both appraisers and appraisees with regard to various aspects of the perfor- mance review system. The results showed that most teacher-appraisers thought that apprais- ers should be line managers rather than peers and that classroom observations should be part of the appraisal system. In a similarly designed study, Lo (1998) conducted a survey to explore the perceptions of both appraisers and appraisees of the teacher appraisal system. His findings showed that both appraisers and appraisees “have a positive attitude towards teacher appraisal” and that while teachers “prefer the formative appraisal,” middle managers advocate both formative and summative appraisal. As for appraisal methods, both groups considered classroom observation, examination of teach- ers' overall performance, and self-evaluation as effective methods. As for criteria, both groups agreed that “skills, attitude, and knowledge are important criteria in teacher appraisal.” The “skill for and knowledge about classroom teaching” are considered the most important criteria in setting up the ap- praisal system. In another study, Kyriacou (1997) exam- ined teacher appraisal from the perspective of 41 appraisers. The results indicated that par- ticipants generally felt that teacher appraisal had been of value to both the appraisee and the appraiser. Almost all participants felt that the classroom observation had been helpful to appraisees in thinking about, and developing, their classroom practice. 2. Method The Diploma program at ION is a three- year program designed for students who hold a secondary school certificate. The language of instruction is English, and the ultimate goal of the Diploma program in the ION, as it appears in its mission statement, is "the preparation of generalist, Arabic-speaking nurses who employ critical thinking skills and the nursing process in meeting the human needs of clients as individuals, families, and communities throughout the life span." (Program Package: p. 6) According to the ION Instructions on Classroom Observation, "The primary purpose of the observation is improving the teaching and learning processes at the Institutes of Nursing." In practice, however, the results of the teacher appraisal process are used for both professional development and management-related purposes. ION nursing teachers are currently evaluated principally through two methods: classroom observation (twice per semester) and Student Evaluation of Teaching (once for each course per semester). Though a bit more lenient than in the English Department in that one observation is announced and one unannounced, the scheduling of unannounced observations for nursing teachers may still be placed at the stricter end of Bailey's (2006) scale where the administration decides the decision about the date and time of the observation, and teachers are not informed of the visit until the session begins. However, teachers have a clear idea of the ION teacher evaluation criteria and have copies of the appraisal forms used (for classroom observation and student evaluation). The Session Appraisal Form is a 23-item, 3-point Likert-type scale. The 23 Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 237 Teacher Appraisers' Perceptions of Teacher Appraisal items, each representing a performance criterion, are divided into the three categories of Teaching/Learning Process, Content, and Teaching/Learning Environment, and the supervisor is expected to react to statements by making one of three possible choices: Accomplished, Partially Accomplished, or Not Accomplished. The Student Evaluation Form covers more or less the same concepts as in the session appraisal form. The 22 items here are written as statements describing teacher behaviors, and students can make one of four possible choices: All the time, Most of the time, Sometimes, or Rarely. Due to the small number of teacher- appraisers that represented the study population, I decided to study the whole population (i.e., all 14 teacher-appraisers at the three branches of the ION). All participants had leadership roles at the ION and had been actively engaged in appraising nursing teachers. All participants are non- native speakers of English with teacher- appraisal experience ranging from 3 to 20 years. All 14 participants hold master's degrees in Education, Nursing Education, or Quality Management. One possible threat to the validity of any study when using personal interviews lies in the area of participant suggestibility. Precautions taken to avoid such contamination of the results included preparing an interview guide (See Table 1) where questions are limited in number and presented in a set way and in the simplest and briefest form to act as cues for the participants to voice their opinions and attitudes. Table 1. Interview Guide Step 1: Introduction Thank the participant for coming and give them an overview of what the interview will cover. Let the partici- pant know that the anonymity of whatever goes on during this interview is guaranteed and that they may withdraw from the interview whenever they think it is necessary. Step 2: Purpose • Do you think it is important/necessary for an institution like the ION to have a teacher appraisal sys- tem? • Why do you think it is important? What do you think is the major purpose served by teacher appraisal? • What are some other benefits/purposes served by teacher appraisal? Look for: major purpose, secondary purposes, individual vs. institutional purposes/ PD vs. quality assurance purposes/ diagnostic purposes. Step 3: Methods • In your opinion, what is the best method for evaluating teachers' performance? • What are some other effective methods for evaluating teachers? Look for: major method, other methods, announced/unannounced observation/ student evaluation. Step 4: Criteria • What are the most important criteria that can be used for evaluating teachers? • What are some other important criteria? Look for: knowledge, skills, and attitude Step 5: Conclusion Moreover, the results are subject to bias due to the fact that the researcher is at the same time the participants' colleague at the same administration level in the same institution, and thus appraisers who might disagree with the way the appraisal system is currently applied at the ION might not want to express their objections. On the other hand, this relationship with the researcher entailed that the researcher already knew their opinions about many of these points. Their voluntary participation in the study and Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 238 Teacher Appraisers' Perceptions of Teacher Appraisal acceptance for the interviews with them to be audio recorded and their honest comments during the interview suggest that such a bias could not have altered the findings much. Ethical considerations of working with fellow appraisers were observed. Appraisers were given the choice to participate without pressure. They were informed of their participation in a study about their perceptions of teacher appraisal before they were given the interview, and I made every effort to ensure that the participants fully understood the procedure. I also explained to participants that everything they said would be confidential. To protect their identities, I used numbers (e.g., P1 for Participant 1) as an identifier throughout the study (i.e., on the transcribed interviews and/or any written manuscripts, including the present report of the study.) In order to gain an in-depth understanding of participants' attitudes towards appraisal and to make sure all three topics of purpose, methods, and criteria are covered, data were collected via semi- structured, face-to-face interviews that were conducted by the researcher. After obtaining approval to conduct the study, I personally contacted the 14 appraisers to determine their interest in being interviewed. At that time, I informed them of the study's objectives and told them that their participation was voluntary and that they were free to stop the interview at any time. All of them agreed to take part in the study. Each participant was also asked if the interview could be audio-recorded. Only one participant did not prefer to have her interview recorded, and notes taken during that interview were used in the analysis. Over two months, participants (N=14) were administered semi-structured interviews, lasting approximately 25 to 35 minutes, on an individual basis. All interviews were conducted using the same interview guide or list of topics and broad questions that must be addressed in each interview (see Table 1). The topic guide was developed on the basis of the research questions of the study, the literature review, as well as my previous research on teacher appraisal (Abdulrazzaq, in press). After conducting the interviews, I transcribed the audiotapes verbatim and moved on to the thematization process. During this process, I analyzed the transcripts using the “editing” analysis style described by Miller and Crabtree (1992). In this analysis style, the researcher acts as an interpreter who reads the transcripts carefully, looking for meaningful units or “segments” of text. When segments are recognized, the researcher develops a categorization scheme and corresponding codes that can be utilized to sort and categorize the data. The researcher then looks for the patterns that connect those categories to understand the views of each participant and in an attempt to create a summary that “reveals the interpretive truth in the text” (Miller and Crabtree, 1992). For example, when analyzing the purpose of evaluation, I began by reading the relevant section across all interview transcripts, attempting to identify meaningful text segments. During this initial reading, I found frequent references and discussions of professional development, school development, teacher-related decisions, school diagnosis, motivation, and initial diagnosis of teachers' strengths. After identifying these segments, I placed these meaningful segments into a preliminary categorization scheme that I had developed before I started analyzing the data of the current study, based on my readings of the literature and my previous research study (Abdulrazzaq, in press). That categorization scheme mainly contained three major Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 239 Teacher Appraisers' Perceptions of Teacher Appraisal categories: professional development (teacher vs. school), managerial decisions (teacher vs. school), and initial diagnosis of teachers' performance. However, after carefully reading the actual data, I felt that the situation was more complicated than I originally thought and that I needed to develop a new, more complicated categorization scheme to sort and categorize the data (See Table 2). Once I developed the new version of the categorization scheme, I started reviewing the transcripts one by one for content and began the process of coding the data for their correspondence to, or exemplifications of, the categories I identified in my scheme Finally, and in order to reduce the possibility of a biased and one-sided interpretation of the data, I resorted to "investigator triangulation" (Denzin, 1978). I asked a fellow researcher to take part in analyzing the data. Accordingly, right after I finished my initial reading and categorization of the data, I asked this researcher, who was familiar with qualitative research methodology, to assist me in analyzing the transcripts. The fellow researcher read the data independently and wrote her notes before we re-read the relevant sections together and collaboratively negotiated and determined the final categories and their contents. Table 2. Coding Schemes Coding Scheme of Purposes of Evaluation Professional Development Codes • Teacher Development • School Development Managerial Decisions Codes • Teacher Decisions • School Decisions Diagnostic Codes • Diagnosis of Teachers' Performance by Self • Diagnosis of Teachers' Performance by School • Diagnosis of School Performance • Diagnosis of Teachers' Attitude Motivation Codes • Motivation • Threat Coding Scheme of Methods of Evaluation Best Method Observation Codes • Direct Observation –Announced/Unannounced • Indirect Observation (Videotaping/ Portfolio/Written Reports) • Peer Evaluation SET Codes • Objective • 50/50 • Subjective • Key Students Student Achievement Codes • Objective • 50/50 • Subjective Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 238 Teacher Appraisers' Perceptions of Teacher Appraisal 3. Result and Discussion In line with findings reported by Lo (1998) and Kyriacou (1997), the results gen- erally showed that appraisers had a very pos- itive attitude towards teacher appraisal and firmly believed in its value for purposes of professional development, managerial deci- sions, and diagnostic purposes. a. Purposes of Appraisal Analysis of the interview transcripts re- vealed that the following four major themes characterized the purposes of teacher ap- praisal as described by participants in this study. 1) Professional Development The first theme that emerged from the in- terviews was that of professional develop- ment. The most frequently mentioned compo- nent of this theme was enabling the ION to improve teachers' performance. All 14 partic- ipants referred to staff development when asked about appropriate purposes for con- ducting teacher appraisal, maintaining that appraisal can help us to “identify weak- ness...to identify who needs supervision” (P1), and “based on the strengths and weak- nesses, you can develop the plan” (P13) to “help them and coach them” (P9), and thus, “improve the performance of the teachers” (P14). Interestingly, while all 14 participants referred to the first purpose given by Darling- Hammond, Wise, and Pease (1983), i.e., “In- dividual staff development,” only two partic- ipants referred to their second purpose, i.e., “school improvement,” stating that in addi- tion to staff development, conducting teacher performance appraisal “is a requirement for your institution development” (P13). 2) Managerial Decisions Again all 14 participants referred to the importance of teacher appraisal in helping management decisions. Interestingly, all par- ticipants referred to “promotions” when thinking of managerial decisions, while four referred to “renewal of contract,” and three to “hiring” new staff and “decisions about salary raises.” On the other hand, only three participants talked about the importance of appraisal in helping managerial decisions regarding ter- minating teachers. 3) Diagnostic purpose Many participants also talked about the importance of conducting teacher appraisal for diagnostic purposes, with most of them talking about how the institution needed “get- ting baseline information about teachers” (P10). One participant explains: We have people coming from different backgrounds. So, we do not have consistency in their knowledge base. So, in the beginning, you need to assess … to get a baseline assess- ment for each individual that you are having. (P1) On the other hand, five participants talked about the importance of teacher ap- praisal in helping teachers themselves, rather than their institution, diagnose their own points of strength and weakness, "so teachers know where they are standing." (P11) On a different note, two participants talked about the role of teacher appraisal in diagnosing school, rather than teacher perfor- mance, explaining that appraisal can help us to “make sure that the teaching-learning pro- cess is going as it should be … that the stu- dents are really learning” (P10). Participant response in the current study thus adds more specific dimensions to Bai- ley's concept of diagnostic evaluation (see 1.4.1 above) by indicating who, or what, is to Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 239 Teacher Appraisers' Perceptions of Teacher Appraisal be the subject of such an evaluation (the teachers or the teaching process at the institu- tion), and who could benefit from the results (the administration or the teachers them- selves). 4) Motivation The final theme that emerged in this sec- tion was that of motivation. Two participants talked about the role that performance ap- praisal plays in motivating teachers and how feedback from such appraisals can “give in- centives, or even appreciation … gives moti- vation to do better” (P12). Participants further elaborated on how appraisal reduces the stress on the part of the teacher since it tells them exactly where they are and what they need to do next: Many employees want their boss to know that their work is ok…to be recognized. Sometimes appraisal means to workers more than money and salary… to know their strengths and weaknesses. And this will de- crease the stress on the workers because if the worker is doing well, that is fine. If they have a weakness and there is a plan for improve- ment, that is also good. But not at the end of the year, suddenly the manager tells that his performance is poor, and there is action to be taken against him (P10). Participants also affirmed that it is im- portant that appraisal “should not be under- stood as a threat; it should give appreciation” (P12). They went on to assert that “teachers need to perceive evaluation as developmen- tal” (P11), warning that, “If they feel they are threatened by the evaluation, they will never improve.” Thus, consistent with previous research findings that appraisers advocate both forma- tive and summative appraisal (Lo, 1998), the current study revealed that participants place equal importance on the use of teacher ap- praisal results for the purposes of both professional development and personnel de- cision-making. b. Methods of Appraisal Most participants started their answers by saying that there was no one best method and that for evaluation to be accurate, “it has to be comprehensive… Everyone must partic- ipate in this evaluation… We should cover all aspects” (P10). Many participants, however, indicated indirectly in their answers what they thought were the best methods, using such expres- sions as “the best source of information” (P10), “one of the best methods,” (P9), “gives the best results” (P11) or even “the best method” (P14), and these mainly revolved around: direct observation (N=6), student evaluation (N=4), and student achievement (N=1). Analysis of the methods of appraisal sec- tion of the interview transcripts revealed the following three major themes or methods of appraisal. 1) Observation a) Direct Observation All 14 participants referred to direct fac- ulty observation, whether in the classroom or clinical settings, as an important, or some- times the most important, method of ap- praisal, with the majority of participants re- garding unannounced direct observation as being "the most objective" (P1), and main- taining that it shows “the true performance of the teacher … in terms of information, prep- aration, time planning, etc.” (P10) and would “tell you really what the teacher does in the classroom” (P1). Another participant says, “if you really want to see if the teacher is consist- ently prepared and performing well, the unan- nounced will reflect better about the teacher's performance” (P9). Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 240 Teacher Appraisers' Perceptions of Teacher Appraisal Nevertheless, participants did admit that unannounced observation is not without dis- advantages and that this mainly had to do with the fact that it “might create some tension in the tutor which may affect their performance” (P1) and that “this might give a threatening atmosphere, which makes teachers unable to function as they usually function without your presence” (P9). On the other hand, most appraisers ques- tioned the credibility of announced observa- tion, stating that these would “give the teach- ers the chance to change their usual trend. They will try to enhance their performance to the maximum because they know they are be- ing observed” (P1). Another participant adds: In the announced [observation], teachers do their best because they know their super- visor is coming. In unannounced, you see them as they are. When you see the same teacher in announced and unannounced ob- servations, you will see a great difference and no consistency (P12). However, some participants conceded that announced observations have some ad- vantages, stating that such observations are not only beneficial because “they show the maximum [that the teacher can give]” (P10), referring to their positive impact on the teach- er's future performance explaining that: “The teacher really tries to show her skills and maybe she will be trying to perform at the same level every time and every ses- sion…This would motivate her to perform at the same level in every session” (P9). In conclusion, appraisers believe that both types are important and that each type serves a different purpose, stating that “if you want to help and coach the teacher and not just observe every single thing they do, in that sense it [announced observation] is more [im- portant]” (P9), but if it is a decision-making thing, then it should be unannounced because there can be changes; people can modify their behavior, so you don't know whether their be- havior or performance consistently is happen- ing at the same level (P13). b) Peer Observation Eight participants considered peer obser- vation appropriate for teacher appraisal, but only “if it is done in the correct way” (P13). For four participants, the correct way had to do with the peer being “experienced” (P12) and “capable” (P1), while four other partici- pants stated that this observation “should be only informal” (P13) and “only for profes- sional development purposes” (P1) warning that otherwise “it could have a negative im- pact.” Though believing in the benefits of peer observation, one participant expressed her pessimism about the current ION situation stating that “peer evaluation is good in a dif- ferent context, but here it is not applicable be- cause teachers are biased and not objective” (P12). c) Indirect Observation Participants also talked about the im- portance of other appraisal methods whereby teachers' performance can be observed indi- rectly “through reports they are submitting, contribution in the development of the total course evaluation” (P13). Another appraiser adds, “I can also observe tutors' documenta- tion – notes about each student. This usually tells me how much a tutor is really following up” (P1). Similarly, another participant sug- gests “asking teachers to prepare a portfolio, with documents and evidence about how much a person is working and developing themselves.” (P9) 2) Student Evaluation of Teaching (SET) Another major theme that emerged when discussing methods of appraisal had to do with students' evaluation of their teachers. Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 241 Teacher Appraisers' Perceptions of Teacher Appraisal Participants' perceptions of the effectiveness of this method ranged from some considering it the "best method and the best source [of in- formation]" (P10), on one side, to others call- ing it subjective and "emotional" (P1) on the other. Out of the 14 participants, four stated that student evaluation “is a very good method of evaluation because students are the ones who really know how much they are benefitting from the teacher, and which areas they feel the teacher could be more helpful to them” (P9). Another participant confirmed that “the best method and the best source is students. If there is an area of weakness, it shows from [student] evaluation” (P10) Six participants expressed concern about the “element of subjectivity” (P13) in student evaluation, maintaining that “there can be bias from students because it depends on your social relationships with the students.” Nev- ertheless, these participants stated that it was still “generally valid” (P14) and that “in the end, you will get some information that will give you some clues about the teacher's per- formance” (P13). The other four participants, on the other hand, expressed total skepticism about the va- lidity of student evaluation, calling it “mostly an emotional evaluation” (P1), and attributing this to the “cultural background of the stu- dents and their maturity level …[which makes them not] qualified to objectively eval- uate a tutor based on that tutor's actual perfor- mance in the classroom.” A different and more beneficial form of student feedback, according to three apprais- ers, can be obtained by getting “information from key students in the class, ones who are mature and objective and are capable of giv- ing you details and specific feedback about the tutor” (P1). 3) Student Achievement The final major theme that emerged in this section had to do with students' outcomes in tests and exams. Again, participants ex- pressed two opposing views. On the one hand, eight participants expressed their full agreement with the use of exam results for purposes of teacher appraisal, stating that stu- dents' exam grades “reflect teachers' perfor- mance” (P10), with one participant consider- ing it the “best method” (P9). On the other hand, and despite not totally objecting to the idea of using students' grades for evaluating teachers, one participant ques- tioned the effectiveness of using this method in the ION, maintaining that: Students' grades may be used in other in- stitutions but not in the ION because I know students' habits here. They only study when they have an exam. …. If you look at the coursework, it is ok, but it is only the exam where they did not do well. This means you judged the teacher only on the last [result] and not look at how she performed from the be- ginning of the year (P12) Interestingly, using the same “context and type of learners” logic, a diametrically opposed opinion is offered by one of the eight participants who agreed with this method ear- lier, saying that this method will be particu- larly effective in the ION context due to ION students' lack of maturity and independence: If you are really teaching mature adult learners, then the teaching has nothing to do with it. But if you don't have such mature in- dependent students, then the grades will tell you about how teachers are working with them and motivating them. (P13) To conclude, it is clear that participants agree with previous research findings (Smith, 1995; Lo, 1998; Kyriacou, 1997) regarding the importance of classroom observation in teacher appraisal. Though participants re- ferred to most of the other evaluation methods Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 242 Teacher Appraisers' Perceptions of Teacher Appraisal mentioned in the literature (except for com- petency tests and self-evaluation), they were divided regarding the validity of using those methods for decision-making purposes. c. Criteria The third and final section of the inter- view had to do with what participants thought were the best criteria for teacher appraisal. Answers to this section covered Brown's (2001) four major groups of technical knowledge, pedagogical skills, interpersonal skills, and personal qualities, though not in equal proportions, with pedagogical skills re- ceiving the most emphasis, while all three other categories received more or less the same amount of attention. 1) Technical Knowledge Nine participants talked about the im- portance of “knowledge of the subject mat- ter” (P13) as an important criterion in evalu- ating teachers, with one of them giving it top priority, stating that “the first thing is knowledge” (P13). Besides theoretical subject matter knowledge, only two participants talked about the importance of having a different type of technical knowledge, i.e., clinical ex- perience. They talked about the importance of having “a good clinical background … be- cause every teacher needs to put their insight into the discussion, and without good experi- ence, you can't bring situations and insight into the discussion” (P9). The same two participants mentioned yet another type of technical knowledge, having a good command of English, since English is the language of instruction at the ION, with all students and all nursing teachers being non-native speakers of the language. Ensuring that technical knowledge is up to date through keeping up with the field and “updating themselves with the latest development in their fields” (P10) is another important criterion, according to one partici- pant. 2) Pedagogical Skills These criteria received the most attention from participants. Participants talked about most teaching skills exercised throughout the different stages of a regular class, referring to the importance of planning, delivering, as- sessing, and even reflecting on what hap- pened in the session. While only five participants referred to “preparation” (P1) or “planning activities” (P2) as important criteria to be used in a teacher appraisal, most participants seemed to focus on the actual delivery of the session. Eight appraisers stressed the importance of teachers' “way of explaining” (P13) and whether teachers are “able to present the con- tent clearly and comprehensively” (P9) and to “make the information easy” (P13). Issues of variety were also addressed dur- ing the interviews, with six participants talk- ing about using a variety of “methods” (P10) or “strategies” (P9) as a criterion for teacher appraisal. Six appraisers also referred to teachers' ability to “adapt strategies” (P13) in order to deal with “different [learning] styles” (P9) as another criterion. Another criterion mentioned by three ap- praisers was classroom management, or “con- trol” (P11), as well as teachers' ability to “handle conflict” (P1). Six teachers also men- tioned teachers' time management. Four participants also talked about the importance of relating content to students' “real life” (P14), and two talked about the im- portance of teachers' ability to “stimulate stu- dents to think” (P10). As for the assessment phase of the ses- sion, eight appraisers referred to teachers' "questioning techniques" (P9), and three talked about how important it is to monitor Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 243 Teacher Appraisers' Perceptions of Teacher Appraisal students' progress and "to make sure students are understanding [sic]" (P1) Finally, two participants referred to “re- flection” (P1) and the teacher's “ability to re- flect on what went wrong and work on devel- oping herself” as another criterion to be used in teacher appraisal. 3) Interpersonal Skills Ten participants talked about the im- portance of having the necessary interper- sonal skills that would enable teachers to in- teract with their students successfully. Partic- ipants mainly stated that teachers should be “pleasant” (P12), “close, available” (P9), and “caring and supportive to students” (P10). Three participants also talked about the importance of teachers' awareness of cross- cultural differences, stating that teachers should be “culturally sensitive” (P9). Another participant referred to teachers' ability to “deal with students of different age groups and different backgrounds” (P11). Appraisers also mentioned that the teacher should be "pa- tient" (P11) and should “respect students, but still be strict” (P9) and “assertive” (P1). 4) Personal Qualities The final set of teacher appraisal criteria had to do with teachers' personal qualities. All participants referred to one or more personal qualities of teachers that can be used as a cri- terion for teacher appraisal. Generally, teach- ers need to be “confident, receptive, innova- tive” (P1), “active, not sleepy, [and] fair” (P10). The teacher should also be able to “cope with stress” (P11) and “be a role model” (P14). Eight participants also referred to organ- ization and being able to “work in an orga- nized manner” (P14), and five referred to pro- fessionalism as other criteria for teacher ap- praisal. Finally, six participants referred to teach- ers' attitude as another criterion, referring to teachers' ability to “accept new ideas” (P9), “accept criticism” (P11), and adapt their per- sonal philosophy to conform with the “insti- tutional philosophy” (P13) Thus, consistent with available literature (Acheson & Gall, 1997; Goodwin and Ste- vens, 1993; Ramsden, 1992; Brown; 2001), participants seem to pay attention to all as- pects of the teaching-learning aspects when thinking of criteria of evaluation, with special emphasis on those teaching skills utilized during the actual delivery of teaching. In par- ticular, and despite some references to the im- portance of technical knowledge, the results largely agree with Danielson and McGreal’s (2000) argument that when educators start classifying teaching criteria, they seem to ig- nore “what teachers know” and divide these criteria in terms of “what teachers do in the course of their professional practice” (inputs) and the results they achieve (outputs). 4. Conclusion The research findings indicate that ION appraisers have a positive attitude towards teacher appraisal. In terms of purpose, appraisers seemed to view both formative appraisal and summative evaluation as being of equal importance. They also talk about the value of teacher appraisal in providing baseline data about teachers’ performance and providing motivation to teachers. Regarding evaluation methods, almost all appraisers seemed to agree that (unannounced) direct observation is the best method for getting an objective picture of teachers' performance. Appraisers were divided regarding the validity of using SET and student achievement as teacher appraisal methods, especially for personnel decision- making purposes. Appraisers also talked about the value of using peer evaluation and Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 244 Teacher Appraisers' Perceptions of Teacher Appraisal indirect methods of observation for professional development purposes. As for the evaluation criteria, the results revealed that while appraisers looked at teachers' knowledge, skills, and personal qualities, their main focus was mainly on pedagogical skills, especially those skills utilized during the actual delivery of teaching. Based on the findings of the study, the overall recommendation that can be proposed for the preparation and application of a teacher appraisal system in the English Department at the ION would be the implementation of an EFL teacher appraisal system designed to encourage teachers to reflect on their teaching and improve their professional growth; a program which benefits the teacher in terms of professional development, through which strengths are commended and areas where support is necessary are identified. At the same time, the appraisal program should be able to help the administration get an accurate idea about teachers’ performance and the teaching- learning process at the ION in general to make informed decisions that enhance the quality of education at the ION. The findings also suggest that a variety of appraisal methods need to be implemented, with classroom observation being the major method for purposes of personnel decision- making along with SET and student achievement, whose results should be treated with extra care. These methods, in addition to peer observation and other indirect forms of observation, could all be utilized for purposes of helping teachers develop professionally. Finally, according to the findings, the appraisal criteria in the new program should cover all aspects of the teaching process (i.e., knowledge, skill, and attitude) with special emphasis on classroom teaching skills. According to the study's findings, such a program is very likely to gain the positive support of ION teacher-appraisers. 5. References Abdulrazzaq (in press). EFL teachers' percep- tions of teacher appraisal. Journal of Ed- ucation College Wasit University. Acheson, K., & Meredith D. (1997). Tech- niques in the clinical supervision of teachers: Telling tales about school. Ed- ucational Leadership, 41(7), 16-18. Bailey, K. (2006). Language teacher supervi- sion: A case-based approach. Cam- bridge: Cambridge University Press. Braskamp, L., & Ory, J. (1994). Assessing faculty work. New York: John Wiley and Sons, Inc. Brown, H D., & Lee, H. (2015). Teaching by principles: An interactive approach to language pedagogy (4th ed.). New York: Pearson Education. Casey, R. J., Gentile, P., & Bigger, S. W. (1997). Teaching appraisal in higher ed- ucation: An Australian perspective. Higher education, 34(4), 459-482. Danielson, C., & McGreal, T. (2000). Teacher evaluation to enhance profes- sional practice. New Jersey: ETS. Daresh, J. (2001). Supervision as proactive leadership (3rd ed.). Prospect Heights, IL: Waveland Press. Darling-Hammond, L., Wise, A. E., & Pease, S. R. (1983). Teacher evaluation in or- ganizational context: a review of litera- ture, Review of Educational Research, 53, 285–328. Denzin, N. K. (1978). The research act: A theoretical introduction to sociological methods. NewYork: McGraw-Hill. Fayol, H. (1949). General and industrial management. New York: Pitman Pub- lishing. Genesee, F. (2001). Evaluation. In Carter, Ronald and Nunan, David (Eds.), The Indonesian Journal on Learning and Advanced Education (IJOLAE)| p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 4 (3) (2022) 233-245 245 Teacher Appraisers' Perceptions of Teacher Appraisal Cambridge guide to teaching English to speakers of other languages. Cambridge: Cambridge University Press. Goodwin, L., & Stevens, E. (1993). The in- fluence of gender on university faculty members' perceptions of "good" teach- ing. Journal of Higher Education, 64(2), 166-85. Hutchinson, B. (1995). Appraising appraisal: Some tensions and some possibilities. Higher Education, 29(1), 19-35. Institutes of Nursing, Ministry of Health, UAE. (2012). Diploma nursing pro- gram: Program package. Sharjah, UAE: Author. Kyriacou, C. (1997). Appraisers' views of teacher appraisal. Teacher Development, 1(1) 35-41. Lo, C. (1998). Teacher appraisal - Percep- tions of appraisers (middle managers) and appraisees (teachers) in Hong Kong secondary schools. (Unpublished M. Ed. Thesis, The University of Hong Kong, 1998). McGreal, T. (1988). Evaluation for enhanc- ing instruction: Linking teacher evalua- tion and staff development. In S. Stanley, & W. Popham (Eds.), Teacher Evalua- tion: Six prescriptions for success. Alexandria, VA: Association for Super- vision and Curriculum Development. Miller, WL., & Crabtree, BF. (1992). Primary care research: A multimethod typology and qualitative roadmap. In BF Crabtree and WL Miller (Eds.) Doing Qualitative Research. London: Sage. Missouri Department of Elementary and Sec- ondary Education (MODESE). (1999). Guidelines for performance-based teacher evaluation. Jefferson City, MO: Author. Pennington, M. and Young, A. (1989). Ap- proaches to faculty evaluation for ESL. TESOL Quarterly, 23(4), 619-646. Ramsden, P. (1992). Learning to teach in higher education. London: Routledge. Quoted in Casey, R. John, Gentile, Patri- cia, and Bigger, Stephen W. (1997). Teaching appraisal in higher education: An Australian perspective. Higher Education, 34(4), 459-482. Smith, R. (1995). Staff appraisal in higher ed- ucation: A study of performance review at Nene College, Northampton. Higher Education, 30(2), 189-205 Ur, P. (2012). A course in language teaching. Cambridge: Cambridge University Press.