121 © 2023 by the authors; licensee Asian Online Journal Publishing Group Asian Journal of Social Sciences and Management Studies Vol. 10, No. 4, 121-131, 2023 ISSN(E) 2313-7401/ ISSN(P) 2518-0096 DOI: 10.20448/ajssms.v10i4.5076 © 2023 by the authors; licensee Asian Online Journal Publishing Group Faculty members’ perceptions about the effectiveness of the procedures for applying the appraisal of academic staff at Sultan Qaboos University Shubair Abdullah1 Abdelrahman El-Haj2 Sameh Said Ismail3 Hussain Alkharusi4 Wajeha El-Aani5 Amal Al-Badi6 ( Corresponding Author) 1,2,3,4,5College of Education, Sultan Qaboos University, Muscat, Sultanate of Oman. 3Faculty of Graduate Studies for Education, Cairo University, Egypt. 1Email: shubair@squ.edu.om 2Email: abdoelhaj@squ.edu.om 3Email: samehsaid@squ.edu.om 4Email: hussein5@squ.edu.om 5Email: wajeha@squ.edu.om 6Military Technological College, Muscat, Sultanate of Oman. 6Email: amal11bh@gmail.com Abstract The appraisal of academic staff at Sultan Qaboos University (SQU) has been in place for many years as a quality assurance measure. Its primary objective is to enhance staff performance by identifying commendable areas and those requiring improvement in daily activities. Over the years, this appraisal has contributed in faculty development and quality assurance as well as in the continuous improvement. This study explores the perceptions of faculty members at the College of Education concerning the procedures for applying the academic appraisal form at SQU. Utilizing a mixed- method approach (quantitative and qualitative), including questionnaire and semi-structured interview, the research examines the faculty members' views on the effectiveness of the appraisal procedures. The sample participating in this study was 54 faculty members. Out of the 54 participants, 48 completed the questionnaire, while 6 were chosen for in-depth semi-structured interviews. The sample included a wide range of participants to reflect the diversity within the academic staff. The findings may offer valuable insights into the current status of these perceptions and potentially assist policy makers at SQU in enhancing the existing procedures and system for applying the academic appraisal. Furthermore, the results may lead to a set of recommendations and directions for future research, helping to improve the robustness and applicability of the appraisal system within the university context. Keywords: Academic accreditation, Academic appraisal, Assessment, Faculty perceptions, Quality assurance, Sultan Qaboos University. Citation | Abdullah, S., El-Haj, A., Ismail, S. S., Alkharusi, H., El- Aani, W., & Al-Badi, A. (2023). Faculty members’ perceptions about the effectiveness of the procedures for applying the appraisal of academic staff at Sultan Qaboos University. Asian Journal of Social Sciences and Management Studies, 10(4), 121–131. 10.20448/ajssms.v10i4.5076 History: Received: 17 August 2023 Revised: 28 September 2023 Accepted: 12 October 2023 Published: 20 October 2023 Licensed: This work is licensed under a Creative Commons Attribution 4.0 License Publisher: Asian Online Journal Publishing Group Funding: This research is supported by Sultan Qaboos University, Sultanate of Oman (Grant number: IG/EDU/TECH/22/01). Institutional Review Board Statement: The Ethical Committee of the Sultan Qaboos University, Sultanate of Oman has granted approval for this study on 19 January 2022 (Ref. No. IG/EDU/TECH/22/01). Transparency: The authors confirm that the manuscript is an honest, accurate, and transparent account of the study; that no vital features of the study have been omitted; and that any discrepancies from the study as planned have been explained. This study followed all ethical practices during writing. Competing Interests: The authors declare that they have no competing interests. Authors’ Contributions: The design and implementation of the research, S.A., A.E.H. and S.S.I.; analysis of the results, H.E.K. and W.E.A.; writing of the manuscript, A.A.B. All authors have read and agreed to the published version of the manuscript. Contents 1. Introduction .................................................................................................................................................................................................122 2. Literature ......................................................................................................................................................................................................122 3. Method...........................................................................................................................................................................................................124 4. Results and Discussion .............................................................................................................................................................................126 5. Conclusion ....................................................................................................................................................................................................130 References .........................................................................................................................................................................................................130 mailto:shubair@squ.edu.om mailto:abdoelhaj@squ.edu.om mailto:samehsaid@squ.edu.om mailto:hussein5@squ.edu.om mailto:wajeha@squ.edu.om mailto:amal11bh@gmail.com https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ https://www.doi.org/10.20448/ajssms.v10i4.5076 https://orcid.org/0000-0002-1133-5979 https://orcid.org/0000-0001-6576-9791 https://orcid.org/0000-0003-2718-5159 https://orcid.org/0000-0002-5822-9188 https://orcid.org/0000-0001-6354-4214 https://orcid.org/0000-0002-4422-9857 Asian Journal of Social Sciences and Management Studies, 2023, 10(4): 121-131 122 © 2023 by the authors; licensee Asian Online Journal Publishing Group Contribution of this paper to the literature This paper addresses the faculty perceptions of appraisal procedures at Sultan Qaboos University. Using a mixed-method approach, it addresses a gap in Oman's higher education research. The findings guide policy makers towards system refinement and provide insights for global institutions seeking to enhance their faculty evaluation processes. 1. Introduction Higher education institutions (HEIs) have been independent organizations supported by the notion that encourages academic staff to anticipate and enjoy high degrees of autonomy and independence (Egginton, 2010). According to Dasanayaka, Abeykoon, Ranaweera, and Koswatte (2021) universities undertake a significant responsibility to convey knowledge to society for economic growth through high-quality research and the production of skilled graduates. Also, students are primarily concerned with the reputation and quality of the university, regardless of its location, world ranking, facilities available, and world-class recognition. Dasanayaka et al. (2021) believe that competing for any financial allocations are typically made depending on the performance of these universities. Universities worldwide use staff appraisal as a management technique to aid employees' professional growth. However, it is still controversial whether staff appraisal improves an institution's human resource management procedures in terms of soft measures such as work satisfaction (Sułkowski, Przytuła, Borg, & Kulikowski, 2020). Also, despite the importance of the staff appraisal process in the HEIs for increasing job satisfaction and productivity of their faculty members, only a limited number of studies works related to this scope has been reported so far for both developed and developing countries (Dasanayaka et al., 2021). Furthermore, most of the currently available appraisal measures at universities primarily focus on the research performance of their academic staff, with no or little emphasis paid to measure teaching performance (Lohman, 2021). Locally, the education sector in Oman has been under more scrutiny and control forced by Oman vision 2040’s strategic direction of inclusive education, lifelong learning, and scientific research that led to a knowledge society and competitive national talents. This strategic direction aimed at creating a high-quality educational system and it entails changes in human resource policy, which is echoed in the appraisal of the academic staff in HEIs. The purp ose of appraisal is to offer a chance to explore how staff may improve their job tasks and remove any barriers to growth. Appraisal is designed to be a helpful and creative reflection of personal progress. Appraisal of academic staff has been in existence since the inception of the SQU in 1986. Its primary goal is to define the duties and responsibilities of the academic staff, identifying their strengths and weaknesses, receive feedback from staff, provide them with performance feedback, and enhance communication. Therefore, the purpose of this study is to investigate faculty members' perceptions of the processes for using the academic appraisal system at Sultan Qaboos University (SQU). The research questions of this study are as follows: 1. What are faculty members’ perceptions about the system of academic staff appraisal at the College of Education in SQU? 2. Are there statistically significant differences or variations in faculty members’ perceptions about the system of the appraisal of academic staff at the College of Education in SQU due to gender and department? 2. Literature 2.1. Performance Appraisal With the rapid expansion and growth of HEIs nowadays it is very essential for these institutions to assess the performance of their staff. The procedure of appraisal of academic staff is generally known in the literature as performance appraisal process. According to Fashoto, Amaonwu, and Afolorunsho (2018) performance appraisal is defined as “a systematic method used in identifying and determining employee’s performance within a given organization”. It involves all the staff in the institution and it is often used to determine professional growth and based on it both promotions and payment increases are determined (Dangol, 2021). If a faculty member receives higher evaluations, it shows that the staff is performing well, and that the institution is growing. If a staff has lower ratings, the institution can provide them additional training to increase their skills and expertise (Lohman, 2021) which in turn improve their performance and assures the quality of the HEI. It is conducted on a timely basis in order to analyze staffs' growth and success in their work and job tasks (Dangol, 2021). 2.2. The Performance Appraisal Process Performance appraisal system is “a formal interaction that exists between an employee and the supervisor or management conducted periodically to identify the areas of strength and weakness of the said employee” (Fashoto et al., 2018). Research classified performance appraisal process into four steps. Performance appraisal systems include performance criteria that have been defined, a mechanism for evaluating staff performance, a comparison to the standards, and an evaluation of performance based on the comparison (Dangol, 2021; Dasanayaka et al., 2021; Dewi, Astawa, Ernawati, & Suarta, 2019; Lohman, 2021; Sułkowski et al., 2020). The first stage in setting performance criteria is to identify the duties of the staff. Faculty member’s performance is used to determine work criteria. Fashoto et al. (2018), for instance, stated that the management in Kampala International University (KIU) makes use of five factors as the major criteria for evaluating an employee performance such as personal skills, initiatives, teaching quality, method of teaching and research. However, according to AlKubaisi, Aziz, George, and Al-Tarawneh (2019), four factors may be identified as potential discriminators in staff appraisal: employment history to assess experience; job testing to evaluate knowledge; a personality measure to judge friendliness; and lastly college GPA to evaluate success at college. The second phase is to measure staffs’ performance by using certain techniques. Universities use different methods for evaluating faculty members such as the traits approach, behavioral approach, ranking methods, alternation ranking, and outcomes methods, productivity metrics, 360-degree assessment, and Management by Objectives (MBO) (Dangol, 2021). A study conducted by Lohman (2021) considered students’ experience as a major criteria in evaluating faculty and he proposed for faculty cohorts so that educators may learn from one another in the field and receive feedback from peers. Third, there is the comparison against criteria. The individual work record is compared to the job criteria. Fourth, a performance appraisal is produced based on the comparison. Asian Journal of Social Sciences and Management Studies, 2023, 10(4): 121-131 123 © 2023 by the authors; licensee Asian Online Journal Publishing Group The accuracy with which that judgment is applied determines the integrity of appraisal systems. Problems in judgment are always inevitable, and the literature on performance appraisal process has long identified a number of basic problems in judgment that may impact negatively on the appraisal process. Lin and Kellough (2019) identified five problems in performance appraisal process as following: 1. The “halo effect”: it means a supervisor observes a subordinate doing one task well and rates that person high on all tasks (the opposite is possible). 2. The “first impression error”: it reflects that the rating is determined by the initial impression the supervisor forms of the employee. 3. The “similar-to-me effect”: it occurs when supervisors rate employees higher when those employees exhibit behaviors similar to the supervisor’s own actions. 4. Employee “comparison or contrast effects”: it means that employees are rated relative to each other rather than relative to specific criteria and standards. 5. The “central tendency error”: it occurs when employees are rated at the mid-point of rating scales used. 2.3. Perceptions of Faculty on the Performance Appraisal Process Although some have resisted performance appraisal process and argued it is untrustworthy due to human influence, lack of understanding it and feeling unthreatened by it Dauda (2018), it is argued that staff responses to performance appraisals are critical to appraisal success and effectiveness because they are linked to subsequent job attitudes, motivation, and performance (Pichler, 2019). Therefore, it is wise to explore the faculty members’ perceptions on the performance appraisal. Phan, Jones, Person, and Sparks (2021) investigated faculty perceptions of the appraisal process and its overall influence on their teaching performance. The qualitative case study aimed to explore full-time faculty members' perceptions of the present performance appraisal procedure at a local institution (i.e., private university in Ho Chi Minh City, Vietnam). Data from individual, semi-structured interviews with 12 full-time faculty members were evaluated. Following data analysis, key conclusions addressing the improvement of teaching quality were revealed. Second, significant factors influencing faculty favorable opinions of the PA process and teaching approaches were identified. Following on from this, the researchers identified important criteria that teachers feel are essential in an effective performance review process. Furthermore, the leadership tools needed to make the whole process successful. Also, Lin and Kellough (2019) investigated supervisors' perceptions of individual employee performance appraisal in the federal government of the United States in an attempt to uncover structural and operational issues with that system as seen by those responsible for making it operate. Data are obtained from a Merit Systems Protection Board (MSPB) Merit Principles Survey, which included a part asking supervisors to identify nine possible performance appraisal difficulties. A total of 36,926 public employees from 59 federal agencies filled the survey. According to the responses, the most problematic components of the appraisal process were inflated ratings, incorrect criteria, and a lack of assistance. The utilization of performance-based rewards by an agency, a supervisor's opinion that his or her own performance is objectively evaluated, and supervisor age are all consistent predictors of supervisors' perceptions. Similarly, Roine (2018) explored some academics' perceptions on performance appraisal at a Finnish university. A qualitative case study used theme interviews to investigate employee perceptions of performance appraisals throughout the spring and summer of 2017. The interview data was then evaluated by comparing it to Bowen and Ostroff theory. According to the findings of this study, the perceptions of the case faculty members are fairly similar. When the performance appraisal process is studied from a content standpoint, the organizational climate may be viewed as extremely robust. This conclusion is based on the fact that the target faculty members placed a high priority on collegial engagement. When performance appraisal is considered from a process standpoint, however, the outcome is exactly the reverse. A formal procedure was not viewed to play a vital role, but rather as bureaucratic and something to "get over and done with." The findings of this study suggest that, in order to raise the relevance of the process, performance appraisal should be freed of any formality. Different disciplines have also looked at the perceptions of employees on the performance appraisal process. Worku (2019); for example, collected data from 198 Transnet Engineering employees as part of the company's attempt to describe and quantify Transnet Engineering employees' perceptions of the degree to which the company's performance appraisal system was suitable enough for its intended purpose. Employee perception was tested using a predefined benchmark. According to the survey, around 63 % were satisfied with the company's performance appraisal system, while around % had a negative perception. The findings indicated that employees' perceptions of the performance appraisal method employed at Transnet Engineering were influenced by their perceptions of fairness and the assessment of employees' performance based on key performance indicators. Also, Petasis, Christodoulou, and Louca (2020) studied how employees at the Cyprus Electricity Authority perceive performance appraisal. Data were obtained from 161 Organization employees using an online semi-structured questionnaire and processed using SPSS; qualitative data were manually analyzed. The study's findings focused on respondents' perceptions of the Organization's performance management procedures, which have generated a culture of mistrust toward the performance management system in use. The Organization's current performance appraisal system appears to have more flaws than positives, and it does not appear to be integrated or linked with the Authority's organizational goals and missions. 2.4. Summary The expansion of the educational system at all levels, as well as the enhancement of its outputs, has become essential for the development of Omani HEIs. Performance appraisal process is one of the methods that HEIs implement to grantee the quality of the educational system. The reviewed literature showed scholars investigated different areas of performance appraisal process and that HEIs varied in their implementation of performance appraisal. The previous studies investigated performance appraisal either following quantitative approach (Lin & Kellough, 2019) or qualitative approach (Dauda, 2018). This study aimed to investigate the perceptions of faculty members in college of education at SQU on the performance appraisal system using a mixed method approach with a questionnaire and interview. Asian Journal of Social Sciences and Management Studies, 2023, 10(4): 121-131 124 © 2023 by the authors; licensee Asian Online Journal Publishing Group 3. Method The success of an educational institution is inherently tied to the efficacy and quality of its faculty members. To understand and improve upon this vital component, an accurate and insightful assessment of academic staff is crucial. This study specifically focuses on exploring the faculty members’ perceptions of the effectiveness of the appraisal procedures at College of Education in SQU. 3.1. Research Design The research design followed in this paper is mixed-mode design. The methodology implemented in this study was chosen to provide a view of the current appraisal procedures and to offer an in-depth understanding of the faculty members' experiences and opinions. A questionnaire was designed to collect quantitative data on College of Education faculty members' perceptions of the appraisal process. Closed-ended questions were used to collect data that is easily quantifiable and to provide a clear snapshot of the prevailing opinions on the effectiveness of the current procedures. In addition to the questionnaire, semi-structured interviews were conducted with selected faculty members i.e. heads of departments at the College of Education. These interviews provided qualitative insights, allowing the researchers to acquire a clear snapshot of the intricate feelings, experiences, and suggestions of the faculty members. 3.2. Population and Sample The focus of this research was the academic staff at College of Education in SQU, which served as the population for the study. The total population consisted of 114 faculty members. The sample participating in this study was 54 faculty members. Out of the 54 participants, 48 completed the questionnaire, while 6 were chosen for in-depth semi- structured interviews. The sample included a wide range of participants to reflect the diversity within the academic staff. Participants varied in: • Gender: Both male and female faculty members were included to ensure gender diversity. • Qualification: Participants ranged from those with master's degrees to doctoral qualifications, representing a variety of educational backgrounds. • Teaching experience: Faculty members with diverse teaching experiences were included. • Department and specialization: The sample encompassed faculty members from various departments and specializations, reflecting the multidisciplinary nature of the university. Table 1 shows detailed categorization of the sample. 3.3. Instruments To gain a comprehensive understanding of faculty members' perceptions about the effectiveness of the procedures for applying the appraisal of academic staff at College of Education in SQU, two instruments were used: a questionnaire and semi-structured interviews. 3.3.1. Questionnaire The questionnaire was designed to collect quantitative data and was divided into three main parts: 1. Demographic part: This part gathered information about the participants' gender, qualification, teaching experience, department, and specialization. 2. Academic appraisal form (13 questions): This part focused on the faculty members' perceptions and experiences with the academic appraisal form itself, probing aspects such as clarity, relevance, and comprehensiveness. 3. Academic appraisal procedure (12 questions): This part delved into the procedures followed in the academic appraisal process, including aspects like fairness, transparency, and efficiency. The questionnaire consisted of a total of 25 questions and was administered using Google Forms. A link to the Google Form was sent via email to the population, allowing for convenient access and completion. 3.3.2. Semi-Structured Interviews The semi-structured interviews were conducted with heads of departments and were designed to provide qualitative insights. The interviews were divided into four main parts in addition to the demographic part: 1. Validity part (3 questions): The questions aimed to understand the perceived validity of the appraisal process and form. 2. Understandability (3 questions): This part probed how easily faculty members could comprehend the appraisal form and process. 3. Consensus (2 questions): This part examined the extent of agreement among faculty members regarding the appraisal form and procedures. 4. Future (3 questions): This part sought to explore thoughts and suggestions about future improvements to the appraisal process. The semi-structured interviews consisted of a total of 11 open-ended questions, allowing the interviewees to express their detailed opinions and insights. 3.3.3. Validity and Reliability In order to ensure the validity and reliability of the questionnaire, the instrument was reviewed by two experts within the College of Education. After, sending their notes and suggestions within, several modifications were made, including grammar corrections, rephrasing, and adding/removing items. The reliability of the questionnaire was further confirmed by calculating Cronbach's Alpha. The calculated value, 0.82, indicated strong internal consistency. Similarly, the interview guide was reviewed by different two experts, and necessary modifications were made based on their feedback. Asian Journal of Social Sciences and Management Studies, 2023, 10(4): 121-131 125 © 2023 by the authors; licensee Asian Online Journal Publishing Group Table 1. Research sample categorization (N = 54). Category Number Rate Gender Male 40 74% Female 14 26% Qualification Master degree 4 7% Assistant professor 20 37% Associate professor 16 30% Professor 14 26% Teaching experience 1-5 years 2 4% 6-10 years 6 11% 11-15 years 2 4% 16-20 years 4 7% Above 20 years 38 70% Department Department of art education 5 9% Instructional and learning technologies department 10 19% Department of curriculum and instruction 13 24% Educational foundation and administration department 8 15% Psychology department 6 11% Early childhood education department 2 4% Physical education & sport sciences department 5 9% Islamic education department 5 9% 3.4. Data Collection Procedures The data collection for this study was conducted through a carefully planned and systematic approach. The procedures for both the questionnaire and the semi-structured interviews are outlined the subsequent sections. 3.4.1. Questionnaire Administration The Google Forms were used to facilitate a smooth and efficient process for collecting and organizing the quantitative data. After a Google Form for the questionnaire, the link was shared via email to the targeted population of 114 faculty members at the College of Education in SQU. Out of the entire population, 48 faculty members completed the questionnaire. The online format allowed for convenient and confidential participation. The responses were collected electronically and were subsequently analyzed using statistical methods. 3.4.2. Semi-Structured Interview Process Six heads of departments at the College of Education were selected for in-depth semi-structured interviews. The selection was purposeful and to target position holders with insightful perspectives on the appraisal process. The interviews were scheduled at convenient times for the interviewees, and were divided into four main parts along with a demographic part. They were conducted using open-ended questions, allowing for an exploration of the participants' thoughts and experiences. In consideration of ethical principles, the privacy of the participants was maintained throughout the study, with no personal or identifiable information disclosed in the research findings. 3.5. Data Analysis Methods Data analysis in this study was conducted through a combination of quantitative and qualitative methods to ensure a comprehensive interpretation of the findings. The questionnaire responses were processed and analyzed using the Python programming language. For the analysis of the semi-structured interview data, we employed a thematic analysis approach, utilizing Atlas.ti software to facilitate the process. Peel (2020) six-stage thematic analysis was followed. Figure 1 shows the stages followed. Figure 1. Six-stage thematic analysis was followed in this study. Note: Peel (2020). Asian Journal of Social Sciences and Management Studies, 2023, 10(4): 121-131 126 © 2023 by the authors; licensee Asian Online Journal Publishing Group 4. Results and Discussion The primary aim of this study was to explore and understand the faculty members' perceptions of the appraisal procedures at the College of Education in SQU. Specifically, the research sought to address the following questions: 1. What are faculty members’ perceptions about the system of academic staff appraisal at the College of Education in SQU? 2. Are there statistically significant differences or variations in faculty members’ perceptions about the system of the appraisal of academic staff at the College of Education in SQU due to gender and department? In the subsequent sections, the findings related to these questions will be presented and analyzed. 4.1. Faculty Members’ Perceptions about the System of Academic Staff Appraisal 4.1.1. Quantitative Analysis Table 2 summarizes the responses to the questions related to the appraisal procedure, including the means, medians, and modes of responses. With regards to the appraisal form part, the faculty members generally seem to find the appraisal form well-defined, clear, and easy to understand, as reflected in the higher mean values for these items. The organization of the appraisal form (timeline/areas of evaluation/feedback) and its relevance to job tasks is rated moderately positively. However, some aspects of the appraisal form appear to need improvement, such as the perception that the form is not easy to complete, requires time to fill in, and is a lengthy paper-based process. These areas have means closer to the mid-point of the scale. The item regarding the appraisal form being eco-friendly received a lower mean score of 2.67, possibly indicating concerns about environmental impact. As for the appraisal procedure, some figures refracted positive opinions. For example, the clarity and specificity of the appraisal procedure are viewed positively, as indicated by mean scores above 3.5. One more example is the aspects related to self - reflection, feedback, and instructional improvement are also seen favorably, suggesting that respondents see value in the appraisal process for personal and professional growth. Beside these positive opinion indications, some items related to the appraisal procedure have meant close to the mid-point, such as the frequency of conducting the appraisal, the link to promotion and rewards, and flexibility in generating reports. These might reflect a lack of strong consensus among respondents. Some dissatisfaction or room for improvement in the areas of engagement, motivation, and ongoing performance as their results show slightly lower mean scores. Figure 2 shows bar chart for the mean scores for 13 items related to the appraisal form. It shows varying levels of agreement. The highest mean score appears to be for the statement that the "appraisal form's purpose is well- defined," while the lowest score is observed for the statement that the "appraisal form is eco-friendly." Overall, most items related to the clarity and relevance of the appraisal form have received mean scores above 3, suggesting a generally positive perception among faculty members. Figure 2. The mean scores for the appraisal form items in the questionnaire. Table 2. Descriptive analysis. Part Item Mean Median Mode St. dev. Min. Max. A p p ra is al f o rm The appraisal form’s purpose is well-defined. 3.67 4 4 1.03 1 5 The appraisal form has clear guidelines. 3.56 4 4 0.74 2 5 The appraisal form is easy to understand. 3.74 4 4 0.71 2 5 The appraisal form’s criteria reflect real performance. 2.96 3 4 1.21 1 5 The appraisal form criteria are satisfactory. 3.41 4 4 0.88 1 5 The appraisal form is well organized (Timeline/Areas of evaluation/ Feedback or discussion) 3.26 4 4 1.12 1 5 The appraisal form is helpful and relevant to job tasks 3.41 4 4 0.84 2 5 The appraisal form is not easy to complete, and the process of filling in the form needs to be more simplified 3.30 3 2 1.25 2 5 The appraisal form requires time to fill in the evaluation details. 3.37 4 2 1.26 2 5 The appraisal form is a lengthy paper-based process 3.26 4 4 1.39 1 5 The appraisal form is very costly (Printing, distributing, scanning, rekeying, filing & archiving) 2.96 3 3 1.30 1 5 The appraisal form is eco-friendly 2.67 3 2 1.17 1 5 Asian Journal of Social Sciences and Management Studies, 2023, 10(4): 121-131 127 © 2023 by the authors; licensee Asian Online Journal Publishing Group Part Item Mean Median Mode St. dev. Min. Max. The appraisal form allows for effective capture and storage of feedback 3.00 3 3 0.99 1 5 A p p ra is al p ro ce d u re There is a clear timeline for the evaluation. 3.67 4 4 0.91 2 5 The appraisal procedure is clear and specific 3.56 4 4 0.74 2 5 The frequency of conducting the appraisal is appropriate. 3.22 4 4 1.14 1 5 The appraisal helps faculties to indicate their strengths and weaknesses in teaching 3.63 4 4 0.92 1 5 The appraisal is linked to my promotion and rewards 3.22 3 3 1.11 1 5 There is beneficial feedback in the performance appraisal 3.63 4 4 0.96 1 5 The appraisal procedure leads to self-reflection. 3.74 4 4 0.85 2 5 The appraisal procedure leads to instructional improvement. 3.44 3 3 1.00 1 5 The appraisal form is a meaningful active ongoing evaluation. 3.56 4 4 0.92 2 5 The appraisal procedure allows for more frequent communication and assessment. 3.22 3 4 1.00 1 5 The appraisal procedure increases engagement, motivation and ongoing performance. 3.15 3 4 0.94 1 5 The appraisal procedure is flexible in terms of generating reports. 3.26 3 3 0.97 1 5 Figure 3 focuses on the remaining items that concern the procedure of the appraisal system. It reveals that respondents generally agree that aspects such as "self-reflection," "clear timeline for the evaluation," and "meaningful active ongoing evaluation" are present in the system. However, there are also concerns, such as the flexibility of the appraisal form in generating reports, reflected in lower mean scores. Figure 3. The mean scores for the appraisal procedure items in the questionnaire. Figure 4 represents the distribution of the most frequent responses (modes) across all items. It provides a high- level view of the dominant opinions shared by the respondents. For example, the highest percentage (68.0%) associated with a response of 4 indicates that a majority of the items were rated with this value by the respondents, reflecting a general agreement or satisfaction with the aspects evaluated. Figure 4. The distribution of the most frequent responses (#2, #3, and #4) across all items. 4.1.2. Qualitative Analysis The qualitative analysis aimed at providing more insights about the reasons for the faculty members’ opinions, especially in areas where there is more variation or where the mean scores are closer to the mid -point of the scale such as such as the frequency of conducting the appraisal, the link to promotion and rewards, and flexibility in Asian Journal of Social Sciences and Management Studies, 2023, 10(4): 121-131 128 © 2023 by the authors; licensee Asian Online Journal Publishing Group generating reports. We interviewed six academic staff with various qualifications, teaching experience, and areas of specialization. All respondents have a PhD in education and specialized in different majors in college of education at SQU. They have 17 to 30 years of teaching experience and have been in the current position between 3 -5 years. The analysis of the results highlighted four major themes related to the validity of the academic staff appraisal form, its clarity and understandability, academic staff’s consensus of the appraisal form and some suggestions for the future. In the following sections, the respondents are identified as respondent 1, respondent 2, etc. 4.1.3. Validity The validity of the academic staff appraisal form was explored through interviews, revealing diverse sub-themes: • Achievements and accomplishments: Respondents 3 and 6 highlighted that the appraisal process helps gather accomplishments, giving a record of milestones. • Understanding of job performance: Respondents 3 and 6 expressed that the appraisal contributes to understanding job performance, while respondent 3 also pointed out the lack of a students' survey as a limitation. • Identifying priority areas and setting objectives: Respondent 6 positively regarded these aspects, emphasizing how the process assists in focusing on improvement areas and developing future goals. • Electronic Support: Respondent 5 appreciated the electronic appraisal system for allowing easy access to past evaluations. Some divergence was noted in how the appraisal influences performance and skills development. While respondent 1 felt it didn't contribute, respondents 2 and 4 saw some assistance in their performance development, and respondent 5 emphasized self-responsibility over the appraisal process. The respondents' assessments further covered: • Defining expectations: Respondent 1 believed that defining expectations assists in achieving desired results. • Academic freedom and flexibility: Respondent 4 stressed not limiting academic freedom, while respondent 6 noted the importance of a clear plan for teaching skills. • Progress assessment and goal setting: Respondent 5 spoke about analyzing success and setting new milestones. Overall, the individual responses indicate that the academic staff appraisal process's validity is multifaceted, with different respondents emphasizing different aspects, both positive and negative, of the procedures at the College of Education in SQU. 4.1.4. Understandability Interview responses revealed sub-themes related to the understandability and clarity of academic staff assessment. These sub-themes encompass achievement, self-assessment and personal growth, suitability assessment, documentation of achievements, and the improvement of weaknesses and development of strengths. Respondent 1 found the purpose of the evaluation process unclear, viewing it primarily as a means to consolidate achievements into a single document. Respondent 2, however, saw it as a tool for self-appraisal and personal growth. Respondent 5 linked the process to aligning individual aims with college objectives, emphasizing the documentation of accomplishments. Respondent 3 perceived the evaluation as assessing suitability for roles, while respondent 6 believed it helped in identifying and improving weaknesses and enhancing strengths. Some interviewees expressed concerns about how the assessment process addressed behaviors and attitudes. While respondent 1 felt it lacked specificity, respondent 2 found it informative about strengths and areas for growth. Respondent 3 noted the provision of feedback to aid in behavior modification, but respondent 4 emphasized the focus on outcomes rather than professional conduct. Results also illuminated issues with the appraisal form's instructions and guidelines, including a lack of clarity (respondent 1), length and redundancy (respondent 4), and a preference for an electronic system (respondent 3). These observations suggest variations in the user-friendliness of the form, with some finding it overwhelming and others seeing it as clear and straightforward. 4.1.5. Consensus The respondents expressed diverse opinions on the fairness and accuracy of the academic staff appraisal process. In terms of fairness, varied clarifications were evident. Respondent 1 doubted the process could provide a fair idea about the instructor, while respondent 2 found the outcomes fair. Respondent 3 had reservations due to the absence of a student survey, and respondent 4 deemed it outright unfair. Respondent 5 questioned what "fair" meant in the context, noting the absence of incentives or consequences, while respondent 6 highlighted the lack of both quantitative and qualitative evaluation methods. These varying viewpoints accentuate the need to address issues to enhance the appraisal process's fairness. As for accuracy, opinions ranged from optimism to uncertainty. Respondent 1 expressed a lack of faith in the process, questioning the reliability of the information used. Respondent 2 had some trust in the procedure but saw room for improvement. Respondent 3 displayed a balanced view, acknowledging potential benefits despite possible inaccuracies. Respondent 4 voiced strong doubts, attributing them to biases and subjectivity. Respondent 5 commended the process for providing accurate data, while respondent 6 held a moderate belief in its correctness, acknowledging both potential accuracy and error. 4.1.6. Future The interviews provided insights on potential improvements for the procedure of appraising faculty members in the future: • Simplification and electronic handling: Respondent 1 suggested that the process be simplified and managed electronically to reduce complexity and facilitate effective data retrieval. • Focus on actual performance: Respondent 2 emphasized aligning the appraisal process closely with academic staff performance, concentrating on essential performance indicators and relevant topics. Asian Journal of Social Sciences and Management Studies, 2023, 10(4): 121-131 129 © 2023 by the authors; licensee Asian Online Journal Publishing Group • Quality over quantity: Respondent 4 urged a shift from quantity-based evaluation to an assessment of the impact, effectiveness, and overall quality of the work. • Clear evaluation and data retrieval system: Respondent 6 called for clear evaluation methods and a solid framework for an objective review process, backed by an effective data retrieval system for easy access to essential information. Collectively, these insights suggest a need for a more streamlined and user-friendly experience, with a focus on quality and clear, accurate evaluation. The goal is to create a meaningful and fair process that promotes the growth of academic staff and aligns their performance with organizational objectives. Additional recommendations include eliminating unnecessary components, giving equal credit to various types of publications, and streamlining the structure of the assessment form. There is also a call for more accurate Key Performance Indicators to enhance the evaluation's objectivity. In summary, the future development of the evaluation process should concentrate on streamlining, focusing on crucial features, and enhancing objectivity. 4.2. Differences or Variations in Perceptions The second research question is investigating whether there are statistically significant differences in faculty members' perceptions of the appraisal procedure at the College of Education in SQU. These perceptions are being examined based on several variables including gender, specialization, qualification, and academic degree. The subsequent sections present the statistical tests were performed to answer this research question. 4.2.1. Normality Check Before investigating the statistical significance of differences in faculty members' perceptions of the appraisal procedure at the College of Education in SQU, a normality check was performed. The Shapiro-Wilk test, a widely- used method for testing the normality of a distribution, was applied to the perceptions measured by the questionnaire. The results of the Shapiro-Wilk test indicated that none of the tested questionnaire items followed a normal distribution, as all p-values were found to be less than the chosen significance level (alpha) of 0.05. The P values ranged from 0.0000000142369 - 0.000305991125. This finding has implications for the selection of subsequent statistical tests, as it recommends the use of non-parametric methods. These results guided the next steps in the analysis, leading to the selection of appropriate non-parametric tests to further investigate the faculty members' perceptions of the appraisal procedure. Table 3 summarizes the results of the non-parametric tests performed in this research. 4.2.2. Mann-Whitney U Test Based on the research question and the number of groups (2 groups) for the gender variable, the Mann-Whitney U Test was selected as non-parametric test. The Mann-Whitney U Test allows for the comparison of two different independent groups i.e. gender. Table 3. Results of the non-parametric tests. Variable: Gender Department Test conducted: Mann-Whitney U Kruskal-Wallis P values: − Min: 0.0032 − Max: 0.973 − Min: 0.0013 − Max: 0.9046 Statistically significant differences item: − # of items: 6 − % of items: 24% − # of items: 5 − % of items: 20% No statistically significant differences items: − # of items: 19 − % of items: 76% − # of items: 20 − % of items: 80% Results: The gender variable has no impact on faculty members’ perceptions of the appraisal system The department variable has no impact on faculty members’ perceptions of the appraisal system The results of the Mann-Whitney U test on the gender variable indicate that, for the majority of the items (19 out of 25), there is no statistically significant difference between the two gender groups in their perceptions of the appraisal procedure at the College of Education in SQU. This suggests that gender does not have a strong or consistent impact on perceptions of the appraisal system. However, for 6 specific items, there were statistically significant differences. These particular aspects may warrant further investigation to understand why gender might influence perceptions in these areas. It's possible that these differences are due to other underlying factors or specific characteristics of the appraisal process related to those items. For example, the item of “The appraisal form requires time to fill in the evaluation details” and the item of “The appraisal form is a lengthy paper-based process”. 4.2.3. Kruskal-Wallis Test Although the College of Education consists of 8 departments, faculty members from only 5 departments participated in the study. Therefore, department variable consists of 5 different groups. As the responses to the appraisal-related items are non-normally distributed and there 5 groups, the Kruskal-Walli’s test was selected to analyze the differences between the departments. The Kruskal-Walli’s test is a non-parametric statistical test that is used to compare the medians of two or more independent groups. It is often used to determine if there are statistically significant differences between the medians of the groups being compared. In our study, the Kruskal-Walli’s test was applied to the Department variable to compare the perceptions across the five departments concerning 25 different items related to the appraisal process. The results indicate that there are no statistically significant differences between the departments for the majority of the items (20 out of 25). This suggests a general consensus on those aspects of the appraisal process. Therefore, further pairwise comparisons were not conducted. As for the 5 items, the Asian Journal of Social Sciences and Management Studies, 2023, 10(4): 121-131 130 © 2023 by the authors; licensee Asian Online Journal Publishing Group test revealed significant differences (p < 0.05), indicating that opinions varied significantly across departments on those specific aspects of the appraisal process. These items may need further investigation to understand the underlying differences. 5. Conclusion This study set out to explore the perceptions of faculty members at the College of Education about the procedures for applying the academic appraisal form at SQU. Through a mixed-methods approach using both questionnaire and semi-structured interview techniques, and engaging with 48 participants in the questionnaire and 6 interviewees, several key findings were revealed. With regard to the results of quantitative analysis, the faculty members generally perceived the appraisal form as well-defined, clear, and easy to understand. However, areas such as ease of completion, time required, and the lengthy paper-based process suggest a need for improvement (Aggarwal & Thakur, 2013). While the clarity and specificity of the appraisal procedure were positively viewed, areas like frequency of conducting the appraisal, link to promotion and rewards, and flexibility in generating reports reflected a lack of strong consensus or some dissatisfaction. This study found no statistically significant differences in faculty members’ perceptions of the appraisal system based on gender or department. This aligns with or contrasts with Kim and Holzer (2016) other studies in the field. As for the qualitative analysis, the results showed that the multifaceted nature of the academic staff appraisal process's validity, as seen in this study, resonates with broader discussions in the literature (Steers & Lee, 2017). The varying opinions on the user-friendliness of the form mirror similar findings in previous research (Aggarwal & Thakur, 2013). The diverse views on fairness and accuracy may reflect the broader debate in the field about these issues (Iqbal, Akbar, & Budhwar, 2015). 5.1. Findings The study's findings indicate that while there are positive aspects of the current appraisal system, there are clear areas for improvement. The insights gathered from this study can inform future refinements of the academic appraisal form and procedure at SQU. Moreover, the lack of significant differences in perceptions based on gender and department may reflect consensus among the faculty members. The findings may suggest also future development to focus on simplification, electronic handling, and enhancing objectivity echoes calls in the existing literature for more streamlined and transparent evaluation processes. 5.2. Limitations The current study acknowledges certain limitations that may have influenced the findings. Firstly, the small sample size may limit the generalizability of the results to the broader population of faculty members. Secondly, the interview process did not include key decision-makers within the college, such as the dean and dean's assistants, which might have provided a more comprehensive understanding of the appraisal system from different perspectives. 5.3. Recommendations Based on the study insights, there is a clear recommendation for the development of a comprehensive web-based platform for faculty appraisal. Such a system would potentially facilitate the process, increase its validity, and increase the acceptance among faculty members. Emphasizing user-friendliness, transparency, and alignment with the real needs of the academic staff, a web-based platform could significantly enhance the existing appraisal practices and contribute to more meaningful evaluations. References Aggarwal, A., & Thakur, G. S. M. (2013). Techniques of performance appraisal-a review. International Journal of Engineering and Advanced Technology, 2(3), 617-621. AlKubaisi, M., Aziz, W. A., George, S., & Al-Tarawneh, K. (2019). Multivariate discriminant analysis managing staff appraisal case study. Academy of Strategic Management Journal, 18(5), 1-12. Dangol, P. (2021). Role of performance appraisal system and its impact on employees motivation. Quantitative Economics and Management Studies, 2(1), 13-26. https://doi.org/10.35877/454ri.qems119 Dasanayaka, C. H., Abeykoon, C., Ranaweera, R., & Koswatte, I. (2021). The impact of the performance appraisal process on job satis faction of the academic staff in higher educational institutions. Education Sciences, 11(10), 1-20. https://doi.org/10.3390/EDUCSCI11100623 Dauda, Y. (2018). A review of performance appraisal systems in different countries: The UK, India, South Africa and Ghana. International Journal of Applied Environmental Sciences, 13(2), 203-221. Dewi, P. E. O., Astawa, I. K., Ernawati, N. M., & Suarta, I. K. (2019). The impact of hotel staff appraisal to career development. International Journal of Green Tourism Research and Applications, 1(1), 73-84. https://doi.org/10.31940/ijogtra.v1i1.1632 Egginton, B. E. (2010). Introduction of formal performance appraisal of academic staff: The management challenges associated with effective implementation. Educational Management Administration & Leadership, 38(1), 119-133. https://doi.org/10.1177/1741143209351660 Fashoto, S. G., Amaonwu, O., & Afolorunsho, A. (2018). Development of a decision support system on employee performance appraisal using AHP model. JOIV: International Journal on Informatics Visualization , 2(4), 262-267. https://doi.org/10.30630/joiv.2.4.160 Iqbal, M. Z., Akbar, S., & Budhwar, P. (2015). Effectiveness of performance appraisal: An integrated framework. International Journal of Management Reviews, 17(4), 510-533. Kim, T., & Holzer, M. (2016). Public employees and performance appraisal: A study of antecedents to employees’ perception of the process. Review of Public Personnel Administration, 36(1), 31-56. https://doi.org/10.1177/0734371x14549673 Lin, Y.-C., & Kellough, J. E. (2019). Performance appraisal problems in the public sector: Examining supervisors’ perceptions. Public Personnel Management, 48(2), 179-202. https://doi.org/10.1177/0091026018801045 Lohman, L. (2021). Evaluation of university teaching as sound performance appraisal. Studies in Educational Evaluation, 70, 101008. https://doi.org/10.1016/j.stueduc.2021.101008 Peel, K. L. (2020). A beginner's guide to applied educational research using thematic analysis. Practical Assessment, Research, and Evaluation, 25(1), 2. https://doi.org/10.7275/ryr5-k983 Petasis, A., Christodoulou, S., & Louca, A. (2020). Employees perception on performance appraisal effectiveness in the Cyprus public s ector (EAC). International Journal of Human Capital Management, 4(1), 26-41. Phan, N., Jones, V., Person, D., & Sparks, S. L. (2021). Perceptions of faculty on the performance appraisal process. Universal Journal of Educational Research, 9(9), 1649–1655. https://doi.org/10.13189/ujer.2021.090905 https://doi.org/10.35877/454ri.qems119 https://doi.org/10.3390/EDUCSCI11100623 https://doi.org/10.31940/ijogtra.v1i1.1632 https://doi.org/10.1177/1741143209351660 https://doi.org/10.30630/joiv.2.4.160 https://doi.org/10.1177/0734371x14549673 https://doi.org/10.1177/0091026018801045 https://doi.org/10.1016/j.stueduc.2021.101008 https://doi.org/10.7275/ryr5-k983 https://doi.org/10.13189/ujer.2021.090905 Asian Journal of Social Sciences and Management Studies, 2023, 10(4): 121-131 131 © 2023 by the authors; licensee Asian Online Journal Publishing Group Pichler, S. (2019). Performance appraisal reactions: A review and research agenda. Feedback at Work, 75-96. Roine, A. (2018). Employee perceptions of performance appraisal. Retrieved from https://core.ac.uk/download/pdf/250151953.pdf Steers, R. M., & Lee, T. W. (2017). Facilitating effective performance appraisals: The role of employee commitment and organizational climate. Performance Measurement and Theory , 75-93. https://doi.org/10.4324/9781315211947-5 Sułkowski, Ł., Przytuła, S., Borg, C., & Kulikowski, K. (2020). Performance appraisal in universities—Assessing the tension in public service motivation (PSM). Education Sciences, 10(7), 1-19. https://doi.org/10.3390/educsci10070174 Worku, Z. (2019). A study of employee perceptions about performance appraisal at transnet engineering, South Africa. Journal of Applied Business Research, 35(5), 145-156. https://doi.org/10.19030/jabr.v35i5.10307 Asian Online Journal Publishing Group is not responsible or answerable for any loss, damage or liability, etc. caused in relation to/arising out of the use of the content. Any queries should be directed to the corresponding author of the article. https://core.ac.uk/download/pdf/250151953.pdf https://doi.org/10.4324/9781315211947-5 https://doi.org/10.3390/educsci10070174 https://doi.org/10.19030/jabr.v35i5.10307