Review of Economics and Development Studies Vol. 4, No 2, December2018 179 Volume and Issues Obtainable at Center for Sustainability Research and Consultancy Review of Economics and Development Studies ISSN:2519-9692 ISSN (E): 2519-9706 Volume 4: No.2,December 2018 Journal homepage: www.publishing.globalcsrc.org/reads Student Evaluations of Teaching in Universities of Pakistan: Analysis from the Perspective of Closing the Feedback Loop 1 Hakim Ali, 2 Bashir Hussain 1 PhD Scholar, Department of Education, Bahauddin Zakariya University, Multan &Associate Professor, Government WHISL College, Multan, Pakistan hakimalimphil@gmail.com 2 Assistant Professor, Department of Education,Bahauddin Zakariya University, Multan, Pakistan bashirhussain@bzu.edu.com ARTICLEDETAILS ABSTRACT History Revised format: November 2018 Available Online: December 2018 Internationally, centralized systems for collecting students’ feedback have become an increasingly common practice in higher education institutions [HEIs] for monitoring quality of teaching as well as for professional development of faculty members. The collection, analysis and reporting of evaluation results are carried out systematically in many HEIs across the globe. However, how to effectively close the feedback loop with students as well as teachers on the results of student evaluations of teaching [SETs] remain an issue to be addressed. Consistent with global trend, HEIs in Pakistan are also supposed to conduct SETs. In this context, the main intention of this study was to determine whether the cycle of teaching evaluation process is completed, and feedback loop proceeds effectively towards closing around the SETs or not in Pakistani HEIs. To achieve the objective, the triangulation design was used in which an online search was carried out in 130 Pakistani HEIs’ official websites to collect qualitative data. Concurrently, a questionnaire comprising 13 close-ended items, with “yes-and-no” scale, was administered in online format to collect quantitative data from a sample of 507 faculty members and 110 administrators from 130 Pakistani HEIs. Based on content analysis of documents and descriptive analysis of participants’ responses, this paper concludes that universities in Pakistan are bound by HEC to collect feedback from students using central system of Quality Enhancement Cells, but limited attention has been placed to close the feedback loop with students and teachers to inform improvements. Finally, this paper recommends the need for universities in Pakistan to genuinely listen to students’ voices and to act on their feedback as part of quality assurance. © 2018 The authors, under a Creative Commons Attribution- NonCommercial 4.0 Keywords Student evaluations of teaching, feedback loop, engagement with feedback, quality assurance, Pakistani HEIs. JEL Classification: A20, I20, I23 Corresponding author’s email address: hakimalimphil@gmail.com Recommended citation:Ali, H. and Hussain, B. (2018). Student Evaluations of Teaching in Universities of Pakistan: Analysis from the Perspective of Closing the Feedback Loop. Review of Economics and Development Studies, 4 (2), 179-187 DOI: 10.26710/reads.v4i2.402 http://www.publishing.globalcsrc.org/reads mailto:bashirhussain@bzu.edu.com Review of Economics and Development Studies Vol. 4, No 2, December 2018 180 1. Introduction and Literature Review Formal systems of evaluating faculty teaching, typically comprising student feedback employing varied instruments and mechanisms, are common in higher education institutions [HEIs] across the globe. SETs are generally required by HEIs as a means to explore students’ perspectives on the faculty instruction (Hativa, 2013); to evaluate various aspects of faculty teaching quality and to “compare it across different courses, teachers, departments and institutions” (Goos & Salomons, 2016). Stein et al. (2013) have revealed that faculty members are usually supposed to support student evaluations of their teaching as a part of an institutional evaluation system. These studies, however, also indicate that a substantial difference exists in how institutions practically implement a formal evaluation system (Stein et al., 2013); how students, teachers and administrators engage with the practice of evaluation (Moskal, Stein & Golding, 2016) and use the evaluation results for closing the feedback loop. The literature on SETs is extensive and wide-ranging which ranges from the variety of the experiences of different cohorts of students (Tucker, 2013) to effect of students’ feedback on teaching and learning (Hattie & Timperley, 2007); “the need for effective student engagement” (Krause & Coates, 2008) and on the importance of seeking experiences of first year students (Kift, Nelson, & Clarke, 2010). Furthermore, literature also reveals the “link between student satisfaction and student feedback” (Grebennikov & Skaines, 2009); procedures used and their effect on students’ satisfaction and response rates (Moskal et al., 2016) and the strategies used for improving students’ satisfaction (Nelson, Smith & Clarke, 2012). Even though the substantial studies have been conducted formerly, there is paucity of knowledge about the practices that HEIs adopt in relation to the systematic use of students’ feedback for improving teaching, enhancing students’ experiences and renewing course designs (Tucker, 2013). A number of scholars (i.e., Leckey & Neill 2001; Nair, Mertova, & Pawley, 2010; Shah & Nair, 2009; Watson, 2003) found that students are disinclined to take part in upcoming SETs surveys, if they do not see any improvements ensuing from their feedback. Earlier scholars argued that failure of universities in systematically closing the SETs feedback loop leads to various hazards comprising decreasing response rates from students (Leckey & Neill, 2001); their disinterest in the feedback process (Nair et al., 2010; Shah & Nair, 2009) and lack of trust between students, academics and universities on improvements owing to their voice (Tucker et al., 2008; Watson, 2003). This failure to close the feedback loop on SETs creates such environment that students do not provide their feedback seriously and can also result in frustrating students, rather than providing constructive feedback (Tucker, Jones & Straker, 2008). Moreover, “failing to close the loop on feedback questions the quality assurance framework in institutions and the extent to which they are used to enhance educational quality” (Shah, Cheng & Fitzgerald, 2017). The concept of ‘closing the loop’ is based on the notion that there is a cycle of feedback and action. Figure 1 presents the generic feedback cycle with allied key moments illustrated eloquently by Harvey (2003). Figure 1: The cycle of feedback Note: Adapted from Harvey, L. 2003, ‘Editorial’, Quality in Higher Education, 9(1), 3-20 Student feedback is viewed as a significant measure for assurance of teaching as well as institutional quality (Goos & Salomons, 2016) and majority of the HEIs worldwide consider it vital to their annual monitoring and periodic Generic FeedbackCycle Stakeholde r develop questionna ire Questionnai re administrati on Analysis of data Report noting areas for action Consultati on process Action plan Implementati on and monitoring Feedback to stakeholder s Review of Economics and Development Studies Vol. 4, No 2, December2018 181 review processes. However, simply collecting and analyzing such feedback is unlikely to lead to improvements (Shah & Nair, 2009) unless its results are communicated in a way that is informative to stakeholders, mainly students and teachers i.e., closing the loop. Symons (2006) emphasized that universities need to complete the feedback cycle not only with students but strategies should also be developed for ensuring the successful closing of the loop with teachers. Writing in the same vein, Scott (2006) advocates that simply collecting students comments and providing each faculty member a list of these comments would not be sufficient to provide them with adequate information on experiences of students. Vital to the dissemination of student feedback comments to stakeholders, is their “analysis into meaningful data” (Symons, 2006) and timely provision of feedback results. ‘Closing the feedback loop’ is the term, generally, refers to the “process of informing respondents about what happens to the results of any survey” (Watson, 2003). In this article, closing the feedback loop refers to systematic processes in collecting, analyzing, reporting and sharing the student feedback results with stakeholders particularly students and faculty members; and “timely actions taken...as a direct result of student feedback” (Shah et al., 2017) as well as implementing and “monitoring the effectiveness of actioned improvements” (Shah et al., 2017). How to close this loop effectively has been an issue that universities have struggled with since the 1990s (Watson, 2003). The rationale behind this paper is that universities/HEIs worldwide are using surveys at institutional and national level to assess students’ experiences regarding faculty’s teaching effectiveness. For example, the National Student Survey (NSS) is employed in the UK, the Course Experience Questionnaire (CEQ) in Australia and the National Survey of Student Engagement (NSSE) is employed in the USA and Canada (Price & Baker, 2012). In the context of international scenario, Higher Education Commission [HEC] of Pakistan also introduced a system for monitoring and carrying out periodic teaching evaluations of university faculty through students to assess faculty’s teaching effectiveness as well as students’ engagement in learning (Batool, Qureshi & Raouf, 2010). Institutions, using the results of SETs surveys, identify their weak and strong areas and advertise their strengths for marketing themselves to prospective students. Furthermore, these results are used for recognizing, rewarding and reviewing teachers and in some instances “distributing internal funding” (Shah & Nair, 2013). Likewise, there is also evidence of using student feedback results for making decisions about institutional rankings and appraising institutional performance (Johnson, 2000). However, how HEIs effectively use students’ feedback to improve teachers’ professional practices and students’ learning outcomes remains an area to be explored, specifically in the context of expansion and diversification of higher education as well as higher students’ expectations. Furthermore, there is lack of evidence concerning effective engagement of stakeholders in communicating feedback results and in taking actions for improvement i.e., closing the feedback loop effectively; despite the fact, this is an important strategy to enhance students’ response rates and to identify areas in need of improvement (Grebennikov & Shah, 2013). 1.1 The Present Study The literature demonstrates that the cycle of closing the loop on SETs involves four key aspects i.e., data collection, analysis, reporting and finally providing feedback to stakeholders. Based on international practices, this paper analyzes SETs practice in universities of Pakistan from the perspective of closing the feedback loop and argues for the need to develop innovative ways to use feedback results for faculty professional development and engage academics as well as students in quality assurance activities. However, to date, there is limited research on the usage patterns of results of student feedback in Pakistan for improving faculty instructional practices and the students’ learning experience. Keeping in view this background, present study mainly analyzed the practice of closing the feedback loop on SETs in Pakistani universities at policy and practice level to genuinely know that how results from student feedback have been utilized for improving students’ learning as well as faculty instructional practices. The rationale behind this research work was also the emphasis of Pakistani higher education sector on quality assurance mechanism, management of academic quality, the enhancement in faculty professional development as well as improving learning experiences of students. 1.2 Research Questions The major purpose of this study was to determine whether teaching evaluation process is completed from holistic perspective and feedback loop proceeds effectively towards closing around SETs or not in Pakistani HEIs. Following research questions were formulated for analyzing practice of Pakistani universities regarding closing the feedback loop around SETs: Review of Economics and Development Studies Vol. 4, No 2, December 2018 182 a. Do policies of HEIs provide any guidelines for reporting of results to various stakeholders and closing the feedback loop on SETs? b. What is the nature of data collection process in HEIs of Pakistan in relation to closing the feedback loop on SETs? c. Are the data analyzed in meaningful format to report the stakeholders? d. Are the results of analysis reported to all the stakeholders? e. Is the feedback resulting from SETs survey communicated to teachers and students to effectively close the loop on SETs in HEIs of Pakistan? 2. Design and Methods To achieve the objective, concurrent triangulation approach (Creswell, 2009) was used in this study. The key objective of this design is to use qualitative and quantitative approaches simultaneously to explore in depth the same aspects of the research problem (Creswell, 2009). The major benefits of this traditional mixed methods design include: the shorter data collection time and well-validated findings (Creswell, 2009). The population for this study, both for qualitative and quantitative components, comprised such 130 Pakistani HEIs in which Quality Enhancement Cells [QECs] have been established before 2014. For qualitative data collection, the researchers carried out an online search in all the 130 HEIs’ official websites to collect publicly accessible documents (QUAL) related to key aspects of SETs process to gain as full a picture as possible about closing the loop on SETs. For quantitative component, a questionnaire (QUAN) was administered with faculty members and administrators to further confirm and triangulate the results of content analysis of documents. The participants from all the 130 HEIs were invited to complete the survey in online format using a ‘census approach’. From the 6933 email invitations that were sent to obtain informed consent of teachers and administrators including staff members of QECs, 1783 (25.7%) participants (1417 teachers and 366 administrators) from 91 universities across Pakistan agreed to participate. Finally, link to online survey questionnaire, administered via Google forms, was e-mailed to these 1783 potential participants. After 2 to 5 reminders with the interval of 10 days, total 617 completed questionnaires were received from 507 teachers and 110 administrators including QEC staff members. Respondents were representative from the general, engineering, agriculture, medical, business and arts HEIs with a range of positions. The response rate was about 36% and 30% respectively which is reasonable for a voluntary survey. Table 1 presents the demographic information of all the participants who participated in this study. Table 1: Showing Demographic Information of Respondents Position Percentages University Category Percentages Lecturer 34.2 General 64.2 Assistant Professor 40.4 Engineering 19.6 Associate Professor 4.5 Agriculture 7.3 Professor 2.6 Medical 0.8 Chairman/HoD 7.8 Business 7.9 Dean 1.8 Arts 0.2 Director/Registrar 8.3 A self-developed questionnaire comprising 13 close-ended items, with “yes-and-no” scale, was used to elicit perceptions of the participants. Section A of the questionnaire asked for demographic information while section B explored current practices (Q1–13). The questionnaire was piloted with 17 participants from the five faculties of one public university i.e., social sciences, natural sciences, commerce and business management, engineering, agriculture and veterinary along with registrar and director quality enhancement cell (QEC). A number of changes were incorporated on the bases of respondents’ feedback to make the language and format of items more understandable and for ensuring the alignment of items with research questions. Expert opinion was also sought for improving the questionnaire items’ content validity. For the analysis of qualitative data (i.e., online accessible documents), a summative approach to qualitative content analysis (Hsieh & Shannon, 2005) was used. This approach involves both the ‘manifest content analysis’ and ‘latent content analysis’ aspects. ‘Manifest content analysis’ quantifies the particular concepts in textual material and deals with the descriptive and objective overview of the “surface meaning of the data” (Dornyei, 2007, p. 246). Review of Economics and Development Studies Vol. 4, No 2, December2018 183 Latent content analysis refers to the process of interpretation of content (Hsieh & Shannon, 2005). As the details of targeted concepts and key words were unavailable in accessible documents, the analysis could not proceed to ‘latent’ level and stopped at ‘manifest’ level (Kondracki & Wellman, 2002). Participants’ responses on close-ended questionnaire were analyzed by applying descriptive statistics (i.e., Frequency Counts and Percentage) and the results were presented in Table 3, Table 4 and Table 5. 3. Findings 3.1 Findings regarding Manifest Content Analysis In total, 47 online accessible documents and consolidated teaching evaluation reports were examined. The analysis specifically focused on searching key words related to all the aspects of SETs process i.e., data collection, analysis, reporting, use of results and provision of feedback with a specific focus on exploring the existence of policy guidelines for closing the loop on feedback. It is worth noting that even a trivial indication of some evaluation activity was considered as an evidence/example of the existence of SETs process and frequency of occurrences was presented in Table 2. Table 2:Manifest Content Analysis of Accessible Documents Aspects of SETs Process Number of documents mentioning a specific aspect Documents mentioning detailed procedure Data Collection 47 4 Data Analysis 13 0 Reporting and use of Results 6 0 Use of Results 1 0 Provision of Feedback 0 0 The analysis of documents displayed in Table 2 revealed that all the HEIs in Pakistan were bound to conduct SETs as mandatory policy by HEC and instrument for evaluation of faculty teaching by students was available on the websites of all the 130 HEIs. The content analysis of documents further showed that detailed information and policy guidelines regarding all the aspects of SETs process as well as about the process of closing the feedback loop with students and faculty members was unavailable in documents obtained through search of institutional websites. 3.2 Findings regarding Participants’ Perceptions The participants were asked to respond in ‘yes’ or ‘no’ to key aspects of the SETs process (i.e., data collection, analysis, reporting and provision of feedback) to determine the extent to which these aspects are effectively and efficiently attended during SETs. The other intent of this section was to determine whether SETs process is completed and feedback loop proceeds effectively towards closing around student evaluations or not. Table 3 summarizes the frequency counts and percentages of “yes” responses concerning the process of data collection; Table 4 related to the process of data analysis and reporting of results while Table 5 about provision of feedback. Table 3: Summary of Perceived Processes Used to Collect Data on SETs Aspects of Process “yes” % age Regularity of use Have you ever experienced/used student evaluations of teaching? 416 67.4 Methods of Data collection Student evaluations of teaching (through questionnaire) 383 92.1 Course evaluations by students 236 56.7 Student interviews 37 8.9 Informal student opinions 86 20.7 Students as observers 41 9.8 Nature of Instrument Used HEC Prescribed/Developed 177 42.5 Self-developed by the university 103 24.8 Not Known 136 32.7 Frequency of Using Completed at the end of the each semester 366 88.1 Twice per semester 15 3.6 Completed Annually 27 6.5 Completed as required by the teacher 13 3.1 Completed as required by the administration 48 11.5 Review of Economics and Development Studies Vol. 4, No 2, December 2018 184 Completed at the time of best teacher award decisions 12 2.9 Delivery of SETs Form via Teacher 3 0.7 Department representative 79 19.0 QEC Staff 221 53.1 Online access to student 113 27.2 Mode of collection Online 233 56.0 Manual 174 41.8 Not Known 9 2.2 Response Rate by Students 0-25% 186 44.7 26-50% 40 9.6 51-75% 102 24.5 76-100% 20 4.8 Not Known 68 16.4 In response to the question (Have you ever experienced/used student evaluation of teaching?) asked to assess the actual practice/regularity in the use of SETs, the analysis of data in Table 3 revealed that majority of the respondents (i.e., 67.4%) believe that SETs are conducted on regular basis in their institutions. It shows, however, almost one third (32.6%) of the participants perceive that this practice is still missing in their institutions and not being actually done regularly though exist in documented policy. The less than half of the participants (42.5%) reported that their universities use HEC prescribed/developed student evaluation of teaching instrument, 24.8%, the lowest number of participants, also claimed that they use self- developed/adapted as per format of HEC student evaluation form while a considerable number of participants (32.7%) reported that they have no knowledge about the nature of instrument used. This result suggests that all the stakeholders either not involved/ consulted or do not take interest in development/selection of instrument. Analysis further revealed with 88.1% responses that almost all the HEIs in Pakistan use student evaluation instrument at the end of each semester. The percentage of ‘yes’ responses for the delivery of instrument depicts that there is general consistency across the country that faculty members are kept away/at arms-length from the process of data collection during SETs. Most often, QEC staff administers the student evaluation instrument for collection of data either manually (41.8%) or online (56%). Majority of the participants (9.6+44.7%) reported response rate below 50%. Table 4: Summary of Perceived Process for Analysis of Data and Reporting of Results Aspects of Process ‘yes’ % age Data Analyzed by Teacher 06 1.4 Department representative 75 18.0 Staff of QEC 295 70.9 Not Sure 40 9.6 Reporting of Data Is the aggregated data reported to stakeholders? 173 41.6 Mode of Reporting of Aggregated Data Means 149 86.1 Standard Deviation 08 4.6 Graphic form 11 6.4 Percentage 05 2.8 Table 4 reveals that collected data is analyzed by staff of QEC and teachers as well as departments are mostly kept away from the teaching evaluation process at this step also in majority of the HEIs. The question regarding reporting of data attracted 41.6% ‘yes’ responses which indicates that aggregated and analyzed data is not reported to stakeholders in majority of the HEIs in which student evaluation is actually conducted. Further analysis revealed that data is mostly reported in the form of mean scores (86.1% yes), graphic form (6.4% yes) or in standard deviation (4.6% yes). Review of Economics and Development Studies Vol. 4, No 2, December2018 185 Table 5: Summary of Perceived Process for Provision of Feedback on Evaluation Results Aspects of Process ‘yes’ % age Access to Students’ Comments Teacher 157 37.7 Department Chair 261 62.7 QEC Staff 291 69.9 Not Provided 48 11.5 Access to End Results Students 12 2.9 Teacher 204 49.0 Department Chair 249 59.9 QEC Staff 210 50.5 Not Provided 89 21.5 Provision of Feedback on Evaluation Results To students 23 5.5 To teacher 137 33.0 Not Provided 256 61.5 Table 5 indicates that often students’ comments are available to chair (62.7% yes) and staff of QEC (69.9% yes) and teachers (37.7% yes + not provided 11.5% yes) are rarely provided access to students’ comments. Almost same situation can be observed regarding access to end results of student evaluations. These results point out a big weakness in the existing SETs process i.e., no access to students’ comments and end results to all the faculty members which is the most frequent and effective source of formative feedback to teachers. Analysis in Table 5 further revealed that evaluation results are never fed back to students (5.5% yes) and rarely to their teachers (33.0% yes). This result suggests that the results are simply filed away and not disseminated to those who could use them at both faculty and institutional level. It means that the time and energy spent in the collection and analysis of this valuable information is wasted and that reports on the student feedback results do not reach to students and faculty members who are the most relevant stakeholders in university community. 4. Discussion This mixed method study analyzed the Pakistani universities’ practices of closing the feedback loop on SETs. A number of scholars (i.e., Hattie & Timperley, 2007; Shah & Nair, 2013) argue that universities ought to develop and disseminate detailed guidelines for all the stakeholders regarding their roles and responsibilities in SETs process, use of evaluation reports; interpretation of results as well as provision of feedback for effective implementation of institutional teaching evaluation system. Writing in the same vein, Shah and Nair (2013) emphasize that similar guidelines should also be developed for members of promotion panels regarding the use of SETs results. The results of content analysis of documents in this study, however, revealed that clear-cut policies and detailed guidelines are unavailable in accessible documents regarding all the aspects of SETs process i.e., closing the feedback loop. Limited evidence is available in documents on systematic usage of feedback from SETs surveys by academics in Pakistani universities to review curriculum content, assessment designs and teaching methods in a timely manner. The analysis of participants’ responses also confirms the results of content analysis. Moreover, provision of feedback resulting from SETs is increasingly being considered essential across the globe as a means to guide teaching practice, (Hattie & Timperley, 2007), to enhance faculty professional development (Blair & Noel, 2014), to improve students’ learning and to strengthen faculty development policies and practices (Catano & Harvey, 2011). However, the findings herein suggest that Pakistani universities do not make systematic usage of the student feedback results to inform improvements. The results of this study further postulate that SETs process in Pakistani HEIs, is generally limited to the collection and analysis of data, and results are filed away, not fed back and communicated to stakeholders particularly students and teachers. Overall, the results of this study are aligned well with the studies conducted by (Scott, 2006; Shah & Nair, 2009; Symons, 2006) who concluded that generally universities simply collect and analyze SETs data and do not complete feedback cycle with stakeholders. But these results contradict with the suggestions of Harvey (2003) who advocates that “views of students should be integrated into a regular and continuous cycle of analysis, reporting, action and feedback” to make an effective contribution to quality improvement in HEIs. Harvey (2003) further Review of Economics and Development Studies Vol. 4, No 2, December 2018 186 stressed that ensuring an appropriate action; providing feedback in transparent, objective as well as consistent manner and making reports publicly available to all the stakeholders is more important to gain support and trust from all the stakeholders than only concentrating on having mechanisms for data collection. 5. Conclusion The results of content analysis of documents and survey responses evidently suggest that closing the feedback loop is an area which requires improvement in Pakistani HEIs. Although, all the universities in Pakistan are bound by HEC for collecting, analyzing and reporting student survey results, but there is limited evidence of effective use of student feedback to enhance faculty professional development and improve students’ learning experiences. It was concluded that though SETs exist in Pakistani HEIs as a documented policy but the extent of their actual use was notably low and there were problems/challenges of actual implementation on regular basis. There also appears to be little meaningful feedback of student evaluation results to students and faculty members, mainly because universities have not developed clear-cut policies and detailed guidelines for execution of SETs process. Universities in Pakistan conduct SETs only to fulfill HEC administrative requirements not with any formative or summative intention. It was also concluded that HEIs in Pakistan are experiencing low response rates on SETs, which is worldwide concern of HEIs (Smithson et al., 2015). The matter of low response is reflective of the skepticism of the students regarding the surveys and feedback. This aspect of low response, therefore, should be examined. This situation calls for the development and implementation of innovative ways by universities to listen to the students’ voices because students, being fee payers, are constantly demanding that their opinions should be heard and acted upon. References Batool, Z., Qureshi, R.H. & Raouf, A. (2010). Performance evaluation standards for the HEIs. Islamabad: Higher Education Commission, Pakistan. Blair, E., & Noel, K. V. (2014). Improving higher education practice through student evaluation systems: Is the student voice being heard? Assessment & Evaluation in Higher Education, 39, 879–894. Catano, V. M., & Harvey, S. (2011). Student perception of teaching effectiveness: Development and validation of the Evaluation of Teaching Competencies Scale (ETCS). Assessment & Evaluation in Higher Education, 36(6), 701– 717. Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3 rd ed.). Thousand Oaks, CA: Sage Publications. Dornyei, Z. (2007). Research methods in applied linguistics: Quantitative, qualitative and mixed methodologies. Oxford, UK: Oxford University Press. Goos, M., Salomons, A. (2016). Measuring teaching quality in higher education: assessing selection bias in course evaluations. Res High Educ 58: 341– 364. DOI 10.1007/s11162-016-9429-8 Grebennikov, L., & Skaines, I. (2009). University of Western Sydney students at risk: Profile and opportunities for change. Journal of Institutional Research, 14(1), 58–70. Harvey, L. (2003). Student feedback. Quality in Higher Education, 9(1), 3-20. Hativa, N. (2013). Student ratings of instruction: A practical approach to designing,operating, and reporting. Oron Publications. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. Hsieh, Hsiu-Fang & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15 (9), 1277-1288. Johnson, R. (2000). The authority of the student evaluation questionnaire. Teaching in Higher Education, 5(4), 419–434. Kift, S., Nelson, K., & Clarke, J. (2010). Transition pedagogy: A third generation approach to FYE—A case study. The International Journal of the First Year in Higher Education, 1(1), 1-20. Kondracki, N. L.,&Wellman, N. S. (2002). Content analysis: Review of methods and their applications in nutrition education. Journal of Nutrition Education and Behavior, 34, 224-230. Krause, L. K., & Coates, H. (2008). Students’ engagement in first-year university. Assessment and Evaluation in Higher Education, 33(5), 493-505. Leckey, J., & Neill, N. (2001). Quantifying quality: The importance of student feedback. Quality in Higher Education, 7(1), 19-32. Moskal, A. C.M., Stein, S. J. & Golding, C. (2016). Can you increase teacher engagement with evaluation simply by improving the evaluation system?Assessment and Evaluation in Higher Education, 419(2), 286-300. Nair, C. S., Mertova, P., & Pawley, D. (2010). Quality in action: Closing the loop. Quality Assurance in Education, Review of Economics and Development Studies Vol. 4, No 2, December2018 187 18(2), 144-155. Nelson, K. J., Smith, J. E., & Clarke, J. A. (2012). Enhancing the transition of commencing students into university: An institution-wide approach. Higher Education Research and Development, 31(2), 185–199. Price, K., & Baker, N. S. (2012). Measuring students’ engagement on college campuses: Is the NSSE an appropriate measure of adult students’ engagement? The Journal of Continuing Higher Education, 60(1), 20–32. Scott, G. (2006). Accessing the student voice: using CEQuery to identify what retains and promotes engagement in productive learning in Australian higher education. Canberra: DEST. Shah, M., Cheng, M. & Fitzgerald, R. (2017). Closing the loop on student feedback: The case of Australian and Scottish universities. Higher Education, 74(1), 115-129. Shah, M., & Nair, C. S. (2009). Using student voice to improve student satisfaction: Two Australian universities— The same agenda. Journal of Institutional Research (South East Asia), 7(2), 43–55. Shah, M., & Nair, C. S. (2013). Enhancing student feedback and improvement systems in tertiary education. CAA Quality Series No. 5 (Vol. June, pp. 16– 33). Abu Dhabi: Ministry of Higher Education and Scientific Research. Smithson, J., Birks, M., Harrison, G., Sid Nair, C., & Hitchins, M. (2015). Benchmarking for the effective use of student evaluation data. Quality Assurance in Education, 23(1), 20–29. Stein, S. J., Spiller, D., Terry, S., Harris, T., Deaker, L., & Kennedy. J. (2013). Tertiary teachers and student evaluations: Never the Twain shall meet? Assessment and Evaluation in Higher Education, 38 (7), 892– 904. Symons, R. (2006). In their own words: Finding out what students think about their university learning experience. Synergy, (23), 34‐35. Tucker, B. (2013). Student evaluation to improve the student learning experience: An Australian university case study. Educational Research and Evaluation, 19(7), 615–627. Tucker, B., Jones, S., & Straker, L. (2008). Online student evaluation improves Course Experience Questionnaire results in a physiotherapy program. Higher Education Research and Development, 27(3), 281-296. Watson, S. (2003). Closing the feedback loop: Ensuring effective action from student feedback. Tertiary Education Management, 9(2), 145–157.