Microsoft Word - Tracz-Vol1Num2.docx Vol. 1, No. 2, January 2013, pp. 63-82 Effectiveness of Teacher Education Programs: A Descriptive Aggregation of Supervisor Ratings of Teachers Trained by The Renaissance Group (TRG) and the Teacher Education Council of State Colleges and Universities (TECSCU) Institutions Susan M. Tracz, Ph.D. California State University, Fresno Abstract This study aggregated supervisor’s ratings of teachers trained at The Renaissance Group (TRG) and the Teacher Education Council of State Colleges and Universities (TECSCU) institutions. Fourteen studies conducted by 12 universities or states were gathered, and 12 that met inclusion criteria were included in this analysis. The total number of survey items in all studies (N = 374) was coded into 13 variables. Frequencies of the percentages of ratings falling into below average, average, and above average for those variables were tabled. Supervisors consistently gave teachers very high ratings. 64 S. Tracz Background Of paramount importance to P-12 education in this country is the quality of its teachers. Not only are public and private schools concerned with hiring the best teachers available, but schools and departments of education at universities are fully cognizant that their own credibility rests on the performance of the teachers they train. Research has demonstrated over and again that teachers do have a great impact on the education of the students they teach. The ratings supervisors give teachers about their knowledge and skills have often been used as indicators of teaching effectiveness, and by extension, of the effectiveness of university education programs. Although numerous universities routinely conduct internal studies by surveying principals or other teacher supervisors, these studies are not well disseminated, and the results are largely unknown. In contrast, a conservative group is currently using a single study which purportedly found that a large proportion of teacher supervisors are highly unsatisfied with their teachers because their university training was lacking. The overt agenda of this message is that university education programs are ineffective. Purpose Therefore, the purpose of this study was to aggregate the results of studies conducted by The Renaissance Group and TECSCU university education programs to determine how supervisors rate the teachers trained at these universities. Methods Study Design. Typically, when researchers want to determine the global effect of a phenomenon, they combine individual studies that examined the topic of concern. This type of design is called meta-analysis, which is generally presents effect sizes in terms of a standardized difference between the means for two groups or correlation coefficients. The data that is available for aggregation does not conform to the requirements for a meta-analysis. However, it was possible to combine this data in with descriptive aggregation technique that is described below. Studies Included. The Director of The Renaissance Group solicited studies and reports of supervisor ratings of teachers and teacher ratings of their training. Fourteen studies were submitted in a timeframe that enabled them to be included in this aggregation study. Each study was considered a separate study if it was send as a separate file, although some studies have multiple parts and some universities send separate files for their various studies. Ultimately, the aggregation of the data will make these distinctions irrelevant. The universities and states represented in this sample of studies follow: Effectiveness of Teacher Education 65 Table 1 Description of the studies available for this descriptive aggregation report University Study Components N Rating Scale N of Items Study Included 1 Eastern Kentucky Univ Principals eval teachers, Superintendents eval principals, Graduates eval education program 193 5 pt scale: Poor, Below aver, Aver, Above aver, Outstanding 27 Yes 2 Montana State Univ Billings Employers eval teachers 85 5 pt scale: Strongly disagree, Disagree, Neutral, Agree, Strongly agree 11 Yes 3 Montana State Univ Billings Teachers eval education program Not available 3 pt scale: Low preparation, Moderate preparation, High preparation 6 No 4 Ball State Univ Employers eval teachers 29 3 pt scale: Disagree, Agree, Strongly Agree 4 Yes 5Emporia State Univ Administrators eval teachers, Graduates eval education program Not available 2 pt scale: Need help or not prepared, Acceptable level of preparation 1 Yes 6California State Univ System Supervisors eval teachers, Graduates eval education program 3781 1 pt scale: Well or adequately prepared 15 Yes 7Kean Univ Employers eval teachers Not available 5 pt scale: Insufficient, Beginning, Adequate, Competent, Strong 72 Yes 8Univ of Mary Washington Employers eval teachers 75 5 pt scale: Not satisfactory, Needs improvement, Satisfactory, Good, Excellent 35 Yes 9Radford Univ Principals eval teachers 25 5 pt scale: Poor, Below aver, Aver, Above aver, Outstanding 23 Yes 10 Public & Private Univ, State of Illinois Supervising employers eval teachers 8144 2 pt scale: Other, Mostly or extremely prepared by teacher education program to be a successful teacher 108 Yes 66 S. Tracz University Study Components N Rating Scale N of Items Study Included 11 Lamar Univ at Beaumont Supervisors eval teachers 28 4 pt scale: Not at all prepared, Not sufficiently prepared, Sufficiently prepared, Well prepared 30 Yes 12 Lamar Univ at Beaumont Teachers eval education program 4484 4 pt scale: Not at all prepared, Not sufficiently prepared, Sufficiently prepared, Well prepared 30 No 13 Longwood Univ Supervisors eval teachers 201 5 pt scale: 1 Not prepared at all, 2, 3, 4, 5 Outstanding 15 Yes 14 Longwood Univ Supervisors eval teachers 58 3 pt scale: Below aver, Aver, Above aver 39 Yes The majority of these studies report supervisor ratings of teachers after their initial teacher training, and only those study results are combined here. Some studies report teacher ratings of their teacher education programs or supervisor ratings of teachers receiving graduate training in counseling, leadership, special education, and those results are not combined here. Of the 14 studies, 2 studies and parts of other studies were not included, and 12 studies are combined. Variables Coded. A wide variety of survey items examining teacher competence were included in these 12 studies, and no two surveys were even closely similar in these items. The number of items per survey ranged from 1 to 108. In an effort to code survey items into consistent variables, the following 13 items were developed: 1. Dispositions – personal characteristics or tendencies of teachers, not necessarily affected by teacher education programs, 2. Knowledge Base – Content knowledge demonstrated by teachers, 3. Pedagogy – Knowledge of and ability to apply the methods and principles of teaching, 4. Classroom Management/Environment – Knowledge of students and ability to effectively manage classroom environment and activities, 5. Technology – Knowledge of and ability to teach effectively using technology, 6. Assessment – Ability to correctly ascertain student knowledge and learning, 7. Reflective Practice – Continuous self-examination of one’s teaching practices in an effort to improve, 8. Diversity/Equity – Cultural competence and ability to teach so all students are treated in an equitable manner, Effectiveness of Teacher Education 67 9. Disabilities – Knowledge of and ability to teach students with disabilities effectively, 10. Professional Workplace/Relations – Ability to act in a professional manner and to have good relationships with co-workers, 11. Parents/Guardians/Community – Ability to have good relationships with parents and the community, 12. Enhancing teacher quality – Processes that encourage and support teacher quality, 13. Overall Effectiveness – Comprehensive evaluation of teaching effectiveness. Reliability. The author coded all items into the variables described above. In an effort to ascertain reliability, the Director of The Renaissance Group, the dean and associate dean from CSU Fresno, were also asked to rate 2 to 5 studies. After discussion of the meanings of the 13 variables, the inter-rater reliability as a percent of agreement of 93% was achieved. This was deemed to be acceptable for this aggregation study. Rating Scales. As can be seen in Table 1, a wide variety of rating scales were chosen in the individual studies. Reported scales ranged from 1 to 5 points. In many studies, only part of the entire scale was reported. One study did not report a below average rating, one study did not report an average or midpoint rating, and one did not report a high rating. Another reported a combined the midpoint and high rating, while still another reported only an “other” rating in addition to the high rating. The only commonality is that most studies reported at least one above average point and most studies reported a midpoint rating, so those will be reported in the summary over all studies. Finally, the graph only reports the high ratings. Data Collection Procedures The following procedures were done for each individual study. Each survey item was coded into one of the 13 variables listed above. A data set consisting of each survey item, the rating scale for that study, and the variable code was developed. Also input into that data set were the sample sizes, the frequencies of all responses, the percentages of all responses, and means as were available. Studies rarely had all that information reported. Analysis After the data set was compiled, it needed to be put into a format that allowed for aggregation. A set of consistent collapsing of ratings was developed. Each study’s ratings were combined into below average, average and above average categories. AS discussed in the Ratings section, not all studies had all these categories, so only the ones that were available were compiled. When only frequencies were available, percentages were calculated. Once each study had a percentage of responses for each item in below average, average and above average categories, those percentages were tallied for each of the 13 variables. The aggregate ranges of those percentages were reported by study for each of the variables when the number of studies combined was 1 to 5. When more than 5 68 S. Tracz studies were aggregated per variable, the lowest and highest percentages were eliminated and the resulting ranges were reported. The reason for this is that a percentage can be calculated on a very few number of supervisor ratings so it is possible to greatly skew the results with only a few items. Results Tables 2 through 13 present the ranges of percentages of variables in rating categories over the individual studies. Table 14 presents the ranges of percentages for the average and the above average categories aggregated over all studies. The number of items combined to produce these results was 374. Figure 1 presents a graph of the above average categories aggregated over all studies. Except for the Disability variable, the ranges of above average percentages for all other variables have a low of 49.3% to 100.0% indicating substantial percentages of supervisors believe that their teachers are highly competent. The ranges for the average or competent category are also high, and the vast majority of ratings fall into those two categories. The range for the Disability variable is 21.4% to 87.0%. ! Figure 1. Ranges of Percentages for Above Average Supervisor Ratings of Teachers on Items from 12 Studies from TRG and TECSCU Institutions. Effectiveness of Teacher Education 69 Conclusions In general, supervisors give their teachers high marks. Most of teachers are rated as either average or above average. Only a small percentage of teachers are not considered to be competent. Recommendations The value of aggregating the evaluations supervisors give teachers over teacher education institutions cannot be overemphasized. This information would allow the public, school districts and researchers to assess the institutions that train the teachers their schools hire. In order to accomplish this goal, some recommendations follow. These recommendations are not meant as criticism of the studies whose results are reported here. Although many education programs at universities conduct their own research studies, combining their studies is difficult due to the plethora of methods and instruments used. 1. A common instrument should be developed. Such an instrument could have some questions that are the same for all institutions and also allow for institutions to ask additional questions to address their individual research interests. 2. A common rating scale should be agreed upon. This scale should have at least 3 ratings: below expectation, meets expectation, and above expectation. 3. Common definitions or rubrics for the ratings should be written so evaluating supervisors can approach a common understanding of what the ratings mean. Most notably, what is truly considered above average is often defined in numerous ways: some believing that a teacher has to reach an outstanding level of performance much like that of a veteran teacher to be considered above average, while others view anything above the minimal acceptable level of performance to be above average. 4. Instruments should be administered at a similar time during the academic year. 5. Supervisors, principals or other persons who rate teachers should be very familiar with their teaching practices. 6. Data should be input into a common data template that asks for items, codes, frequencies, and percentages. 7. A format for presenting the aggregation of the results should be agreed upon. 70 S. Tracz Table 2 Results for Eastern Kentucky University (Study #1) Codes for Variables Range of % for Items N of Items Combined Poor or Below Average Average Outstanding or Above Average 1 Dispositions -- -- -- 0 2 Knowledge Base 2.6 21.2 76.2 1 3 Pedagogy 2.6-11.5 24.1-34.7 55.5-72.3 9 4 Classroom Mgt/Environment 2.6-5.1 23.8-31.3 63.6-73 3 5 Technology 3.6-5.3 32.5-34.7 60.0-63.9 2 6 Assessment 3.1-6.2 26.6-41.9 52.3-69.7 5 7 Reflective Practice 3.8-6.8 33.5-38.6 57.6-59.7 2 8 Diversity/Equity 5.9 33.9 60.2 1 9 Disabilities -- -- -- 0 10 Prof Workplace/ Relations 1-2 23.4-28.5 69.5-75.5 3 11 Parents/Guardians/Community 2.6 32.8 64.6 1 12 Enhancing Teacher Quality -- -- -- 0 13 Overall Effectiveness -- -- -- 0 Study Citation: Principals’ Perceptions of Eastern Kentucky University (EKU) Teacher Effectiveness of Teacher Education 71 Table 3 Results for University Montana State University Billings (Study #2) Codes for Variables Range of % for Items N of Items Combined Disagree or Strongly Disagree Neutral Agree or Strongly Agree 1 Dispositions -- -- -- 0 2 Knowledge Base 7.0 14.0 79.0 1 3 Pedagogy 6.0-9.0 16.0-18.0 74.0-76.0 3 4 Classroom Mgt/Environment 4.0 26.0 69.0 1 5 Technology 7.0 18.0 75.0 1 6 Assessment -- -- -- 1 7 Reflective Practice 8.0 23.0 69.0 1 8 Diversity/Equity 8.0-12.0 20.0-42.0 50.0-69.0 2 9 Disabilities -- -- -- 0 10 Prof Workplace/ Relations -- -- -- 0 11 Parents/Guardians/Community 5.0 29.0 66.0 1 12 Enhancing Teacher Quality -- -- -- 0 13 Overall Effectiveness -- -- -- 0 Study Citation: Follow-Up Studies of Graduates and Employers Comments: * Data entered incorrectly for this item. Only beginning teachers evaluated by supervisors are reported. Evaluations for teachers and counselors who received advanced degrees are not reported here. 72 S. Tracz Table 4 Results for Ball State University (Study #4) Variables Range of % for Items N of Items Combined Disagree Agree or Strongly Agree 1 Dispositions -- -- 0 2 Knowledge Base 0.0 100.0 1 3 Pedagogy 3.6 96.4 1 4 Classroom Mgt/Environment -- -- 0 5 Technology 0.0 100.0 1 6 Assessment -- -- 0 7 Reflective Practice -- -- 0 8 Diversity/Equity 3.4 96.6 1 9 Disabilities -- -- 0 10 Prof Workplace/ Relations -- -- 0 11 Parents/Guardians/Community -- -- 0 12 Enhancing Teacher Quality -- -- 0 13 Overall Effectiveness -- -- 0 Study Citation: Ball State University Teacher Fair Survey Summary Effectiveness of Teacher Education 73 Table 5 Results for University Emporia State University (Study #5) Variables Range of % for Items N of Items Combined Need Help or Not Prepared Acceptable Level of Preparation 1 Dispositions -- -- 0 2 Knowledge Base -- -- 0 3 Pedagogy -- -- 0 4 Classroom Mgt/Environment -- -- 0 5 Technology -- -- 0 6 Assessment -- -- 0 7 Reflective Practice -- -- 0 8 Diversity/Equity -- -- 0 9 Disabilities -- -- 0 10 Prof Workplace/ Relations -- -- 0 11 Parents/Guardians/ Community -- -- 0 12 Enhancing Teacher Quality -- -- 0 13 Overall Effectiveness 3.8-5.0 96.2-95.0 3 Study Citation: Follow-up Study of Graduates and Employers, Emporia State University Comments: Data presented for 6 years from 2000 to 2006 74 S. Tracz Table 6 Results for California State University System (Study #6) Variables Range of % for Items N of Items Combined Adequately and Well Prepared 1 Dispositions -- 0 2 Knowledge Base 81.5-85.0 3 3 Pedagogy 81.5-84.3 5 4 Classroom Mgt/Environment 84.2 1 5 Technology 81.4 1 6 Assessment 81.1 1 7 Reflective Practice -- 0 8 Diversity/Equity 78.6-80.5 2 9 Disabilities 78.6 1 10 Prof Workplace/ Relations -- 0 11 Parents/Guardians/Community -- 0 12 Enhancing Teacher Quality -- 0 13 Overall Effectiveness 81.3 1 Study Citation: Learning to Teach: Comparing the effectiveness of three pathways Comments: Data presented for 6 years from 2006 to 2011 Effectiveness of Teacher Education 75 Table 7 Results for Kean University (Study #7) Codes for Variables Range of % for Items N of Items Combined Insufficient or Beginning Adequate Competent or Strong 1 Dispositions 0 0-20 80-100 9 2 Knowledge Base 0 7.7-20 80-92.3 3 3 Pedagogy 0.0-20.0 0.0-20.0 69.3-100.0 21 4 Classroom Mgt/Environment 0.0-10.0 0.0-20.0 70.0-100.0 6 5 Technology 0.0-10.0 15.4-20.0 70.0-84.6 3 6 Assessment 0.0-10.0 10.0-30.8 60.0-90.0 9 7 Reflective Practice 0.0 7.7-30.0 70.0-92.3 3 8 Diversity/Equity 0.0-10.0 10.0-30.8 69.3-80.0 3 9 Disabilities -- -- -- 0 10 Prof Workplace/ Relations 0.0 0.0 100.0 3 11 Parents/Guardians/Community 00.0-200. 0.0-30.0 50.0-100.0 9 12 Enhancing Teacher Quality -- -- -- 0 13 Overall Effectiveness 0.0 0.0 100.0 3 Study Citation: Survey of Employers Comments: Just tables are provided for this study. 76 S. Tracz Table 8 Results for University of Mary Washington (Study #8) Codes for Variables Range of % for Items N of Items Combined Needs Improvement or Not Satisfactorya Satisfactoryb Excellent or Goodc 1 Dispositions 3.57-4.65 4.3-14.3 82.1-95.5 2 2 Knowledge Base 0.0 13.0-24.1 75.9- 100.0 3 3 Pedagogy 0.0-3.57 8.7-17.9 82.1- 100.0 6 4 Classroom Mgt/Environment 3.6-8.3 4.3-17.9 78.6-95.8 6 5 Technology 0.0-10.7 13.0-14.3 75.0- 100.0 3 6 Assessment 0.0-4.2 17.4-17.9 78.6-95.8 3 7 Reflective Practice -- -- -- 0 8 Diversity/Equity 0.0-7.1 13.0-21.4 75.0- 100.0 6 9 Disabilities -- -- -- 0 10 Prof Workplace/ Relations 0.0 0.0 100.0 1 11 Parents/Guardians/Community 0.0-4.3 8.7-10.7 85.7- 100.0 3 12 Enhancing Teacher Quality 0.0-7.4 7.4-13.0 85.2-87.0 2 13 Overall Effectiveness -- -- -- 0 Study Citation: University of Mary Washington, Mary Washington College, Education Department, Employer Satisfaction Survey Comments: Data for this study was collected between 2009 and 2011. a for part of this study, this category was labeled Not Very Effective or Ineffective. b for part of this study, there was no middle category. c for part of this study, this category was labeled Effective or Highly Effective. Effectiveness of Teacher Education 77 Table 9 Results for Radford University (Study #9) Codes for Variables Range of % for Items N of Items Combined Poor or Below Average Average Above Average or Outstanding 1 Dispositions -- -- -- 0 2 Knowledge Base -- -- -- 0 3 Pedagogy 0.0-4.2 24.0-50.0 50.0-76.0 8 4 Classroom Mgt/Environment 0.0 20.0-40.0 60.0-80.0 5 5 Technology 0.0 8.0 92.0 1 6 Assessment 0.0 25.0-41.7 58.3-75.0 3 7 Reflective Practice -- -- -- 0 8 Diversity/Equity 0.0 23.8-41.7 58.3-76.2 2 9 Disabilities -- -- -- 0 10 Prof Workplace/ Relations 0.0-4.0 12.0-20.0 80.0-84.0 2 11 Parents/Guardians/Community 9.1 36.4 54.6 1 12 Enhancing Teacher Quality 0.0 15.0 85.0 1 13 Overall Effectiveness -- -- -- 0 Study Citation: Radford University Survey of Program Graduates and Principals Comments: Data for this study was collected in 2010. 78 S. Tracz Table 10 Results for 12 Public and 3 Private Universities in Illinois (Study #10) Variables Range of % for Items N of Items Combined Other* “Mostly” / “Extremely” prepared by teacher educ prog to be successful new teacher 1 Dispositions -- -- 0 2 Knowledge Base -- -- 0 3 Pedagogy 4-20 80-96 18 4 Classroom Mgt/Environment 14-20 80-86 12 5 Technology 10-19 81-90 6 6 Assessment 10-14 86-90 6 7 Reflective Practice -- -- 0 8 Diversity/Equity 9-32 68-91 18 9 Disabilities 13-17 83-87 6 10 Prof Workplace/ Relations 7-15 85-93 24 11 Parents/Guardians/ Community 11-32 68-89 18 12 Enhancing Teacher Quality -- -- 0 13 Overall Effectiveness -- -- 0 Study Citation: Teacher Graduate Assessment: A Statewide Assessment of Professional Teacher Education Programs in Illinois. Illinois Association of Deans of Public Colleges of Education Comments: *Categories for responses other than “Mostly” or “Extremely” prepared by teacher education program to be a successful new teacher are not reported. N2006 = 1454, N2007 = 1501, N2008 = 1564, N2009 = 1433, N2010 = 1352, N2011 = 840 Effectiveness of Teacher Education 79 Table 11 Results for Lamar University at Beaumont (Study #11) Codes for Variables Range of % for Items N of Items Combined Not Sufficiently Prepared or Not At All Prepared Sufficiently Prepared Well Prepared 1 Dispositions -- -- -- 0 2 Knowledge Base -- -- -- 0 3 Pedagogy -- -- -- 0 4 Classroom Mgt/Environment 3.6-7.1 28.6-46.4 46.4-67.9 4 5 Technology 0.0-3.9 37.0-80.8 15.4-59.3 10 6 Assessment -- -- -- 0 7 Reflective Practice -- -- -- 0 8 Diversity/Equity 10.7-36.4 50.0-63.6 13.6-30.4 7 9 Disabilities 14.3-28.6 35.7-64.3 21.4-42.9 7 10 Prof Workplace/ Relations -- -- -- 0 11 Parents/Guardians/Community 7.1 42.9 50.0 1 12 Enhancing Teacher Quality -- -- -- 0 13 Overall Effectiveness -- -- -- 0 Study Citation: Texas Education Agency SB 174 Principal Pilot Survey Comments: Data collected in 20010. Unusual rating categories with 2 categories for below average with only 1 category for above average may possibly have skewed the results. 80 S. Tracz Table 12 Results for Longwood University (Study #13) Codes for Variables Range of % for Items N of Items Combined Insufficient or Beginning Adequate Competent or Strong 1 Dispositions 15.9 0.0 84.8 1 2 Knowledge Base 0.0 0.0 100.0 1 3 Pedagogy 0.0 2.0-27.4 72.6-98.0 6 4 Classroom Mgt/Environment 0.0-18.9 25.9-43.8 49.3-74.1 3 5 Technology 0.0 23.4 76.6 1 6 Assessment 0.0 25.9 74.1 1 7 Reflective Practice 0.0 35.8 64.2 1 8 Diversity/Equity -- -- -- 0 9 Disabilities -- -- -- 0 10 Prof Workplace/ Relations -- -- -- 0 11 Parents/Guardians/Community 1.5 47.8 50.7 1 12 Enhancing Teacher Quality -- -- -- 0 13 Overall Effectiveness -- -- -- 0 Study Citation: Longwood University Professional Educational Unit Employer Survey 2010-2011 Comments: Data collected in 2009. Effectiveness of Teacher Education 81 Table 13 Results for Longwood University (Study #14) Codes for Variables Range of % for Items N of Items Combined Below Averagea Averagea Above Averagea 1 Dispositions 0.0-4.0 19.0-35.0 66.0-81.1 10 2 Knowledge Base -- -- -- 0 3 Pedagogy 0.0 20.0-38.0 61.0-80.0 10 4 Classroom Mgt/Environment 0.0 31.0-50.0 50.0-70.0 5 5 Technology 0.0 27.0 73.0 1 6 Assessment 0.0 27.0-31.0 69.0-73.0 3 7 Reflective Practice 0.0 31.0 69.0 1 8 Diversity/Equity 0.0 31.1-42.0 58.0-69.0 3 9 Disabilities 0.0 35.0 65.0 1 10 Prof Workplace/ Relations 0.0 31.0 69.0-70.0 2 11 Parents/Guardians/Community 0.0 38.0 62.0 1 12 Enhancing Teacher Quality 0.0 27.0-31.0 70.0-74.0 2 13 Overall Effectiveness -- -- -- 0 Study Citation: Longwood University Professional Educational Unit Employer Survey 2010-2011 Comments: Data collected in 2010-2011. a Ratings were reported on a 5 point scale, but descriptors for those numbers were not reported. 82 S. Tracz Table 14 Ranges of Average and Above Average Percentages for All Items and All Variables Aggregated Over 12 Studies Codes for Variables Range of % for Items N of Items Combined Average Min – Max Above Average Min - Max 1 Dispositions 0.0 - 35.0 66.0 - 100.0 22 2 Knowledge Base 0.0 - 21.2 76.2 - 100.0 22 3 Pedagogy 2.0 - 38.0 55.5 - 100.0 82 4 Classroom Mgt/Environment 4.3 - 46.4 49.3 - 95.8 42 5 Technology 13.0 - 34.2 60.0 - 100.0 32 6 Assessment 17.4 - 41.7 58.3 - 90.0 20 7 Reflective Practice 7.7 - 38.6 57.6 - 80.0 32 8 Diversity/Equity 13.0 - 42.0 50.0 - 91.1 32 9 Disabilities 35.0 - 64.3 21.4 - 87.0 32 10 Prof Workplace/ Relations 0.0 - 31.0 69.5 - 100.0 28 11 Parents/Guardians/Community 8.7 - 42.9 50.0 - 100.0 18 12 Enhancing Teacher Quality 7.4 - 31.0 70.0 - 87.0 5 13 Overall Effectiveness* 0.0 - 95.0 a 81.3b - 100.0 7 *The interpretability of this item is difficult as only three studies had items in this category. One of these studies had low, middle and high ratings, one only had one category of “adequately and well prepared,” and the last one only had two ratings “not prepared” and “acceptable level of preparation.” a Based on only one study which combined only had “acceptable level of preparation.” b Based on only one study which one category that combined adequately and well prepared.