TOWARDS AN OBJECTIVE EVALUATION OF TEACHER PERFORMANCE Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for non- commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-nd/2.5/. All other uses must be approved by the author(s) or EPAA. EPAA is published jointly by the Mary Lou Fulton College of Education at Arizona State University and the College of Education at the University of South Florida. Articles are indexed by H.W. Wilson & Co. Please contribute commentary at http://epaa.info/wordpress/ and send errata notes to Sherman Dorn (epaa-editor@shermandorn.com). EDUCATION POLICY ANALYSIS ARCHIVES A peer-reviewed scholarly journal Editor: Sherman Dorn College of Education University of South Florida Volume 14 Number 24 September 27, 2006 ISSN 1068–2341 Funding for Performance and Equity: Student Success in English Further Education Colleges Ozan Jaquette University of Oxford England Citation: Jaquette, Ozan. (2006). Funding for Performance and Equity: Student Success in English Further Education Colleges. Education Policy Analysis Archives, 14(24). Retrieved [date] from http://epaa.asu.edu/epaa/v14n24/. Abstract The impact of performance funding on community college student outcomes is a contested issue. Performance funding policies in most U.S. states involve too small a proportion of funding to change college behavior. English further education colleges are similar to U.S. community colleges. 1992 policy reforms in England centralized policy control, and implemented a per-pupil funding formula; 10% of all funding is based on student success but other components of the funding formula pay colleges more money for enrolling disadvantaged students. This research uses five years of student level data to test the impact of these policies. Overall student success rates rose by 10% during the five-year period, with the largest gains made by ethnic minorities, adult basic education students, and students from disadvantaged neighborhoods. Although the English system depends on regulatory agencies that do not exist in the U.S., the major assertion of this research is that market-based funding policies—if properly designed—can promote equity in educational achievement. Keywords: performance funding, performance accountability, student success, community colleges Education Policy Analysis Archives Vol. 14 No. 24 2 Financiamiento por desempeño y por equidad: Éxito de los estudiantes en escuelas de enseñanza superior inglesas. Resumen El impacto del financiamiento por desempeño en los resultados académicos de los estudiantes universitarios es una cuestión muy disputada. Las políticas de financiamiento por desempeño en la mayoría de los estados de EE.UU. son muy escasas como para tener un peso sustantivo en la modificación de políticas universitarias. Las escuelas de enseñanza superior inglesa son similares a las universidades comunitarias de EE.UU. En 1992 se aprobaron reformas a las políticas de control centralizado en Inglaterra y se comenzó a implementar una fórmula de financiamiento por estudiante: 10% de todo el gasto se basaba en el éxito del estudiante y otros componente otorgaban mas dinero a las universidades por inscribir estudiantes de grupos desfavorecidos. Esta investigación utiliza cinco años de información sobre los estudiantes para testear el impacto general de estas políticas. El nivel de éxito general de los estudiantes aumento en un 10% durante esos 5 años, con incrementos mayores en los estudiantes provenientes de minorías étnicas, adultos estudiando su escolaridad básica, y estudiantes de barrios desfavorecidos. Aun cuando el sistema inglés depende de agencias de control desconocidas en el caso de EE.UU. la afirmación mas importante de este estudio es que cuando están debidamente diseñadas, las políticas de financiamiento basadas en principios de mercado pueden promover equidad en los logros educacionales. Community colleges provide the pathway to a better career for students who lack the time, financial resources, or educational background to attend a four-year institution, but the benefits of a community college education are much greater for those who graduate from a degree or certificate program than those who dropout (Grubb, 2002). Unfortunately, national graduation rates for community college students are quite low. Six years after initial enrollment only 36% of community college students received any degree or certificate. The results are worse for African American and Hispanic students, 26% and 30%, respectively (U.S. Department of Education, 2002). The picture is much brighter in England. English further education colleges—referred to as further education colleges—are similar to U.S. community colleges—referred to as community colleges—in that they are the primary education providers for low income adults. Success rates in further education colleges—defined as whether or not a student successfully passes the qualification they are enrolled in - increased by an impressive 18% from the 1997–98 to 2003–04 academic year (Learning & Skills Council, 2004a, 2005). The gains were strongest for ethnic minorities, students from “deprived” areas, Adult Basic Education (ABE) students, and students with learning disabilities.1 How did England achieve these results? This article focuses on the role of performance funding and regulatory control, which are the components of its performance accountability system. 1 Deprived areas refer to geographic areas that receive high scores on a multidimensional index of deprivation (Learning & Skills Council, 2002). Funding for Performance and Equity 3 Literature Review Performance accountability has been identified as one solution to the problem of unsatisfactory student outcomes in community colleges. Traditionally, accountability has focused on fulfilling legal requirements, such as civil rights legislation, whereas the contemporary accountability is focused on performance (Behn, 2001). Performance accountability refers to public policies which attempt to align college goals with government goals. Three categories of performance accountability are performance funding, performance budgeting, and performance reporting (Burke & Minassians, 2003). In performance funding, a portion of funding is directly tied to performance on key indicators through a funding formula. In performance budgeting, policymakers may consider performance on key indicators when considering funding allocations. In performance reporting, funding is not tied to performance, but public reporting of performance is theorized to create incentives for improvement. Recently U.S. state policy has moved away from performance funding in favor of performance reporting. In 2003, 15 states had performance funding programs compared to 17 in year 2000, 21 states had states performance budgeting programs compared to 28 in year 2000, and 46 states had performance reporting programs compared to 30 in 2000 (Burke & Minassians, 2003). Some states are still experimenting with radical performance accountability programs. For example, Colorado has instituted a statewide voucher program in its higher education system. Instead allocating state funding to institutions, all students receive a flat-rate voucher which may be used towards tuition at any college or university (Harbour, Davies, & Lewis, 2006). There is a surprisingly small amount of research on the impact of performance accountability programs on community college student outcomes. One nine-state study by Dougherty and Hong (2005) found no relationship between the strength of performance accountability system and community college student outcomes relating to remediation, retention, graduation, transfer rates, and job placement. The study highlighted an important problem with performance funding; in isolation performance funding rewards a very limited number of outcomes to the detriments of other aspects of educational quality. Therefore, performance accountability can lead to unintended consequences such as a decline in academic standards and creating a disincentive to serve students who have lower likelihoods of success. U.S. literature on the impact of performance accountability on student outcomes is limited for several reasons. First, U.S. performance accountability policies generally involve too small a proportion in overall funding to induce behavioral changes in colleges. Second, research, especially cross-state studies, tends to oversimplify performance accountability systems. More attention should be paid to individual mechanisms within the system, and how these mechanisms interact with each other as a whole. Clearly, performance accountability policies can have unintended consequences, but are these consequences unavoidable or can they be overcome by regulatory agencies and a well- designed system? This article provides insight into these issues by presenting results from a quantitative study on the impact of performance accountability on student outcomes in further education colleges. The funding system in England attempts to balance its strong funding component with additional funding for colleges that serve “disadvantaged” students.2 In addition, regulatory controls— 2 “Disadvantaged” students are defined as students coming from “backgrounds which have disadvantaged them” (Learning & Skills Council, 2002), and are eligible for additional funding. These include Education Policy Analysis Archives Vol. 14 No. 24 4 specifically college inspection and external assessment of student work—are designed to overcome the perverse incentives created by performance funding. The two-part research question is, first, how has the structure of funding policy and regulatory control created incentives and mechanisms to increase student success? Second, what has been the impact of performance accountability on student success over time? Because the impact of performance funding is theorized result from the economic incentives it creates, the theoretical framework of quasi-markets provides a useful lens. The analysis of English policy is followed by statistical modeling using five years of student level data to ascertain the impact of policy on student success. The final section discusses policy implications for the U.S. Suitability of a U.S.—England Comparison A basic understanding of further education colleges is prerequisite for a comparison of U.S. and English policymaking. Further education colleges began as vocational training institutions and, in a country obsessed with class, were held in low regard by the more genteel echelons of society (Pratt, 2000). As educational opportunity reached a larger proportion of society, and as jobs increasingly required a stronger foundation of academic knowledge, enrollment in further education colleges skyrocketed and provision became a mix of academic and vocational instruction (Melville, 2000). Today, more than 4 million students are enrolled in further education colleges (Learning & Skills Council, 2004b), out of a population of 50 million people (National Statistics, 2005a).3 By far the largest set of further education institutions are the general further education colleges, with 3 million students enrolled in the 2003–04 academic year.4 The analyses presented in this article focus on general further education colleges, but will refer to them as further education colleges. Like community colleges, further education colleges have a mission to serve disadvantaged students. Both types of institution are the main education providers for low-income adults, students seeking vocational training, and students who need ABE, such as literacy instruction. In both countries, enrollment is highest for courses in business, information technology, and health care (Learning & Skills Council, 2004b; National Center for Educational Statistics, 2004). There are important differences as well. First, further education colleges typically offer qualifications, with coursework ranging from a few weeks to several years, as opposed to degrees. These qualifications resemble certificates offered by community colleges. For example, further education colleges offer full-time, full-year qualifications in database management, which are analogous to earning an advanced certificate in community colleges. Further education colleges generally do not offer degree programs, such as the Associate’s degree, and they generally lack the well-articulated transfer function that exists in Community colleges. adult basic education students, those living in deprived areas, those with mental health problems or drug dependencies, political asylum seekers, and others (Learning & Skills Council, 2002). 3 Further education colleges are one provider of English post-compulsory education, which ends at age 16. The other types are universities, work-based learning providers (apprenticeships), and adult and community learning centers. 4 Further education colleges can be divided into four types: sixth-form colleges, which educate 16- to 18-year-olds; specialist colleges, which focus on specific fields such as horticulture or performing arts; and general further education colleges. Funding for Performance and Equity 5 Second, although 83% of students enrolled in further education are older than 18 (Learning & Skills Council, 2004b), further education colleges also educate a large proportion of the country’s 16–18 year olds. Third, England has a strong national policymaking system, whereas in the U.S. each state has an autonomous community college policymaking framework (Education Commission of the States, 2000). Therefore, the policy implications from this English case-study relate to U.S. state policymaking, not federal policymaking. Theoretical Framework Quasi-markets provide a useful framework for understanding funding and regulatory control policies in England. Johnson (1999) states that the four aspects of social services are strategy, funding, regulation, and provision. Generally, quasi-markets involve the separation of state finance from state provision, alongside the introduction of competition in the provision of services (Johnson, 1999; Walsh, 1995). Voucher systems in education are one example; funding follows students in their choice of schools, competition to attract students leads to increases efficiency (in theory), and institutions failing to attract enough students are forced to exit the market. Market based policies can be used to align the incentives of providers with the overall goals of central government. The principal-agent relationship plays a central role here: the principal (central government) pays the agent (individual colleges) to perform pre-specified services (Bartlett, Roberts, & Le Grand, 1998a). The principal must decide on the set of performance indicators to assess performance. Performance funding is generally resisted by colleges because it reduces budget stability, it undermines autonomy, and because the performance indicators chosen are often too simplistic to be valid measures of performance at college level. Analyzing the incentives created by principal-agent funding arrangements is central to the analysis of funding policy. The smaller the amount of performance funding and the greater the number of performance indicators this funding is divided by, the less incentive colleges have to improve their performance on these indicators. The ineffectiveness of performance funding in many U.S. states can be partially attributed to having too small an amount of funding tied to too many indicators (Burke & Minassians, 2003). Creating incentives for one kind of behavior can have unintended side effects. When performance is based on a single indicator—for example student success—providers have an incentive to focus their energy on that indicator, potentially at the expense of other activities that may be important to provision. Funding solely on the basis of student success would create strong incentives to lower academic standards. Additionally, performance funding for student success tends to exacerbate educational inequalities because colleges serving disadvantaged populations are likely to receive less performance funding than colleges serving affluent populations. Allocating additional funding for colleges that serve disadvantaged populations can help counterbalance this unintended side effect. Monitoring/regulation costs are the amount spent to ensure that agents (colleges) are acting in the interest of the principal (government). Monitoring costs decrease when the incentives of the agent are aligned with the incentives of the principal, or when both parties can agree on a shared mission. Similarly, when trust between the principal and agent increases, monitoring costs decrease. Monitoring costs increase when performance is difficult to measure and involves many outcomes. Walsh (1995) describes a specific quasi-market form in which a central organization has control of overall strategy, and ultimate control of funding while semi-autonomous regulatory agencies are responsible for the different aspects of regulation. Monitoring costs are the costs incurred by these regulatory agencies. The idea is that the central organization “steers” but does not Education Policy Analysis Archives Vol. 14 No. 24 6 “row” (Bartlett, Roberts, & Le Grand, 1998b). 1992 policy reforms in England created a similar policymaking system for further education colleges. The Department for Education and Skills (DfES) retained control of overall strategy; separate agencies, accountable to the DfES, became responsible for funding, inspection, and external assessment of student grades. The goal of these agencies was to ensure that agents (the colleges) act in the interest of the principal, the DfES. Funding Policy and Regulatory Control in England Overview of Policy Reform in English Further Education Before 1992, funding and policymaking for further education colleges was very similar to the current system used by California community colleges (Jaquette, 2005); colleges were funded through a combination of local property taxes and block grants from the national government to local education districts. These districts were responsible for hiring and firing, curriculum design, financial administration, and allocation of funds to individual colleges (McLure, 2000). In 1992 there were dramatic reforms in English further education. National legislation moved policymaking control from the local districts to the central government. Centralized regulatory agencies were created to regulate the different aspects of provision, such as funding, inspection, external assessment of student grades, etc. As a part of this effort, the new funding agency devised a performance-based per-pupil funding formula. Before this reform, block grant funding was given to local districts. After the reform, funding followed individual students to the college they decided to attend, essentially creating a national voucher system. The funding formula pays institutions additional funding for enrolling disadvantaged students and, currently, 10% of total funding depends on student success (Learning & Skills Council, 2002). The government created a centralized data system to track the progress of all students in the country and to allocate formula funding to institutions based on individual student progress. This data system, called the Individualized Learner Record, was the source of quantitative data used in this study. After the reform, each college became responsible for its own financial administration and solvency. These responsibilities had previously been the domain of local education authorities. Additionally, within each college, the lay board of governors was given increased oversight power of school finances and employment of senior management. Collectively, the 1992 reforms are commonly referred to as Incorporation Reform because they forced colleges to behave more like private corporations. The passage of Incorporation Reform legislation was quickly followed by a high-profile report entitled Unfinished Business which detailed the low retention and success rates under the prior further education funding and control framework (Audit Commission & Office for Standards in Education, 1993). The evidence from Unfinished Business was instrumental in securing the support of colleges during the implementation of the strong external accountability policies created in the Incorporation legislation (Davies & Rudden, 2000). The Per-Pupil Funding Formula This section utilizes the quasi-market framework to analyze Incorporation funding policies and regulatory agencies created incentives and mechanisms to increase student success rates. Table 1 Funding for Performance and Equity 7 below shows the total number of full time equivalent students (FTES) in all of English further education, as well as total funding, and average funding per FTES student.5 Total funding for further education has risen consistently since the 1999–00 academic year, while average funding per student peaked in the 2001–02 academic year. Table 1 Full-time Equivalent Students (FTES) and Funding in Further Education Colleges, 1994–95— 2003–04 Year FTES students Total funding ($millions) Average funding per FTES ($) 1994–95 914,000 5,326 5,827 1995–96 989,000 5,372 5,432 1996–97 1,027,000 5,425 5,282 1997–98 1,020,000 5,289 5,186 1998–99 1,004,000 5,093 5,072 1999–00 977,000 5,302 5,427 2000–01 953,000 5,514 5,786 2001–02 970,000 6,078 6,266 2002–03 1,051,000 6,483 6,169 2003–04 1,117,000 6,899 6,176 Funding columns use constant 2002 £ converted using average 2002 exchange rate. Source: National Statistics (2005b). The per-pupil funding formula is the tool by which this funding is allocated from the central government to individual colleges. The formula was first implemented in the 1993–94 academic year (McLure, 2000). The funding formula would more precisely be called a per-qualification funding formula because colleges are paid on the basis of each qualification a student undertakes. However, throughout the paper I will refer to the per-pupil funding formula to stress the fact that funding follows each student. The amount calculated in the formula is intended to cover teaching as well as fixed costs such as building and equipment. To increase stability, colleges are guaranteed at least 90% of previous year’s funding (Learning & Skills Council, 2002). Analysis of the funding formula is based on funding guidance documents sent from the central funding agency to individual colleges from 1998–2003 (Further Education Funding Council, 1999b, 2000, 2001; Learning & Skills Council, 2002). Although there have been incremental changes from year to year, the formula remained largely the same until the 2002–03 academic year when the policy reforms of Blair’s Labour government took effect (Jaquette, 2005). Due to space limitations this article focuses on the most recent funding formula and not on how funding policy changed over time.6 The 2002–03 funding formula is presented in equation (1). Each of its components is explained in turn. 5 These funding amounts differ from funding calculations shown later in the paper, because later calculations are based on a subset of adult students, while Table 1 is based on all students in further education colleges, including 16–19 year old students. 6 Detailed analyses of how the formula changed over time appear in Jaquette (2005). Education Policy Analysis Archives Vol. 14 No. 24 8 2002–03 Funding Formula (1) £ per qualification = [ (Base Rate - Achievement Funding) x Programme Weighting Factor x Disadvantage Uplift x London Weighting Factor] - Tuition + Additional Learning Support Base Rate = Base funding for each qualification Achievement Funding = Deduction valued at 10% of the base rate if the student fails Programme Weighting Factor = Higher weighting for more costly programs. (A=1, B= 1.12, C= 1.3, D=1.6, E= 1.72, Adult Basic Education = 1.4) Disadvantage Uplift = Additional funding to reflect that some students require more resources than others (Postcode disadvantage uplift = 1.1 (on average), Homeless students disadvantage uplift = 1.12, All others = 1.1) London Weighting Factor = Additional funding to account for higher cost of provision in London (Central London = 1.18, Inner London = 1.12, Outer London = 1.06). Tuition = 25% of Base Rate. If the student is not eligible for tuition remission, then government funding is reduced by 25% of the base rate, and the student pays tuition directly to the college. If the student is eligible for tuition remission, than the government does not subtract tuition from college funding. Additional Learning Support = Additional funding for students with special learning needs. Value depends on needs of individual student. Source: Learning & Skills Council (2002). Funding for Performance and Equity 9 Base rate. The base-rate is the core amount of funding for each qualification. This amount is predominantly determined by the number of guided learning hours (instructional hours) for that qualification.7 90% of base-rate funding is dependent on retention and 10% is dependent on student success. The academic year is divided into three funding periods and an institution receives retention funding for a qualification only if the student is present on the census date for that period. For example, imagine a student enrolled in a qualification that begins in September and ends in July. If that student drops out on December 15th (which is after the first census date but before the second census date) then the institution only receives 30% of base rate funding. Utilizing the quasi-market framework, the funding formula forces these public providers to focus their resources on student retention and achievement as opposed to initial enrollment. Achievement funding. Student success is valued at 10% of base-rate funding. For example, in 2002–03 a qualification listed as having 440 guided learning hours had a base rate of £1,594 (Learning & Skills Council, 2002). If the student successfully completes the qualification, then the institution receives the full £1,594 base-rate funding. If the student is present throughout the qualification, but does not pass examination, then the institution receives 90% of base-rate funding [(£1,594 – 10%*£1,594 = £1435)]. Before the 2002–03 academic year achievement funding fell into three categories: qualifications deemed relevant to the needs of the economy received achievement funding equivalent to 7% of total funding, other qualifications received 5% achievement funding, and certain qualifications (such as those not externally assessed) received no achievement funding (Further Education Funding Council, 1999b) (Further Education Funding Council, 2000, 2001). Beginning in the 2002–03 academic year all qualifications received 10% achievement funding. This represents a shift in funding emphasis toward student success and, in theory, gives colleges a stronger financial incentive to ensure their students are successful.8 Program weighting factor (PWF). Program weighting factors are another component of the funding formula in equation (1) above. These weighting factors give higher funding for provision that is deemed more costly. Table 2 below shows the different weighting factors. Providing medical technician training, for example, is more costly than teaching history and thus funding for students enrolled in these qualifications has a higher weighting factor. The goal of weighting factors is to eliminate the disincentive against providing costly provision. The funding formula presented in equation (1) shows that higher weighting factors are multiplied through achievement funding, which can lead to dramatic increases in total funding (Jaquette, 2005). Prior to the 2002–03 this was not the case; two qualifications with different weighting factors but the same instructional hours received the same achievement funding. Therefore, starting in 2002–03 institutions received a greater financial incentive increase success rates for qualifications that had higher weighting factors. 7 Base-rate funding for certain qualifications is listed explicitly as opposed to being determined by the number of guided learning hours. However, these amounts of base-rate funding for these listed courses do not differ greatly from the amount of funding they would receive had base-rate funding been determined by guided learning hours. 8 Most qualifications consist of only a single component, but colleges can receive partial achievement funding when a single qualification has multiple components. For example, if a student enrolled in an Advanced Vocational Certificate of Education—one of England’s longer qualifications—successfully completes three out of the five modules, the college would receive three-fifths of the achievement funding. In the 2002–03 academic year, just 1.5% of the qualifications received partial achievement funding. An analogous program in the United States would pay achievement funding for each individual course a student successfully completed. Education Policy Analysis Archives Vol. 14 No. 24 10 Table 2. Program Weighting Factors Factor Weight Example A 1 Accounting, history, economics, psychology B 1.12 Information technology, teacher training, dance, pharmacology, chemistry C 1.3 Hair styling, photography, catering, interior design, metallurgy D 1.6 Music technology, food preparation, animal care, engineering E 1.72 Gardening, fish production Adult Basic Education (ABE) 1.4 Adult literacy, numeracy Source: Learning & Skills Council (2002). Table 2 shows that ABE qualifications receive higher program weighting. This policy is part of the Skills for Life initiative, which is a massive effort to increase adult literacy and numeracy (Department for Education and Employment, 2001). The higher weighting factor for ABE contributes to the initiative by increasing the financial incentive for colleges to serve these students.9 Disadvantage uplift. Performance funding was simultaneously implemented with several policy efforts to increase enrollment of low-income students (Kennedy, 1997). The disadvantage uplift is one of these policies. It pays institutions premium funding for enrolling disadvantaged students. The stated goal of disadvantage uplift funding in England is “to ensure that certain learners attract a funding enhancement, which reflects their relative disadvantage and the expected additional costs incurred by institutions in attracting and retaining such learners” (Learning & Skills Council, 2002, p. 9). The funding formula presented in equation (1) shows that disadvantage uplift funding is multiplied through achievement funding. This creates a financial incentive to increase success rates for disadvantaged students. The disadvantage uplift was introduced in the 1998–99 academic year. It was initially applied to homeless students and students living in deprived postcodes (Kennedy, 1997). Starting in the 1999–00 academic year the uplift was extended to adult basic skills students, students receiving means-tested benefits, those with mental health problems and drug dependencies, asylum seekers, refugees, ex-offenders, and others (Further Education Funding Council, 1999a) (Further Education Funding Council, 2000). Table 3 below shows that over time the average monetary value of the uplift increased as did the percentage of students receiving the uplift. 9 Adult basic education students have been defined as those who are “undertaking programmes where the primary learning goal is adult basic education or English for speakers of other languages” (Learning & Skills Council, 2002). ABE qualifications refers to coursework in literacy, numeracy, access to further education, courses for students with learning disabilities, and other basic education. Therefore, ABE students are those whose primary learning goal is ABE/ESOL, but they may take non-ABE qualifications in addition to ABE qualifications. Funding for Performance and Equity 11 Table 3 Disadvantage Uplift Mechanisms and Consequences, 1998–99—2002–03 Effects 1998–99 1999–00 2000–01 2001–02 2002–03 Value of disadvantage uplift by student type a Postcode disadvantage uplift (average value) 1.06 1.06 1.08 1.1 1.1 Homeless and residential care disadvantage uplift 1.09 1.09 1.12 1.12 1.12 All othersa NA 1.06 1.08 1.1 1.1 Percentage of students receiving disadvantage uplift by student typeb Postcode disadvantage uplift 29.2 26.9 26.7 26.1 29.1 Any disadvantage uplift (including postcode) 29.2 37.0 38.8 39.1 41.7 Source: Jaquette (2005). a The following groups of students became qualified for a disadvantage uplift beginning in the 1999– 00 academic year: adult basic skills students, those receiving means-tested benefits, those with mental health problems, those recovering from alcohol or drug dependencies, political asylum seekers, political refugees, ex-convicts, those whose statutory education has been interrupted, those in or who have recently left mental or physical healthcare, those taking care of children/relatives as a full-time job. b The percentage of students receiving a disadvantage uplift was calculated using a sub-sample of the further education population. This sub-sample is defined in the quantitative modeling section to follow. Additional learning support. Additional learning support (ALS)—another component of the funding formula—is funding for special support in addition to what is normally provided in a standard learning program (Further Education Funding Council, 2000). ALS funding facilitates the employment of specialist staff including additional teachers to reduce class size (used especially in basic skills), personal care assistants, mobility assistants, readers, note-takers, and educational psychologists (Further Education Funding Council, 2001). The most common types of students utilizing ALS were basic skills students and those with sensory impairment, dyslexia, learning difficulties, or physical impairment (Faraday, Fletcher, & Gidney, 2000). ALS has existed since 1993. Two evaluations of ALS have stated that the program is very popular amongst providers because there is no limit to the amount of ALS funding an institution may receive for serving a single student and because ALS is funded on an uncapped, per-pupil basis by the central funding agency rather than by operating revenue of individual colleges (Faraday & Fletcher, 2003; Faraday et al., 2000). If, instead, each college was given a lump sum of ALS funding for all students, this could create a disincentive for enrolling students with costly support needs. In the 2002–03 academic year 9.3% of the population analyzed in this study received additional learning support. The average amount of ALS funding for each qualification receiving ALS (note that a single student can be enrolled in more than one qualification) was equivalent to $1080 using a January 2003 exchange rate (Jaquette, 2005). Logistic regression modeling, presented below, shows that the presence of ALS had a significant positive effect on the likelihood of student success, especially for disabled students and basic skills students (Jaquette, 2005). Tuition remission. Tuition fees are the final component of the funding formula. Generally, students are expected to pay their colleges a tuition fee equal to 25% of the national base rate for their qualification. Referring to the funding formula above, any tuition fees paid to the institution by Education Policy Analysis Archives Vol. 14 No. 24 12 the student is deducted from the amount the central funding agency pays to the college. However, many students are eligible for tuition remission. If the student is eligible for tuition remission, the central funding agency does not deduct the tuition fee from the amount it pays the college. In the five years of data analyzed in this study, 29% of students paid full tuition fees. The rest were either eligible for tuition remission or the individual college decided to reduce or waive tuition costs as part of their internal policy. In this latter case, the institution does not receive tuition fees from either the student or the central funding agency. Regulating Further Education The potential unintended consequences of performance funding include lowering academic standards, restricting open access, and focusing resources only on the measurable outcomes which are subject to performance funding. Performance funding policies implemented in isolation of countervailing policies and institutions are more likely to exhibit these unintended consequences. In England several semi-autonomous government agencies help reduce these unintended side-effects and also put pressure on institutions to improve performance. This section discusses three important regulatory agencies in English further education, all of which are funded by and accountable to the Department for Education and Skills (DfES). The Learning and Skills Council. The Learning and Skills Council (LSC) is in charge of funding and planning further education, which includes administering the funding formula described above. The LSC was created under the Labour Government by the Learning and Skills Act of 2000 (Department for Education and Employment, 1999). The predecessor to the LSC was the Further Education Funding Council (FEFC), which was created by the Conservative government in the wake of 1992 Incorporation Reform. There are important differences between these two organizations. First, while FEFC was responsible only for further education, the LSC expanded its policymaking remit to include work-based learning providers and adult community education learning providers as well. Second, while FEFC was responsible for inspection, LSC lost this inspection responsibility because the DfES wanted an arms-length relationship between funding and inspection. Third, under the FEFC enrollment growth was encouraged by market mechanisms. The entry funding element of the FEFC funding formula created a “grow or die” mentality for colleges leading to dubious recruitment practices (Ball, Maguire, & Macrae, 2000; Rospigliosi, 2000). When the Labour government came to power in 1997 market incentives for growth were reduced. The LSC funding formula introduced in 2002–03 eliminated entry funding and LSC satellite offices took a stronger role in planning enrollment growth at the local level. This exemplifies the shift away from free market policies and towards centralized planning. Therefore, LSC has a planning remit that the FEFC lacked. Inspection agencies. Inspection is seen by many to be the most influential regulatory force in English further education (G. Pine, personal communication, August 5, 2005). The Adult Learning Inspectorate and the Office for Standards in Education are jointly responsible for inspecting colleges. Inspection of colleges is guided by seven core questions, three of which relate directly to student success. Each college was inspected once between April 2001 and summer 2005 (Office for Standards in Education & Adult Learning Inspectorate, 2001). Inspection teams grade soft outcomes and processes to guard against the unintended consequence of colleges focusing resources only on performance funding measures. These include quality of instruction, student engagement, quality of their guidance counseling and tutoring services, fulfillment of equal-opportunity responsibilities, etc (Learning & Skills Development Agency, 2003; Funding for Performance and Equity 13 Office for Standards in Education, 2002) Additionally, inspection teams review a sample of individual learning plans which colleges are required to create for each student (Learning & Skills Development Agency, 2003). Inspections have sharp teeth. Poor inspection grades for a particular program area can lead to a freeze in enrolment, the closing of that program area, and even closure of the institution. When college programs or services are deemed in need of improvement, they must develop action plans and report on their progress (Office for Standards in Education, 2002). Inspection is also a strategic mechanism used to increase student success. High or improving success rates are prerequisite for good inspection grades. Results of inspections are posted on the Office for Standards in Education website.10 This public report on performance creates an incentive for institutions to increase success rates in order to maximize institutional prestige. Additionally, several college principals have been fired by their local board of governors because of poor inspection results, which are largely dependent on improving success rates (G. Pine, personal communication, August 5, 2005). Qualifications and Curriculum Authority. The Qualifications and Curriculum Authority is a regulatory agency that “maintains and develops the national curriculum… and accredits [externally assesses] and monitors qualifications in colleges and at work” (Qualifications and Curriculum Authority, 2005).11 This study is primarily concerned with the external assessment role; exams and projects are sent to third-party graders who determine whether students pass the qualification and the grade that students receive. The guiding principle behind external assessment is that when institutions face strong pressure to increase student success, they should not determine what constitutes success. In the absence of external assessment colleges would have a strong incentive to lower academic standards in order to increase success rates. Therefore, external assessment helps ensure that gains in student success rates are not due to declining academic standards. Summary This section sought to address the first research question: how has the structure of funding policy and regulatory control created incentives and mechanisms to increase student success? Analysis of the per-pupil funding formula showed that institutions had a strong financial incentive to ensure their students were successful. In absence of other measures this could lead some colleges to restrict enrollment to students that had a high likelihood of success. However, the funding formula contained other components—specifically the disadvantage uplift and additional learning support— which mediated against such enrollment restriction. Regulatory agencies were also shown to play a key role in reducing unintended consequences and catalyzing improvements in student success. The Department of Education and Skills controls the strategic goals of these agencies to ensure they compliment one another. In the next section I analyze five years of student level data to see whether funding policy and regulatory control was actually successful in raising student success rates. 10 This website has inspection reports for all FE colleges available at http://www.ofsted.gov.uk/reports/index.cfm?fuseaction=listColleges&type=fecollege. 11 In England the process of using third-party graders to assess student work is external accreditation. In this article I have replaced the term external accreditation with external assessment. The rationale for this action is to reduce confusion for readers because in the U.S. the term accreditation refers to peer-evaluation of entire institutions, as opposed to external assessment of student work. http://www.ofsted.gov.uk/reports/index.cfm?fuseaction=listColleges&type=fecollege Education Policy Analysis Archives Vol. 14 No. 24 14 Descriptive Statistics and Modeling This section uses five years of student level administrative data to gain insight on the second research question: What has been the impact of funding and regulatory control on student success? A description of the data will be followed by analysis of descriptive statistics and regression modeling. Unfortunately, these analyses using data from 1998–99 to 2002–03 cannot fully answer the second research question because the data years analyzed postdate the dramatic policy reforms implemented beginning in 1993. Additionally, incremental changes in funding policy and regulation have often been implemented simultaneously with changes in the national curriculum, changes in teacher training, etc. which lie outside the already wide scope of this research. However, these are shortcomings which plague most analyses that are not experimental in nature. There is another reason to be cautious results which attempt to isolate the effect of specific funding policies on student success; these policies are intended to work in concert rather than isolation from one another. Disadvantage uplift funding and additional learning support funding are intended to offset the additional financial burden of helping disadvantaged students become successful. The grading system used by the inspection regime is intended to give colleges the incentive to focus resources on increasing student success. Indeed, a major criticism of performance accountability efforts in U.S. states is that they tend to be tacked on to existing systems without thought to how policies reinforce or conflict with one another. Therefore, when thinking about what drives trends in success rates, readers are encouraged to think holistically about the incentives and checks and balances created by the entire funding and regulatory system. Data and Sample The following analyses are based on student level administrative data from 1998–99 to 2002–03 academic years for the entire population of further education students. The data have never before been used in academic research. Further education reform in 1992 mandated that all colleges return data for all students. The resulting dataset is called the Individualized Learner Record (ILR), which is similar to student data tracking systems which exist in most U.S. states. The data is qualification level data rather than student level data because an individual enrolled in three qualifications would have three observations in the ILR. Several additional datasets were merged with the ILR. First, institution level data was retrieved from an LSC administrative dataset. Secondly, ILR data was merged with data from the Learning Aim Database12 to get qualification level variables, including the program weighting factor. Third, data on local area population density and local area educational attainment were retrieved from the 2001 UK census. Finally, the English Indices of Deprivation 2004 were merged to the ILR by student postcode. These indices combine seven measures of deprivation (income deprivation, 12 The Learning Aim Database provides information on qualifications and learning aims for institutions. The software provided by the Learning Aim Database is used by colleges to determine how much funding they will receive for each qualification. The Learning Aim Database Website is http://providers.lsc.gov.uk/lad/default.asp http://providers.lsc.gov.uk/lad/default.asp Funding for Performance and Equity 15 crime deprivation, housing deprivation) into a single index, which shows how deprived a local area is.13 Variables were chosen for the following analysis based on a literature review of factors that influence adult student success rates (Alfonso, Bailey, & Scott, 2005; Bailey et al., 2004; Davies, 2001; Davies & Rudden, 2000; Grubb & Lazerson, 2004; Martinez, 2001; St. John, 1999). These factors were divided into individual demographic determinants (such as ethnicity, gender), qualification level determinants (such as difficulty level, mode of instruction), institution level characteristics (such as number of students, and population density of college geographic area), and, finally, funding determinants (Jaquette, 2005). These funding determinants were based on the above analysis of funding policy, and are the main determinants of interest. Unfortunately, key variables such as previous educational attainment, parental income, and parental education were not present in the ILR. The population was restricted to one that would be most comparable to American community colleges. The sample kept students from General Further Education Colleges and Tertiary Colleges, dropping sixth-form colleges (which educate 16–18 year olds), specialist colleges (for example horticulture, or drama colleges), external providers, and work-based learning providers (apprenticeships). This analysis only retained students who were 19 or older at the beginning of the academic year. Because the analysis of funding policy focused on the funding policies under the FEFC/LSC, qualifications not funded by FEFC/LSC were dropped. Qualifications in franchised provision were dropped in order to focus on a more homogenous group of providers.14 Additionally the sample only kept students on courses of 20 or more guided learning hours. The rationale was to exclude “taster courses” and other courses that would be expected to have high success rates due to their short duration. The data was further limited to qualifications where student success was known. Observations were deleted if the qualification was continuing to the next academic year, if the exam results were unknown, or if the students were partially successful. To illustrate, in the 2002–03 ILR data student success was known for 86.5% of qualifications. This 86.5% would be kept in the sample. Of the remaining 13.5%, 1.5% had partial achievement, 2.5% exam not taken/result not known, and 9.5% qualification continuing to the next academic year. There was right censoring, but not left censoring; a student who started a two-year course in year X-1 would be dropped from the data in year X-1, but would appear in year X when their outcome was known. This ensures that there is no duplication of student qualifications from one year to the next. Observations in the analysis dataset have three possible outcomes: first, they withdrew from the course; second, they were present throughout the duration of the qualification (this is called retention), but failed to pass examination; third, they were present throughout the qualification, and 13 The 2004 Indices of Deprivation were commissioned by the Office of the Deputy Prime Minister to the Social Disadvantage Research Centre at the Department of Social Policy and Social Work at Oxford University. The Indices of Deprivation are based on the idea that distinct measures of deprivation can be measured at the output area level and aggregated into an index that measures the total amount of deprivation experienced by individuals living in a particular super output area. Each super output area has about 1,500 people. The seven domains of deprivation are: income deprivation; employment deprivation; health and disability deprivation; education, skills and training deprivation; barriers to housing and services, living environment deprivation, and crime deprivation. These indices are different from the ones which are used to determine whether students receive a postcode disadvantage uplift. 14 Franchised provision is when an institution contracts an external provider to provide instruction on behalf of the institution. Education Policy Analysis Archives Vol. 14 No. 24 16 they successfully passed examination. The analyses presented here only consider the binary outcome called success, where 0 = withdrawn or failed examination, 1 = passed examination. Analyses of student retention can be found in Jaquette (2005) . Descriptive Statistics Table 4 on the next page shows success rates for the overall population and for sub-groups from the 1998–99 to 2002–03 academic year. In the appendix, Table A-1 shows the corresponding sample sizes. Overall success rates increased by 10% over this time period and the gains were distributed fairly equally over time, growing by 2.8%, 2.3%, 2.2% and 2.7% in each respective year. Subgroup analysis shows which groups made the strongest gains. The local level of deprivation is analyzed first. A higher deprivation score means that the student comes from a more deprived postcode. Looking down the columns, less deprived areas are always associated with higher success rates, but looking across the rows the biggest gains over time were made by students in highly deprived areas. This result suggests that the achievement gap between affluent and deprived areas is decreasing over time. Looking next at ethnicity, 2002–03 non-whites made up 26% of the sample as compared to 22% in 1998–99.15 White and Indian students generally have the highest success rates. The overall picture is that gains were strong for all ethnic groups, with the white vs. non-white achievement gap closing over time, from 7% in 1998–99 to 5% in 2002–03.16 It is important to note that non-whites are making these gains despite residing in more highly deprived areas. For example, in this study Bangladeshis had the highest average level of deprivation and their success rates increased by 20%. The findings for deprivation and ethnicity are important considering research in the United States, which indicates ethnic minorities and deprived students persistently have low and stagnant success rates (Bailey et al., 2004). Looking next at gender, women generally have higher success rates than men. However, success rates for women have increased by 9%, while success rates for men have increased by 12%. Looking next at age, older students have higher success rates and stronger gains over time than younger students. The next set of variables focus on the type of qualification a student enrolls in (a single student can enroll in multiple qualifications). The variable qualification level is a measure of course difficulty as determined by the Qualification and Curriculum Authority.17 The gains are strongest for “other” and level 1, which are generally low level qualifications.18 15 Author’s calculation. 16 Author’s calculation. 17 Note that we cannot make a valid comparison to the 2002–03 data because the level assigned to some courses was changed as part of the transition towards a national qualification framework. 18 Other qualification variables, such as area of learning (i.e. science, business, construction, etc) and qualification type (i.e. A-level, NVQ, GNVQ, etc) are included in Jaquette (2005). Critics have state that the rise in success rates is due to declining rigor in new curriculum. Jaquette (2005) devotes considerable attention to this assertion but finds that this is not the case. Funding for Performance and Equity 17 Table 4 Qualification Success Rates, 1998–99 to 2002–03 Population 1998–99 1999–00 2000–01 2001–02 2002–03 Overall 56.9 59.7 62.0 64.2 66.9 Deprivation Index Deprivation LT 10 60.9 62.6 64.4 66.8 69.4 Deprivation 10-<20 59.5 61.8 63.9 65.9 68.7 Deprivation 20-<30 56.6 59.4 61.7 63.7 66.7 Deprivation 35-<50 52.9 56.8 59.4 61.5 64.5 Deprivation 50+ 49.4 54.2 57.6 60.1 62.9 Ethnicity Bangladeshi 46.6 51.5 57.2 61.1 67.0 Black-African 44.4 51.8 53.7 56.9 59.5 Black-Caribbean 46.3 50.3 53.1 56.3 57.5 Black-Other 45.2 48.3 51.7 55.2 56.8 Chinese 52.2 56.4 59.5 63.1 67.9 Indian 55.5 59.0 60.4 64.5 68.3 Pakistani 48.9 52.0 50.3 58.1 60.8 White 58.4 61.1 63.0 65.7 68.3 Asian-Other 51.3 55.7 58.8 61.4 62.3 Gender Male 53.0 56.4 59.4 61.8 64.6 Female 59.3 61.7 63.5 65.7 68.4 Age Age 19–25 52.1 54.7 55.4 57.1 60.2 Age 26–34 56.8 59.9 62.4 64.6 66.3 Age 35–44 60.8 63.4 65.9 67.7 69.6 Age 45–54 59.7 62.1 65.2 67.2 70.5 Age 55+ 54.8 57.9 61.5 65.5 70.9 Disabled 56.6 62.1 63.3 65.5 70.1 Qualification levela Entry & level 1 55.7 59.1 60.7 64.0 68.5 Level 2 55.2 58.1 59.7 61.3 62.6 Level 3 55.1 57.8 60.6 62.6 64.5 Level 4 or Higher 54.3 50.2 51.3 54.7 60.3 Other 61.9 68.9 71.7 71.7 76.9 External assessment Not externally assessed 60.0 65.9 71.0 74.3 75.9 Externally assessed 56.1 59.0 60.8 62.0 63.9 Qualification duration LT 24 Weeks 66.0 67.0 70.2 73.6 74.9 24-<48 Weeks 53.8 57.6 58.1 59.5 62.1 48+ Weeks 46.6 47.8 52.0 50.4 56.5 Mode of attendance Full-time, full-year 56.7 59.9 57.2 58.0 62.2 Full-time, part-year 66.2 69.6 71.9 74.8 75.5 Part-time 56.3 58.9 62.3 64.6 67.3 Education Policy Analysis Archives Vol. 14 No. 24 18 Population 1998–99 1999–00 2000–01 2001–02 2002–03 Open/distance learning 44.1 47.8 50.8 47.3 52.0 Funding Determinants Receive access funding 57.2 59.1 65.0 Postcode disadvantage uplift 52.0 55.6 58.4 60.4 63.7 On benefit 50.1 53.1 55.2 58.2 60.3 ABE student 55.8 64.4 70.0 73.1 72.8 Asylum seeker 47.8 51.1 59.1 58.4 57.5 Additional Learning Support 61.4 65.6 66.3 69.3 73.8 N 1,509,393 1,687,464 1,692,394 1,646,138 1,771,842 a Percentages for 2002–03 not comparable to previous years due to change in definitions. External assessment has special theoretical importance to this research; when institutions face such strong pressure to increase success rates, the cheapest solution is to institute more lenient grading policies. However, colleges do not have this power when standards are controlled by the national curriculum authority and student success rates are determined by external graders. Generally, about 80% of qualifications analyzed in this study were externally assessed. Over the five year period success rate gains for non-externally assessed qualifications were 16% compared to 8% for externally assessed qualifications. Therefore, non-externally assessed qualifications have higher success rates are higher and make stronger gains over time, but are not the driving force in overall success rate gains because they represent a minority of the population. The final set of variables in table 4 focuses on qualifications which were subject to additional funding initiatives. Most of these funding initiatives—whether they were aimed at the college or directly to the student—focused on students who historically have had low success rates. Access Funding is given directly to students who are low income or in receipt of means tested benefit and can be used to offset the childcare costs, transportation costs, books and equipment, and examination fees. In theory, such programs help adult students balance work, family obligations, and education (Grubb & Lazerson, 2004). Access funding was not implemented nationally until the 2000–01 academic year. Success rates for students receiving access funding grew 8% over the three years of data.19 The postcode disadvantage uplift is a disadvantage uplift for students who live in a deprived postcode. Between the academic years 1998–99 and 2002–03 success rates for students receiving a disadvantage-postcode uplift increased by 11.7% compared to 10% for the national average. This finding may be especially important for policymakers. Research on Community colleges has found it is difficult to increase success rates for disadvantaged students (Bailey, Calcagno, Jenkins, Kienzl, & Leinbach, 2005; Grubb & Lazerson, 2004). Further education colleges, however, are given a dual financial incentive to serve these students and ensure that they are successful. The value of the disadvantage uplift increased from an average of 6% of total funding in 1998–99 to 8% in 2000–01 to 10% in 2002–03. Furthermore, as explained in the previous section, the total value uplift funding is increased when students are successful. 19 In analysis restricted to students receiving means tested benefits, those receiving access funding had higher success rates than those not receiving access funding (Jaquette, 2005). Therefore, when restricting analysis to students which are—theoretically—eligible for access funding, those that actually receive access funding have higher success rates. Funding for Performance and Equity 19 Students receiving means tested benefits and political asylum seekers have been eligible for fee remission—a funding policy directed at the student—in all years studied. Beginning in 1998–99 both groups began receiving disadvantage uplift funding at 6% of total funding. This amount rose to 8% in 2000–01, and to 10% in 2002–03. Therefore, these students receive direct financial support (no-tuition), and in addition institutions are paid more money for serving these students. Although the success rates remain below the national average, both groups showed gains of 10% which is significant considering the special challenges facing these disadvantaged students. ABE students have benefited from a number of financial incentives aimed at both students and providers. To summarize, they received tuition remission since 1998–99,20 beginning in 1999–00 they became eligible for a disadvantaged uplift, and beginning in 2002–03 ABE qualifications received a higher program weighting. ABE students were also eligible to receive additional learning support. Over the five years studied success rates for ABE students increased by 17%. Finally, students receiving Additional Learning Support (ALS) have much higher success rates than those not receiving ALS. ALS students also make stronger gains over time. Further subgroup analysis shows that this result is partly explained by the fact that ALS students are more likely to be enrolled in non-externally assessed qualifications (Jaquette, 2005). However, even after controlling for external assessment, ALS is strongly associated with higher success rates as shown in the following regression results. Simulating the funding formula. The per-pupil funding formula was simulated by combining student level data from the Individualized Learner Record with the rules from funding policy documents (Further Education Funding Council, 1999b, 2000, 2001; Learning & Skills Council, 2002).21 Figure 1 below shows funding per instructional hour for select student groups from 1998– 99 to 2002–03 using constant 2002 £ converted using average 2002 exchange rate. Because this research focused on incentives created by funding policy, Figure 1 shows how much money institutions receive for each student if the student is successful. Figure 1 yields several important results. First, average funding for all students rose steadily from $8.7 per instructional hour in the 1998–99 academic year to $10.4 per instructional hour in 2002–03. Second, colleges receive more funding for students with higher resource needs. Students receiving a disadvantage uplift are funded 7% higher on average than those that do not receive a disadvantage uplift. ABE students are funded 30% more and average than non-ABE students.22 Students receiving additional learning support receive nearly twice the funding on average as compared to those that do not receive additional learning support. Funding per instructional hour for ABE students receiving additional learning support (not shown) and disabled students receiving additional learning support (not shown) is even higher. In conclusion, the per-pupil funding formula 20 Tuition remission was the ILR variable used to identify basic skills learners. 21 Details on the construction of the simulation can be found in the appendix of the full report (Jaquette, 2005). 22 It may appear contradictory that funding for ABE students fell in 2002–03, the same year that the Skills for Life initiative came into effect. The reason for this is as follows: the new funding formula which came into effect in 2002–03 assigned a program weighting factor of 1.4 for ABE students. This weighting factor was only applied to ABE qualifications that followed the new ABE national curriculum which came into effect in 2002–03 (Department for Education and Skills, 2003b). However, colleges were considerably confused about this policy and only a minority of ABE students were actually enrolled in qualifications that met the new national curriculum (Learning & Skills Council, 2003). Education Policy Analysis Archives Vol. 14 No. 24 20 which emphasizes student success, allocates additional funding for students that need more resources to become successful. $5 $10 $15 $20 1998-99 1999-00 2000-01 2001-02 2002-03 Overall Receives additional learning support Adult basic education student Receives disadvantage uplift Figure 1. Funding per instructional hour, 1998–99 to 2002–03, using constant 2002 £ converted using average 2002 exchange rate Statistical Modeling The preceding trends in success rates by subgroup are partially driven by their correlation with particular variables. Modeling helps control for the effect of other variables. The analyses presented below utilize logistic regression modeling. The model was built in stages, adding student- level socioeconomic determinants first, followed by qualification-level determinants, institution-level determinants, and finally funding-determinants. Table 5 below shows selected variables from the final model in which all variables were included. Due to space constraints, results of following variables appear in Table A-2 (in the appendix): geographic region, area of learning, qualification type, institution size, population density, and program weighting. The results for all models are shown in the appendices of Jaquette (2005) which can be obtained by the author upon request. Table 5 shows odds ratios and p-values for selected variables. The odds ratios show the odds of success compared to that of the reference group, controlling for other factors. An odds ratio greater than one means that group is more likely to be successful than the reference group, while an odds ratio of less than one means that group is less likely to be successful than the reference group. P-values are measures of statistical significance. With sample sizes so large, most effects are highly significant. Therefore, the value of the odds ratio gives a better indication of whether one group has significantly higher or lower success rates than another. Funding for Performance and Equity 21 Table 5 Odds Ratios for Student Success Logistic Regression Variable 1998–99 1999–00 2000–01 2001–02 2002–03 Deprivation index (ref= LT 10) Deprivation 10-<20 0.96*** 0.95*** 0.94*** 0.94*** 0.94*** Deprivation 20-<30 0.90*** 0.88*** 0.86*** 0.88*** 0.89*** Deprivation 30-<50 0.82*** 0.81*** 0.78*** 0.81*** 0.82*** Deprivation 50+ 0.71*** 0.73*** 0.71*** 0.75*** 0.77*** Female 1.25*** 1.20*** 1.20*** 1.18*** 1.21*** Disabled 0.93*** 0.98 0.93*** 0.89*** 0.97*** Ethnicity (ref= white) Asian-other 0.81*** 0.82*** 0.80*** 0.81*** 0.77*** Bangladeshi 0.81*** 0.75*** 0.80*** 0.83*** 0.92*** Black-African 0.74*** 0.82*** 0.76*** 0.76*** 0.74*** Black-Caribbean 0.72*** 0.76*** 0.76*** 0.76*** 0.71*** Black-other 0.71*** 0.73*** 0.75*** 0.75*** 0.69*** Chinese 0.79*** 0.79*** 0.82*** 0.83*** 0.87*** Indian 0.90*** 0.90*** 0.91*** 0.95*** 1.00 Pakistani 0.76*** 0.74*** 0.64*** 0.77*** 0.81*** Age (ref= 19–25) Age 26–34 1.17*** 1.20*** 1.25*** 1.28*** 1.26*** Age 35–44 1.28*** 1.32*** 1.39*** 1.44*** 1.44*** Age 45–54 1.17*** 1.21*** 1.30*** 1.38*** 1.45*** Age 55+ 0.93*** 1.05*** 1.13*** 1.27*** 1.43*** Qualification levela (ref= level 1) Level 2 1.09*** 1.02*** 0.96*** 0.96*** 0.91*** Level 3 1.18*** 1.20*** 1.15*** 1.09*** 1.00 Level 4 & 5 1.07*** 0.80*** 0.75*** 0.78*** 0.86*** Other Level 1.49*** 1.48*** 1.37*** 0.92*** 1.47*** Externally assessed 0.82*** 0.88*** 0.92*** 0.68*** 0.70*** Mode of Attendance (ref= full-time full-year) Full-time Part-year 1.06*** 1.14*** 1.32*** 1.47*** 1.29*** Part-time 0.82*** 0.83*** 0.99 1.06*** 0.96*** Qualification duration (ref= LT 24 weeks) 24-<48 Weeks 0.67*** 0.71*** 0.63*** 0.56*** 0.61*** 48+ Weeks 0.51*** 0.47*** 0.47*** 0.38*** 0.47*** Open/Distance Education 0.64*** 0.79*** 0.65*** 0.50*** 0.55*** Funding Determinants Postcode disadvantage uplift 0.99 0.99 1.00 0.97*** 0.98*** Additional learning support 1.31*** 1.26*** 1.24*** 1.43*** 1.52*** Access funding 1.00 0.98 1.13*** Means tested benefit 0.69*** 0.71*** 0.75*** 0.80*** 0.73*** ABE students 0.81*** 0.91*** 1.08*** 1.08*** 1.16*** Asylum seeker 0.85*** 0.71*** 0.90*** 0.82*** 0.80*** N 1,244,985 1,608,931 1,622,600 1,629,278 1,771,656 Pseudo R2 .09 .079 .084 .104 .096 Number of Parameters 74 74 75 75 75 Education Policy Analysis Archives Vol. 14 No. 24 22 Table 5 notes: Results for geographic region, area of learning, qualification type, institution size, population density, and program weighting not shown. See Table A-2 in appendix for full model. a Percentages for 2002–03 not comparable to previous years due to change in definitions for how qualification level was defined. b The reference group for each funding determinant is qualifications not having that characteristic. For example, the odds ratio for Additional Learning Support shows the regression adjusted odds of success in comparison to qualifications that do not receive Additional Learning Support. c The access funding program did not begin until the 2000–01 academic year * p<.01, ** p<.001, *** p<.0001 Regression models were run separately for each academic year. Therefore, year to year comparisons should be made with caution. Recall that success rates increased by 10% over the five year period. If the odds ratio for a particular subgroup remains the same from year to year, it does not mean that the odds of success for that subgroup did not change from the previous year. Rather, it means that compared to the reference group, the subgroup has the same likelihood of success compared to the reference group as it did in the previous year. Comparing odds ratios over time for a particular row can, however, show how success rates for that group changed over time compared to the reference group. If a particular subgroup consistently had an odds ratio of .8, this means that their odds of success are lower than the reference group but that their gains in success rates are keeping pace with that of the reference group. Regression results in Table 5 show findings similar to the descriptive statistics. First, for level of deprivation, the odds ratios are all lower than one. This makes intuitive sense; it means that, after controlling for other factors, success rates are higher for the lowest deprivation band, which is the reference group. However, the odds ratios for the high deprivation groups get larger in each year, meaning that, controlling for other variables, success rates for more deprived students are catching up to success rates for less deprived students. The same can be said about ethnic minorities in comparison to white students, which are the reference group; despite having lower success rates in each year (exhibited by odds ratios less than one), odds ratios for ethnic minorities generally grow higher each year. There are several notable differences between the descriptive statistics and the regression results. When the model is run without qualification level variables (not shown), the odds ratios for high deprivation bands increase, which shows that these students are disproportionately enrolled in low-level qualifications which have higher success rates for all students (Jaquette, 2005). This line of thinking can be employed to explain the results for other sub-groups; Bangladeshis have higher odds ratios than whites (reference group) when only socioeconomic variables such as deprivation are included. This is because Bangladeshis are disproportionately living in high deprivation areas, a characteristic correlated with low success rates. However, the odds ratios for Bangladeshis decline once qualification level variables are added because Bangladeshis are disproportionately enrolled in entry level qualifications which have high success rates for all students. Such analysis shows the value of statistical modeling over descriptive statistics. Moving to age, even after controlling for other variables, such as difficulty level and qualification duration, older students have much higher success rates than younger ones. Additionally, longer course duration and open/distance education are both associated with lower success rates. As predicted in the analysis of funding policy, qualifications that were externally assessed had lower rates of success than non-externally assessed qualifications. The strength of this trend increased over time. Funding for Performance and Equity 23 Externally assessed qualifications much lower odds of success than externally non-externally assessed qualifications. These findings support the statement that funding for achievement can lead to a decline in academic standards. At the same time, this decline is not inevitable. External assessment precludes this possibility. This study is centrally concerned with funding determinants which are shown at the bottom of Table 5. Here, results for the individual components of the funding formula are shown, because it proved difficult to model the entire funding formula in any meaningful way. Institutions received financial rewards (described above) for increasing the success rates of students on means tested benefits, asylum seekers, and students receiving a postcode disadvantage uplift. Despite having odds ratios less than one, success rates for these students kept pace with the strong gains of their respective reference groups.23 Success rates for ABE students increased over time, and were especially strong in 2002–03 with an odds ratio of 1.16. This means that ABE students made strong gains in comparison to non- ABE students, who were the reference group. This is an important finding considering the number of funding initiatives which have been aimed at ABE students. The positive results for ABE students are stronger (1.24 odds ratio in 2002–03) when external assessment is not included in the model (model not shown). This is because ABE students were disproportionately enrolled in non- externally assessed qualifications. The results for students receiving Additional Learning Support (ALS) are even stronger. ALS has a large, positive impact on the odds of success, and this effect has grown over time. For example in 2002–03 students receiving ALS were 1.52 times more likely to be successful than those not receiving ALS. When external assessment is not included in the model, the odds ratio rises slightly to 1.55. The results for ABE students and ALS students are positive from a policy perspective. They suggest that financial policies targeted at institutions can help increase student success rates. Conclusion: Policy Learning for U.S. Community Colleges? To summarize, overall success rates for the population analyzed rose by 10% over a five year period. Gains were especially strong for ABE students, disadvantaged students, and those in need of additional learning support. How was this achieved? This study has focused on the role of funding policy and regulatory control. 1992 Incorporation Reform created a quasi-market in further education. Utilizing the theoretical discussion of quasi-markets, government policy generally sought to retain public providers, but created incentives for these providers to act competitively. These colleges became responsible for their own financial solvency. Colleges were given performance funding contracts. Institutions only receive funding if they are able to attract students. If their students dropped out or were not successful, funding would decrease. Additionally, high or improving success rates were prerequisite for “good inspection grades,” which in turn determined the job security of a college’s senior management. The quantitative analyses presented yield optimistic results. We are not powerless in the face of socioeconomic factors. The unintended consequences of performance accountability are not inevitable. External assessment can protect against declining academic standards. A disadvantage 23 In order to check for high collinearity between “level of deprivation” and students receiving a “postcode disadvantage uplift,” an additional model (not shown) was run without “level of deprivation.” The results for this model were very similar to the model shown. Education Policy Analysis Archives Vol. 14 No. 24 24 uplift can protect against the disincentive against enrolling students with higher resource needs. Inspection can protect against the incentive to focus resources only on a small number of outcomes. However, it would be difficult to incorporate these policies to U.S. states. Five reasons are listed. First, all public funding for English further education comes from the central government, while public funding for community colleges comes from federal, state and local governments. From an organizational theory perspective, the more that organizations in a sector rely on a single funding source, the more leverage that funding source has to demand performance (DiMaggio & Powell, 1983; Scott & Meyer, 1991). The relative dilution of public funding sources for community colleges and the small amount of funding devoted to performance makes it unlikely that performance funding will be sufficient to induce behavioral change in community colleges. Second, a fundamental priority of any organization is survival (DiMaggio & Powell, 1983). Voucher funding systems, which use the threat of market-exit as an inventive for increased efficiency, inherently increase budget instability in comparison to traditional “base + enrollment growth + inflation” funding policies. On Incorporation Day (April 1, 1993) there were 465 further education colleges but by 2003 there were only 435 colleges due to mergers and closures (Cope, Goodship, & Holloway, 2003). Such closures would not be tolerated in the U.S. because the American Association of Community Colleges is much stronger than its English counterpart, The Association of Colleges. Third, although the equity components of the per-pupil funding formula would be supported by champions of social justice the U.S. political climate that increasingly values merit funding over need-based funding. Furthermore, need-based funding policies in the U.S. are usually eroded over time by political pressure to cater to middle-class voters, as has been the case with Federal Pell Grants (Callan, 2001) and the Georgia Hope Scholarship (Henry & Rubenstein, 2002). Fourth, the English brand of performance funding could not be imported to the U.S. because English qualifications are generally much shorter and more discrete than U.S. degrees. Interestingly, the English system provides performance funding when students successfully pass sub-components of a single qualification (Learning & Skills Council, 2002). By contrast, U.S. performance funding entire degrees (which can take community college students upwards of six years to complete) seems ridiculous. A U.S. performance funding system analogous to the English system would provide performance funding for each individual course successfully completed, but lack of external assessment would make such a policy problematic. Fifth, the U.S. lacks the strong regulatory agencies that underpin the English system. Perhaps the most intractable obstacle to importing English policies is the fundamental difference between the U.S. and English welfare states. England regulates further education through a sophisticated bureaucracy of state-owned regulatory agencies. The Department for Education and Skills controls overall strategy and can continually reorganize the regulatory agencies under its remit so that their individual missions balance the sector on the whole. Other areas of English social policy, such as welfare and unemployment benefits, operate similarly (Cope et al., 2003). The U.S., by contrast, has a much smaller welfare state, and historically relies on voluntary agencies to regulate social policy. In his 19th century observations of America Alexis de Tocqueville said “Americans of all ages, all conditions, and all dispositions constantly form associations…. Wherever at the head of some new undertaking you see the government in France, or a man of rank in England, in the United States you will be sure to find an association” (Tocqueville, 1862, p. 198). Accreditation of postsecondary education institutions provides an example of one such voluntary agency. Accreditation is America’s substitute for English inspection and external assessment. However, accreditation associations are run by college and university presidents. This violates the English principle that regulation and provision should have an arms-length relationship. U.S. accreditation associations cannot be expected to hold a tough line with regard to performance Funding for Performance and Equity 25 accountability standards, especially when public funding is at stake. In absence of strong state regulatory agencies, performance funding for student success is likely to decrease academic standards. A frequent lesson from international comparative policy research is that individual social policies work because they are buoyed by a complex welfare state and social structure that has evolved over time in that particular country (Esping-Anderson, 1990). Wholesale policy borrowing rarely works. However, certain components of English policymaking may be more feasible in some states than others. Florida, for instance, has a strong regulatory environment and 69% of community college operating revenue comes from the state (Education Commission of the States, 2000), a trait that permits considerable leverage to demand performance. Despite these heavy-handed caveats, the English story is a positive one that U.S. policymakers can learn from. First, performance accountability based is merely a means to convince colleges to focus their resources on outcomes deemed important by external stakeholders. Performance accountability policies are typically unsuccessful when colleges are coerced into compliance, or if they think the performance indicators or performance targets to be unreasonable. Policy mandates cannot increase student success alone; before any dramatic gains in student success, colleges must internalize the value of student success. My informal interviews with policymakers and college presidents in England convinced me that English policy was successful in convincing colleges to internalize the importance of student success. In England the report Unfinished Business (Audit Commission & Office for Standards in Education, 1993), which showed the low retention and success rates in further education colleges, was instrumental in convincing college administrators and faculty that these problems needed urgent repair. Furthermore, equity funding components, such as the disadvantage uplift and ALS funding, show college administrators that the government is a reasonable partner and that it will provide the additional resources to make student success a reality. By contrast, U.S. policies which demand better student outcomes without additional funding that considers educational inputs have been viewed with skepticism (Burke & Associates, 2002; Dougherty & Hong, 2005; Harbour & Nagy, 2005). Although the English story is a positive one, a note of caution is necessary. Since the 2002– 03 academic year, the English government has increasingly coerced further education colleges to serve national economic ends. England has a centralized, as opposed to federalized, system of governance and the central government controls nearly all funding for further education colleges. In 2001 the Department of Education was merged with the Department of Employment to become the Department for Education and Skills (DfES). The recent education policies of the DfES—for example raising fees, cutting funding for adult education, and pressuring colleges to provide training in certain industries (Department for Education and Skills, 2003)– are clearly economically motivated and are resented by many colleges as an infringement on autonomy and a diminished commitment to disadvantaged students. In the coming years the coalition between colleges and the government may crumble. A more thorough discussion is outside a scope of this paper, but one general insight emerges; although centralized education policy can make impressive progress, it can also be hijacked by a narrow economic focus which can hurt the system in the long run. The emerging story will be interesting to watch. The final policy lesson from this research concerns the use of market-based policies— specifically voucher systems—in education. Interestingly, voucher systems in education were first theorized by American economist Milton Friedman (1955) and first implemented by U.S. school districts (Halsey, Lauder, Phillip, & Stuart Wells, 1997). English further education colleges are funded through a national voucher system in which funding follows students to whatever institution they decide to attend. Similar voucher systems are used to fund English compulsory education Education Policy Analysis Archives Vol. 14 No. 24 26 (Glennerster, 2002), and compulsory education in Belgium (Vandenberghe, 1998), and Sweden (Lundahl, 2002). Despite their widespread use in other countries, voucher systems have not taken root in U.S. education, and the voucher debate remains polemical. Some critics contend that voucher systems will only exacerbate social stratification (Newman, Couturier, & Scurry, 2004). Indeed, this can be true. For example, the new voucher system for higher education in Colorado pays a flat rate to all residents, which wealthy families may use to supplement their existing income (Harbour et al., 2006). Clearly, voucher systems pose legitimate questions of concern. Will for-profit institutions be eligible for voucher funding? Will religious institutions be eligible for voucher funding, thereby diluting the separation between church and state? What about the stratification of educational achievement? School choice tends to create a sorting hat where the best students—typically having the most informed and engaged parents—are concentrated in the same schools while the worst students are concentrated in others (Vandenberghe, 1998). Indeed, this has been a problem in English compulsory education (Ball et al., 2000). However, does this problem persist in adult education, where there are often few affordable providers within a reasonable geographic distance? On the other hand, voucher systems can tailor funding to individualized student needs in a way that block funding cannot. Depending on the particular funding formula used, voucher systems can exacerbate educational inequalities or they can promote educational equality. The English voucher system promotes vertical equity. It pays tuition for disadvantaged students and, as Figure 1 showed, colleges receive more funding for enrolling students that require additional resources to be successful. In conclusion, this research calls for a more nuanced discussion of voucher systems in education. Voucher systems are policy tools that give institutions incentives to achieve outcomes that are rewarded by the funding formula. They are neither inherently good nor inherently bad; the policy details matter a great deal. Funding for Performance and Equity 27 References Alfonso, M., Bailey, T., & Scott, M. (2005). The educational outcomes of occupational sub- baccalaureate students: Evidence from the 1990s. Economics of Education Review, 24(2), 197–212. Audit Commission, & Office for Standards in Education. (1993). Unfinished business: Full-time educational courses for 16–19 year olds: Norwich: Her Majesty's Stationery Office. Bailey, T., Alfonso, M., Calcagno, J. C., Jenkins, D., Kienzl, G., & Leinbach, T. (2004). Improving student attainment in community colleges: Institutional characteristics and policies. New York: Community College Research Center, Teachers College, Columbia University. Bailey, T., Calcagno, J. C., Jenkins, D., Kienzl, G., & Leinbach, T. (2005). The effects of institutional factors on the success of community college students. New York: Community College Research Center. Ball, S. J., Maguire, M., & Macrae, S. (2000). Choice, pathways and transitions post-16: New youth, new economies in the global city. London: RoutledgeFalmer. Bartlett, W., Roberts, J. A., & Le Grand, J. (1998a). Lessons from the experience of quasi- markets in the 1990s. In W. Bartlett, J. A. Roberts & J. Le Grand (Eds.), A Revolution in Social Policy (pp. 275–290). Bristol, UK: The Policy Press. Bartlett, W., Roberts, J. A., & Le Grand, J. (Eds.). (1998b). A revolution in social policy: Lessions from the development of quasi-markets in the 1990s. Bristol, UK: The Policy Press. Behn, R. D. (2001). Rethinking democratic accountability. Washington, DC: The Brookings Institution. Burke, J. C., & Associates. (2002). Funding public colleges and universities for performance: Popularity, problems, and prospects. Albany, NY: The Rockefeller Institute. Burke, J. C., & Minassians, H. P. (2003). Performance reporting: "Real" accountability or accountability "lite": Seventh annual survey 2003. Albany: State University of New York, Rockefeller Institute of Government. Callan, P. (2001). Reframing access and opporunity: Problematic state and federal higher education policy in the 1990s. In D. E. Heller (Ed.), The states and public higher education policy: Affordability, access, and accountability (pp. 83–99). Baltimore: Johns Hopkins University Press. Cope, S., Goodship, J., & Holloway, D. (2003). Regulating the new governance: The case of further education. Journal of Education and Training, 55(2), 183–207. Davies, P. (2001). Closing the achievement gap: Colleges making a difference. London: Learning and Skills Development Agency. Education Policy Analysis Archives Vol. 14 No. 24 28 Davies, P., & Rudden, T. (2000). Differential achievement: What Does the ISR profile tell us? London: Learning and Skills Development Agency. Department for Education and Employment. (1999). Learning to succeed: a new framework for post-16 learning (No. Cm 4392). London: Department for Education and Skills. Department for Education and Employment. (2001). Skills for life: the national strategy for improving adult literacy and numeracy skills. London: Department for Education and Employment. Department for Education and Skills. (2003). 21st century skills, realising our potential (No. Cm 5810). Norwich: The Stationery Office. DiMaggio, P., & Powell, W. W. (1983). The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48(2), 147– 160. Dougherty, K. J., & Hong, E. (2005). State systems of performance accountability for community colleges: impacts and lessons for policymakers. Boston, MA: Jobs for the Future. Education Commission of the States. (2000). State funding for community colleges: A 50-state survey. Denver, CO: Center for Community College Policy, Education Comission of the States. Esping-Anderson, G. (1990). The three worlds of welfare capitalism. Oxford, UK: Polity Press. Faraday, S., & Fletcher, M. (2003). Funding additional learning support. London: Learning and Skills Development Agency. Faraday, S., Fletcher, M., & Gidney, M. (2000). Evaluation of the additional support mechanism. London: Further Education Development Agency. Friedman, M. (1955). The role of government in education. In R. A. Solo (Ed.), Economics and the public interest (pp. 123–144). New Brunswick, NJ: Rutgers University Press. Further Education Funding Council. (1999a). Extension of the widening participation factor for 1999–2000 (No. 99/42). Coventry: Further Education Funding Council. Further Education Funding Council. (1999b). Tariff 1999–2000 (No. 99/01). Coventry: Further Education Funding Council. Further Education Funding Council. (2000). Guidance on the tariff 2000–01. Coventry: Further Education Funding Council. Further Education Funding Council. (2001). Guidance on further education funding and eligibility rates 2001–02. Coventry: Further Education Funding Council. Glennerster, H. (2002). United Kingdom education 1997–2001. Oxford Review of Economic Policy, 18(2), 120–136. Funding for Performance and Equity 29 Grubb, N. (2002). Learning and earning in the middle, part I: national studies of pre- baccalaureate education. Economics of Education Review, 21, 299–321. Grubb, W. N., & Lazerson, M. (2004). The educational gospel: the economic power of schooling. Boston, MA: Harvard University Press. Halsey, A. H., Lauder, H., Phillip, B., & Stuart Wells, A. (Eds.). (1997). Education: Culture, economy, and society. Oxford, UK: Oxford University Press. Harbour, C. P., Davies, C. P., & Lewis, C. W. (2006). Colorado's voucher legislation and the consequences for community colleges. Community College Review, 33(3/4), 1–18. Harbour, C. P., & Nagy, P. (2005). Assessing a state-mandated institutional accountability program: The perceptions of selected community college leaders. Community College Journal of Research and Practice, 29, 445–461. Henry, G. T., & Rubenstein, R. (2002). Paying for grades: Impact of merit-based financial aid on educational quality. Journal of Policy Analysis and Management, 21(1), 93–109. Jaquette, O. (2005). Funding, control, and student success: A comparative study of English further education colleges and California community colleges. Oxford University, Oxford, UK. Johnson, N. (1999). Mixed economies of welfare: A comparative perspective. Herfordshire, UK: Prentice Hall Europe. Kennedy, H. (1997). Learning works: widening participation in further education. Coventry, UK: Further Education Funding Council. Learning & Skills Council. (2002). Funding guidance for further education in 2002/3. Coventry, UK: Learning & Skills Council. Learning & Skills Council. (2004a). Further education and work based learning for young people - learner outcomes in england 2002/2003. Coventry, UK: Learning & Skills Council. Learning & Skills Council. (2004b). Further education, work based learning for young people and adult and community learning—Learner numbers in England 2003/4. Coventry, UK: Learning & Skills Council. Learning & Skills Council. (2005). Further education and work based learning for young people— Learner numbers in England 2002/2003. Coventry, UK: Learning & Skills Council. Learning & Skills Development Agency. (2003). Quality Matters: March 2003. London: Learning and Skills Development Agency. Lundahl, L. (2002). Sweden: Decentralization, deregulation, quasi-markets - and then what? Journal of Education Policy, 17(6), 687–697. Martinez, P. (2001). Improving student retention and achievement: What do we Know and what do we need to find out? London: Learning and Skills Development Agency. Education Policy Analysis Archives Vol. 14 No. 24 30 McLure, R. (2000). Recurrent funding. In A. Smithers & P. Robinson (Eds.), Further Education Reformed : Shaping the Future. London: Falmer Press. National Center for Educational Statistics. (2004). Digest of education statistics, 2003. Washington, DC: National Center for Education Statistics. National Statistics. (2005a). Population: UK population grows to 59.6 million. Retrieved April 26, 2005, from http://www.statistics.gov.uk/cci/nugget.asp?id=760. National Statistics. (2005b). Statistics of education: Education and training expenditure since 1993–94. London: National Statistics. Newman, F., Couturier, L., & Scurry, J. (2004). The future of higher education: Rhetoric, reality, and the risk of the market. San Francisco: Jossey-Bass. Office for Standards in Education. (2002). Handbook for inspecting colleges (No. HMI 464). London: Office for Standards in Education. Office for Standards in Education, & Adult Learning Inspectorate. (2001). The common inspection framework for inspecting post-16 education and training. London: Office for Standards in Education. Qualifications and Curriculum Authority. (2005). What we do. Retrieved 27 April, 2005, from http://www.qca.org.uk/3657.html. London: Author. Rospigliosi, G. (2000). Franchising. In A. Smithers & P. Robinson (Eds.), Further education reformed: Shaping the future. London: Falmer Press. Scott, R. W., & Meyer, J. W. (1991). The organization of societal sectors. In P. J. DiMaggio & W. W. Powell (Eds.), The new institutionalism in organizational analysis (pp. 108–140). Chicago: University of Chicago Press. St. John, E. P. (1999). Evaluating state student grant programs: a case study of Washington's Grant Program. Research in Higher Education, 40(2), 149–169. Tocqueville, A. d. (1862). Democracy in America. Cambridge: Sever and Francis. U.S. Department of Education. (2002). Descriptive summary of 1995–96 beginning postsecondary students: Six years later. Washington, DC: Author. Retrieved 21 April, 2005, from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2003151. Vandenberghe, V. (1998). Educational quasi-markets: The Belgian experience. In W. Bartlett, J. A. Roberts & J. Le Grand (Eds.), A Revolution in Social Policy (pp. 275–290). Bristol, UK: The Policy Press. Walsh, K. (1995). Public services and market mechanisms: Competition, contracting and the new public management. Basingstoke: Macmillan Press. http://www.statistics.gov.uk/cci/nugget.asp?id=760 http://www.qca.org.uk/3657.html http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2003151 Funding for Performance and Equity 31 Appendix Table A-1 Sample Size: Qualifications by Year and Subgroup Variable 1998–99 1999–00 2000–01 2001–02 2002–03 Overall 1,509,393 1,687,464 1,692,394 1,646,138 1,771,842 Deprivation Index Deprivation LT 10 329,810 365,731 365,396 369,394 371,588 Deprivation 10-<20 436,658 483,795 480,302 470,287 491,029 Deprivation 20-<30 278,732 309,921 307,696 295,890 317,918 Deprivation 35-<50 322,034 362,924 366,418 349,254 396,008 Deprivation 50+ 142,159 165,093 172,582 161,313 195,299 Ethnicity Bangladeshi 7,755 9,114 10,295 11,604 15,677 Black-African 33,780 38,409 45,118 49,322 64,450 Black-Caribbean 30,092 33,403 31,710 34,666 41,738 Black-Other 11,243 12,882 13,599 10,587 11,632 Chinese 8,899 10,437 9,944 9,665 12,785 Indian 29,268 37,855 35,799 35,916 43,477 Pakistani 31,013 33,540 35,455 34,505 40,668 White 1,177,380 1,260,058 1,210,861 1,129,889 1,311,190 Asian-Other 17,681 23,764 29,420 30,670 36,637 Gender Male 569,048 631,287 636,922 626,456 673,948 Female 940,345 1,056,177 1,055,472 1,019,682 1,097,894 Age Age 19–25 363,485 390,459 388,885 367,617 412,494 Age 26–34 426,861 454,255 434,282 394,028 418,390 Age 35–44 378,978 424,700 421,507 403,925 426,266 Age 45–54 221,082 255,302 257,197 256,781 261,555 Age 55+ 118,987 162,748 190,523 223,787 253,137 Disabled 83,927 108,009 139,531 157,727 204,372 Qualification levela Entry & level 1 439,825 621,675 624,377 619,177 858,984 Level 2 372,908 476,299 448,887 431,416 434,292 Level 3 272,344 322,010 318,419 293,362 294,696 Level 4 or Higher 51,581 36,115 31,324 25,233 34,092 Other 372,735 231,365 269,387 276,950 149,778 External assessment Not externally assessed 331,466 164,134 199,966 293,008 448,577 Externally assessed 1,177,927 1,523,330 1,492,428 1,353,130 1,323,265 Qualification duration LT 24 Weeks 494,352 603,543 641,736 659,468 748,634 24-<48 Weeks 830,882 875,065 861,214 818,652 844,314 48+ Weeks 184,159 208,856 189,444 168,018 178,894 Education Policy Analysis Archives Vol. 14 No. 24 32 Variable 1998–99 1999–00 2000–01 2001–02 2002–03 Mode of attendance Full-time, full-year 302,482 323,105 312,304 285,149 319,136 Full-time, part-year 82,854 98,716 111,348 118,090 114,797 Part-time 1,124,057 1,265,643 1,268,742 1,242,899 1,337,909 Open/distance learning 83,674 61,012 78,916 123,820 127,065 Funding Determinants Receive access fundingb 70,526 74,232 149,949 Postcode disadvantage uplift 440,432 454,575 451,883 429,449 516,141 On benefit 254,439 276,424 253,293 226,068 271,579 ABE student 119,546 149,385 155,942 160,031 243,046 Asylum seeker 5,438 14,197 24,311 27,710 35,323 Additional Learning Support 81,054 105,700 133,967 127,717 164,280 a Qualification level in 2002–03 not comparable to previous years due to change in definitions for how qualification level was defined. b The access funding program did not begin until the 2000–01 academic year Funding for Performance and Equity 33 Table A-2 Full-Model Regression Results Variable 1998–99 1999–00 2000–01 2001–02 2002–03 Socioeconomic determinants Deprivation (ref = LT 10) Deprivation 10-<20 0.96*** 0.95*** 0.94*** 0.94*** 0.94*** Deprivation 20-<30 0.90*** 0.88*** 0.86*** 0.88*** 0.89*** Deprivation 30-<50 0.82*** 0.81*** 0.78*** 0.81*** 0.82*** Deprivation 50+ 0.71*** 0.73*** 0.71*** 0.75*** 0.77*** Female 1.25*** 1.20*** 1.20*** 1.18*** 1.21*** Disabled 0.93*** 0.98 0.93*** 0.89*** 0.97*** Ethnicity (ref= white) Asian-other 0.81*** 0.82*** 0.80*** 0.81*** 0.77*** Bangladeshi 0.81*** 0.75*** 0.80*** 0.83*** 0.92*** Black-African 0.74*** 0.82*** 0.76*** 0.76*** 0.74*** Black-Caribbean 0.72*** 0.76*** 0.76*** 0.76*** 0.71*** Black-other 0.71*** 0.73*** 0.75*** 0.75*** 0.69*** Chinese 0.79*** 0.79*** 0.82*** 0.83*** 0.87*** Indian 0.90*** 0.90*** 0.91*** 0.95*** 1.00 Pakistani 0.76*** 0.74*** 0.64*** 0.77*** 0.81*** Age (ref= 19–25) Age 26–34 1.17*** 1.20*** 1.25*** 1.28*** 1.26*** Age 35–44 1.28*** 1.32*** 1.39*** 1.44*** 1.44*** Age 45–54 1.17*** 1.21*** 1.30*** 1.38*** 1.45*** Age 55+ 0.93*** 1.05*** 1.13*** 1.27*** 1.43*** Region (ref= South East) East Anglia 1.00 1.05*** 1.10*** 0.90*** 0.93*** East Midlands 1.17*** 1.27*** 1.12*** 1.09*** 1.07*** Greater London 0.86*** 0.94*** 1.07*** 0.91*** 1.01 North East 1.21*** 1.35*** 1.57*** 1.22*** 1.10*** North West 1.02 1.24*** 1.38*** 1.28*** 1.16*** South West 1.23*** 1.18*** 1.19*** 1.09*** 1.05*** West Midlands 1.12*** 1.24*** 1.25*** 1.11*** 1.03*** Yorkshire Humberside 1.14*** 1.20*** 1.26*** 1.06*** 1.04*** Qualification determinants Qualification levela (ref = level 1) Level 2 1.09*** 1.02*** 0.96*** 0.96*** 0.91*** Level 3 1.18*** 1.20*** 1.15*** 1.09*** 1.00 Level 4 & 5 1.07*** 0.80*** 0.75*** 0.78*** 0.86*** Other level 1.49*** 1.48*** 1.37*** 0.92*** 1.47*** Externally assessed 0.82*** 0.88*** 0.92*** 0.68*** 0.70*** Education Policy Analysis Archives Vol. 14 No. 24 34 Variable 1998–99 1999–00 2000–01 2001–02 2002–03 Area of learning (ref= tourism/hospitality) ABE 0.82*** 0.77*** 0.97 0.90*** 0.71*** Agriculture 0.72*** 0.81*** 0.64*** 0.68*** 0.69*** Art & design 0.68*** 0.65*** 0.73*** 0.77*** 0.79*** Business 0.75*** 0.73*** 0.92*** 0.71*** 0.84*** Construction 1.05* 0.98 1.01 0.91*** 0.79*** Engineering 0.96* 0.81*** 0.95** 1.04 0.93*** Health & care 1.21*** 1.06*** 1.16*** 1.07*** 1.14*** Humanities 0.66*** 0.60*** 0.65*** 0.55*** 0.58*** Science, math, IT 0.68*** 0.60*** 0.66*** 0.54*** 0.61*** Qualification type (ref= “other”) A/AS 0.62*** 0.55*** 0.67*** 0.77*** 0.82*** GNVQ 0.84*** 0.94* 0.81*** 0.79*** 0.93* HE access 1.04 0.83*** 0.89*** 0.92*** 0.90*** NVQ 0.74*** 0.67*** 0.61*** 0.64*** 0.69*** pre GNVQ 1.11*** 1.02*** 0.88*** 0.82*** 1.12*** Student mode of attendance (ref= full-time full-year) Full-time part-year 1.06*** 1.14*** 1.32*** 1.47*** 1.29*** Part-time 0.82*** 0.83*** 0.99 1.06*** 0.96*** Qualification duration (ref= LT 24 weeks) 24-<48 weeks 0.67*** 0.71*** 0.63*** 0.56*** 0.61*** 48+ weeks 0.51*** 0.47*** 0.47*** 0.38*** 0.47*** Open/distance education 0.64*** 0.79*** 0.65*** 0.50*** 0.55*** Dedicated employer provision 0.89*** 1.13*** 1.50*** 1.41*** 1.88*** Employee release 1.28*** 1.19*** 1.30*** 1.38*** 1.48*** Qual is not highest level taken 1.20*** 1.23*** 1.17*** 1.14*** 1.00 Institution level determinants College size (ref= LT 15,000) 15,000–25,000 students 1.02* 0.94*** 0.96*** 0.96*** 0.89*** 8,000–15,000 students 0.99 0.99 0.97*** 0.95*** 0.90*** GT 25,000 students 0.82*** 0.96*** 0.94*** 0.83*** 0.89*** Population density (ref= urban) Rural 0.98 1.00 1.03*** 1.06*** 1.06*** Town 0.99 0.98* 1.01 1.06*** 1.03*** Tertiary college (ref= general further education college) 1.06*** 0.97*** 0.96*** 0.90*** 0.99 Funding determinantsb Postcode disadvantage uplift 0.99 0.99 1.00 0.97*** 0.98*** Additional learning support 1.31*** 1.26*** 1.24*** 1.43*** 1.52*** Access fundingc 1.00 0.98 1.13*** Means tested benefit 0.69*** 0.71*** 0.75*** 0.80*** 0.73*** Basic skills learner 0.81*** 0.91*** 1.08*** 1.08*** 1.16*** Asylum seeker 0.85*** 0.71*** 0.90*** 0.82*** 0.80*** Funding for Performance and Equity 35 Variable 1998–99 1999–00 2000–01 2001–02 2002–03 Program weighting factor (ref = A = 1.00) B=1.12 1.06*** 1.04*** 1.15*** 1.07*** 1.00 C=1.3 0.94*** 1.05*** 1.21*** 1.05*** 1.17*** D=1.6 0.85*** 0.90*** 1.19*** 0.86*** 1.10*** E=1.72 0.95 0.88* 1.47*** 1.21*** 1.10* F=1.4 0.70*** 0.78*** 0.65*** 0.69*** 0.87*** Model Fit Statistics Number of cases 1,244,985 1,608,931 1,622,600 1,629,278 1,771,656 Raw student success rate 56.4% 59.6% 61.8% 64.2% 66.9% -2 log L 1,619,075 2,073,899 2,055,472 1,997,139 2,121,875 Pseudo r-square 0.09 0.079 0.084 0.104 0.096 Chi-square 86,148 96,399 103,144 128,778 126,941 Degrees of freedom 74 74 75 75 75 a Percentages for 2002–03 not comparable to previous years due to change in definitions for how qualification level was defined. b The reference group for each funding determinant is qualifications not having that characteristic. For example, the odds ratio for Additional Learning Support shows the regression adjusted odds of success in comparison to qualifications that do not receive Additional Learning Support. c The access funding program did not begin until the 2000–01 academic year * p<.01, ** p<.001, *** p<.0001 Education Policy Analysis Archives Vol. 14 No. 24 36 About the Author Ozan Jaquette University of Michigan Email: ozanj@umich.edu Ozan Jaquette is a doctoral student in the Department of Higher Education at the University of Michigan. He received an MPhil. in Comparative Social Policy from the Department of Social Policy and Social Work at the University of Oxford. The research presented in this article was part of his MPhil. thesis “Funding, control, and student success: A comparative study of English further education colleges and California community colleges.” mailto:ozanj@umich.edu Funding for Performance and Equity 37 EDUCATION POLICY ANALYSIS ARCHIVES http://epaa.asu.edu Editor: Sherman Dorn, University of South Florida Production Assistant: Chris Murrell, Arizona State University General questions about appropriateness of topics or particular articles may be addressed to the Editor, Sherman Dorn, epaa-editor@shermandorn.com. Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Robert Bickel Marshall University Gregory Camilli Rutgers University Casey Cobb University of Connecticut Linda Darling-Hammond Stanford University Gunapala Edirisooriya Youngstown State University Mark E. Fetler California Commission on Teacher Credentialing Gustavo E. Fischman Arizona State Univeristy Richard Garlikov Birmingham, Alabama Gene V Glass Arizona State University Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Ohio University William Hunter University of Ontario Institute of Technology Daniel Kallós Umeå University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Berkeley Michele Moses Arizona State University Anthony G. Rud Jr. Purdue University Michael Scriven Western Michigan University Terrence G. Wiley Arizona State University John Willinsky University of British Columbia http://epaa.asu.edu/epaa/board/apple.html http://epaa.asu.edu/epaa/board/berliner.html http://epaa.asu.edu/epaa/board/camilli.html http://epaa.asu.edu/epaa/board/darling-hammond.html http://epaa.asu.edu/epaa/board/fetler.html http://epaa.asu.edu/epaa/board/fischman.html http://epaa.asu.edu/epaa/board/garlikov.html http://epaa.asu.edu/epaa/board/green.html http://epaa.asu.edu/epaa/board/howleya.html http://epaa.asu.edu/epaa/board/howleyc.html http://epaa.asu.edu/epaa/board/hunter.html http://epaa.asu.edu/epaa/board/kallos.html http://epaa.asu.edu/epaa/board/levin.html http://epaa.asu.edu/epaa/board/mauhs.html http://epaa.asu.edu/epaa/board/mclean.html http://epaa.asu.edu/epaa/board/mintrop.html http://epaa.asu.edu/epaa/board/moses.html http://epaa.asu.edu/epaa/board/rud.html http://epaa.asu.edu/epaa/board/scriven.html http://epaa.asu.edu/epaa/board/wiley.html http://epaa.asu.edu/epaa/board/willinsky.html Education Policy Analysis Archives Vol. 14 No. 24 38 EDUCATION POLICY ANALYSIS ARCHIVES English-language Graduate-Student Editorial Board Noga Admon New York University Jessica Allen University of Colorado Cheryl Aman University of British Columbia Anne Black University of Connecticut Marisa Cannata Michigan State University Chad d'Entremont Teachers College Columbia University Carol Da Silva Harvard University Tara Donahue Michigan State University Camille Farrington University of Illinois Chicago Chris Frey Indiana University Amy Garrett Dikkers University of Minnesota Misty Ginicola Yale University Jake Gross Indiana University Hee Kyung Hong Loyola University Chicago Jennifer Lloyd University of British Columbia Heather Lord Yale University Shereeza Mohammed Florida Atlantic University Ben Superfine University of Michigan John Weathers University of Pennsylvania Kyo Yamashiro University of California Los Angeles Funding for Performance and Equity 39 Archivos Analíticos de Políticas Educativas Associate Editors Gustavo E. Fischman & Pablo Gentili Arizona State University & Universidade do Estado do Rio de Janeiro Founding Associate Editor for Spanish Language (1998—2003) Roberto Rodríguez Gómez Editorial Board Hugo Aboites Universidad Autónoma Metropolitana-Xochimilco Adrián Acosta Universidad de Guadalajara México Claudio Almonacid Avila Universidad Metropolitana de Ciencias de la Educación, Chile Dalila Andrade de Oliveira Universidade Federal de Minas Gerais, Belo Horizonte, Brasil Alejandra Birgin Ministerio de Educación, Argentina Teresa Bracho Centro de Investigación y Docencia Económica-CIDE Alejandro Canales Universidad Nacional Autónoma de México Ursula Casanova Arizona State University, Tempe, Arizona Sigfredo Chiroque Instituto de Pedagogía Popular, Perú Erwin Epstein Loyola University, Chicago, Illinois Mariano Fernández Enguita Universidad de Salamanca. España Gaudêncio Frigotto Universidade Estadual do Rio de Janeiro, Brasil Rollin Kent Universidad Autónoma de Puebla. Puebla, México Walter Kohan Universidade Estadual do Rio de Janeiro, Brasil Roberto Leher Universidade Estadual do Rio de Janeiro, Brasil Daniel C. Levy University at Albany, SUNY, Albany, New York Nilma Limo Gomes Universidade Federal de Minas Gerais, Belo Horizonte Pia Lindquist Wong California State University, Sacramento, California María Loreto Egaña Programa Interdisciplinario de Investigación en Educación Mariano Narodowski Universidad Torcuato Di Tella, Argentina Iolanda de Oliveira Universidade Federal Fluminense, Brasil Grover Pango Foro Latinoamericano de Políticas Educativas, Perú Vanilda Paiva Universidade Estadual Do Rio De Janeiro, Brasil Miguel Pereira Catedratico Universidad de Granada, España Angel Ignacio Pérez Gómez Universidad de Málaga Mónica Pini Universidad Nacional de San Martin, Argentina Romualdo Portella do Oliveira Universidade de São Paulo Diana Rhoten Social Science Research Council, New York, New York José Gimeno Sacristán Universidad de Valencia, España Daniel Schugurensky Ontario Institute for Studies in Education, Canada Susan Street Centro de Investigaciones y Estudios Superiores en Antropologia Social Occidente, Guadalajara, México Nelly P. Stromquist University of Southern California, Los Angeles, California Daniel Suarez Laboratorio de Politicas Publicas-Universidad de Buenos Aires, Argentina Antonio Teodoro Universidade Lusófona Lisboa, Carlos A. Torres UCLA Jurjo Torres Santomé Universidad de la Coruña, España http://epaa.asu.edu/epaa/board/acosta.html http://epaa.asu.edu/epaa/board/bracho.html http://epaa.asu.edu/epaa/board/canales.html http://epaa.asu.edu/epaa/board/casanova.html http://epaa.asu.edu/epaa/board/epstein.html http://epaa.asu.edu/epaa/board/kent.html http://epaa.asu.edu/epaa/board/schugur.html http://epaa.asu.edu/epaa/board/torres2.html http://epaa.asu.edu/epaa/board/torres.html Literature Review Suitability of a U.S.—England Comparison Theoretical Framework Funding Policy and Regulatory Control in England Overview of Policy Reform in English Further Education The Per-Pupil Funding Formula Regulating Further Education Summary Descriptive Statistics and Modeling Data and Sample Descriptive Statistics Statistical Modeling Conclusion: Policy Learning for U.S. Community Colleges? References Appendix Production Assistant: Chris Murrell, Arizona State University