THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION1 FEATURED ARTICLE Student Talent Enrichment Program: Improving Underfunded Students’ Retention Outcomes Trevor Francis, Associate Vice Provost, Director of Student Success, University of Arkansas Don T. Nix, Director of ASAP & 360 Advising, University of Arkansas Danielle Dunn, Associate Director for Student Success, University of Arkansas Brett E. Rankin, Research Coordinator, University of Arkansas The Student Talent Enrichment Program was an experimental grant program conducted at a major research university to improve underfunded students’ retention outcomes. This program provided scholarships to 151 first- year students with $5,000 or more in unmet financial need. Participants received a one-time grant in the fall of 2017, with award amounts ranging from $2,500 to $10,000. Grant recipients’ retention rates exceeded the control group by small margins. However, these grants failed to make a difference only among the few grantees already in academic jeopardy. For grant recipients who had early GPAs of at least 1.5 prior to receiving financial awards, term- to-term retention rates surpassed the control group by a substantial, expected, and statistically significant margin. Introduction The federal government, state governments, and employers, along with colleges and universities, have long used need-based financial aid to assist students in their academic endeavors. The Higher Education Act of 1965 was passed by Congress to provide financial aid to students for post-secondary education (McCants, 2003). Need-based aid serves as a method to help families who may otherwise not be capable of sending their students to postsecondary institutions cover the cost of tuition, fees, and living expenses along with various other expenditures. In 2012-13, the federal government provided 45% of undergraduate grant aid compared to 39% in 2002. In 2013, the federal government issued $185.1 billion dollars in aid to undergraduates, which more than doubled in a ten-year period from 2003 (Baum & Payea, 2013). Pell grants grew from approximately $11 billion in the 2000-2001 academic year to $39 billion during the 2010-2011 academic year, although total dollars spent on need-based grants fell to roughly $29 billion in 2017. In 2017, approximately $100 billion in student loans, $30 billion in need-based grants, and $30 billion in income tax preferences were provided to students and their families. In all, the federal government awarded Pell grants to approximately 39% of undergraduates during the 2015-16 academic year (Alsalam & Carrington, 2018). There is no doubt that need-based aid provides students from low-income families opportunities to pursue higher education when they would likely face financial hardships otherwise. However, there are often academic benchmarks students must meet to maintain their need-based aid. Pell grants and other forms of need-based financial aid provided by state and institutional programs are no exception. The academic requirements from these funding sources vary from state to state and among institutions, and oftentimes must be met to remain eligible for future financial assistance. Within this article we will explore the impact that need-based and merit- based aid has on students, both with and without academic standards they must meet during their college 2VOLUME 28 NUMBER 1 career. Additionally, we will offer our results and conclusions from providing need-based aid to a subset of freshmen students at a large, research-based public institution located in the Southern United States. The purpose of this study is to investigate the effects of need-based aid on retention rates of first-time, full-time freshmen students in order to identify the population of freshmen who will benefit the most from supplemental grants. The research questions are as follows: 1) What impact does one-time, variable amount funding have on freshmen with demonstrated financial need? 2) Does one-time, variable amount funding have a significant impact on one-year retention rates? 3) What role, if any, does academic preparation play when considering need-based funding? Review of the Literature Financial Aid Considerations There are several reasons why students are motivated to accept financial aid. Alsalam and Carrington (2018) explain three financial barriers that drive students to accept federal aid. First, students oftentimes accept financial aid because they would be unable to fund higher education without it. Second, some students would choose to not pursue education because potential future financial benefits and education costs are too uncertain. Moreover, financial aid awards help reduce the uncertainty of costs and encourage students to pursue, even persist in higher education. Lastly, some students may not consider education worth the cost, even though they and society writ large will gain benefits from their increased education. Need-based grants such as Pell grants help reduce the cost for students with low incomes who are pursuing higher education and oftentimes provide relief from working part-time jobs to fund their education (Stinebrickner & Stinebrickner, 2008). Despite easing the financial burden on low-income students, some research suggests Pell grants have only modest effects, excepting non-traditional students, on the rates low-income students enroll in higher education. Several reasons exist that may lead to students giving up on free federal money - complex application processes along with late, untimely award notifications may contribute to many students passing up Pell grants (Alsalam & Carrington, 2018). Low-income students may ultimately decide the obstacles to enrolling in postsecondary education are not worth the effort and enter the workforce instead of pursuing education. Conversely, state grants may be more effective at attracting low-income students to college because state grants are often well known to the student population. Need-based grants provided by universities also give a considerable amount of assistance to undergraduate students – institutional money has been shown to help increase enrollment, student retention, persistence, and degree completion (Bettinger, 2004; Castleman & Long, 2016; Dynarski, 2003; Goldrick-Rab et al., 2013; Heller, 2003; Kane, 2007). Mettler (2014) pointed out local and state budgets have traditionally kept tuition prices lower than the full cost of higher education; however, states are devoting a smaller proportion of their budgets to higher education even though enrollments have increased. Furthermore, tuition costs have increased by 113% while state funding for full-time students has decreased by 26%. Consequently, public institutions producing the majority of bachelor’s degrees have become more reliant on tuition as a revenue source – the proportion of revenue from fees and tuition has increased from approximately 30% in 2000 to almost 50% in 2012 (Baum et al., 2014). Even when considering the positive impact of financial aid, the increase in tuition and fees has escalated the financial burden on families. A student at the 20th percentile of family income who attends a four-year institution uses an average of 45% of the family budget (Page & Scott-Clayton, 2016). Baum et al. (2014) discussed how institutions THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION3 and states have responded to ever increasing tuition and fees by offering financial aid packages to help offset the cost of higher education using a high-tuition, high-aid strategy (for some students). Financial aid data from 2014-15 indicates that 7 out of 10 undergraduates across the United States receive some type of financial aid. Full-time undergraduates received an average of $8,080 each in grants, $1,260 in tax credits and work-study assistance, and $4,840 in federal loans (Baum et al., 2014). Despite considerable grant funding and tax credits, students from low-income families still have significant unmet financial need. Several studies have reviewed the impact that need-based grants have on student success, enrollment, persistence, and degree completion. Need-Based Award Programs Clotfelter, Hemelt, and Ladd (2018) studied an institutional aid program for low-income students initially focused on need-based aid that evolved to include other nonfinancial supports. The Carolina Covenant Program began in the fall of 2003 at the University of North Carolina and covered 100% of unmet need for students from low-income families. Beginning in 2007, the Carolina Covenant Program added nonfinancial supports such as faculty and peer mentoring, professional development opportunities, and summer support for students struggling academically. From 2007 to 2010 the students who received need-based aid and nonfinancial support were more likely to earn 30 credits in their first year (and more likely to earn 60 credits by the end of their second year) by seven percentage points when compared to a control group. The 2003 to 2006 cohort only received need-based aid and showed no differences when compared to a control group. While later cohorts of the Carolina Covenant Program earned higher grade point averages and were more likely to graduate on time (four years), overall the results were not conclusive (p. 279). Kane (2007) reported District of Columbia residents were eligible for grants to cover the difference between in-state and out-of-state tuition at public universities. The author found the D.C. Tuition Assistance Grant Program significantly increased enrollment rates for students who were eligible for the program. Between 1998 and 2000 the number of D.C. residents attending public institutions in Maryland and Virginia more than doubled, and between 2000 and 2002 colleges outside Maryland and Virginia were allowed which resulted in another two-fold increase of students enrolling in public institutions. Similarly, Wilbert van der Klaauw (2002) discovered that for each $1,000 in aid (2001 dollars) the likelihood of students attending a major Eastern university increased by three to four percentage points. These results were similar to findings by Dynarski (2003) who studied the impacts of the Social Security Student Benefit Program in 1982. Under this program, students of deceased, disabled, or retired Social Security beneficiaries were able to receive benefits if enrolled in college full time. Average annual payments in 1980 totaled $6,700 and accounted for 12% of full-time college students from 18 to 21 years old. After the program was cut, college attendance probabilities dropped by over one-third. Dynarski (2003) discovered the probability of attending college increased by 3.6% for $1,000 in grant aid and this additional money also appeared to increase school completion. Another study conducted by Castleman and Long (2016) examined completion rates for students at a Florida public university and found that need-based grants of $1,300 (in 2000 dollars) increased enrollment at a public four-year university by 3.2 percentage points and also increased persistence through the spring semester of the freshman year by 4.3 percentage points. Moreover, the $1,300 grant increased graduation rates by almost five percentage points, or 22% (p. 1026). A separate study by Heller (2003) discovered that first-year college students who received a $1,200 need-based grant were six percentage points more likely to persist than non- recipients. Even students who received non-need awards increased persistence by six percentage points until completion of the academic year during 2000-2001. Heller (2003) stated, “Awarding of institutional aid early in a 4VOLUME 28 NUMBER 1 student’s career was an important predictor of much later persistence or attainment of a credential (certificate, associate’s degree, or bachelor’s degree).” Heller’s (2003) findings are similar to those found by Singell (2004) where the author found that students are more likely to graduate the higher their levels of need-based awards are in the first year. Goldrick-Rab et al. (2013) examined the impacts of need-based grants on college persistence. The Wisconsin Scholars Grant, which included a renewable grant (up to five years) of $3,500, was distributed to eligible first- year undergraduates at 13 public universities across the state. The authors studied four different cohorts and discovered suggestive findings that the grants increased completion of a full-time credit load as well as retention rates for a second year of college. An increase of $1,000 in total aid increased second year enrollment rates by 2.8 to 4.1 percentage points. Similarly, Bettinger (2015) reviewed the impacts of the Ohio College Opportunity Grant during the 2006-2007 academic school year in Ohio. Changes to this financial aid award increased need- based grants by 60% for some students. After one year, students who received more aid after changes were implemented were less likely to drop out of college or transfer from their original institution. Overall, there was a 2% decrease in dropout rates for students who received higher financial awards. Additionally, students who benefited from the new policy were more likely to attend a four-year institution and had higher grade point averages after one year (Bettinger, 2015). Castleman et al., (2018) conducted a study examining the impacts of need-based aid on STEM attainment using a quasi-experimental research design and expected family contribution (EFC). Expected family contribution is an index number that colleges use to determine how much financial aid a student is eligible to receive. The EFC factors in a family’s taxed and untaxed income, assets, and other benefits such as Social Security. Family size and the number of family members who attend college during the year are also considered to determine award amounts (Sallie Mae, 2020). This research focused on the implications of receiving the need-based Florida Student Assistant Grant (FSAG) and STEM credit attainment and degree completion. The authors found positive effects of need-based aid on STEM credit attainment between 3.7 and 7.3 credits after seven years (among college-ready STEM students). This STEM credit attainment represents gains of 20% and 34.5% respectively when compared to students who were just above the need-based aid cutoff. Conversely, there was not a definite correlation between receiving the FSAG and STEM degree attainment, but study results did show students who received the FSAG increased STEM degree attainment by three percentage points. This increase represented gains of 50-60% relative to students just above the EFC threshold (Castleman et al., 2018, p. 138). Merit-Based Award Programs There are several examples of merit award programs, and Georgia’s Helping Outstanding Pupils Educationally (HOPE) scholarship is perhaps one of the most well-known and often studied programs. The HOPE scholarship is entirely merit based and does not require a student to pay back money (Campbell & Finney, 2005). For students that earn a 3.0 grade point average in high school, the HOPE scholarship reduces the price of college by providing annual aid to cover tuition and fees for Georgia high school graduates at any public college or university. Students who choose to attend a private school receive an annual grant that varies depending on the institution. Additionally, college students must earn a 3.0 grade point average at the 30, 60, and 90 credit hour marks to remain eligible for the HOPE scholarship (Campbell & Finney, 2005). Dynarski (2004) studied the impacts of the HOPE scholarship and found the merit-based award program significantly increased college attendance by 8.6% compared to other Southern states that did not award merit-based scholarships. The HOPE THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION5 scholarship caused moderate increases in college attendance and shifted students from two-year schools toward four-year schools. The data also suggest that Georgia’s merit aid program has increased the probability that a student will attend college in his or her home state (p. 89). While there are benefits to merit-based award programs there are also consequences. Negative impacts of the HOPE scholarship and other merit-based aid programs were presented by the Harvard Civil Rights Project that found merit-based aid overly assists White and Asian American students resulting in widened racial differences in college attendance rates (Dynarski, 2004; Henry, Rubenstein, & Bugler, 2004). Two important factors which help explain the widened racial differences are standardized test and grade point average requirements. Dynarski (2004) found that both standardized test score and grade point average requirements have negative impacts on African American eligibility. For example, 31% of Michigan students in 2000 scored high enough on standardized tests and grade point averages to warrant an award, but only 7.9% were African American. Similarly, merit-based award programs with only a grade point average requirement will see racial differences in eligibility as well. Forty percent of high school seniors have a 3.0 grade point average or higher; however, only 15% of African Americans and Hispanics will meet this benchmark. Dynarski (2004) states, “Blacks, Hispanics, and low-income youths are relatively unlikely to attend college, so any subsidy to college students will flow disproportionately to white, upper-income youth. Even among nonwhite, Hispanic, and low- income youths who do attend college, academic performance is a barrier to merit aid eligibility (p. 68). Henry et al. (2004) conducted a study comparing HOPE scholarship recipients who barely qualified for the merit-based award and those who barely missed qualifying for the award. There was a significant difference between HOPE recipients and nonrecipients at four-year institutions with the predicted odds of persistence 13% greater for recipients than nonrecipients. However, students who lost the scholarship were not more likely to persist after four years than nonrecipients. There was no significant difference between the two groups at two- year institutions. Another interesting finding from this study is the number of students who retained the merit- based award for the entire four years of college. Eighty-five percent of students who qualified to receive HOPE lost the scholarship after completing 30 credit hours while only 3.5% of students kept the scholarship for four years (p. 700). The Case for Academic Progress The federal Pell grant program has evolved from a grant awarded to low-income students with “no strings attached” to a program that requires satisfactory academic progress (SAP). Schudde and Scott-Clayton (2016) pointed out that college student enrollment and student body diversity, mainly in terms of academic preparedness, have increased since the SAP policy was first enacted. In 1976, the term satisfactory academic progress was inserted to the Higher Education Act of 1965 legislation without having a clearly defined meaning. However, the amendment did articulate the need for “satisfactory progress” towards a degree in order to receive financial assistance under Title IV (Bennett & Grothe, 1982). In 1978, the Interim Final Regulations stated that institutions should evaluate academic progress by establishing a “normal time frame” to complete course requirements and by measuring grades “against a norm” (Bennett & Grothe, 1982). Over the years, the requirements for SAP have been clarified while allowing institutions some leeway on how best to measure academic progress. Federal regulations currently mandate that a school’s SAP policy must be in writing and consist of qualitative and quantitative measures. By the end of the student’s second year they must possess a “C average or its equivalent” or the student must have a satisfactory academic standing that meets requirements to graduate. 6VOLUME 28 NUMBER 1 Additionally, schools must check satisfactory academic progress at least once each academic year or by the midpoint of the academic program if less than one year (U.S. Department of Education). Quantitatively, school policy must indicate a maximum time frame where students are expected to complete their academic programs. The maximum completion time allowed by federal policy is 150% of the length of time required to complete the program. This time is measured by academic years, terms, credit hours attempted, or clock hours completed. Schools may set a more restrictive program completion policy, but the policy cannot allow more than 150% than the time it should take to complete (U.S. Department of Education). While the federal Pell grant program provides aid to low-income students, many of these students may not be academically prepared to succeed in college. The lack of college readiness combined with first-generation student status, transition challenges, and other obstacles can result in a student who matriculates but does not persist to completion. Schudde and Scott-Clayton (2016) conducted a study on the impact of holding students to a standard for academic achievement in a large state community college. They examined national trends in SAP grade point average failure during the first year of college across institution types and Pell grant status. The authors discovered that approximately 21% of first-year Pell recipients were at risk of losing their grant due to the SAP grade point average requirement in 2012. Approximately 25% of first-year Pell recipients in their sample did not meet the 2.0 grade point average, which was in line with estimates from national data. Furthermore, the first-year SAP failure rate was almost 40% after accounting for credit completion requirements. The regression-discontinuity method did not yield statistically significant results while the difference-in-difference model estimates that failing to meet SAP grade point averages discourages persistence after one year in college (Schudde & Scott-Clayton, 2016). Although this study obtained mixed results, the findings are conclusive in that SAP failure was most prevalent in the first term of enrollment (40%), followed by a 16% SAP failure rate by the fall of the second year, and by the third year the rate was approximately 8%. The steadily decreasing SAP failure rate was not due to increased grade point averages (some students did raise their grade point average), but mainly because many students did not reenroll. Requirements for satisfactory academic progress seem to negatively affect student persistence in both merit-based award programs and Pell grant programs although the intent of the financial aid is to assist students in completing postsecondary education. If the amount of financial aid a student expects to receive is reduced it may lower the maximum tuition price that students are willing to pay, which could lead the student to transfer to a cheaper school or drop out (Bettinger, 2004). Methodology Experimental Design The Student Talent Enrichment Program (STEP) was an experimental scholarship program meant to increase retention of students with high unmet financial need (i.e., cost of attendance minus free aid and estimated family contribution). In fall 2017, a major research university awarded $650,000 in STEP grants. Grants were offered to roughly half of the eligible freshmen who, based on their amount of unmet financial need, were projected as most likely to benefit. The university defined as a control group the remaining eligible freshmen most needing aid. STEP grants reduced recipients’ unmet financial need for the freshman year but did not offset recipients’ need for subsequent years. Full-time, first-year, degree-seeking students at the university were eligible for STEP only if they were in-state residents, had a high school grade point average meeting the university’s general admission requirement, were not NCAA athletes, and, according to a FAFSA completed by early October, had $5,000 or more in unmet financial THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION7 need (for the first term). These criteria delimited 773 eligible students from a freshman cohort of over 5,000. The freshmen eligible for STEP’s grant component were stratified into pairs by their scores on a measure that predicted probability of one-year retention from factors including high school academic achievement, freshman financial situation, housing type, and date of university enrollment. Candidacy for a treatment group was then randomly assigned to one member of each pair, while candidacy for a control group was assigned to the other. Next, the university determined how various possible grant amounts would change each student’s predicted probability of retention, with probabilities in this case predicted by a model based only on amount of unmet need and amount owed to the university. First for the treatment group and again for the control group, an imaginary $650k was then allocated in $2.5k increments to those 150-plus candidates for the group and in those amounts (up to a maximum of $10k per recipient) that most improved the group’s sum of predicted probabilities of retention (i.e., that most improved the full freshman cohort’s predicted retention rate). The resulting treatment group of 151 students was awarded grants in the amounts specified by this process and compared to a control group of 162 students selected via the same process. Limitations. Two aspects of STEP’s design limit quality of program analysis and confidence of conclusions. First, student selection and aid distribution were ultimately determined by a process intended to maximize change to students’ predicted probabilities of retention, rather than by random assignment. Because defined by eligibility criteria imposed at the selection process’ initial stage and by the stratified randomization undertaken at the selection process’ penultimate stage, treatment and control groups differed little in terms of characteristics (e.g. high school academic records, amounts of unmet financial need, and financial holds) that predict retention outcomes. However, because the treatment and control groups were ultimately defined by changes to treatment and control candidates’ projected probabilities of retention, rather than solely through random assignment, the groups do differ somewhat on those incoming student characteristics that are predictive of retention. That is, the treatment and control groups are less similar than would have been the case if random selection were the selection process’ final stage and differ thus because the university prioritized the attempt to optimize grants’ apparent retention impact over randomization. Second, the university did not exclude as ineligible those students who, before grants were awarded, were already in academically tenuous or non-viable situations. First-year early progress grades were already available when the selection process was run and many (16%) of the students selected for treatment or control groups were already unlikely to persist due to non-viably low (<1.5) early progress grade point averages. Analysis of outcomes suggests that better retention results and more confident conclusions would have been possible had STEP excluded these students. Instead, no measures of university academic progress were considered in the selection process. Results At first glance, STEP grant recipients have been retained at rates only slightly higher than those of the control group. Recipients were retained to spring 2018 at a rate 3.5 percentage points higher than that of the control group (88.1% grant vs. 84.6% control) and retained to fall 2018 at a rate 1.2 points higher than the control group’s (72.8% vs. 71.6%). Neither difference in outcomes is statistically significant. The control group began the year somewhat better prepared than the treatment group. According to the strongest available predictor of retention from fall to spring, the predicted one-term retention rate of the control group was 83.4%, while that of the treatment group (absent any grants) would have been 82.7%. The treatment group thus beat its predicted one-term retention rate by 5.3 percentage points (88.1% - 82.7%), and this margin 8VOLUME 28 NUMBER 1 is 4.1 points larger than the 1.2 points (84.6% - 83.4%) by which the control group beat its predicted rate (Figure 1). Even after adjusting for those incoming student characteristics that best predict one-term retention, the difference between the treatment and control groups’ overall one-term retention outcomes is not statistically significant. However, the treatment group’s gains relative to predicted rates and relative to the control group are not equally distributed across measures predictive of retention. Consider first the treatment group’s actual performance by predicted first-term retention rate. Among the few students whose projected probability of one- term retention, absent STEP grants, was below 80%, grant recipients actually underperformed slightly relative to their control group counterparts and relative to predicted rates. However, among those majority subsets of the treatment and control groups with 80% or better projected probabilities of one-term retention, treatment group members outperformed both predicted values and their control group counterparts by substantial margins. The grant recipients predicted to have an 80% or better predicted probability of one-year retention absent grants were actually retained at a rate 8.5 percentage points higher than projected (94.6% actual vs. 86.1% predicted). In contrast, the control group students in this range were actually retained to the spring at a rate (87.3%) within one percentage point of their predicted rate (86.6%). More remarkable is the treatment group’s performance by high school GPA. Consider the 73% of STEP study participants with high school GPAs of at least 3.3. Among these students, the grant recipients’ one-term retention rate exceeded that predicted from incoming student characteristics by 10.3 percentage points (95.32% actually retained one term vs. 85.07% predicted to be retained one term if no STEP grants had been awarded; Figure 5), and that of their control group counterparts by 11.7 percentage points (95.3% treatment vs. 83.6% control; Figure 1). Figure 1 One-Term Retention by HS GPA STEP Grant Treatment vs. Control. Note. Students who received STEP Grants and had a HS GPA over 3.3 outperformed students with a HS GPA over 3.3 but who did not receive STEP Grants by over 11 percentage points. 1 Because the treatment and control groups were defined by changes to treatment and control candidates’ projected probabilities of retention, rather than solely through random assignment, the groups differ somewhat on incoming student characteristics, such as high school GPA, that are predictive of retention. THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION9 This difference between members of the treatment and control groups with high school GPAs of at least 3.3 is statistically significant (p < .01) before and after considering covariates. Among members of the STEP study in this high school GPA range, no other factor or set of factors examined, including predicted first-term retention rate, is as strong a predictor of one-term retention as the STEP grant treatment. Nor were STEP grant recipients with high school GPAs of at least 3.3 significantly better prepared than their control group counterparts in terms of incoming student characteristics known to predict one-term retention. Upon further investigation, we found that outcomes differ more substantially among the majority of students who were not in academic jeopardy when grants were offered. Among the more than four-fifths of study participants who had earned early progress GPAs of at least 1.5, grant recipients were retained one term at a rate 7.7 percentage points higher than that of control group members (95.2% grant vs. 87.5% control). Likewise, among the 73% of the studied group with high school GPAs of at least 3.3, grant recipients were retained one term at a rate 11.7 points higher than that of their control group counterparts (95.3% grant vs. 83.6% control). Most striking, among the 65.2% of the studied group with early progress GPAs of 1.5 or better and high school GPAs of 3.3 or better, grant recipients had a nearly perfect (99%) one-term retention rate, 13.8 points higher than that of control group students (Figure 2). These differences register as statistically significant before and after taking covariates into account. Figure 2 Retention by Academic Preparation and Whether STEP Grant Recipient Note. STEP Grant recipients who had early progress GPAs of 1.5 or better and HS GPAs of 3.3 or better had a 99% one-term retention rate. 2 On the contrary, the treatment group members in this high school GPA range began their first term with slight disadvantages relative to their control group on some important measures of preparation. As compared to the 122 members of the control group with high school GPAs of at least 3.3, the 107 members of the treatment group in the same high school GPA range are more likely to be first-generation college students (53.3% vs. 46.7%) and were more likely to have a financial hold as of early October (47.7% vs. 41%). Moreover, the two groups earned similar mean high school GPAs (3.964 treatment vs. 3.965 control), had similar amounts of pre-STEP Grant unmet need ($7,887 treatment vs. $7,908 control), and began their first term living on campus at similar rates (82.2% treatment vs. 81.1% control). 10VOLUME 28 NUMBER 1 Among students with early progress GPAs of at least 1.5, grant recipients were even retained to the second fall, for which no STEP grant funding was available, at a rate 6.9 points higher than that of their control group counterparts (83.3% grant vs. 76.5% control). Figures 8 and 9 show how STEP grants’ apparent impact on retention varies by early progress GPA. Among the students with early progress GPAs below 1.5, grant recipients have been retained at rates well below those of their control group counterparts. But among the 83.7% of study participants with early progress GPAs of at least 1.5, grant recipients in almost every early progress GPA range were retained one term (Figure 8) and one year (Figure 9) at rates above those of their control group counterparts. Figure 3 One-Term Retention by Early Progress GPA and Whether STEP Grant Note. One-term retention rates for STEP Grant recipients with early progress GPAs above 1.5 out-performed their control group counterparts. Figure 4 One-Year Retention by Early Progress GPA and Whether STEP Grant Note. One-year retention rates for STEP Grant recipients with early progress GPAs above 1.5 out-performed their control group counterparts. STEP grant recipients with early progress GPAs of at least 1.5 were not more likely to persist than similarly progressing control group students before grants were awarded. On the contrary, grant recipients in this early progress GPA range began their first term with slight disadvantages relative to their control group counterparts on some important measures of preparation. Figures 5 and 6 show by what margins STEP Grant recipients have outperformed control group students with similar pre-grant probabilities of persistence, in comparisons that THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION11 exclude the few students who earned early progress GPAs below 1.5. Figure 5 One-Term Retention by Predicted Probability of One-Year Retention and Whether STEP Grant – Excluding Students with Early Progress GPAs Below 1.5 Note. Pre-Grant predicted probability of one-year retention given HS academic achievement, unmet financial need, financial hold, housing type, and enrollment date. Figure 6 One-Year Retention by Predicted Probability of One-Year Retention and Whether STEP Grant - Excluding Students with Early Progress GPAs Below 1.5 Note. Pre-Grant predicted probability of one-year retention given HS academic achievement, unmet financial need, financial hold, housing type, and enrollment date. Under the assumption that STEP grants caused these differences in retention outcomes, 9.7 recipients enrolled the first spring and 8.6 recipients enrolled the second fall (out of 126 recipients who earned early progress GPAs of at least 1.5) would not have so enrolled without a grant. Discussion For research questions one and two: what impact does one-time, variable amount funding have on freshmen with demonstrated financial need; and does one-time, variable amount funding have a significant impact on one- year retention rates? We found that one-time, variable amount funding does have a positive impact on retention for those students who are academically prepared who enter college as first-time, full-time freshmen. By looking 12VOLUME 28 NUMBER 1 at the treatment and control groups with greater specificity on academic preparedness and progress we found a statistically significant relationship between STEP grant funding and rate of retention for students that had high school grade point averages of 3.3 or higher and that were not in academic jeopardy when grants were offered. Our findings are different than those found in a previous STEP grant program study by Nichols (2018) who found no statistically distinguishable effects in semester to semester enrollment, year to year enrollment, spring 2018 grade point average, spring 2018 credits completed, or likelihood of good academic standing after the spring 2018 semester. Nichols (2018) looked at all students who were eligible for the STEP grant program, including many not delimited within the study’s treatment or control groups, and reported no analysis differentiating students who were in academically non-viable situations when grants were offered, while we found that grants were associated with retention among students that had high school grade point averages above 3.29 or among students who were academically viable when STEP began. Research question three asked, what role, if any, does academic preparation play when considering need-based funding? We found that academic preparation does effect retention when considering need-based funding. Grant recipients who had an 80% or better predicted probability of one-year retention were actually retained (94.6%) at a rate 8.5 percentage points higher than projected (86.1%), whereas the control group in the same range were actually (87.3%) retained at a rate within one percentage point of the predicted rate (86.6%). Likewise, we found that grant recipients who had viable (1.5 or better) early progress GPAs outperformed comparable control group students by 7.7 points on one-term retention and by 6.9 points on one-year retention. Overall, our retention rate findings thus resemble those of Castleman and Long (2016), which found that need-based grants of $1,300 increased persistence through the spring semester of the freshman year by 4.3 percentage points. Our findings also parallel findings by Heller (2003) which discovered that first-year college students who received a $1,200 need-based grant were six percentage points more likely to persist than non- recipients. Goldrick-Rab et al. (2013) found that $1,000 grants of need-based aid increased college persistence slightly (one percentage point) but not significantly. Our preliminary finding shows a positive correlation between need-based aid packages and retention when students are academically viable when aid is offered. Conclusion We examined the impact of need-based scholarships on the retention of first-time, full-time freshmen. A major finding in this study is the impact of need-based aid on students who had academically viable GPAs of 1.5 or higher. We discovered that, excepting the relatively few students who lacked academically viable GPAs when awards were offered, need-based aid had a statistically significant association with retention among the remaining majority subset of students. In the current environment of shrinking college allocations for need-based aid and increasing tuition rates the implications from this study can help administrators determine the students who will not only benefit from need-based aid but stand to gain the most in terms of persistence in college. Conclusions from the STEP grants study call for additional research on two fronts. First, for validation of the preliminary finding that STEP grants improved one-term retention outcomes among students who were making adequate academic progress, another experiment could exclude students with inadequate GPAs from its treatment and control groups. More broadly, given the STEP grants study’s finding that grants had significant associations with retention just over the period that the aid covered, further research is invited to determine differences between one-time financial awards and need-based aid provided over the course of a student’s college career. THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION13 References Alsalam, N., & Carrington, W. (2018). Federal aid for postsecondary students. The Congressional Budget Office. Washington, D.C. Baum, S.R., Payea, K. (2013). Trends in student aid, 2013. New York, NY: The College Board. Baum, S. R., Elliott, D. C., & Ma, J. (2014). Trends in student aid, 2014. New York, NY: The College Board. Bennett, W., & Grothe, B. (1982). Implementation of an academic progress policy at a public urban university: A review after four years. Journal of Student Financial Aid, 12 (1), 33-39. Bettinger, E. (2004). How financial aid affects persistence. National Bureau of Economic Research. NBER Working Paper No. 10242. Bettinger, E. (2015). Need-based aid and college persistence: The effects of the Ohio college opportunity grant. Educational Evaluation and Policy Analysis 37(1S), 102S-119S. Campbell, N., & Finney, R. Z. (2005). Mitigating the combined distributional consequences of the Georgia lottery for education and the hope scholarship. Social Science Quarterly, (86)3, 746-758. Castleman, B. L., & Long, B. T., (2016). Looking beyond enrollment: The causal effect of need-based grants on college access, persistence, and graduation. Journal of Labor Economics, 34(4). Castleman, B. L., Long, B. T., & Mabel, Z. (2018). Can financial aid help to address the growing need for STEM education? The effects of need-based grants on the completion of science, technology, engineering, and math courses and degrees. Journal of Policy Analysis and Mangagement, 37(1), 136-166. Clotfelter, C.T., Hemelt, S.W. & Ladd, H.F. (2018), Multifaceted aid for low-income students and college outcomes: Evidence from North Carolina. Economic Inquiry, 56, 278-303. Dynarski, S. M. (2003). Does aid matter? Measuring the effect of student aid on college attendance and completion. American Economic Review, 93(1), 279-288. Dynarski, S. M. (2004). “The New Merit Aid.” In College Choices: The Economics of Where to Go, When to Go and How to Pay for It. Ed. Caroline Hoxby, 63-100, Chicago: University of Chicago Press. Expected Family Contribution. (2020, September 20). Sallie Mae. Retrieved April 2, 2021, from https://www. salliemae.com/college-planning/financial-aid/fafsa/expected-family-contribution/ Goldrick-Rab, S., Harris, D. N., Kelchen, R., & Benson, J. (2013). Need-based financial aid and college persistence: Experimental evidence from Wisconsin. SSRN Electronic Journal. 10.2139/ssrn.1887826. Heller, D. E. (2003). Informing public policy: Financial aid and student persistence. Western Interstate Commission for Higher Education. Retrieved from http://www.wiche.edu/info/publications/InformingPublicPolicy.pdf. Henry, G. T., Rubenstein, R., & Bugler, D. T. (2004). Is HOPE enough? Impacts of receiving and losing merit-based financial aid. Educational Policy (18)5, pp. 686-709. Kane, T. J., (2007). Evaluating the impact of the D.C. tuition assistance grant program. Journal of Human Resources, 42(3) pp. 555–582, http://jhr.uwpress.org/content/XLII/3/555.short. McCants, J. (2003). The early history of the U.S. higher education act of 1965. National Trio Clearinghouse. Washington, D.C. Mettler, S. (2014). How U.S. higher education promotes inequality-and what can be done to broaden access and graduation. Cambridge, MA: Harvard University, Scholars Strategy Network. 14VOLUME 28 NUMBER 1 Nichols, M. A. (2018). Mindset, mentor, and money: How each influences college success. (Publication No. ED597030 [Doctoral dissertation, University of Arkansas]. ProQuest Dissertations Publishing. Page, L. C. & Scott-Clayton J. (2016). Improving college access in the United States: Barriers and policy responses. Economics of Education Review, (51) pp. 4-22. Schudde, L. & Scott-Clayton J. (2016). Pell grants as performance-based scholarships? An examination of satisfactory academic progress requirements in the nation’s largest need-based aid program. Research in Higher Education, (57) pp. 943-967. Singell, L. D. (2004). Come and Stay a While: Does Financial Aid Effect Enrollment and Retention at a Large Public University?” Economics of Education Review, 23(5): 459-71. Stinebrickner, R. & Stinebrickner, T. (2008). The effect of credit constraints on the college dropout decision: A direct approach using a new panel study. American Economic Review, (98) pp. 2163-2184. U.S. Department of Education (2019). Ch. 1 School-Determined Requirements in Federal Student Aid. Washington, D. C. Van der Klaauw, W. (2002). Estimating the effect of financial aid offers on college enrollment: A regression- discontinuity approach. International Economic Review 43(4).