THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION1 FEATURED ARTICLE VALIDATING Pathways to Student Success and Retention: Viable and Accessible Links of Investment and Delivery through Active and Timely Interventions to Navigate to Graduation Paul J. Croft, Kean University Neva Lozada, Kean University Retention interventions focus on support programs, measures of success, and demographics. This study offers an alternate student-based narrative relative to student college preparation and self-diagnosed functionalities and compatibilities. We examine metrics that portray economic and environmental settings and social frameworks – as well as interactions, behaviors, and responses – that comprise first-year students’ experiences for the spectrum of engagement behaviors. We analyze the return on investment for student retention efforts at a four-year public university by applying a predictive and proactive retention model. We also discuss ensuring accessibility to services, diagnosing student intervention, and considering subsequent engagement behaviors. Attempting to understand the impact of retention intervention efforts in higher education is often made difficult by student responses and behaviors that don’t line up neatly. In fact, the idea of engagement – and the expected outcomes – are often disparate as students seek meaningful returns on their investments of time in support services whereas those units providing the same are delivered in a programmatic and systematic manner. This situation is further complicated by attributes of individual students and groupings of students, such as those in their first year of college. These complexities raise the question of whether it is possible or desirable to fully analyze or predict student retention given the multitude of contradicting and confounding variables (Croft & Lozada, 2020). In spite of these challenges, this study presents the creation of an operational predictive retention model. This model relies upon a diagnostic operational platform of metrics – both quantitative and qualitative – that relate to underlying cognitive and non-cognitive factors. Through the analysis of demographic, behavioral, and basic student data, we develop a model construct according to specific workflows that consider both student success (retention to graduation) and assessments of interventions. With this model, a proactive rather than reactive strategy may be used, modified at any time, and applied to any student or group of students. In the process, it is possible to more closely examine metrics that may drive purposeful interventions according to an institution’s desired returns on investment – while also accounting for students’ circumstances. Considering the multitude of metrics (economic, social, and environmental) that tie to student retention and success, we find that the underlying processes and systems involved here are comparable to aspects of sustainability science and its application. In the practice of sustainability science, the outcomes of actions (or inactions) are considered collectively according to the often competing interests of three key systems: economic, social, and environmental. This means that each system’s characteristics and processes are considered in terms of their own autoregressive responses, their feedback, and their interactions with the other systems. 2VOLUME 28 NUMBER 1 In the case of student retention and success, we consider a student’s (or cohort’s) situation financially (economic), how the student engages with academic support services (social), and their interactions within the context of university services and resources (the campus environment). For individuals and groups, these fundamental structures will evolve during a semester, and are used to create each student’s frameworks and behaviors. These behaviors are linked to students’ self-determination and level of success. It is within these structures where students learn and adapt new behaviors according to specific interactions and experiences. These frameworks also depend on students’ own competing interests (e.g., working, commuting, caring for a family member) which define each student’s openness to support and evaluation of its effectiveness. Thus for any student the underlying support structures – and that student’s responses, interactions, and feedback to the same – drive accessibility to support services and retention programs. Indeed, the student’s response to support structures is ultimately responsible for the success or failure of interventions regardless of the administrative costs borne by an institution. For this reason another element in the practice of sustainability science is the direct consideration of competing interests to assess whether interventions are equitable (economic vs. social), viable (environment vs. economic), and bearable (social vs. environment). From a student perspective, one must consider whether academic support is equitable (cost-effective and compatible with their behaviors), viable (offered in appropriate ways and when/where they need assistance), and bearable (e.g., services are inviting, comfortable, and not too time-consuming). PRIME: An Operationally-Oriented Workflow and Outcomes Model In terms of retention, integrated and overarching community engagement clearly makes a difference and necessitates tracking of student cohorts to better understand the specific combinations of metrics that contribute to student engagement and retention behaviors. Our development of a Proactive Retention Through Integrated Modeling of Engagement (PRIME) model identifies multiple measures and retention variables involved in data collection and analysis, including anticipated impacts on specific workflows and student support services. Through this model, we make use of the entire landscape of community engagements to make strides toward redesigning retention strategies from reactive to proactive, conceptualizing interdependencies of retention intervention metrics and student behaviors, and identifying the actions needed to achieve them in practice through real-time collaborative campus partnerships. In our PRIME model approach (see Figure 1), we rely upon multiple measures and variables connected with retention to link student success to more than the obvious student demographic information. The model also provides for the underlying framework of systems and competing processes found in sustainability science. When depicting this entire landscape from a student point of view – situationally rather than using administrative or programmatic foci – it is possible to examine student retention behaviors, identify relevant metrics, and consider how equitable, viable, and bearable intervention strategies are from their perspective. Given the wide diversity of student needs and circumstances encountered in today’s student populations, the PRIME model has made significant strides toward achieving proactive retention through integrated modeling of engagement as applied operationally. The PRIME model expresses student retention behaviors within a phased sequence approach. The approach: 1) makes use of statistical information and student data to provide specific empirical foundations known to be relevant to retention and supported by the institutional data and metadata; 2) identifies what is happening on a finer scale (in real time) and how various actions and reactions comprise student success and retention and associated behaviors; THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION3 3) represents the interactive and autoregressive aspects of retention (i.e. how does a student’s behavior change with time with or without intervention) and the metrics often used to quantify student success; 4) captures the students’ own self-reported experience during their first year at college; and 5) drives individual and group interventions at any time independent of student body, semester, time within a semester, or as a function of both prior and ongoing student support services and outcomes or experiences. Together these key performance indicators provide valuable insight when defining overall student success. Figure 1 PRIME Model Note. At the outset, the PRIME model presents the question of whether students thrive in or feel failed by the support environment, are secure financially or pre-destined to exit their studies, and whether the interface of the societal retention framework is conducive to their retention (i.e. does it fit and meet their expectations). The interplay of each of these define how viable, equitable, and bearable support and retention interventions are from the student perspective -- and thus ultimately how successful. In the application of the PRIME model, the particularistic aligns with student-focused interventions and engagements that must be viable, equitable, and bearable for students. Demographics and student evolution through participation in the process in real-time and the resulting outcomes (e.g., grades, retention) determine these interventions and behaviors. The descriptive nature of this study explores interactions between and among the wide variety of data collected within the environmental, economic, and societal landscape. The heuristic products of this study link to how workflows and actions provide specific insights into the success of students. Purpose of Study This paper fills a void in the current research by outlining specific steps taken in moving from concept to practice through proactive and predictive retention modeling. The big picture of retention connects the broader definition of student success to specific metrics that are quantitative, qualitative, cognitive, non-cognitive, and interdependent. The approach examines metrics that portray economic and environmental settings and social 4VOLUME 28 NUMBER 1 frameworks that comprise first-year students’ experiences for two cohorts. These metrics identify the spectrum of engagement behaviors and illustrate the application of a predictive and proactive retention model that fosters integrated investments and engagements for student success. The framework allows analysis of the university’s return on investment and attendant workflows involving student retention. Thus the guiding questions of the current study explore the perceived impact of retention interventions and whether these interventions foster greater student investments in their own persistence and academic performance. Framed from a sustainability science point of view, these questions seek to understand whether interventions made a quantitative difference. In other words, we seek to offer more than circumstantial evidence that retention and support services and programs make a difference for student success when guided by modeling that is robust enough to allow corrective actions and real-time evolution of services and interventions during a semester, from term to term, and across or between student cohorts. The unit of analysis here is students from the fall 2018 and fall 2019 entering cohorts at a four-year public institution located in the Mid-Atlantic region of the United States. This includes students both retained and not retained after their first year, in an effort to ensure that participant responses satisfactorily represent the range of behaviors that correlate with both retention and attrition. Data collection methods chosen for the current study include surveys, semi-structured interviewing and focus groups in addition to extensive demographic and background (i.e. baseline) information and data. The following research questions guide this study: 1. Have retention interventions, events, and engagements changed behaviors of students, as evidenced by their academic performance and self-reporting and associated metrics? 2. Have those same intentional behaviors become embedded as a series of thought-change processes such that students take ownership of their academic performance as an investment in their own retention as evidenced by collected data? It is through these two questions the lens of this study and PRIME model are employed. This is similar to how the fundamental elements of sustainability science (i.e. environmental, economic, and societal aspects) are considered according to their own metrics, interactions with one another, and self-evolving processes. In the present case, we may consider the university services and support systems as the environment, student success/ retention as the economic key, and the embedding of behaviors as societal response. Our two research questions focus on the interplay among workflows that deliver proactive student support, their reception by students, and how they become ingrained in behaviors that ultimately lead to retention or attrition. To understand how these elements commingle and impact the ultimate success of a student (due to interventions, engagements, and intentional behaviors) requires examination of interactive relationships. This can be considered with regard to sustainability principles and according to how viable, equitable, and bearable the proactive interventions are from a student perspective. In this study, viability is measured according to accessibility and timeliness of services for retention given the environmental context of student support and student economy such as financial aid or hardship. Equity is measured as the distribution of support and modalities linked with student cohorts (and individual students) and their behaviors. And tolerability is measured according to whether responsive behaviors are in-sync with the support and services provided, which is an indication of whether the environment and context of support are compatible with needs and expectations of students. Based on prior research, we anticipate that higher levels of institutional engagement during a student’s first year in college positively correlate with a higher likelihood of being retained the following year (Kuh, THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION5 2001, 2003; Tinto, 1975, 1993, 2012). We are also interested in how additional factors may contribute to an individual’s likelihood to be retained as we work toward building a predictive model in an effort to provide students more personalized support and increased access to resources that meet their specific needs as they complete their college degree program over time. The multitude of intervention types and their frequencies, those directed at variously sized audiences versus one-on-one experiences, and the student demographics involved reflect both the broader community’s roles in retention as well as the large variance found in student responses and among outcomes. Together these shape the metrics that delineate student success. These metrics are modified with specific and measurable engagements and interventions towards instilling retention behaviors. In turn, this information can lead to subsequent changes in administration, institutional process, and workflows. Literature Review Early attrition models in the field of higher education identified factors that inhibited the academic potential of students with a specific focus on social engagement and sense of belonging. Tinto’s (1975) foundational Student Integration Model and Bean and Metzner’s (1985) Student Attrition Model (from which many other models have emerged) explored characteristics and predispositions of dropout behaviors, suggesting that pre- enrollment motivations should be considered in studying college persistence. Such pre-enrollment factors have been at the forefront of more recent retention models (Kerby, 2015; Noel et al., 1985; Singell & Waddell, 2010) including predictive models (Davis et al., 2019) as seen in the current study. However, previous works in student persistence have yet to venture beyond identification of characteristics and toward studying changed behavior over time that may influence the complex system of student retention (Forsman et al., 2015). Moving beyond pre-enrollment characteristics, later research examined behaviors of students, mostly first-year students, as they began to navigate college. These studies sought to determine which behaviors may lead to retention or attrition in the subsequent year. Such behaviors included a sense of belonging (Hurtado & Carter, 1997) to the campus as a whole, or to specific institutional constructs (Hoffman et al., 2002; Lee & Davis, 2000). These studies concurred that the combination of academic and social experiences was a better predictor of student persistence in college than when either factor was studied on its own. Davis et al. (2019) suggest that a comprehensive, focused outreach protocol that encompasses both social and academic factors could have a positive impact on student retention. Thus, a multi-pronged approach, such as this, encourages institutions to undertake holistic transformation that benefits all students, including those previously identified as not at risk (Ortiz-Lozano et al., 2020). While other studies conceptualize predictive modeling, this study presents the practical application of such modeling and subsequent interventions and connects to specific workflows, which allows for replication, thus serving a unique contribution to the field of higher education research concerning student retention. While the literature is replete with specifics of retention variables and case studies, there are only the hints or glimpses of clues that tie behaviors, support services, workflows, and processes together—including those across community-driven inputs—in a comprehensive manner. What that means is that, if we intend to truly attempt modeling of retention interventions and their impacts, the study’s guiding questions must be multi-pronged and considered in a phased examination. This is essential to move from basic diagnostics and baselines to predictive modeling if the intention is to harvest the benefits of the multitude of data collected not only for student cohort groups but also across all groupings of students. If we are truly to understand the connections between support and responses, we must acknowledge the steps along the way and the multi-perspective parameters that give different points of view. 6VOLUME 28 NUMBER 1 Further, the current study uses a nontraditional theoretical framework through which retention and attrition behaviors can be explored. Previous research on student retention models are predominantly grounded in student engagement theory (Astin, 1985; Hurtado & Carter, 1987; Hoffman et al., 2002; Lee & Davis, 2000); sociological theory (Fishbein & Ajzen, 1975; Kerby, 2015); customer service theory (de Rosa & de Oliveira, 2020); and survival theory (Murtaugh et al., 1999); however, the current study adapts a theoretical framework from sustainability science to establishing understandings of the interconnectedness between sense of belonging, social integration, and pre-enrollment characteristics, building on the work of Spady’s (1970) path model that suggested “the result of the entire model may lead to changes in students’ attitudes, interests, goals, or motivation that will have either positive or negative effects at later stages of the college or university career” (as cited in Kerby, 2015). Methodology A particularistic, mixed-methods case study methodology examines the impact of retention interventions, events, and engagements on the behaviors of students from incoming freshman cohorts at a four-year, public university in the Mid-Atlantic region of the United States. The methodological approach aligns with Merriam’s (1998) definition of case study research, which characterizes case studies in three ways: particularistic, descriptive, and heuristic. First, the case study is particularistic in that it focuses on a particular situation, event, program, or phenomenon. Second, the case study is descriptive in nature, as it provides a rich, thick description of the phenomenon under study. Third, the case study is heuristic because it “can bring about the discovery of new meanings, extend the reader’s experience, or confirm what is known” through an exploration of the background of the phenomenon and the reasons for the problem, as well as an analysis of the findings. Collectively these methodologies offer an additional advantage - the assessment of intervention strategies and their related workflows. In this application of the PRIME model, the particular situation being studied is student-focused interventions and engagements that must be viable, equitable, and bearable. These interventions and behaviors are determined by demographics, student evolution through real-time participation in the process, and the resulting outcomes (e.g., grades, retention). The descriptive nature of this study deeply explores interactions between and among the wide variety of data collected within the environmental, economic, and societal landscape. The heuristic products of this study link to how workflows and their interactions provide specific insights about the success of students. Data Collection Our ongoing study documents, tracks, and examines aliased select student cohorts, subgroups, and individuals with regard to their derived retention profile. This retention profile is measured and depicted by a wide variety of quantitative and qualitative variables and datasets. These variables and datasets are currently used within a routine, cross-divisional operational and administrative framework and are built into the community infrastructure for the express purpose of student retention and degree completion. The community infrastructure enables both coordination among units and easy electronic access for analysis and consistent coding across service groups (e.g., demographics, use of services, and timing or frequency). The information is used operationally during the academic year to directly notify and assist students needing support services to improve their academic performance. Student Support Services provide the assistance. Data use and exchanges are compliant with FERPA rules and regulations and also make use of THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION7 software/online resources (e.g., Ellucian Colleague and CRM Advise; Ruffalo Noel Levitz College Student Inventory [CSI] and Mid-Year Student Assessment [MYSA]) and confidentiality is maintained. From an administrative perspective, the data create a baseline. From this baseline, the data are used to establish benchmarks and assess programming, events, and support services. Holistically, these portray how student motivators tie directly to support encounters which are intended to help with student academic performance (e.g., financial concerns about continuing their education). To collect data on students’ first-year experiences, the authors designed and distributed a 32-question survey to all students from the previously identified freshmen cohorts through a communication plan built in the Ellucian CRM Advise platform. Survey questions consisted of Likert scale, multiple choice, and short answer questions. Eleven non-returning students and 337 returning students responded. Of the 348 students surveyed, 110 (106 returning; 4 non-returning) students responded “yes” to be part of a focus group or interview. In addition to survey data, the authors included interview and retained-student responses from focus groups in a comprehensive data dashboard, with the goal of identifying the collective experience of these students through data convergence. The data dashboard developed for the purpose of the study is a data management tool that tracks, analyzes, and displays key performance indicators (KPI) and critical data points in real time. It measures the essential data points and shows a customized report to understand students. In other words, the data dashboard provides the opportunity to tie quantitative and qualitative information that connects to specific workflows, events, programming, and interventions – and their collective interactions as received by students. These elements create the operational working model (PRIME) that ties the empirical evidence to best practices, interventions, and behaviors in real time. These define and help us to analyze and understand retention intervention impacts and student behaviors in any specific context and to take proactive steps whenever necessary. Participants The initial selection of subjects is based upon their membership in an incoming cohort of students. The subjects are tracked by various metrics thereafter. Results of the current study represent students from the entering 2018 and 2019 cohorts (n = 3,467) of which 348 (10%) completed the survey (of the 2,357 who opened the email invitation to participate). Each student’s information is obtained according to their admit status/code via the Ellucian Colleague system, and their attendant data and information are used to create a file for analysis. There is no action on the part of the students for these aspects of the study. As part of the study the students are then provided notifications (generic and specific) relative to retention and intervention student support services and programs via email and other platforms, such as ADVISE, CSI/MYSA, or TutorTrac. The modality of messaging is important given the digital native sensibility of students. The right kind of messaging can be timely and viable in raising students’ awareness levels of support services (Bilgiç et al., 2016; Jaschik, 2019). From a behavioral point of view, messages are designed to engage students with the desirability of accessing support services and programs before they are having problems in any of their courses. The generic notifications help notify all students of the assistance available from other campus stakeholders. These support services are part of the institution’s graduation efforts across campus, and they provide both remediation and academic assistance. Specific notifications provide selective messaging such as Academic Early Alerts, Never Attended reporting, Athletics, and similar information for the University as a whole. In these cases specific students are sent messages that tailor to their circumstances (e.g., based on an Early Alert 8VOLUME 28 NUMBER 1 on their performance in a class) and that may suggest and/or recommend they take specific actions (such as visiting one of the student support services available). As this ongoing study considers individual behaviors and cohort performance, there is also planned interaction between students and the researchers (or their support providers). For example, the authors regularly invite subjects to participate in focus groups, exit interviews, surveys, and questionnaires (autoregressive elements of retention behaviors and processes). Students receive an email near the end of their first spring semester inviting them to participate in the survey, which outlines the purpose of the study as well as informed consent and next steps, such as optional focus groups for those retained and exit interviews for those not retained. These interactions elicit further information about the usefulness and perceived value of retention support services, engagement and interaction methods, events, programs, and messaging. Findings Based on institutional data, retention rates for the fall 2018 and fall 2019 cohorts were 73.7% and 74% respectively. Trends in first-to-second year retention over the past six years have, for the most part, hovered between 72% and 76%, maintaining status quo with the national average for similar institutions (71.2%), meaning those with the same Carnegie Classification and a similar acceptance rate (National Student Clearinghouse Research Center, 2019). In following the proposed methodology to identify retention and attrition trends, data collection methods used for traditional metrics, pre-enrollment characteristics, additional measures of interaction, and consideration of conditional contingencies yielded the following results. Pre-Enrollment1 In looking at the racial and ethnic distribution for students who were not retained and did not seek out any academic support services as compared to those who were retained and did seek out at least one support service, no statistical significance exists – in fact, the distributions are almost identical and reflect the same distribution of race and ethnicity as found within the student body as a whole. Similar to the equal distributions of race and ethnicity across both fall 2018 and fall 2019 incoming freshman cohorts, students’ self-reported identification of first-generation status is also equally distributed across both groups: 47.5% of students retained identified as first-generation college students as compared to 44.5% of their not retained peers, which is representative of the larger population of first-generation students, which ranges between 40-50% each year. Of all the students who were not retained who also did not seek out any support services, freshmen admitted through one of the university’s specialized admission programs, PASSPORT, make up 17%. This is significant given the small number of students (200-250) in the PASSPORT freshman cohort. Even though these students are provided additional support from the PASSPORT program, this data (as well as prior year retention rates for the PASSPORT student population) indicate we need to do more to support these students who we already know are coming to us more academically underprepared. 1 The Ruffalo Noel Levitz College Student Inventory, or CSI, first implemented at the institution which serves as the context for this case study in 2019, identifies at-risk students in the incoming class using the leading non-cognitive indicators of college student success, which include items related to student academic motivations, areas of risk, and receptivity to specific student services, as compared to national norms for similar institutions. Prior to 2019, similar pre-enrollment data was collected for each entering cohort via an in-house constructed Entering Student Survey (ESS), deployed through CampusLabs. Data retrieved from the ESS and CSI instruments for the purpose of this study contributed to identifying key pre-enrollment characteristics of both the fall 2018 and fall 2019 freshman cohorts respectively. THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION9 In response to how well the high school attended by the student prepared them for college, ESS data yielded equal distributions across both retained and not retained populations from the entering fall 2018 cohort. The majority of students from each group (83.5% for those retained and 80.6% for those not retained) reported that their level of preparedness for college was “good” or “excellent” on a four-point Likert scale. The CSI measures preparedness by students’ self-reported confidence in two key areas: math and science or verbal and writing. Although these measures for the fall 2019 cohort cannot be equally correlated with the measures identified by the ESS for the fall 2018 cohort, students’ responses remained equal, regardless of they were retained or not, thus suggesting that self-reported levels of preparedness across both cohorts did not yield statistically significant results. However, from a longitudinal perspective, we anticipate that these responses will provide a portion of the insights needed about how a student’s retention behaviors change as well as their response to support services. Desire to transfer for the fall 2018 cohort did not accurately predict student attrition, suggesting that the students who were ultimately not retained did not leave to pursue their higher education at another institution: 90.2% of students who were retained and 88.9% of students who were not retained indicated they planned on completing their degrees. Data from fall 2019 entering freshmen revealed a greater disparity in expressed desire to transfer between retained (21%) and not retained students (31%) before they even started their first semester. Similarly, their ranking of the institution as their first, second, third or fourth (or more) choice did not reveal any significant finding; 53-60% of students from both retained and not retained populations across both cohorts identified the institution as their first choice school. Further, the responses for second, third, and fourth choice options, again across both retained and not retained populations as well as across both fall 2018 and fall 2019 cohorts, were equally distributed. The pre-enrollment information provides the fundamental input of any student or cohort that is often used to create and administer support services for retention at any institution. However it is clear from the information above that these must be considered on an ongoing basis (incoming cohorts) to ensure that the anticipated tuning is appropriate for incoming students (Bean & Metzner, 1985; Tinto, 1975). Indeed, it will also be essential to the longitudinal aspects of applying the PRIME approach to track original cohorts through to graduation. The analysis is critical to determining whether institutional constructs, resources, and workflows are truly appropriate to ensure a productive experience with significant return on investments as students self- evolve (Hoffman et al., 2002; Lee & Davis, 2000). These will shape the student environment put forth by the institution, student economy construct in place, and social framework that establishes the interactions necessary to promote success and retention while instilling effective and proactive behaviors among students. 10VOLUME 28 NUMBER 1 Figure 2 PRIME Model Findings Student Economy Data concerning students’ sense of financial security, however, did yield clear differences, specifically in regard to students’ plans to work during college as well as the number of hours they planned on working per week. While 86% of students not retained from the fall 2018 cohort and 93% not retained from the fall 2019 cohort planned on working during their freshman year, the percentage was lower (by 10%) for their retained peers. An even greater significance lies in the number of hours each group of students planned on working per week: 19% of students not retained from fall 2018 and 27% not retained from fall 2019 planned on working 20 or more hours per week as compared to 10% and 20% of those retained (respectively, half as many). The need for these students to work more than 20 hours per week may also suggest that they are not financially secure, which speaks to attrition issues due inability to pay tuition or associated costs needed for college persistence. This is further compounded by the fact that more than 34% of our fall 2019 first-year students fell within the bottom 50th percentile nationally in regard to sense of financial security as measured by the CSI. Further, students who do not make satisfactory academic progress toward degree completion due to low academic performance are unable to maintain financial aid eligibility, which creates a difficult situation for many THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION11 of these students from the not-retained population. Even with institutional support and programs that assist students under such duress, the measure suggests a barrier that may not be easy to overcome. Student Environment The data analysis for the fall 2018 cohort also identifies distinct differences in levels of engagement between students who were retained as compared to those not retained. ESS data for the fall 2018 cohort revealed the following engagement interests among students who were retained: community service (33.5%); leadership opportunities (29.8%); and clubs and organizations (34.8%). For those not retained, the numbers were lower in all categories: community service (26.4%); leadership opportunities (26.6%); and clubs and organizations (11%). However, whether or not the students resided on campus (42.2% for those retained versus 42.8% for those not retained) was not statistically significant. While incoming freshmen express a desire to seek involvement opportunities on campus (54% of the fall 2019 entering cohort ranked within the top quartile for social engagement receptivity), oftentimes they fail to actually engage in such activities. Regarding on-campus involvement, 34.7% of survey respondents indicated involvement in only one club, organization, or student group on campus during their first year, while 40% reported no involvement. It is important to understand that students may not perceive this form of participation in a college setting as part of the support system designed to assist them in their studies. Indeed, they may view such experiences as strictly social activities or an investment in extracurricular or career development contexts. Despite this viewpoint, their involvement is an investment that reflects their behavioral evolution in the college environment, as noted by Forsman et al. (2015). This involvement can change their situational context (i.e. perhaps from “many factors working against” to “my network is helping” to succeed, thrive, and grow). Social Framework The contributions from student economy and student environment can thus make a difference in how students react and respond to the social framework that is in place to support their success. In order for students to use these supports, there must accessibility, timeliness, relevance, and modalities of delivery that meet their specific situational contexts and allow room for growth. Students must also make a conscious effort on their part to use support services and to consider using multiple types as they create their own retention behaviors to increase and/or improve their academic performance. In the present study, students from both the fall 2018 and fall 2019 cohorts reported equal levels of receptivity to the use of academic support services with 84.5% of those retained and 82.8% of those not retained from fall 2018 indicating they would seek out academic support during their first year of college. In fall 2019, this number was 80% for both retained and not- retained students, thus identifying that the majority of freshman from both cohorts were receptive to using academic support services. A simple fall-to-fall comparison of incoming freshman student cohorts revealed that nine of every ten students not retained one year later – and who did not make use of student support services of any kind – left the university with a probationary GPA (below 2.0), indicating that they did not or were not engaged for success. In contrast, one of every ten students retained after completing their first year of college who also accessed at least one student support service during their first year achieved a GPA twice that of their cohort peers. Of those who accessed services, 59% reported their frequency of usage as one to four visits, and 80% were referred by a faculty member or advisor, which supports the notion that by helping students feel that 12VOLUME 28 NUMBER 1 they are cared for by the institution, faculty and advisors are subsequently preventing student attrition by fostering behavioral change (O’Keeffe, 2013). This level of referral from faculty members or advisors also supports targeted messaging and specification by group needs to assist students’ behavioral transformation. The satisfaction level among students accessing services shows that while 60% were satisfied with their first- year academic experience, nearly 30% were neutral. Continuing this analysis, the study looked at the usage of academic support services for both incoming fall cohorts relative to demographic and qualitative data, moving beyond whether students accessed services to which combinations of services and activity behaviors they engaged in to yield higher GPAs. One such combination (or cross-product) of metrics and variables revealed that tutoring and supplemental instruction (SI) was the answer. Of students surveyed, 35.7% of respondents accessed tutoring and 20% accessed SI during their first year, while 19% did not utilize any academic support services. Students from the fall 2018 cohort who utilized both tutoring services and SI during their first year had an average cumulative GPA of 3.15, and 100% were retained in fall 2019. Comparatively, students from the fall 2019 cohort who accessed both support services had an average cumulative GPA of 3.13 after their first year, and 80% were retained in fall 2020. An analysis of the students not retained from both cohorts who also did not access either tutoring or SI yields similar results: 2.98 average cumulative GPA with a 77% retention rate for fall 2018 students, and 2.84 average cumulative GPA with a 71% retention rate for fall 2019 students. Discussion An initial drill-down of pre-enrollment data was necessary to identify which internal or external factors impacted retention behaviors, as well as to confirm that we do not bias in advance our services and programs with regard to student cohorts and populations. Such data can provide situational awareness; however, they do not drive the retention outcomes, nor do they interfere with the viability of support programs or students’ ability to seek support. Pre-enrollment data, when analyzed for recent cohorts, can help to evaluate whether the social framework of support is properly tuned to incoming students and fosters a sense of belonging from their very first moment on campus (Hurtado & Carter, 1997). Initial observations of the PRIME model findings identify the following as observable characteristics (based on the first two years of cohort data) that are more likely to yield higher attrition among our first-year populations: • conditional or special program admission; • high desire to transfer; • low financial security; • plans to work 20+ hours during first year; • lack of campus involvement; • inability to establish connections with others; • low usage of academic support services (particularly tutoring and SI). While some of these findings may not appear to be surprising, such a view suggests that students are not adapting to offered services or not seeking assistance, building on the work of Astin (1985) in that students who engage are more likely to be retained; however, institutions must also focus on those who choose the path of lesser engagement. The PRIME model of proactive engagement suggests the need to focus resources and services at the root of problems common to first-generation cohorts. Support via an institutional network (i.e. peers, faculty, staff, advisors) serves as “a significant predictor of students’ confidence in graduating, confidence in THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION13 achieving educational goals, and satisfaction with academic career” (D’Amico Guthrie & Fruiht, 2020). If support systems are not accessible, timely, viable, or equitable, the students are less likely to be retained, leading to a poor return on investment. While it is important to explore the correlation between retention and pre-enrollment characteristics, it is also necessary to examine how students’ behaviors change once they assimilate (or not) into the university culture (Elder, 2020). The lack of engagement observed may be attributed to the need for those not retained to prioritize work obligations over campus involvement. This finding is significant because the prioritization of working, in some cases even full-time, may result in students struggling to keep up with the academic rigor of college. Studies in higher education are replete with evidence that the more students engage on campus the higher the likelihood of their retention and timely graduation (Astin, 1999; Harper & Quaye, 2008; Manning et al., 2014; Tinto, 1993, 1999, 2000, 2007; Tinto & Pusser, 2006). However, some students simply do not have the time due to work and family responsibilities outside of the classroom. The above suggest that there is a definable breakpoint or limit that separates students with a capacity for viable engagement from those who cannot engage. This indicates a need for alternate approaches (and modalities) for the latter group of students, including alternate mediums of targeted outreach to increase such engagement (Davis et al., 2019). Thus, these findings represent the disadvantage that students have when they need to prioritize work over campus life; their subsequent lack of engagement may ultimately lead to lower levels of retention for this population. If these findings suggest higher rates of attrition, then the converse is also true regarding which observable behaviors yield higher retention among our first-year student cohorts. The research on academic support in higher education strongly suggests that such efforts have a positive effect on student performance: college students who engage in academic support services tend to have higher course grades than students who do not engage in these services (Arendale, 1994; Cohen et al., 1982; Hendriksen et al., 2005; Lidren et al., 1991; Navarra-Madsen & Ingram, 2010; Ogden et al., 2003; Rheinheimer et al., 2010; Topping, 1996). While engagement is a desirable behavior/intent, it is also important to note that depending upon a student’s particular experiences with one-on-one support or events-driven interventions, the viability of the services provided may or may not be as tolerable with time, as measured by a student’s satisfaction and return visits. This may be a function of time management that impacts a student’s response behaviors (i.e. raising or lowering the bearability limit of investment). A predisposition or likelihood to transfer suggests that definable limitations to intervention strategies will exist according to specific cohorts and individual circumstances beyond the control of an institution. Findings suggest that the majority of students who were not retained did not leave after admission into a school that outranked their current institution on their list of applications – and perhaps a change or evolution in their behaviors. Focus group responses revealed that students who were retained connected early on during their freshman year with a faculty member, advisor, club, organization, or peer leader, which may make the difference in their decision to continue at the university. Students must be held to high standards for success, while, at the same time, provided with academic, financial, and social support systems that are built with their success in mind. In addition to institutional commitment, student involvement, which is often described as academic and social integration in relevant literature, is central to student success. Increased student engagement positively correlates with increased rates of persistence and graduation, especially throughout a student’s first year, during which a sense of belonging on campus is critical (Tinto & Pusser, 2006). The institution cannot expect students to engage themselves; rather, 14VOLUME 28 NUMBER 1 the college or university must provide its students educationally purposeful experiences that make them feel connected to the campus and encourage them to engage in such experiences, with the ultimate result of higher levels of student satisfaction and persistence. While the use of the PRIME model, within the context of sustainability science, offers clear benefits, it is not a panacea. Retention and the success of key student academic support services and interventions will always be an ongoing iterative process that evolves and devolves from one student group/cohort to the next – and even within a given group over time. The fact that the approach relies upon live data and continuous adjustments or adaptations of workflows suggest that the methods are high maintenance given the always-responsive mode of engagement. This can be debilitating to both staff and budgets, result in burnout, and skew baselines when using moving targets across administrative structures. If too much change occurs too frequently, there is a potential for reductions in both the overall quality and effectiveness of intervention strategies, resources, and programs. A further limitation is that ensuring this approach is realistic and has innate flexibility requires a staff that can pivot quickly, respond to feedback efficiently, and has the requisite ongoing training or professional development opportunities to do so. Conclusion This study intended to pilot, demonstrate, and assess the use and operationalizing of the PRIME approach – as well as its potential for portability and implementation elsewhere and among varying cohorts. The unit of student employed here is essential as the individual and cohort flows may be likened to fish swimming upstream and making adjustments as they avoid whirlpools, rocks, and fishers. Thus the diagnostic aspects, while important, are quickly overwhelmed by the situational context of the individual student and his/her response to the prevailing support environment, financial contingencies, and the social fabric of the institution’s support network of services, programs, deliveries, types, and modalities. The key performance indicators, including empirical foundations and metadata, fine detail in real time, specific interactions, student self-reporting, and the results of retention interventions, define student success in a fluid manner. Student behaviors and steps towards success are constantly evolving both in a fishbowl (i.e. the student’s own realm) and within the stream (the university). The data analyzed thus far suggest that what we are doing has changed behaviors (and as supported by student interviews); it is not yet clear whether these have become embedded because that requires longitudinal study. Initial results suggest that the PRIME approach has allowed for confirmation of methods and strategies of support while allowing for direct improvements, enhancements, and changes to specific support mechanisms and underlying workflows. This validation illustrates the particularistic aspects (types and modalities of support and engagements), descriptive elements (landscapes, interactions), and heuristics (workflows, actions, student inputs through output/success) of the study. It confirms that mere demographics do not define cohorts (or their level of success) nor do they imply interventions based solely on such characteristics. Key variables and predictors should be modified operationally and fitted to specific students, cohorts, or subgroups according to other metrics tied to retention behaviors. And while confounding and aliasing of data will always be present in such a dataset lacking internal independent measures, sufficient evidence will be visible to offer proof that support programs and services made a difference in student success. To completely operationalize this approach, which is essentially a living multiple analysis of variance non- parametric model, a real-time dashboard of metrics (quantitative, qualitative, cognitive, et cetera) is essential. Over time this dashboard will allow for refined measures, the identification and incorporation of other salient THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION15 factors, and moving the discussion away from anecdotal evidence toward an appropriate form for full numeric analysis. This transitioning of previous methods of data collection away from asking “how students can build cultural capital to leverage their success within an institution,” and toward what the institution should be doing “to become inclusive to all students, staff, and the wider community” continues to foster student-focused interventions and engagements that are viable, equitable and bearable for students, as the present study suggests (Naylor, 2020). This capital extends to accessibility (and thus equity) in terms of obstacles or limitations due to technology, awareness, location, language and other factors. It links with the viability of support to cohorts and individual students as a balance between their own economic and environmental circumstances and constraints. Regardless of support services and programs or events in place, there are critical thresholds for students that drive their engagement and their decision to seek assistance. The situational context is daunting and speaks to inequities and social justice beyond the scope of this study. While all students have rights to access and engage, they may be prevented from doing so by factors beyond their control including intervention mechanisms. These kinds of roadblocks are a call to action for systemic change in higher education. This study – and the very nature of the PRIME model itself – also serves as a stepping stone for longitudinal research through the operationalization of data collection and analysis of each new cohort from admission through graduation. As the comprehensive data dashboard continues to evolve through additional inputs of key performance indicators and critical data points for each cohort, the big picture of the collective experience of students will emerge, as verified by data convergence and trends in retention and attrition through the years, which will serve to inform future structural and administrative changes. Whether these are cyclical, linear, non- linear, or non-stationarity, the strength of the PRIME approach is its flexibility. As it relates to and derives from the delivery of services to students, an assessment of workflows is also possible within the framework of this study to truly measure the return on investment. The application of the PRIME model should not be exclusive to the university that serves as the context for the current study, as it can be effectively replicated at other institutions of higher education with minor adjustment based on campus needs. By further exploring the impact of pre-enrollment characteristics, the student’s environment, economic factors, and social frameworks, which encompasses inputs and behaviors, program administrators will gain a better sense of how such non-cognitive factors impact retention. Identifying potential barriers and roadblocks for student success proactively rather than reactively allows more targeted deterrence of student attrition (Elder, 2020). The PRIME model can also prove effective for students beyond their first year in an effort to continue to track retention and engagement behaviors from the first year through graduation. 16VOLUME 28 NUMBER 1 References Arendale, D. R. (1994). Understanding the Supplemental Instruction model. In D. C. Martin, & D. R. Arendale (Eds.), Supplemental Instruction: Increasing student achievement and retention. Jossey-Bass. Astin, A. W. (1985). Involvement: The cornerstone of excellence. Change, 17, 35-39. http://dx.doi.org/10.1080/0 0091383.1985.9940532 Astin, A. W. (1999). Student involvement: A developmental theory for higher education. Journal of College Student Development, 40(5), 518–529. Bean, J. P., & Metzner, B. S. (1985). A conceptual model of nontraditional undergraduate student attrition. Review of Educational Research, 55(4):485-540. https://doi.org/10.3102/ 00346543055004485 Bilgiç, H., Doğan, D., & Seferoglu, S. S. (2016). Digital natives in online learning environments: New bottle old wine. The Design of Online Learning Environments for Today’s Generation. https://doi.org/10.4018/978-1- 5225-0039-1.ch009 Cohen, P. A., Kulik, J. A., & Kulik, C. (1982). Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research Journal, 19(2), 237-248. https://doi.org/10.3102/00028312019002237 Croft, P. J., & Lozada, N. (2020). Proactive retention through integrated modeling of engagement. The Learning Assistance Review (TLAR), 25(2), 81-110. D’Amico Guthrie, D., & Fruiht V. (2020). On-campus social support and hope as unique predictors of perceived ability to persist in college. Journal of College Student Retention: Research, Theory & Practice, 22(3), 522-543. Davis, G. M., Hanzsek-Brill, M. B., Petzold, M. C., & Robinson, D. H. (2019). Students’ sense of belonging: The development of a predictive retention model. Journal of the Scholarship of Teaching and Learning, 19(1), 117- 127. https://doi.org/10.14434/josotl.v19i1.26787 De Rosa, F., & de Oliveira, L. (2020). Student retention at a higher educational institution: A comparison between evaded and nonevaded groups using partial least square. Journal of College Student Retention: Research, Theory & Practice, 1-27. https://doi.org/10.1177/1521025120924663 Elder, A. C. (2020). Holistic factors related to student persistence at a large, public university, Journal of Further and Higher Education. https://doi.org/10.1080/0309877X.2020.1722802 Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Addison-Wesley. Forsman, J., Van den Bogaard, M., Linder, C., & Fraser, D. (2015). Considering student retention as a complex system: A possible way forward for enhancing student retention. European Journal of Engineering Education, 40(3), 235-255. http://dx.doi.org/10.1080/03043797.2014.941340 Harper, S. R., & Quaye, S. J. (2008). Beyond sameness, with engagement and outcomes for all: An introduction. In S. R. Harper and S. J. Quaye (Eds.), Student engagement in higher education: Theoretical perspectives and approaches for diverse populations (1-13). Routledge. Hendriksen, S. I., Yang, L., Love, B., & Hall, M. C. (2005). Assessing academic support: The effects of tutoring on student learning outcomes. Journal of College Reading and Learning, 35(2), 56-65. Hoffman, M., Richmond, J., Morrow, J., & Salomone, K. (2002). Investigating “sense of belonging” in first-year college students. Journal of College Student Retention: Research, Theory and Practice, 4(3), 227–256. https:// doi.org/10.2190/DRYC-CXQ9-JQ8V-HT4V THE JOURNAL OF COLLEGE ORIENTATION, TRANSITION, AND RETENTION17 Hurtado, S., & Carter, D. F. (1997). Effects of college transition and perceptions of the campus racial climate on Latino college students’ sense of belonging. Sociology of Education, 70(4), 324–345. https://doi. org/10.2307/2673270 Jaschik, S. (2019). Instagram on the rise. Inside Higher Ed. https://www.insidehighered.com/admissions/ article/2019/09/23/prospective-students-social-media-preferences-have-changed-two-years Kerby, M. B. (2015). Toward a new predictive model of student retention in higher education: An application of classical sociological theory. Journal of College Student Retention: Research, Theory & Practice, 17(2), 138- 161. https://doi.org/10.1177/1521025115578229 Kuh, G. D. (2001). Assessing what really matters to student learning: Inside the National Survey of Student Engagement. Change: The Magazine of Higher Learning, 33, 10-13. Kuh, G. D. (2003). What we’re learning about student engagement from NSSE: Benchmarks for effective educational practices. Change: The Magazine of Higher Learning, 35(2), 24-32. Lee, R. M., & Davis, C. III. (2000). Cultural orientation, past multicultural experience, and a sense of belonging on campus for Asian American college students. Journal of College Student Development, 41(1), 110–115. Lidren, D. M., Meier, S. E., & Brigham, T. A. (1991). The effects of minimal and maximal peer tutoring systems on the academic performance of college students. The Psychological Record, 41(1), 69–77. Manning, K., Kinzie, K., & Schuh, J. H. (2014). One size does not fit all: Traditional and innovative models of student affairs practice (2nd ed.). Routledge. Merriam, S. B. (1998). Case study research in education: A qualitative approach. Jossey-Bass. Murtaugh, P. A., Burns, L. D., & Schuster, J. (1999). Predicting the retention of university students. Research in Higher Education, 40(3), 355-371. National Student Clearinghouse. (2019). Persistence & Retention - 2019. NSC Research Center. https:// nscresearchcenter.org/snapshotreport35-first-year-persistence-and-retention/ Navarra-Madsen, J., & Ingram, P. (2010). Mathematics tutoring and student success. Procedia - Social and Behavioral Sciences. 8, 207-212. https://doi.org/10.1016/j.sbspro.2010.12.028 Naylor, R. (2020). Key factors influencing psychological distress in university students: The effects of tertiary entrance scores. Studies in Higher Education, 1-13. Noel, L., Levitz, R., Saluri, D., & Associates. (1985). Increasing student retention. Jossey-Bass Ogden, P., Thompson, D., Russell, A., & Simons, C. (2003). Supplemental Instruction: Short- and long-term impact. Journal of Developmental Education, 26(3). O’Keeffe, P. (2013). A Sense of Belonging: Improving Student Retention. College Student Journal, 47(4), 605–613. Ortiz-Lozano, J. M., Rua-Vieites, A., Bilbao-Calabuig, P., & Casadesús-Fa, M. (2020). University student retention: Best time and data to identify undergraduate students at risk of dropout. Innovations in Education and Teaching International, 57(1), 74-85. https://doi.org/10.1080/14703297.2018.1502090 Rheinheimer, D. C., Grace-Odeleye, B., Francois, G. E., & Kusorgbor, C. (2010). Tutoring: A support strategy for at- risk students. The Learning Assistance Review (TLAR), 15(1), 23-34. Singell, L. D., & Waddell, G. R. (2010). Modeling retention at a large public university: Can at-risk students be identified early enough to treat? Research in Higher Education, 51(6), 546-572. doi: https://www.jstor.org/ stable/40785094 18VOLUME 28 NUMBER 1 Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45, 89-125. Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition. (2nd ed.). University of Chicago Press. Tinto, V. (1999). Taking retention seriously: Rethinking the first year of college. NACADA Journal, 19(2), 5-9. Tinto, V. (2000). Linking learning and leaving: Exploring the role of the college classroom in student departure. In J. M. Braxton (Ed.), Reworking the student departure puzzle (pp. 81-94). Vanderbilt University Press. Tinto, V. (2007). Research and practice of student retention: What next? Journal of College Student Retention: Research, Theory & Practice, 8(1), 1-19. Tinto, V. (2012). Completing college: Rethinking institutional action. Chicago: University of Chicago Press. Tinto, V., & Pusser, B. (2006). Moving from theory to action: Building a model of institutional action for student success. National Postsecondary Educational Cooperative. http://web.ewu.edu/groups/academicaffairs/IR/ NPEC_5_Tinto_Pusser_Report.pdf Topping, K. J. (1996). The effectiveness of peer tutoring in further and higher education: A typology and review of the literature. Higher Education, 32, 321-345. https://doi.org/10.1007/BF00138870