Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023, pp.14-30. doi: 10.14434/josotl.v23i1.32675 The Impact of Undergraduate Research Experience Intensity on Measures of Student Success Donna Chamely-Wiik Florida Atlantic University dchamely@fau.edu Anthony Ambrosio Florida Atlantic University Tracy Baker Lynn University Amrita Ghannes Florida Atlantic University Jennie Soberon Florida Atlantic University Abstract: Despite the growing interest to provide research engagement opportunities to undergraduate students, few studies have investigated how engagement “intensity” impacts measures of student success. A quasi-experimental, matched-subject design was employed to study differences between varying levels of research experience intensity (i.e., Experienced, Novice, Control groups) on Graduating GPA, Time to Graduate, and type of post-graduation experience. Results indicated that experienced students had significantly higher graduating GPAs than novice or control students, and both research groups had significantly lower time to graduate than the control group. Findings also indicated experienced student researchers are significantly more likely to progress to graduate school than either novice research or control students. Implications for implementing research initiatives are discussed. Keywords: Undergraduate research, student success, GPA, post-graduation activities, time to graduation Undergraduate research (UGR) is one of ten high impact practices shown to enhance and improve college student performance and success (Kuh, 2008). The Council on Undergraduate Research (CUR) defined undergraduate research as “an inquiry or investigation conducted by an undergraduate student to make an original intellectual or creative contribution to a discipline” (Council on Undergraduate Research, 2018, para. 3). Undergraduate research experiences can include both curriculum-based opportunities and co-curricular opportunities (Beckman & Hensel, 2009; Batttaglia et al., 2022). Course-based Undergraduate Research Experiences (CUREs), such as capstone courses, integrate undergraduate research into course curriculum and materials (Batttaglia et al., 2022, Chamely-Wiik et al., 2014). Co-curricular undergraduate research involves research activities outside the classroom such as summer research internships or fellowships, participating in summer research programs, presenting at poster sessions, and contributing meaningfully to research projects (Kardash, 2000; Pender, 2010; Thiry et al., 2012). At the individual skill level, curricular and co-curricular activities have been defined as those activities that involve “analyzing data, interpreting findings, and deliberating about next steps of an experiment” (Thiry et al., 2012, p. 264). mailto:dchamely@fau.edu Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu Multiple studies have demonstrated the importance of curricular and co-curricular UGR experiences on promoting inquiry skills and learning outcomes such as critical thinking and communication (Thiry et al., 2011; Monarrez et al., 2020) student success metrics (Baron et al., 2020; Brown et al., 2020; Simmons, 2018), student preparedness, and career clarification (Craney et al., 2011; Frederick et al., 2021). Russell et al. (2007) found that undergraduate research opportunities increased students’ understanding of how to conduct a research project, confidence in their research skills, and awareness of what graduate school entails. Students who participate in UGR are also more likely to develop an expansive skillset. McDevitt et al. (2016) found that undergraduate students who worked with faculty mentors in research improved in the areas of analyzing, writing, and presenting research data. One study found that science undergraduates who participated in a research experience had an increased ability to perform skills at the end of the experience compared to the beginning (Kardash, 2000). Evidence suggests these skills persist through graduate school. Graduate students who experienced UGR were better at finding existing literature, developing testable hypotheses, and selecting, analyzing, and presenting data (Gilmore et al., 2015). Participating in UGR also positively affects students' perceived skill competence. For example, students who participated in the NSF-funded Research Experiences for Undergraduates (REU) program (NSF, 2017) reported higher gains in specific research-based skills (Follmer et al., 2018). Perceived increases in skills and knowledge from research engagement occur even with beginner researchers (Bhattacharyya et al., 2018). Research competence was not only reported by the students, but also noted by their faculty mentors (Kardash, 2000). Likewise, students who participated in UGR at a Hispanic-majority institution were more likely to gain “knowledge and skills, institutional support, overall satisfaction, grade point average, and student-faculty interaction” (Collins et al., 2017, p. 583). Undergraduate research engagement has also led to increased institutional metrics for student success, especially for underrepresented populations including persistence (Simmons, 2018) and higher GPAs (Whittinghill et al., 2019). Bowman and Homes (2018) observed freshmen participation in UGR to be positively related to fourth-year undergraduate GPAs. Another study using student self- reported research experiences in an empirical design found that students who participated in UGR had higher GPAs when compared to those without UGR participation (Baker, 2017). Furthermore, it appears these experiences positively predict higher college GPAs independent of how the research experience occurs (Kim & Sax, 2009). Freshmen and sophomore students, particularly African American students and those with lower GPAs who participated in a targeted UGR experience, were more likely to be retained within the institution compared to a control group (Nagda et al., 1998). Jones et al. (2010) found that Hispanic and African American students who engaged in UGR were more likely to persist in their major and obtain their biology degrees, compared to peers without UGR. In addition, students who participated in UGR were more apt to pursue graduate education, engage in additional research activities, enter doctoral STEM programs, and showed a higher interest in pursuing a research career (Hathaway et al., 2002; Pender et al., 2010; Wolkow et al., 2019). Students at a Minority-Serving Institution were more likely to develop relationships with mentors as well as feel a part of a community through UGR programs, leading them to aspire for Ph.D. and graduate programs (Carpi et al., 2017). Engineering alumni who participated in undergraduate research were four times as likely to pursue doctoral degrees than non-research students (Zydney et al., 2002). Barlow and Villarejo (2004) found that underrepresented minority freshmen who participated in an undergraduate research program for biological sciences were 2.4 times more likely to graduate from the institution, independent of graduating major, and 4.1 times more likely to graduate in Biological Science. Several more recent studies have evaluated the impact of undergraduate research experiences (URE) on academic student success metrics and post-graduation plans specifically at Hispanic or minority-serving institutions (Schneider et al., 2021; Baron et al., 2020; Battaglia et al., 2022). Battaglia 15 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu et al. (2022) found that engagement in undergraduate research of different research modalities (all co- curricular) at a Hispanic-Serving Institution (HSI) is associated with increased number of successfully completed semester credit hours, higher GPAs, increased graduation rates, and higher enrollment in further education. Schneider et al. (2021) showed similar trends of students engaged in UGR having demonstrated increased GPAs as well as entry into graduate school. Schneider et al. (2021) also identified a gap in the literature by stating “We have not looked at this data regarding how many ‘times’ students appeared in our database, but that would warrant further investigation to see if extended involvement impacted GPA (and other factors).” They also identified only a small difference between their institutional and UGR population in terms of years to degree completion, indicating that UGR students have not graduated in fewer semesters than the university undergraduate population, and further suggesting that “it would be interesting to know if other campuses see similar trends” (Schneider et al., 2021). This paper aims to address these gaps in the literature. While many of these studies have examined the benefits of participating in UGR, few have explored how the level or intensity of the undergraduate research experiences affect measures of student success. Zydney et al. (2002) used the number of semesters involved in research to compare student perceptions of research involvement benefits. Their findings suggested that 34 out of 38 students who participated in four or more semesters perceived research involvement as “extremely important” (p.154). Baron et al. (2020) evaluated the impact of the length of participation in their Emerging Scholars program on semester GPA, semester credits earned, persistence and graduation. They found no statistical significance for GPA between those who completed a single versus multiple semesters. They did find that the graduation rate for the single semester participant was significantly lower than the graduation rate for multiple participants, but only for those pursuing an associate degree. Thiry et al. (2012) further defined intensity of engagement in undergraduate research based on the length of time students engaged in research by creating two levels: Novice and Experienced. Novice researchers were defined as “students who had completed two or less semesters of undergraduate research,” and Experienced researchers were defined as “students who had completed more than two semesters plus one summer of undergraduate research” (Thiry et al., 2012, p. 264). Findings indicated differences in the reported experiences between Novice and Experienced researchers and identified Experienced researchers to be more likely to report perceived improvement in data collection techniques, data analysis engagement, and advanced scientific thinking skills. The term Novice researcher has also been used to examine gains in skill and knowledge among undergraduates with little to no prior research experience who began research early in their academic careers (Bhattarcharyya et al., 2018). Although these studies demonstrate multiple positive effects UGR has on student success, these effects have often been measured through self-reports and indirect measures, and impact has been determined by correlations frequently without the benefit of an objective comparison group (Haeger et al., 2020). Additionally, most of these studies have focused solely on STEM students (Haeger et al., 2020). Battaglia et al., (2022) looked at the effect on the type of UGR experience on different metrics, however it was mostly STEM focused and did not factor in semesters involved. Furthermore, the few studies measuring UGR impact empirically have not factored in the possible effect of research experience intensity. This study examines the effect of research experience intensity (i.e., the number of semesters involved in research) between STEM and non-STEM students on several student success metrics. Research experiences consistent with existing literature will be objectively operationalized by participation in well-defined curricular and co-curricular initiatives. 16 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu Institutional Background This study was conducted at Florida Atlantic University, a Hispanic-serving, doctoral granting, public, accredited southeastern university, with a classification of high research activity (Carnegie Classification of Institutions of Higher Education, 2020). This classification identifies the university as a large public university with a high undergraduate enrollment of approximately 24,500. A 2011 analysis of the institution’s research activity revealed that most of the student research conducted was limited to graduate and upper-division undergraduate students, primarily through honors programs. Given the extensive literature on the positive impacts of undergraduate research, a university-wide initiative was developed in 2012 to expand this practice to include more undergraduate students. Florida Atlantic University leveraged its re-accreditation with the Southern Association of Colleges and Universities: Commission on Colleges (SACSCOC) to develop and implement a set of initiatives that would expand student engagement in undergraduate research through a Quality Enhancement Plan (QEP). The university’s QEP, Florida Atlantic University, focused on improving student learning by expanding a culture of undergraduate research and inquiry across all disciplines at the university, and guided by a centralized office. The institution also included undergraduate research metrics as part of their 2015 strategic plan, and as part of faculty promotion and tenure portfolios in 2018. The goal of the Florida Atlantic University was to expand Undergraduate Research and Inquiry (URI) at the university and included the following objectives: a) establish an undergraduate curriculum which provides students necessary URI intellectual skills; b) expand student co-curricular URI opportunities; c) increase support and recognition for engaged faculty and students; and d) enrich and strengthen an institutional URI climate (Chamely-Wiik, 2013). Numerous initiatives were developed to meet these goals such as an undergraduate research grants program, an annual research symposium, a research journal, a summer research fellowship, course-based Research Intensive (RI) and Directed Independent Research (DIR) experiences, student peer mentoring and professional development programs, a faculty liaison program, and student URI clubs. Data collected through several of these programs were examined for this study. Defining Undergraduate Research For the purposes of this study, CUR’s 2015 definition of undergraduate research was adopted and refined to include the following criteria to address the methodological gaps in previous studies and ensure a delineated definition of research: ● student research must be faculty-mentored ● the nature of the inquiry must be original ● students complete the entire cycle of research and inquiry ● the inquiry or research must generate tangible outcomes consistent with the discipline ● the inquiry or research must be communicated to an external audience and/or subject to external or peer review The URI programs that adhered to the above qualifications included the undergraduate research grants program (UGRG), the undergraduate research symposium (URS), the undergraduate research journal (URJ), the Summer Undergraduate Research Fellowship (SURF) program, the National Science Foundation (NSF) Learning Environment and Academic Research Network (LEARN) program, and Research Intensive (RI) and Directed Independent Research (DIR) courses. An undergraduate research committee which includes one faculty member from each college, has 17 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu guided the establishment and implementation of all above initiatives to ensure that STEM and Non- STEM experiences across the university adhere to the above delineated definition as appropriate to their discipline. Below we provide additional details about the different programs for context of the research nature of each of the programs. Undergraduate Research Grants Program (UGRG). Introduced the fall semester of 2012, the UGRG program supports undergraduate student projects where the research or creative activity conducted by the student generally takes diverse forms of directed research, such as independent study, work on an honors thesis or as a part of a larger ongoing study under the direct mentorship of a faculty member. Eligible students must be undergraduates in good academic standing. To be included in this study, undergraduates must have been awarded a grant and received funding. Undergraduate Research Symposium (URS). The URS, which was first hosted in spring 2012, showcases student undergraduate research and scholarship progress through oral, performing arts, poster, or visual arts presentations. Submitted projects must be conducted as an undergraduate, and students whose abstracts were accepted for a presentation were included in this study. Undergraduate Research Journal (URJ). Established in spring 2012, the URJ is an interdisciplinary, double-blind peer-reviewed journal that is published annually, online and in print, and showcases high quality undergraduate research in all fields. Only research projects conducted and written by an undergraduate are eligible for submission. Students whose manuscripts were published were included in this study. Summer Undergraduate Research Fellowship (SURF). SURF projects, which were first awarded the summer of 2015, are ten-week, intensive summer immersion research experiences for faculty-undergraduate student teams. Eligible students must be undergraduates in good academic standing and retain undergraduate status through the following fall semester. Typically, SURF was sought by faculty mentors for experienced undergraduates. To be included in this study, undergraduates must have been awarded a fellowship and received funding. NSF LEARN Program. The NSF LEARN program, which had its first group of participants in the fall of 2016, seeks to increase retention and success of underrepresented students in Science, Technology, Engineering, and Mathematics (STEM) disciplines, by engaging students in undergraduate research. To be eligible to participate in the year-long program, undergraduates must be either transfer students or first time in college (FTIC) freshmen with a declared STEM major. Participants that completed the year-long LEARN program are included in the study. Research Intensive (RI). Research Intensive (RI) course designation was established in the spring of 2015 to expand the practice of curricular-based research opportunities for undergraduate students and to allow documentation of student engagement in research and scholarly activity on their transcripts. All RI courses include evidence of students completing the entire cycle of research and inquiry including learning gains in content knowledge, formulate questions, plan of action, critical thinking, ethics and communication (Chamely-Wiik, et al., 2014). Courses were submitted and reviewed for RI designation by a university committee using a common rubric to ensure assignments satisfied criteria (Author, 2017). Undergraduate students enrolled in an RI course were included in the study. Directed Independent Research (DIR). Directed Independent Research (DIR) courses were established in the spring of 2015 to identify and track through the curriculum, undergraduates engaged in research or creative inquiry under the mentorship of a faculty member. Undergraduate students enrolled in a DIR were included in the study. 18 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu Research Questions A quasi-experimental, matched-subject design was employed to study differences between varying levels of research experience (e.g., Novice and Experienced compared to a Control group) on various measures of student success. The following research guided the analysis: RQ 1: Are there differences in graduation GPA between students with varying levels of undergraduate research experience (Experienced, Novice, Control)? RQ 2: Are there differences in time to graduation between students with varying levels of research experience (Experienced, Novice, Control)? RQ 3: Are there differences in post-graduation activities between students with varying levels of research experience (Experienced, Novice, Control)? Method Participants Participants (n=344) were selected from all undergraduate students involved in undergraduate research (n=2,959) enrolled from spring 2012 through spring 2019 who graduated with a bachelor’s degree and did not transfer credits earned prior to this timeframe. Students who had transfer credits prior to spring 2012 were eliminated from the study to reduce generational and historical effects. This period was chosen to correspond with the inception of the URI programs to allow 4-6-year graduation rates to be realized. Students involved in the selected research programs within the timespan were divided into two groups representing different levels of URI research experience: Novice or Experienced. Novice students engaged in a URI program for one or two semesters prior to graduation. Experienced students engaged for at least three different semesters prior to graduation. With these specifications, the final list of program students (n= 2,959) was submitted to the university’s Institutional Research (IR) office to merge with the subset of participants identified above. After removing personal identification, IR securely returned a complete dataset which included all dependent and independent variables used in this study. Students not previously coded as “Novice” or “Experienced” were identified as part of the Control group, which represented those with no URI experiences. Design A matched-subject design was used to equate these groups based on three factors: First Semester GPA, STEM/Non-STEM-related major, and number of transfer credits at the time of sampling. First Semester GPA was used to equalize students on academic ability, serve as a pre-measure control to graduating GPA, and reduce differences between subjects on potential apriori research interest. Transfer credits were not used in GPA scores. STEM classification was determined by categorizing a student’s graduating major and used to ensure equal numbers in research-heavy disciplines that may place different emphases on the value, utility or definition of a research experience. The number of transfer credits was used to control the degree of exposure to this institution, and to equate groups on their distance to graduation, since time to graduate is a dependent variable in this study. The Experienced group was the smallest of the three groups, so every viable case became a subject in the study (n=86). Each Novice subject was matched to every case in the Experienced group. Twice as many subjects were matched for the Control group to maximize design sensitivity. A random selection procedure was used when multiple Novice or Control subjects were a viable match to a case. 19 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu The final selected group demographics and variable statistics are presented in Table 1. The number of students in the Experienced and Novice groups who participated in each research initiative is presented in Table 2 and these results can be used to gauge differences between the groups in terms of the relative intensity levels. Table 1. Frequencies and Averages of Demographic Characteristics Between Experienced, Novice and Control Groups. Experienced Novice Control Totals N 86 86 172 344 # STEM 55 55 110 220 # Non-STEM 31 31 62 124 Female 62 44 114 220 Male 24 42 58 124 Native American/Alaskan 1 0 0 1 Asian 6 7 10 23 Black/African American 9 8 23 40 Hispanic or Latino 25 24 48 97 White 39 40 82 161 Multiple 0 3 3 6 1st Sem Avg. GPA (SD) 3.60 (0.42) 3.58 (0.45) 3.59 (0.43) --- *Avg Transfer Credits (SD) 22.25 (29.7) 22.30 (29.7) 22.24 (29.6) --- *Median transfer credits were 3.0 for each group Table 2. Number of Participants by Initiative and Group. DIR Florida Atlantic Universi tyRJ LEARN RI SURF URS UGR Novice 19 2 2 22 0 46 30 Experienced 29 6 6 22 24 78 77 Legend: Novice refers to students engaged in an undergraduate research program/course for one or two semesters prior to graduation and Experienced refers to students engaged for at least three different semesters prior to graduation. 20 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu DIR refers to Directed Independent Research; FAURJ refers to Florida Atlantic Undergraduate Research Journal; LEARN refers to the NSF LEARN program; RI refers to Research Intensive courses; SURF refers to the Summer Undergraduate Research Fellowship program; URS refers to the annual undergraduate research and inquiry symposium; and UGR refers to the Undergraduate Research Grants program. Measures This study used three dependent variables in group comparisons: the grade point average of students at the time of graduation (Degree GPA); the number of years to earn a bachelor’s degree (Time to Graduate); and educational experience after graduation (Post-Grad Experience). Postgraduate Experience was obtained through National Clearinghouse Data that was matched with student records. For Postgraduate Experience, students were classified into three experience groups: Graduate/Professional, 2nd-Undergraduate/Certificate, and No-Post Graduate Experience. Results Research Question 1 The first hypothesis sought to test differences between Experienced, Novice and Control groups on their GPA at graduation. A one-way ANOVA was used in this comparison. A Levene’s test validated homogeneity (p = 0.231). Results indicated a significant difference between groups (F (2, 341) = 8.41; p < 0.01). A Bonferroni post-hoc analysis showed that the Experienced group had a significantly higher degree GPA than the Control group (p < 0.01), but not the Novice group (p > 0.05). There was no significant difference between the Novice and Control groups (p > 0.05). Group means and standard deviations are presented in Figure 1. Research Question 2 The second hypothesis examined the difference between Experienced, Novice and Control groups on Time to Graduate. A one-way ANOVA was employed. A Levene’s test indicated no violations in the homogeneity of variance (p = 0.47). Results showed a significant difference between groups (F (2, 341) = 3.31; p < 0.05). A Bonferroni post-hoc analysis indicated a significantly lower time to graduate between the Experienced and Control groups (p < 0.05), but not between the Experienced and Novice groups (p > 0.05). There is a significant difference between the Novice and Control groups (p > 0.05). Figure 1 presents the group means and standard deviations. 21 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu Figure 1. Average Graduating GPA and Years to Graduate by Group. Research Question 3 A 3 X 3 Chi-Square Test for Independence was used to determine the relation between the level of undergraduate research involvement (Experienced, Novice, Control) and type of post-graduate experiences (Graduate/Professional, 2nd-Undergraduate/Certificate, or No Post-Graduate Experience). Results showed a statistically significant difference between groups on post-graduate experiences (X2 (4, N=344) = 14.15, p < 0.05). A follow-up Cramer’s V analysis identified a small overall effect size (φc = 0.14). Post-hoc analysis of adjusted residuals with a Bonferroni correction indicated the Control group had a significantly higher observed versus expected frequency for “No Post-Graduate Experience,” and a significantly lower observed versus expected frequency for “Graduate/Professional.” In contrast, the Experienced group had a significantly lower “No Post- Graduate Experience” count and a significantly higher “Graduate/Professional” count than expected. The Novice group displayed no significant difference in observed and expected values for each post- graduate condition. The observed versus expected frequency for “2nd-Bachelors/Certificate” was comparable for all three groups. Table 3 presents descriptive results of this analysis. Table 3. Frequency of Post-Graduate Experience by Group. Observed Frequency Observed Percent Expected Frequency Control No Post-Graduate Experience 108 62.8 93.5 2nd -Undergraduate/Certificate 24 14.0 24 Graduate/Professional 40 23.3 54.5 Total 172 100.0 172 Novice No Post-Graduate Experience 44 51.2 46.8 2nd -Undergraduate/Certificate 11 12.8 12 Graduate/Professional 31 36.0 27.3 Total 86 100.0 86 Experienced No Post-Graduate Experience 35 40.7 46.8 2nd-Undergraduate/Certificate 13 15.1 12 Graduate/Professional 38 44.2 27.3 Total 86 100.0 86 22 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu Discussion A quasi-experimental, matched-subject design was employed to examine differences between students with differing levels of experience with URI initiatives (Experienced, Novice, and Control groups) on graduating GPA, time to graduation, and post-graduate activities. First-semester GPA, STEM major distribution, and number of transfer credits were controlled to eliminate the possibility of confounding factors such as the perception that high-achieving students pursue undergraduate research more, or that URI is pursued more by students in STEM. Research Question 1: GPA Results indicated a significant difference between the Experienced group compared to both Novice and Control groups when evaluating their graduating GPA. Moreover, the difference between Novice URI students and those with no URI experience was not significant, demonstrating that the intensity or duration of research experience affects student success on this metric. This builds on and provides empirical support for previous research on the positive relationship between GPA and students who participate in research (Whittinghill et al., 2019, Bowman & Holmes, 2018; Collins et al., 2017; Russell et al., 2007, Sell et al., 2018). Since high school GPA varies from institution to institution (weighted, unweighted, etc.), we chose to utilize first-semester GPA within the exact-matched design to eliminate the possibility of confounding factors, further demonstrating a direct role of undergraduate research involvement on gains in GPA at graduation. Experienced researchers may be better equipped to transfer research skills to other courses, resulting in improvements in GPA and other measures of student success. Alternatively, it is possible that students who participate as Novices in research, may be lacking in certain academic skills. Since our study did not evaluate these factors or others such as motivation, further studies would need to be conducted to better understand these differences. An additional analysis was conducted to examine the nature of the relationship between GPA at graduation and the pattern of research involvement in total URI terms. The scatterplot results in Figure 2 reinforce the idea of promoting student participation in multiple semesters of URI for a more significant impact. Student Graduation GPAs have higher variances among students with a single URI experience while students engaging in three or four terms have lower Graduation GPA variances. Additional experience beyond four terms appears to have less effect on Graduation GPA. 23 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu Figure 2. Scatterplot of relationship between Total URI terms and GPA at Graduation. While we controlled for STEM/Non-STEM students in our sample through a matched design, a comparison between these subgroups was warranted to underscore the importance of recruitment of a diverse population of participants. A one-way ANOVA was conducted to compare Degree GPA between STEM and Non-STEM students. A Levene’s test validated homogeneity (p = 0.771). Results indicated no significant difference between the groups (F (1, 342) = 1.04; p > 0.05). These findings are consistent with previous research in this area (Collins et al., 2017; Nagda et al., 1998), suggesting the lift in GPA is beneficial for any student pursuing research opportunities independent of discipline. This has implications for centralized offices to facilitate scaffolded and sustained involvement through/across multiple programs for up to at least three terms and expand their offerings to include STEM and Non-STEM research experiences. To facilitate student engagement in multi-year research opportunities, colleges and universities should consider focusing their efforts and resources on early engagement in undergraduate research (Ishiyama, 2002). Additionally, offices should consider showcasing opportunities such as an annual symposium or a research journal, to support sustained involvement and encourage students to communicate the findings of their inquiry. Departments could also scaffold research skills by offering multiple course-based undergraduate research experiences (CUREs) or other Research Intensive (RI) course experiences within their programs of study, leading to a capstone research experience. Finally, participation in symposia and publication opportunities can be included within CUREs or considered a supplemental requirement for receiving funding from a centralized office. Research Question 2: Time to graduation The second hypothesis of this study examined the impact of URI involvement on student time to graduation. Results indicate a significant difference between the Control group and both Experienced and Novice groups. Even Novice URI experiences (two terms or less) had a positive impact on a participant’s time to graduation. These findings support previous research in this area that found UGR 24 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu participation to be associated with higher retention rates (Nagda et al., 2002), and lower time to graduate when comparing Novice and Experienced groups to a Control group (Thiry et al., 2012). From a student perspective, a lower time to graduation while engaging in undergraduate research could be financially beneficial, resulting in a quicker transition to the workforce or graduate school. Since colleges and universities are being held to more metric-driven standards related to student success efforts such as time to graduation, engagement in high-impact practices like UGR could support student success. Finally, these findings quell the perception that engagement in URI activities may slow a student down academically and supports the idea that even limited URI experiences support student success. The results demonstrate that even Novice researchers who engage in two semesters or less of undergraduate research are more likely to graduate in less time than non-research active students. This could be beneficial for institutions with large transfer student populations. Institutions could encourage these students to get involved, despite the shortened time at the university. Additionally, for smaller institutions or those with limited resources, these findings suggest offering some research experience is still valuable and does positively impact a student’s time to graduation. Since even limited engagement in undergraduate research demonstrates positive outcomes, this also bodes well for supporting small programs with limited opportunities, funding, and time. Research Question 3: Post-graduation activities The third research question explored the relation between intensity of undergraduate research involvement and post-graduate experiences. Results indicated that Experienced research students were more likely to continue to post-graduate experiences than students who did not participate in URI research initiatives. These findings are consistent with previous studies, indicating that research involvement is highly and significantly correlated with higher degree aspirations (Baker & DeDonno, 2020; Bauer & Bennett, 2003; Kim & Sax, 2009; Zydney et al., 2002). While these correlations exist, there is still ongoing research related to causal factors. For example, according to Bowman and Holmes (2018), students who engage in research activities later in their college years may have already made the decision to attend graduate school, and as a result, have decided to participate in these activities to have a better chance of attaining acceptance to a graduate program. This later involvement may lead to a confirmation of a previously decided upon career decision (Seymour et al., 2004). Although it is difficult to know which came first, research involvement or interest in graduate school, participation in undergraduate research activities is positively related to graduate school attendance. While the current sample was matched on STEM/Non-STEM participation, a supplemental analysis was conducted to determine the relationship between major and post-graduation activities of the undergraduate researchers. An additional Chi-Square analysis found a significant relationship between STEM/Non-STEM students and post-graduate experiences (X2 (2, N= 344) = 7.82, p< 0.05). Post-hoc residual analysis with Bonferroni corrections indicated that STEM students were more likely to seek undergraduate, graduate, and professional post-graduation experiences than non-STEM students (p< 0.05), although Cramer’s V indicated the effect size was weak (φc = 0.15). In this analysis, graduate and professional pursuits were combined. Many students pursuing STEM majors often enter this field with intentions of going to professional programs such as medical or dental school. Perhaps students pursuing STEM disciplines come in with those career aspirations which could influence their interest in engagement in research. Since undergraduates involved in research tend to pursue graduate degrees at a higher rate (and even more so in STEM), centralized offices should consider offering graduate education professional development for undergraduates to support efforts of undergraduates who pursue external Research Experiences for Undergraduates (REUs) and introduce them to fellowship 25 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu opportunities like the National Science Foundation Graduate Research Fellowship Program (NSF GRFP) to further optimize their acceptance into graduate programs. Overall, these findings could be used to argue that students engaging in UGR gain cross-competency skills which positively impact their performance in other courses, leading to a higher GPA. However, more research would need to be done to verify this assumption. For example, a longitudinal study where direct measures of student learning outcomes are assessed throughout their educational career could yield which skills are most impactful to student success. Limitations and Delimitations Although a rigorous matching design eliminated many confounding factors in the analysis, there are several limitations of the study that should be noted for subsequent research and when generalizing results. These include potential control group contamination, the varied student experiences within the URI programs, and student motivation or career aspirations. The degree of control group contamination was not known since the degree and type of research experiences or other high impact practices these students had in their other coursework or co-curricular engagements could not be determined. Students in the Novice and Experienced groups had a known degree of research within RI courses and DIR by virtue of a formalized process of course designation. However, there may have been students who completed volunteer research experiences or research internships which would not have been recorded. This could become a factor in reducing significance between the groups, especially when sampling at a research institution. Future studies should attempt to clarify the purity of their control groups in the selection process via student surveys employed on a smaller control group. Students did not engage in equal numbers across the URI initiatives. The symposium and UGR grant experiences accounted for a large percentage of the participation (see Table 2). Part of this was due to these programs being offered in earlier years when compared to other programs which were launched later in the 2012-2019 sampling period. Additionally, research activities such as the institutional symposia, attract larger numbers of students due to the nature of the event. Typically, SURF was sought by faculty mentors for experienced undergraduates, which may account for their superior performance over Novice and Control students. While all these initiatives met the criteria for an intensive research experience, the effect of participation duration and type on the results is unknown. The operational definition of research intensity may need to be further refined. Future studies could delve deeper to analyze the types of URI experiences (i.e., academic versus summer experiences, grant participation versus publishing in a journal) and their specific impact on student success. The third hypothesis, which examined post-graduate experiences, found significant differences between Experienced and Control subjects. However, the causal connection is not clear. It is not certain if students are involved because of their prior intentions of attending graduate/professional post baccalaureate programs, or if the research experiences promoted or motivated their choice to pursue these aspirations. Future studies should consider measuring motivational choice within a pre-post experimental design; with this design, the effect of students’ initial intent could be isolated, and institutions can apply this information into improving their programs by targeting students not internally motivated to do research. Additionally, and as discussed previously, STEM students may have higher post-graduate expectations. Despite controlling for STEM/Non-STEM majors in the study, future studies should consider separating graduate and professional post-graduate activities to determine if differences exist. STEM students’ aspirations to attend professional programs, such as medical school, may affect the percentage of students going to graduate school. 26 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu Conclusions There continues to be growing interest and research on engagement of students in undergraduate research, yet few have studied the impact of “time” or “intensity” on student engagement in research. The primary purpose of this study was to evaluate the impact of the intensity of student engagement in undergraduate research on student success metrics. The metrics evaluated included student GPA at graduation, time to graduation and post-graduation activities. Research findings demonstrate that Experienced undergraduate research students (i.e., those who participated in more than two semesters of undergraduate research activities) had significantly higher GPAs at graduation when compared to Novice and control group students who were matched on first term GPA, STEM/Non-STEM related major, and number of transfer credits at the time of sampling. No difference was identified between STEM and Non-STEM groups. Additionally, our findings indicate that Experienced researchers are significantly more likely to progress to graduate school than either Novice research or non-research students. Experienced STEM research students are also significantly more likely to progress to graduate school compared to non-STEM counterparts. Finally, both Experienced and Novice researchers demonstrated a significantly lower time to graduation when compared to the control group. Acknowledgements The authors would like to acknowledge Ben Silva from Florida Atlantic University’s Institutional Effectiveness and Analysis office, faculty mentors, and the staff from Florida Atlantic University’s Office of Undergraduate Research and Inquiry. References Baker, T. N. (2017). The impact of undergraduate research participation on research self-efficacy. (Publication No. 10642999). [Doctoral dissertation. Florida Atlantic University]. PQDT Open- ProQuest. Baker, T. N. & DeDonno, M. A. (2020). The influence of research participation, field of study, and gender on research self-efficacy. The International Journal of Learning in Higher Education, 27(2), 11-26. doi:10.18848/2327-7955/CGP/v27i02/11-26. Barlow, A. E. L. & Villarejo, M. (2004). Making a difference for minorities: Evaluation of an educational enrichment program. Journal of Research in Science Teaching, 41(9), 861-881. doi: https://doi.org/10.1002/tea.20029. Baron, S. I., Brown, P., Cumming, T., & Mengeling, M. (2020). The impact of undergraduate research and student characteristics on student success metrics at an urban, minority serving, commuter, public institution. Journal of the Scholarship of Teaching and Learning, 20(1), 85-104. doi: 10.14434/josotl.v20i1.25423. Battaglia, S. J., Echegoyen, L. E., & Diaz-Martinez, L. A. (2022). Institution-wide analysis of academic outcomes associated with participation in UGR: Comparison of different research modalities at a Hispanic-serving institution. Scholarship and Practice of Undergraduate Research, 5(3), 8-24. Bauer, K. & Bennett, J. (2003). Alumni perceptions used to assess undergraduate research experience. Journal of Higher Education, 72(2), 210-230. https://doi.org/10.1080/00221546.2003.11777197. Beckman, M. & Hensel, N. (2009). Making explicit the implicit. Council on Undergraduate Research Quarterly, 29(4), 40-44. Bhattacharyya, P., Chan, C. W. M., & Waraczynski, M. (2018). How novice researchers see 27 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu themselves grow. International Journal for the Scholarship of Teaching & Learning, 12(2). doi: https://doi.org/10.20429/ijsotl.2018.120203. Bowman, N. A. & Holmes, J. M. (2018). Getting off to a good start? First-year undergraduate research experiences and student outcomes. Higher Education, 76, 17-33. doi: http://dx.doi.org/10.1007/s10734-017-0191-4. Brown, D. A., Wright, N. B., Gonzales, S. T., Weimer, N. E., & Soto, J. G. (2020). An undergraduate research approach that increased student success at a Hispanic-serving institution (HSI): The SURE program at Texas State University. Scholarship and Practice of Undergraduate Research, 4(1), 52-62. Carnegie Classification of Institutions of Higher Education (n.d.). Florida Atlantic University. Retrieved July 28, 2020, from http://carnegieclassifications.iu.edu/. Carpi, A., Ronan, D., Falconer, H., & Lents, N. (2017). Cultivating minority scientists: Undergraduate research increases self-efficacy and career ambitions for underrepresented students in STEM. Journal of Research in Science Teaching, 54(2), 169-194. doi: https://doi.org/10.1002/tea.21341. Chamely-Wiik, D. (2013). Distinction through discovery: Quality enhancement plan. Florida Atlantic University. Retrieved from https://www.fau.edu/ouri/documents/quality- enhancement-plan-1-11-13.pdf. Chamely-Wiik, D., Holman, M., Kirsch, P., Meeroff, D. & Peluso, J. (2014). Scaffolding the development of student research skills for capstone experiences: A multi-disciplinary approach. Council on Undergraduate Research Quarterly, 34(4), 18-25 Chamely-Wiik D. & Usher, B. (2017). Research-intensive course designation. Council on Undergraduate Research Quarterly, 37(4), 13-15. Collins, T. W., Grineski, S. E., Shenberger, J., Morales, X., Morera, O. F., & Echegoyen, L. E. (2017). Undergraduate research participation is associated with improved student outcomes at a hispanic-serving institution. Journal of College Student Development, 58(4), 583-600. doi: 10.1353/csd.2017.0044. Council on Undergraduate Research (CUR). n.d.a “Mission and Vision.” Accessed April 21, 2021. https://www.cur.org/who/organization/mission_and_vision/. Craney, C., McKay, T., Mazzeo, A., Morris, J., Prigodich, C., & de Groot R. (2011). Cross- discipline perceptions of the undergraduate research experience. The Journal of Higher Education 82(1), 92–113. doi: 10.1353/jhe.2011.0000. Follmer, D. J., Zappe, S., Gomez, E., & Kumar, M. (2018). Student outcomes from undergraduate research programs: Comparing models of Research Experiences for Undergraduates (REUs). Scholarship and Practice of Undergraduate Research, 1(1), 20-27. Frederick, A., Grineski, S., Collins, T. W., Daniels, H.A., & Morales, D.X. (2021). The emerging STEM paths and science identities of Hispanic/Latinx college students: Examining the impact of multiple undergraduate research experiences. Ed. Stanley M. Lo. CBE—Life Sciences Education 20(2), ar18. doi: 10.1187/CBE.20-08-0191. Gilmore, J., Vieyra, M., Timmerman, B., Feldon, D., & Maher, M. (2015). The relationship between undergraduate research participation and subsequent research performance of early career STEM graduate students. The Journal of Higher Education, 86(6), 834-863. doi: https://doi.org/10.1080/00221546.2015.11777386. Haeger, H., Banks, J. E., Smith, C., & Armstrong-Land, M. (2020). What we know and what we need to know about undergraduate research. Scholarship and Practice of Undergraduate Research, 3(4), 62-69. Hathaway, R., Nagda, B., & Gregerman, S. (2002). The relationship of undergraduate research participation to graduate and professional education pursuit: An empirical study. 28 https://www.fau.edu/ouri/documents/quality-enhancement-plan-1-11-13.pdf https://www.fau.edu/ouri/documents/quality-enhancement-plan-1-11-13.pdf Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu Journal of College Student Development, 43(5), 614-631. Jones, M. T., Barlow, A. E. L., & Villarejo, M. (2010). Importance of undergraduate research for minority persistence and achievement in biology. The Journal of Higher Education, 81(1), 82-115. doi: 10.1353/jhe.0.0082. Kardash, C. (2000). Evaluation of an undergraduate research experience: Perceptions of undergraduate interns and their faculty mentors. Journal of Educational Psychology, 92(1), 191-201.doi: https://doi.org/10.1037/0022-0663.92.1.191. Kim, Y. K., & Sax, L. J. (2009). Student-faculty interaction in research universities: Differences by student gender, race, social class, and first-generation status. Research Higher Education, 50, 437-459. doi: 10.1007/s11162-009-9127-x. Kuh, G. D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter. Washington DC: Association of American Colleges and Universities. McDevitt, A. L., Patel, M. V., Rose, B., & Ellison, A. M. (2016). Insights into student gains from undergraduate research using pre- and post-assessments. Bioscience, 66(12), 1070-1078.doi: https://doi.org/10.1093/biosci/biw141. Monarrez, A., Morales, D., Echegoyen, L. E., Seira, D., & Wagler, A. E. (2020). The Moderating Effect of Faculty Mentorship on Undergraduate Students' Summer Research Outcomes. CBE life sciences education, 19(4), ar56. https://doi.org/10.1187/cbe.20-04-0081 Nagda, B., Gregerman, S., Jonides, J., von Hippel, W., & Lerner, S. (1998). Undergraduate student-faculty research partnerships affect student retention. The Review of Higher Education, 22(1), 55-72. National Science Foundation (2017). Research experiences for undergraduates (REU) sites and supplements: Program solicitation. Retrieved from https://www.nsf.gov/pubs/2005/nsf05592/nsf05592.htm Pender, M., Marcotte, D. E., Domingo, M. R., & Maton, K. I. (2010). The STEM pipeline: The role of summer research experience in minority students’ Ph.D. aspirations. Educational Policy Analysis Archives, 18(30), 1-36. Russell, S., Hancock, M., & McCullough, J. (2007). Benefits of undergraduate research. Science, 316, 548-549. doi: 10.1126/science.1140384. Schneider, K., Kuperman, A., Watt, A., Barulich, D., & Campbell, T. (2021). Tracking and assessing undergraduate research campus-wide: Demographics, academic success, and postgraduation plans. Journal of the Scholarship of Teaching and Learning, 21(1), 107-119. Sell, A. J., Naginey, A., & Stanton, C.A. (2018). The Impact of Undergraduate Research on Academic Success. Scholarship and Practice of Undergraduate Research, 1(3), 19-29. Seymour, E., Hunter, A. B., Laursen, S. L., & DeAntoni, T. (2004). Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three‐year study. Science education, 88(4), 493-534. doi: https://doi.org/10.1002/sce.10131. Simmons, Kiyoko N. (2018). Effect of undergraduate research programs on retention of Hispanic students. https://digitalrepository.unm.edu/oils_etds/50 Thiry, H., Weston, T. J., Laursen, S. L., & Hunter, A. (2012). The benefits of multi-year research experiences: Differences in novice and experienced students’ reported gains from Undergraduate Research. CBE-Life Sciences Education, 11, 260-272. doi: 10.1187/cbe.11-11-0098. Whittinghill, J. C., Slovacek, S. P., Flenoury, L. P., & Miu, V. (2019). A 10-Year Study on the Efficacy of Biomedical Research Support Programs at a Public University. Scholarship and Practice of Undergraduate Research, 3(1), 30-38. 29 Chamely-Wiik, Ambrosio, Baker, Ghannes, and Soberon Journal of the Scholarship of Teaching and Learning, Vol. 23, No. 1, April 2023. josotl.indiana.edu Wolkow, T. D., Jenkins, J., Durrenberger, L., Swanson-Hoyle, K., & Hines, L. M. (2019). One early course-based undergraduate research experience produces sustainable knowledge gains, but only transient perception gains. Journal of Microbiology & Biology Education, 20(2). doi: 10.1128/jmbe.v20i2.1679. Zydney, A., Bennett, J., Shahid, A., & Bauer, K. (2002). Impact of undergraduate research experience in engineering, Journal of Engineering Education, 91(2), 151-157. doi: 10.1002/j.2168-9830.2002.tb00687.x 30