NODA Journal 22_1.indd FALL 2014 • VOLUME 22, NUMBER 1 41 The Ronald E. McNair Postbaccalaureate Achievement Program at Cornell University: Survey Results and the Push for Qualitative Research Tremayne O. Waller, Bobby J. Smith, Enongo A. Lumumba-Kasongo, and Daniel J. Lupa There are very few literature reports that have thoroughly defined and evaluated the undergraduate research experience. In response to this scholarly void, we have carried out a study of students’ experience in The Ronald E. McNair Postbaccalaureate Achievement Program at Cornell University. Data were obtained from 17 students who completed a 42-question Qualtrics survey designed to elicit information in three areas: 1) the specific programmatic characteristics that promote an undergraduate’s ability and motivation to conduct research, 2) how the McNair Program could be improved to provide more varied and meaningful research opportunities, and 3) the extent to which their experiences increased their preparedness for graduate school. The data reveal the importance of using qualitative methods to investigate the experiences of participants. Introduction Strategies designed to promote equitable participation in graduate degree programs are not new. Most universities, in fact, can claim several strategies specifically developed to increase the participation of low-income, first-generation, and historically underrepresented students who wish to pursue a graduate degree—or indeed, even consider it as an option. At Cornell University, The Ronald E. McNair Postbaccalaureate Achievement Program (hereafter referred to as the McNair Program) was funded to expand the post graduate opportunities for undergraduate students from these “low participation” categories who might not otherwise consider the possibility of graduate school. Although this multi- CAMPUS NOTES Tremayne O. Waller (tow6@cornell.edu) is the Lecturer and Director of the McNair Scholars Program at Cornell University Bobby J. Smith, II is a Doctoral Student in the Charles H. Dyson School of Applied Economics and Management at Cornell University Enongo A. Lumumba-Kasongo is a Doctoral Candidate in the Department of Science and Technology Studies at Cornell University Daniel J. Lupa is a Counselor for the McNair Scholars Program at Cornell University 42 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION campus federal program was founded more than 25 years ago, there is relatively little research on The McNair Program, and the few studies that do exist can be organized into two categories: studies that describe the general nature and characteristics of McNair programs (Williams, 2004) and a series of targeted reports that provide aggregate data about McNair projects in various parts of the nation (Mansfield, Sargent, & Cahlan, 2002). In an attempt to remedy this research deficit, this mixed-method study explores the experiences and satisfaction levels of 17 students who took part in the McNair Program at Cornell University during the summer of 2013. Specifically, we focused on three components: 1) the specific programmatic characteristics that promote an undergraduate’s ability and motivation to conduct research, 2) how the McNair Program could be improved to provide more varied and meaningful research opportunities, and 3) the extent to which their experiences increased their preparedness for graduate school. The results of this study are expected to be useful for researchers and practitioners currently affiliated with McNair programs, as well as other educators seeking to improve the odds for students who might not otherwise consider themselves capable of earning an advanced degree. The first section of the paper provides a history of The Ronald E. McNair Postbaccalaureate Achievement Program, followed by details about Cornell’s McNair Program. This study then reviews the relevant literature, which is intended to provide a broader context for understanding undergraduate and alumni experiences in McNair Programs. The third section of this paper features methodological details about the survey, including a description of the survey and resulting data. In the final section, we discuss the strengths, limitations, and implications of our findings. History of The Ronald E. McNair Postbaccalaureate Achievement Program The McNair Scholars Program is one of eight federal TRIO programs, which, according to the U.S. Department of Education (US-DOE) website, are “Federal outreach and student services programs designed to identify and provide services for individuals from disadvantaged backgrounds” (U.S. Department of Education, 2014). TRIO programs currently exist at many institutions across the United States and Puerto Rico (McCoy, Wilkinson, & Jackson, 2008). The US-DOE created the Ronald E. McNair Postbaccalaureate Achievement Program (The McNair Program) as a TRIO program in 1986 in order to address the conspicuous lack of ethnic and racial diversity in our nation’s Ph.D. programs and faculty pipelines. In particular, the Department of Education identified undergraduate research and mentoring as critical components of its core strategy for readying underrepresented students for advanced degree programs (McCoy et al., 2008). Thus, the McNair Program was established to encourage and guide undergraduate students—many from economically disadvantaged backgrounds—toward graduate education via a variety of scholastically enriching services and activities designed to prepare them for doctoral studies. Although specific program offerings vary from campus to FALL 2014 • VOLUME 22, NUMBER 1 43 campus (there are currently 152 McNair Programs at the nation’s institutions, with aggregate funding exceeding more than $34M), a McNair scholar will typically have opportunities to engage in academic counseling and mentoring, conferences and workshops, seminars, summer internships, test preparation, tutoring, and financial aid counseling. While it is true that the program is designed for college students from disadvantaged circumstances, they must be able to demonstrate strong academic standing and drive in order to be considered for the McNair Program at Cornell. Once the students are admitted, administrators and faculty associated with the program work diligently to help them meet their college requirements through a variety of strategies, as well as work with them to identify and apply to appropriate graduate programs. A McNair scholar is also tracked after receiving an undergraduate degree to ensure successful completion of an advanced degree. With the program now 25-plus years old, a number of studies have examined the efficacy of the program—both at specific campuses and also more broadly (Beal, 2008; Crowe & Brakke, 2008; Parker, 2003; Williams, 2004). Such an evaluation has not, however, been carried out at Cornell University, thus providing an impetus for potential research described herein. The McNair Scholars Program at Cornell University Cornell University’s Ronald E. McNair Postbaccalaureate Achievement Program was designed in 2011 and implemented in fall 2012, coinciding with the establishment of a new Office of Academic Diversity Initiatives (OADI). Recognizing that Cornell was home to a significant number of talented graduate students who were McNair scholars as undergraduates at other institutions, faculty and administrators felt that the university should contribute to filling the pipeline of qualified undergrads in addition to benefiting from it. And given Cornell’s longstanding commitment to undergraduate research programs across the disciplines, the central tenet of the McNair Program could be well supported by existing structures and committed faculty. As is well known, undergraduate research opportunities are both effective tools for recruiting and retaining students and helping students develop scientific research skills (Elrod, Husic, & Kinzie, 2010). Recruiting the annual cohort of approximately two dozen McNair scholars is conducted on an ongoing basis by program staff, typically in collaboration with faculty, staff, and administrators who assist in identifying highly qualified candidates. Additionally, students can self-nominate for the McNair Program. Potential candidates are required to complete a lengthy written application and undergo a rigorous interview prior to being selected for the program. Those chosen for the program participate in a formal induction ceremony, featuring a keynote speaker, which serves to both emphasize the gravitas of the program, as well as create an initial bonding experience for scholars and the program support team. The first McNair summer research experience at Cornell University in 2013 was a 10-week session designed to engage participants in their field of study. There were 44 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION four juniors and 13 sophomores that participated in the summer research program at Cornell. Research experiences have become somewhat ubiquitous on college campuses across the country, and for good reason. When designed and executed well, summer programs can be critical in enhancing a student’s likelihood for academic success (Williams, 2004). At Cornell, the McNair Program emphasizes a hands-on, mentored research experience for each undergraduate scholar—ranging from intensive laboratory projects to social science projects. Regardless of topical area, McNair scholars become immersed in their research experiences; they work side-by-side with faculty, postdoctoral fellows, and graduate students, who serve as role models and mentors. In an environment that stresses collaboration and a multidisciplinary approach to problem solving, the students discover how research is formulated and carried out. We encourage participants to report their research in refereed journals and at meetings and workshops. It is primarily through this research component that McNair scholars acquire skills critical to success in research and teaching careers and, most importantly, to gain entry into the top graduate programs in the country. The McNair Program: Review of Available Literature As noted earlier, there are relatively few empirical studies, including longitudinal ones, which have thoroughly evaluated the McNair Programs. This research void is somewhat surprising, given the fact that the federal government, universities, and college administrators have been investing substantial funds and human capital toward maintaining the program for more than 25 years. In contrast, research is not lacking as to the critical importance of intensive mentoring and hands-on research opportunities for underrepresented, first-generation, and low-income undergraduates who major in STEM fields; science, technology, engineering, and math (Jones, Barlow, & Villarejo, 2010; Maton, Domingo, Stolle- McAllister, Zimmerman, & Hrabowski, 2009). One available McNair Program assessment, although now 20 years old, was conducted at Rutgers University. Thomas (1994) reported that 88% of McNair scholars at that institution eventually entered graduate programs. In contrast, “On average, close to 27 percent of college grads enrolled in a master’s program within one year of graduation, according to alumni graduate school data reported by 377 ranked colleges and universities in the survey (Sheehy, 2013). In terms of linkages between program participation and significantly higher-than-average graduate school enrollment among McNair students, Thomas cited three factors: 1) the importance of supervised research with a faculty member who directly interacted with the student, 2) ongoing mentoring opportunities with an experienced faculty member, and 3) the availability of a skilled academic advisor who could familiarize the student with graduate school opportunities and assist with the application process—especially with applying for financial aid. As described earlier, these three program elements represent essential components of the McNair Program. A more recent study used the perspectives of McNair Program directors and student alumni to assess program efficacy (Nnadozie, Ishiyama, & Chon, 2001). FALL 2014 • VOLUME 22, NUMBER 1 45 The researchers examined the relationship between the thoroughness of program requirements (e.g., high pre-admission standards, research seminar attendance, and the hands-on research component) and a “successful” graduate school experience. Success for this study was defined as applying to and gaining acceptance into graduate school, obtaining sufficient financial resources to make a graduate degree possible (e.g., scholarships and fellowships), and completing the graduate degree. Nnadozie et al. (2001) stressed the importance of exposing students as early as possible to mentored research opportunities, which ideally persist throughout the student’s undergraduate career. An important point that came across in this study was that McNair scholars uniformly spoke about the criticality of high expectations—both from program administrators and their faculty mentors. They emphasized that high expectations prepared them for graduate school. Survey Results from Survey The main purpose of this section is to present survey results based on student learning outcomes and experiences in the Ronald E. McNair Program at Cornell University. With the recent “matriculation” of the first cohort of 17 students from the Ronald E. McNair Postbaccalaureate Achievement Summer Program (August 2013), program administrators deemed it essential to canvas their opinions on the program so that adjustments could be made as needed. Therefore, the director, in collaboration with participating faculty and staff using Cornell University’s Qualtrics web survey software, developed a 42-question online survey. The survey was subsequently approved by Cornell’s Office of Research Integrity and Assurance Institutional Review Board for Human Participants. The survey relied heavily on the University of Colorado-Boulder’s Undergraduate Research Student Self- Assessment (URSSA)1 and many of the URSSA questions were replicated or edited for our survey at Cornell. The survey, which included basic demographic questions, was designed to assess student perceptions of the summer program in three areas: • Ability and motivation to conduct research • Cultivation of future research agenda through research skills learned • Graduate school preparedness The response format of the questions was either closed choice (e.g., check all that apply, yes/no, and fill-in), or respondents were asked to select from a list of potential answers on a 5-point Likert-type scale. The online survey was sent to all 17 McNair Scholars, all of whom knew that they would be asked to complete a survey at the end of the 10-week program. The survey response rate was 100%. Survey Results The demographic results are as follows: In terms of gender, 14 of the McNair 1URSSA is an online survey instrument for use in evaluating student outcomes of undergraduate research experiences in the sciences. The publicly available evaluation instrument is free and can be used by anyone. http://www.colorado.edu/eer/research/undergradtools.html 46 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION Scholars were female (~78%), and 4 were male (~22%). The ethnic and racial profile of responders included 34% Hispanic, 22% African American/Black, and 22% Asian Pacific Islanders. At the time they completed the survey, all the students were classified as rising juniors and seniors. We also requested information about their academic majors. More than 50% were majoring in some aspect of the biological sciences, and 22% had declared engineering as their major. Also important to note is that over 50% of respondents had completed two summers of research experience prior to participating in the McNair Program. To understand the primary motivations behind the students’ decision to engage in mentored research, we provided a list of potential reasons for choosing to participate in a research program and then asked each student to identify their top three choices. The top three reasons were • I wanted to gain hands-on experience in research (88%) • I wanted to do research to clarify which field I wanted to study (53%) • I wanted to do research to enhance my application to graduate or medical school (53%) The survey also included an open-ended question about how the McNair research experience had impacted their future educational interests and goals. In total, 94% of participants believed that their research experience had cemented their interest in their field of study, with some stating that it had clarified specific areas of interest. Moreover, a significant percentage (88%) reported that they felt ready to pursue graduate education as a result of having conducted summer research as a McNair scholar. Using a Likert-type scale, students were also asked to rank on a scale from “no gain” to “great gain” the skills they had learned in thinking and working like a researcher during their mentored research experience. Table 1 lists the top-ranked gains as reported by the McNair scholars. TABLE 1 Top Reported Gains in Thinking and Working Like a Researcher/Related to research work Understanding the theory and concepts guiding my research project 82% Confidence in discussing concepts with my research faculty member 76% Explaining my project to people outside my field 65% Improved ability to understand journal articles 59% As illustrated in Table 1, a significant proportion of students (82%) agreed that their comprehension of the theoretical underpinnings for their research project had improved as a result of conducting hands-on research with a skilled mentor. A near equal percentage (76%) was more confident in discussing concepts with their assigned research faculty member. Clearly, the skills listed in Table 1 are FALL 2014 • VOLUME 22, NUMBER 1 47 all essential for a successful graduate school experience, regardless of the specific academic program. Strengths, Limitations and Implications for Future Research Strengths This paper’s primary strength lies in its solicitation of student perspectives to evaluate the effectiveness of Cornell University’s McNair program. Over the past ten years, researchers evaluating the McNair Program have failed to incorporate student perspectives in their assessments (Williams, 2004; Keopuhiwa, 2012). Restad’s (2013) article, “Beyond the Program Year: Graduates Students’ Understanding of How McNair Scholars Program Participation Impacts Their Experiences in Graduate School,” she writes, “Typically, McNair program evaluations emphasize the collection and analysis of quantitative data—e.g., academic performance and degree attainment; however, little qualitative research has been conducted on graduates’ perceptions of the impact of program participation on graduate school adjustment and success” (Restad, 2013, p. 2). Although the present study does not target the perspectives of McNair program graduates, a similar argument can be made about the lack of qualitative findings on current McNair scholars. Since 2002, the U.S. Department of Education has published four program profiles on the McNair Program, each of which is based on data from annual performance reports (APR) submitted by the grantee institutions (U.S. Department of Education, 2011). And while APRs are required to detail each institution’s “project-level activities and goals and participant demographics and academic progress,” they do not provide specific insights into the perspectives of student participants (U.S. Department of Education, 2005, p. 1). This study, therefore, builds on the work of scholars like Restad, among others, to incorporate student voices as a tool for evaluating the effectiveness of the McNair Program. Limitations There are two principal limitations that must be considered when interpreting the results of this survey. First, this study relies solely on quantitative data.* As Restad (2013) argues, a focus on quantitative analysis often limits our full understanding of the student experience. The closed-choice format of the survey questions, for example, resulted in participants being unable to articulate in their own words their reasons for pursuing research. Thus, future efforts must be made toward supplementing quantitative survey data with qualitative analysis. A second limitation is that responses were not submitted anonymously. The anonymity of the participants was intentionally de-prioritized in order to ensure * The full survey included open-ended questions, but those questions and responses have not been included in subsequent data generation and analysis 48 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION that all students completed the survey in a timely manner. Thus, since participants were aware that demographic responses were included in their survey responses, they may have felt less inclined to answer the questions in ways that reflected any negative perceptions of the program. Implications for Future Research This paper opens up many potential areas for additional research and gestures toward new methodological approaches for analyzing summer research programs. As has already been suggested, extensive qualitative research is needed to further explore students’ perceptions of the McNair Program’s ability to prepare them for research-based graduate school trajectories. Additionally, this study provides a framework for conducting a longitudinal study that addresses students’ perspectives as they tackle the rigors of graduate education. Rather than focusing on institutional reporting or postbaccalaureate evaluations, as prior researchers have done (Greene, 2007; Willison & Gibson, 2011), this study can reinforce the importance of mining the individualized perspectives of current McNair scholars from year to year. The importance of this kind of data cannot be overstated since it can be used to implement modifications that create overlap between student expectations and program offerings. More broadly, this research may be used as a tool for targeting future applicants. According to our survey data, over 50% of the respondents had two summers of research experience prior to participating in the summer McNair Program. Clearly, further research is warranted to determine if there is a correlation between the perceptions of students with and without prior research experience with respect to the types of gains afforded by being a McNair scholar. If, for example, the students with prior research experience are reporting minimal gains, administrators need to reevaluate their acceptance criterion to ensure that students with the greatest need are being served. In summary, the data generated from this study strongly suggest that the students in the Ronald E. McNair Postbaccalaureate Achievement Summer Program at Cornell University credit their participation with developing skills that are necessary for thinking and working like a researcher. We encourage further research both at Cornell University and other higher education institutions as it relates to the efficacy of the McNair Program. References Bergeron, W. M., Sargent, K. D., Cahalan, M., Belle Jr, R. L., Mansfield, W., Sargent, K. D., & Bergeron, F. (2002). A Profile of the Ronald E. McNair Postbaccalaureate Achievement Program: 1998-99. With Selected Data from 1997- 98 and 1996-97. Washington, DC: US Department of Education. Mathematica Policy Research. FALL 2014 • VOLUME 22, NUMBER 1 49 Beal, R. (2008). “You mean they’ll pay me to think?”: How low-income, first- generation and underrepresented minority McNair students construct an academic identity as scholars. Doctoral dissertation. Retrieved from ProQuest Dissertations and Theses database. Publication No. AAT 3273744. Crowe, M., & Brakke, D. (2008, Summer). Assessing the impact of undergraduate research experiences: An overview of the current literature. Council on Undergraduate Research, 20(4), 43–50. Elrod, S., Husic, D., & Kinzie, J. (2010). Research and discovery across the curriculum. Peer Review, 12(2), 4-8. Greene, K. V. (2007). Alumni perceptions of the McNair scholars program at Kansas universities. Manhattan, KS: Kansas State University. Jones, M. T., Barlow, A.E., & Villarejo, M. 2010. “Importance of undergraduate research for minority persistence and achievement in biology.” Journal of Higher Education 81: 82-115. Keopuhiwa, T. N. (2012). Under the surface: An examination of voice, space, and identity in West Virginia University McNair Scholars. Morgantown, WV: West Virginia University Libraries. Maton, K. I., Domingo, M. R., Stolle-McAllister, K. Zimmerman, J.L., & Hrabowski, F.A. III (2009). Enhancing the number of African Americans who pursue STEM Ph.D.s: Meyerhoff Scholarship Program outcomes, processes, and individual predictors. Journal of Women and Minorities in Science and Engineering, 15, 15-37. McCoy, A., Wilkinson, A., & Jackson, R. (2008). Education and employment outcomes of the Ronald E. McNair Postbaccalaureate Achievement Program alumni. Washington, DC: US Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service. Nnadozie, E., Ishiyama, J., & Chon, J. (2001). Undergraduate research internships and graduate school success. Journal of College Student Development, 42(2), 145–156. Parker, K. D. (2003). Achieving diversity in graduate education: Impact of the Ronald E. McNair Postbaccalaureate Achievement Program. The Negro Educational Review, 54(1–2), 47–50. Resources. In Ronald E. McNair Postbaccalaureate Achievement Program. Retrieved January 11, 2013, from http://www2.ed.gov/programs/triomcnair/resources.html Restad, C. (2013). Beyond the program year: Graduates students’ understanding of how McNair Scholars Program participation impacts their experiences in graduate school. Mcnair Scholars Online Journal, 7(1), Article 16, doi: 10.15760/mcnair.2013.88 Sheehy, Kelsey. (2013). 10 colleges that lead to graduate school. U.S. News and World Report. Retrieved from http://www.usnews.com/ Thomas, E. P. (1994). Taking the first steps toward graduate education: A report on the Ronald E. McNair Postbaccalaureate Achievement Program. (Report No. HE032805). Rutgers, The State University of New Jersey. (ERIC Document Reproduction Service No. ED440588). 50 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION U.S. Department of Education. (February 17, 2011). Programs: Ronald E. McNair Postbaccalaureate Achievement Program. U.S. Department of Education. Retrieved from http://www2.ed.gov/ U.S. Department of Education. (November 24, 2014). Federal TRIO programs— home page. U.S. Department of Education. Retrieved from http://www2.ed.gov/ U.S. Department of Education, Office of Postsecondary Education. (2005). A profile of the Ronald E. McNair Postbaccalaureate Achievement Program: 1997–1998 through 2001–2002. Retrieved from http://www2.ed.gov/programs/triomcnair/ mcnairprofile1997-2002.pdf Williams, E. G. (2004). Academic, research, and social self-efficacy among African American pre-McNair scholar participants and African American post-McNair scholar participants. Blacksburg, VA: University Libraries, Virginia Polytechnic Institute and State University. Willison, S., & Gibson, E. (January 01, 2011). Graduate school learning curves: McNair Scholars’ Postbaccalaureate Transitions. Equity & Excellence in Education, 44(2), 153-168.