NODA Journal 20_2.indd 78 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION The Impact of a Camp Orientation Program on First-Year Academic Engagement and Persistence R. Michael Haynes and T. Wayne Atchley More and more, institutions of higher education are being held accountable for student success measures such as persistence and completion rates. If research indicates that higher levels of student engagement lead to better academic outcomes, it is then reasonable to equate this to improved retention and graduation rates. The present study presents findings associated with a camp orientation program for first-year students and its impact on levels of academic engagement and persistence. Data collected from the Beginning College Survey of Student Engagement (BCSSE) and National Survey of Student Engagement (NSSE) were used to determine if statistically significant differences exist between camp participants and non-participants. Findings indicate no statistically significant difference between camp participants and non-participants on NSSE benchmarks related to academic satisfaction, enriching educational experiences, student-faculty engagement or persistence from fall to spring of the first college year. Astin’s theory of student involvement posits that academic performance and individual development is correlated with the quality and quantity of student engagement (as cited in Pascarella & Terenzini, 2005). Carini, Kuh, and Klein’s (2006) work supported this notion, stating that even lower-ability students demonstrated higher academic outcomes than their classmates as the result of becoming engaged in the college environment; therefore, it should be reasonable to expect that first-year experience (FYE) initiatives, with the goal of increasing student engagement, could result in higher levels of academic engagement and satisfaction. This is particularly salient as more and more institutions are being held accountable in the form of student success outcomes, such as persistence and completion rates. While freshman orientation and first-year seminars have been the interest of the academy for many decades (Crawford, 1926; Gerber, 1970; Guthrie, 1953; FEATURED ARTICLE R. Michael Haynes (rhaynes@tarleton.edu) is the Executive Director of Institutional Research in the Office of Institutional Research and an Assistant Professor of Educational Leadership and Policy Studies at Tarleton State University. T. Wayne Atchley is an Assistant Professor and Graduate Program Coordinator in the Department of Agricultural and Consumer Sciences at Tarleton State University. Correspondence concerning this article should be addressed to R. Michael Haynes. SPRING 2013 • VOLUME 20, NUMBER 2 79 Harriman, 1925; Kronovet, 1969), and there is research related to the impact of these programs on persistence and learning outcomes (Pascarella, Terenzini, & Wolfle, 1986; Porter & Swing, 2006), little is known about the impact of other FYE initiatives, such as freshmen camps. This research will attempt to determine the impact on first-year students’ levels of academic engagement, performance, and persistence by participating in a voluntary, off-campus orientation/leadership camp at a small, regional, public, comprehensive university located in the southwest United States. The host institution is comprised of five colleges and offers 63 undergraduate programs, 22 graduate programs, one doctorate degree, and two associate degrees. Females slightly outnumber the males, and the majority of students are White and first- generation college students. The camp for first-year students is conducted the month prior to the commencement of the fall semester at a location approximately 100 miles from campus. Desired engagement outcomes for the attendees include team and spirit building, establishing relationships with other first-year students, and developing a deeper understanding of academic expectations. Collectively, the holistic purpose of the camp is to place nascent students in a setting away from campus distractions where individual and institutional bonding and commitments can be fostered. The problem this quantitative study addresses is that this summer off-campus program is an expensive proposition for the institution and first-year students alike, while the value of participating in relationship to engagement, academic performance, and the direct impact on retention is unknown. This is particularly problematic considering approximately 80% of the host institution’s student body receives some form of financial assistance, and the camp cost of $140 could be burdensome. McCandless and Barker (1969) reported on a 1951 Texas A&M initiative that provided summer-enrolling freshman an off-campus opportunity to experience the academic rigor of college through enrollment in courses for remediation. Their findings suggested that participation had little impact on subsequent academic performance, but did contribute to increased persistence and graduation rates. Bell and Holmes (Brent Bell, personal communication, April, 2010) conducted a review of articles, theses, and published studies to determine the amount and scope of research related to outdoor orientation activities. Their research revealed that only 53 studies on outdoor orientation programs have been conducted since 1971, and the majority of these were “wilderness” based camps; therefore, the need for this study is evident, not only for institutional improvement, but also as a contribution to the limited extant research. The first aim of this research was to determine if participation in a voluntary off-campus orientation program resulted in greater levels of academic engagement, satisfaction, and performance among first-year students. The second purpose of this study was to determine if program participants persisted from fall to spring at higher levels than non-participants. Identifying a correlation between first-year interventions, such as an off-campus orientation camp and academic engagement and performance, can be critical in not only improving first-to-second- year retention, but also in attending to the unique needs of first-generation and 80 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION underrepresented students. Methodology The research protocol, data collection instruments, and accompanying informative student documents for this quantitative study were approved by the host institution’s Institutional Review Board. A causal comparative research design was selected as first-year students were administered a pre-test in the form of the Beginning College Survey of Student Engagement (BCSSE) during their summer orientation period, allowed to optionally participate in the off-campus orientation camp, then asked to complete a post-test in the form of the National Survey of Student Engagement (NSSE). When random assignment is not possible due to the optional nature of the treatment or when encountering intact groups, a causal comparative design is preferable, as it allows for the inference of limited causation (Gall, Gall, & Borg, 2003). Description of the Population and Sample The population for this study was all first-time-in-college (FTIC) students with less than 30 semester credit hours enrolling at the host institution for the fall of 2010 (N=1,468). These parameters were established in an attempt to only study students who would be classified as new to college and as a freshman. Demographics of the population are presented in Table 1. This population was then contracted to represent only those students who completed the BCSSE pre-test during summer orientation, persisted from fall 2010 to spring 2011, then completed the NSSE post-test during the spring semester of their first-year (n=539). The population was then divided based on participation or non-participation in the off-campus camp and matched on individual High School Academic Engagement scale scores from the BCSSE to control for pre- college differences. This systematic filtration resulted in the final sample for analysis (n=216). Survey Instruments Two instruments developed and administered by the Indiana University Center for Postsecondary Research were used to collect student level data for this research. The Beginning College Survey of Student Engagement (BCSSE) served as a pre- test and was administered to first-year students in the summer prior to their fall enrollment. First administered in 2007, the instrument collects first-year students’ perceptions on six scales related to their high school academic engagement and academic expectations for their nascent first-year of college. For the current research, the respondents’ means score on the High School Academic Engagement scale was used as a matching variable to assist in controlling for pre-college academic engagement differences between the control and experiment groups. SPRING 2013 • VOLUME 20, NUMBER 2 81 TABLE 1 Frequency Distribution of Fall 2012 First Time in College Students by Gender and Ethnicity (N = 1,468) Female Male Ethnicity f % cum % f % cum % American Indian 5 0.57 6 1.01 Asian 5 0.57 1.14 4 0.67 1.68 African-American 37 4.23 5.38 42 7.07 8.75 Hispanic 99 11.33 16.70 78 13.13 21.89 International 5 0.57 17.28 1 0.17 22.05 Not Reported 2 0.23 17.51 1 0.17 22.22 Pacific Islander 1 0.11 17.62 1 0.17 22.39 Two or More Races 18 2.06 19.68 7 1.18 23.57 White 702 80.32 100.00 454 76.43 100.00 Total 874 594 Note. Ethnicity based on Texas Higher Education Coordinating Board race/ethnicity reporting categories. The National Survey of Student Engagement (NSSE) served as the post-test and was administered in first-year students’ spring semester. The NSSE collects students’ perceptions related to first-year levels of engagement in five benchmarks: Level of Academic Challenge, Active and Collaborative Learning, Student Faculty Interactions, Enriching Educational Experiences, and Supportive Campus Environment. The present study utilized all benchmarks as dependent variables with the exception of Supportive Campus Environment. While Indiana University Center for Postsecondary Research reports no reliability information related to the BCSSE, they do present evidence that supports moderate levels of concurrent validity between the BCSSE and NSSE. Published reports indicate acceptable levels of reliability on NSSE scales with Cronbach’s alpha coefficients of .70 or greater on all scales as well as evidence of content validity, construct validity, and predictive validity (National Survey of Student Engagement, 2012a). Data Collection The host institution’s First Year Experience office coordinated the distribution of 1,350 paper BCSSE surveys in the informational packets of students attending 82 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION each of the eight new student orientations during the summer of 2010. While the population consisted of 1,468 FTIC students, only 1,350 BCSSE surveys were ordered based upon spring 2010 matriculation projections for this group. The sampling technique was purposeful and one of convenience as BCSSE instruments were distributed to the first 1,350 orientation attendees. In addition to the survey, the packet included an informed consent form and a letter from the Vice President of Student Life explaining the purpose of the study. Students were asked to return their completed survey during their academic advising period at the conclusion of the orientation session. Of the 1,350 BCSSE surveys distributed, 1,086 were returned, yielding a participation rate of 80.4%. The paper BCSSE surveys were returned to the NSSE offices at Indiana University for processing during September 2010. The post-test, in the form of the NSSE, was administered in the spring of 2011 to those students who completed the BCSSE and persisted from the fall semester. Self-reported institutional identification numbers from the completed BCSSEs were used by the host institution’s Office of Institutional Research to identify these students. Of the 1,086 first-year students who completed the BCSSE during orientation, 884, or 81.4% persisted and became the target population for the spring NSSE administration. The host institution’s Residential Living and Learning (RLL) personnel administered the paper NSSE to on-campus students and personnel from the Recreational Sports Center, and Greek Life offices coordinated the administration to off-campus students. A review of RLL records indicated that of the 884 participant sub-population, 712 were on-campus residents with the remaining 172 residing off-campus. Residential Living and Learning personnel distributed paper NSSE surveys, including a follow-up informed consent, to the target population residing on- campus over a four week period from April 11 through May 6, 2011. Recreational Sports Center staff identified the off-campus population who were participating in intramural softball and volleyball activities and invited them via email to attend a pizza lunch where the survey would be administered. These efforts resulted in 539 students who completed the study’s pre and post-test. The paper NSSE surveys were returned to Indiana University in May 2011 for processing. Variables of Interest The independent variable in this study is categorical and represents participation or non-participation in the camp program. A first level analysis was conducted on five dependent variables, four representing mean benchmark scores from the NSSE and one representing each student’s first-year institutional grade point average. Additionally, mean scores from the BCSSE High School Academic Engagement scale were included in the statistical model in an attempt to control for pre-existing differences between the experimental and control groups. The definition of the four dependent variable benchmarks and one matching scale used in this research and the accompanying questions of each are provided in SPRING 2013 • VOLUME 20, NUMBER 2 83 an Appendix. Finally, persistence from fall 2010 to spring 2011, in the form of a dichotomous dependent variable, was analyzed to determine the impact of group membership. All variables were assigned an abbreviation for computer programming purposes. The variables, their category, associated abbreviations, scales of measurement, and range are presented in Table 2. TABLE 2 Independent and Dependent Variables by Name, Abbreviation, Scale, and Range Category Name Abbreviation Scale Range Independent Camp Participant CP Dichotomous 0-1 Dependent Active and ACL Continuous 0-100 Collaborative Learning Dependent Enriching EEE Continuous 0-100 Educational Experience Dependent Spring 2011 Grade GPA Continuous 0-4 Point Average Dependent Level of Academic LAC Continuous 0-100 Challenge Dependent Persistence from PERST Dichotomous 0-1 Fall 2010 to Spring 2011 Dependent Student-Faculty SFI Continuous 0-100 Interaction Matching High School HSAE Continuous 0-10 Academic Engagement 84 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION Data Inspection Following a recommendation by the American Psychological Association Task Force on Statistical Inference (Wilkerson & APA Task Force on Statistical Inference, 1999), a programmatic inspection of the data set was conducted to identify missing cases or cases outside acceptable response ranges. SPSS Statistics 19.0 was used to identify the number of missing cases and range of values for each of the dependent variables (LAC, EEE, ACL, SFI) as well as the matching variable (HSAE). No values exceeded the acceptable range of 0-100 for the dependent variables LAC, ACL, SFI, and EEE, or the acceptable range of 0-10 for the matching variable, HSAE; however, several missing values were identified across the data set on all five variables. SPSS offers a robust method of managing missing data, which incorporates the basics of regression without the severity of list and pairwise deletion. Utilizing the observed data points, regression coefficients are generated and employed to calculate a value for missing data that most replicates the regression equation. This method for managing missing observations was conducted by selecting the SPSS commands “Transform>Replace Missing Values,” entering the variable names with missing data, and then selecting “Linear trend at point” (SPSS, 2009). This calculation was performed on the four dependent variables as well as the matching variable and missing values imputed where needed. Research Questions The overarching purpose of this study was to determine if a limited causal relationship can be inferred between an institutional first-year initiative that attempts to promote student engagement and various measures of student self- reported academic engagement, satisfaction, and classroom performance. A secondary aim of this research was to determine if differences in fall to spring persistence exist between those students who participated in the initiative and those who did not; therefore, the specific research questions are 1) Is there a statistically significant difference in the mean NSSE benchmark values of LAC, ACL, EEE, SFI, and first-year institutional grade point average between camp participants and non-participants? 2) If a statistically significant difference exists between participants and non-participants, on which means does the difference exist? 3) Of the 1,468 first-time in college cohort, is there a statistically significant difference in the persistence rates from fall 2010 to spring 2011 between camp participants and non-participants? Data Analysis Multivariate analysis of variance (MANOVA) was conducted to determine if a statistically significant difference existed across the means of the four NSSE SPRING 2013 • VOLUME 20, NUMBER 2 85 benchmark variables (LAC, EEE, ACL,SFI) and first-year institutional grade point average (GPA) based upon group membership (camp participants vs. non- participants). In the presence of a statistically significant MANOVA, post-hoc analysis of variances (ANOVA) will be reviewed to determine on which scale the difference(s) exists. Where appropriate, effect sizes were calculated to determine the magnitude of mean differences. When measuring two or more groups on more than two dimensions, MANOVA is superior to individual ANOVAs as it provides a more robust analysis and assists in preventing an inflation of the alpha level and subsequent possibility of committing a Type I error (Meyers, Gamst, & Guarino, 2006). For example, in the present study, the MANOVA design could be decomposed into five separate 2 X 1 ANOVAs, each with an alpha value of .05; however, as each ANOVA is conducted independently, the alpha level associated with finding a comprehensive statistically significant difference across the five dependent variables is inflated to 22.6% (1- (.95)(.95)(.95)(.95)(.95)). It is arguable that the individual members of the population would possess varying levels of attributes that contribute toward a proclivity to participate in the intervention and become more engaged. In a pure experimental design, pre-existing differences among the sample could be controlled for via random assignment addressed through the inclusion of a control variable or co-variate in the statistical design; however, it has been demonstrated that multivariate analysis of co-variance (MANCOVA) or analysis of co-variance (ANCOVA) is not reliable when dealing with intact groups (Henson, 1998). One alternative to the inclusion of a co-variate is matching, where members of the experimental and control groups are paired utilizing some pre-intervention measure. As all students in the final population had participated in the BCSSE, camp participants and non-participants were matched one-to-one on their individual score on the BCSSE scale related to High School Academic Engagement (HSAE). As the name suggests, questions associated with this scale attempt to measure the time and effort first-year students invested in educational activities during high school (National Survey of Student Engagement, 2012b). Matching the sample on this pre-college measure assisted in the limitations associated with a non-experimental design. Finally, the 1,468 first-time in college students were analyzed to determine if there is a statistically significant difference in fall 2010 to spring 2011 persistence rates between camp participants and non-participants. A cross-tabulation and subsequent chi-square statistic were generated to determine if differences in observed and predicted persistence rates were due to chance or group membership. All analyses were conducted utilizing SPSS Statistics 19.0. Results Descriptive statistics were generated for the variables and are presented in Table 3. 86 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION Outcome of Analysis of Mean Differences A multivariate analysis of variance (MANOVA) was conducted to determine if statistically significant differences exist between the means of the four dependent variables based upon group membership. The Box’s M test, which is analogous to Levine’s test in analysis of variance, was not statistically significant, indicating sufficient equality of covariance between the two groups. A review of the omnibus MANOVA indicates there was no statistically significant difference in scores on the dependent variables based upon group membership F, (4, 211) = .555, p = .696; Wilk’s λ = .990, partial ε² = .010. Outcome of Analysis of Persistence Of the 1,468 first-time in college students, 1,240 (84.5%) persisted from fall 2010 to spring 2011. When considering camp attendance, there was no statistically significant difference in persistence between participants and non-participants; X², (1) = 2.780, p = .095. A complete cross-tabulation depicting persistence differences between camp participants and non-participants is presented in Table 4. Discussion This research attempted to determine the impact of participation in an orientation camp on first-year students’ levels of academic engagement and performance and persistence from fall to spring. A MANOVA indicated no statistically significant differences between the means of the five dependent variables based on group membership; therefore, TABLE 3 Descriptive Statistics of the Study Sample Attended Camp Did Not Attend Camp Orientation Orientation Variable Mean Std. Dev. n Mean Std. Dev. N ACL 39.124 14.7365 108 38.125 16.2914 108 EEE 25.844 10.9372 108 25.617 12.0484 108 GPA 2.728 0.6723 108 2.587 0.7982 108 LAC 46.469 11.7308 108 47.314 12.9748 108 SFI 35.196 16.0073 108 32.701 15.8750 108 HSAE 4.922 1.0763 108 4.922 1.0763 108 SPRING 2013 • VOLUME 20, NUMBER 2 87 for this sample, it can be suggested that participation in the camp orientation program had no impact on academic engagement, academic satisfaction, student- faculty interactions, or classroom performance. The MANOVA did not produce a statistically significant model; therefore, no post-hoc interpretations associated with group membership were made. A chi-square statistic indicated no statistically significant difference in persistence rates between camp participants and non-participants; therefore, any differences in persistence are due to chance or some other factor, and cannot be attributed to camp participation. In fact, non-attendees represented a greater portion of students persisting (52.3%) than attendees (47.7%). The findings from this study elicit several challenges and opportunities for the host and institutions of similar student make-up, location, and control. From a challenges perspective, there were no statistically significant differences in self-reported levels of academic engagement between camp participants and non-participants. Additionally, the persistence rate from fall 2010 to spring 2011 of 84.5% was consistent with the average at the host institution for the last eight academic years (approximately 85%). Institutions that invest the resources to administer camp orientations or other first-year initiatives must identify measurable outcomes that are associated with improved levels in these areas. Otherwise, such interventions and programs simply assimilate into an activity more affiliated with a tradition than a student success initiative. Aside from the dearth of positive student success evidence, it is appropriate to also consider the financial investment by the institution and student alike. The TABLE 4 Cross Tabulation of Camp Participation and Persistence from Fall 2010 to Spring 2011 (N=1468) Persisted Fall Did Not Attended Total to Spring Attend Did Not Persist Count 133 95 228 % within Persist to Spring 2011 58.3 41.7 100.0 % within Camp Attendance 17.0 13.8 15.5 % of Total 9.1 6.5 15.5 Persisted Count 649 591 1,240 % within Persist to Spring 2011 52.3 47.7 100.0 % within Camp Attendance 83.0 86.2 84.5 % of Total 44.2 40.3 84.5 88 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION camp is an expensive initiative that includes institutional expenses associated with travel, lodging, and food. From a student perspective, the camp fee is minimal in offsetting these expenses, but is arguably a burden on a population where 80% of students receive some form of financial assistance. Perhaps less costly on-campus initiatives should be considered in an attempt to positively impact persistence and student academic engagement measures for the target population. From an opportunities point of view, intentional, intrusive camp initiatives offer a wonderful platform to introduce and infuse attributes associated with “becoming an educated person,” such as being a citizen, life skills manager, life- long learner, and skilled worker (Hamrick, Evans, & Schuh, 2002). Subject matter can include topics such as independent living, developing effective study skills, career aptitude counseling, and the importance of developing relationships with faculty. Incorporating measurable learning outcomes in this fashion could also provide critical evidence for satisfying accrediting agencies’ core requirements. For example, the Southern Association of Colleges and Schools’ core requirement 2.10 directs institutions to provide student support programs and services that promote learning and development (Southern Association of Colleges and Schools Commission on Colleges, 2012). Additionally, it is worthwhile to consider expanding a pre-college camp initiative to include a first-to-second-year bridge program. While FTIC fall to spring persistence over the last eight academic years at the host institution is approximately 85%, records indicate the fall to fall retention rate for FTIC students has ranged from 61.4% to 67.5% since 2005. Too many times, higher education administrators place a premium on FTIC programs and services while failing to address the “sophomore slump,” which can include apathy, homesickness, and lack of academic success. The development of second-year resources by the National Resource Center for The First-Year Experience and Students in Transition at the University of South Carolina supports the importance of attending to the potentially unique needs of this group (National Resource Center, 2012). Limitations and Delimitations of the Present Study A possible limitation associated with the present study surrounds the generalizability of the findings to institutions of similar classification and control in other geographical regions of the United States. This limitation also extends to differences in student populations. It is reasonable to hypothesize that primarily first-generation students receiving some form of financial assistance might place lower levels of importance on student development and academic engagement activities in favor of work or family commitments. Additionally, as first-generation students, this group could possibly not understand the value of investing in such activities. Another possible limitation is one associated with respondent bias. Other than grade point averages and persistence rates, all other data are self-reported and, therefore, open to personal history, experience, and bias. While the NSSE reports high levels of validity, this form of bias cannot be totally controlled for. SPRING 2013 • VOLUME 20, NUMBER 2 89 A delimitation of the present study is the exclusion of private institutions that typically have greater resources and can provide student services and programs at no charge for their students. Additionally, students attending these institutions may be second-generation students who understand and value the importance of cultivating academic and social capital. Another delimitation can be attributed to omitted variable bias. While the outcome variables in this study were obtained via survey data, persistence, and academic performance, there are certainly other methods by which to measure academic engagement. Additionally, while camp participants and non-participants were paired on a pre-college variable in order to control for pre-existing differences, it still remains possible and highly probable that other external factors contributed to levels of academic engagement and persistence. As the academy continues to face higher levels of accountability for student success outcomes, it becomes imperative that college and university administrators understand what works in retaining and graduating students. This accountability extends to accrediting agencies that require student services and support initiatives that include measurable learning outcomes. Similar research at private and larger institutions would be beneficial in identifying the effectiveness of pre-college initiatives on first-year persistence and academic performance. Positive findings could be used as justification for resources and additional staffing at smaller, public institutions. Additionally, student level data should be analyzed to determine and compare the variance of academic preparation by institution. For institutions that serve primarily first-generation, low socioeconomic populations, front-loading initiatives to improve the probability of student success is critical. References Carini, R. M., Kuh, G. D., & Klein, S. P. (2006). Student engagement and student learning: Testing the linkages. Research in Higher Education, 47(1), 1-32. Crawford, A. B. (1926). The orientation of college freshmen. Bulletin of the American Association of University Professors, 12(8), 597-600. Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An introduction (7th ed.). Boston, MA: Allyn and Bacon. Gerber, S. K. (1970). Approaches to freshman orientation. Improving College and University Teaching, 18(1), 57-60. Guthrie, W. S. (1953). Evaluating the orientation week program. Educational Research Bulletin, 32(1), 11-14. Hamrick, F. A., Evans, N. J., & Schuh, J. H. (2002). Foundations of student affairs practice: How philosophy, theory, and research strengthen educational outcomes. San Francisco, CA: Jossey-Bass. Harriman, P. L. (1925). The orientation of college freshman. Peabody Journal of Education, 3(3), 159-161. 90 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION Henson, R. K. (1998). ANCOVA with intact groups: Don’t do it! Paper presented at the Annual Meeting of the Mid-South Educational Research Association, New Orleans, LA. Kronovet, E. (1969). Current practices in freshman orientation. Improving College and University Teaching, 17(3), 204-205. McCandless, C. E., & Barker, D. G. (1969). Summer camp orientation for freshmen. Improving College and University Teaching, 17(3), 212-214. Meyers, L. S., Gamst, G., & Guarino, A. J. (2006). Applied multivariate research: Design and interpretation. Thousand Oaks, CA: Sage Publications. National Resource Center for The First-Year Experience and Students in Transition (2012). Sophmore-year resources. Retrieved from http://www.sc.edu/fye/ resources/soph/index.html National Survey of Student Engagement (2012a). Psychometric portfolio: NSSE’s commitment to data quality. Retrieved from http://nsse.iub.edu/_/?cid=154 National Survey of Student Engagement (2012b). Quick facts about BCSSE. Retrieved from http://bcsse.iub.edu/_/?cid=232 Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research (Vol. 2). San Francisco, CA: Jossey-Bass. Pascaella, E. T., Terenzini, P. T., & Wolfle, L. M. (1986). Orientation to college and freshman year persistence/withdrawal decisions. The Journal of Higher Education, 57(2), 155-175. Porter, S. R., & Swing, R. L. (2006). Understanding how first-year seminars affect persistence. Research in Higher Education, 47(1), 89-109. Southern Association of Colleges and Schools Commission on Colleges (2012). Principles of accreditation: Foundation for quality enhancement. Retrieved from http://www.sacscoc.org/pdf/2012PrinciplesOfAcreditation.pdf SPSS (2009). Missing data: The hidden problem. Retrieved from http://www.bauer.uh.edu/jhess/documents/2.pdf Wilkinson, L. & Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanation. American Psychologist, 54, 594-604. SPRING 2013 • VOLUME 20, NUMBER 2 91 APPENDIx NSSE Benchmark and BCSSE Scale Definitions and Questions NSSE I. Active and Collaborative Learning (ACL): Students learn more when they are intensely involved in their education and asked to think about what they are learning in different settings. Collaborating with others in solving problems or mastering difficult material prepares students for the messy, unscripted problems they will encounter daily during and after college. • Asked questions in class or contributed to class discussions • Made a class presentation • Worked with other students on projects during class • Worked with classmates outside of class to prepare class assignments • Tutored or taught other students • Participated in a community-based project as part of a regular course • Discussed ideas from your readings or classes with others outside of class II. Enriching Educational Experiences (EEE): Complementary learning opportunities enhance academic programs. Diversity experiences teach students valuable things about themselves and others. Technology facilitates collaboration between peers and instructors. Internships, community service, and senior capstone courses provide opportunities to integrate and apply knowledge. • Hours spent participating in co-curricular activities • Practicum, internship, field experience, co-op experience or clinical assignment • Community service or volunteer work • Foreign language coursework and study abroad • Independent study or self-designed major • Culminating senior experience (capstone course, senior project or thesis, comprehensive exam, etc…) • Serious conversations with students of different religious beliefs, political opinions, or personal values • Serious conversations with students of a different race or ethnicity than your own • Using electronic medium to discuss or complete an assignment • Campus environment encouraging contact among students from different economic, social, and racial or ethnic backgrounds • Participate in a learning community or some other formal program where groups of students take two or more classes together 92 THE JOURNAL OF COLLEGE ORIENTATION AND TRANSITION III. Level of Academic Challenge (LAC): Challenging intellectual and creative work is central to student learning and collegiate quality. Colleges and universities promote high levels of student achievement by emphasizing the importance of academic effort and setting high expectations for student performance. • Hours spent preparing for class • Number of assigned textbooks, books, or book-length packs of course readings • Number of written papers or reports • Emphasis on basic elements of an idea, experience, or theory • Synthesis and organizing of ideas information, or experiences into new, more complex interpretations and relationships • Making judgments about the value of information, arguments, or methods • Applying theories or concepts to practical problems or in new situations • Working harder than you thought you could to meet an instructor’s standards or expectations • Spending significant amounts of time studying and on academic work IV. Student-Faculty Interaction (SFI): Students learn firsthand how experts think about and solve problems by interacting with faculty members inside and outside the classroom. As a result, their teachers become role models, mentors, and guides for continuous, life-long learning. • Discussed grades or assignments with an instructor • Talked about career plans with a faculty member or advisor • Discussed ideas from your readings or classes with faculty members outside of class • Worked with faculty members on activities other than coursework • Received prompt written or oral feedback from faculty on your academic performance • Worked on a research project with a faculty member outside of course or program requirements BCSSE V. High School Academic Engagement (HSAE): Engagement in educationally relevant behaviors during the last year of high school. • Assigned readings • Writing short papers or reports (5 or fewer pages) • Writing longer papers or reports (more than 5 pages) • Time spent preparing for class (studying, doing homework, rehearsing, etc…) • Asked questions in class or contributed to class discussions • Made a class presentation • Discussed grades or assignments with a teacher • Worked with other students on projects during class SPRING 2013 • VOLUME 20, NUMBER 2 93 • Worked with classmates outside of class to prepare class assignments • Prepared two or more drafts of a paper or assignment before turning it in • Discussed ideas from your readings or classes with teachers outside of class • Discussed ideas from your readings or classes with others outside of class (students, family members, etc…)