27Fall 1998 • Volume 6, Number 1 ARTICLE Student Satisfaction with Orientation: Toward a Framework for Program Effectiveness Daniel P. Nadler, Michael T. Miller, and Jennifer Casebere Orientation programs are vital to a new student’s acculturation to the campus communi- ty, and have been linked to students’ long-term academic and personal success on cam- pus. The Council for the Advancement of Standards (CAS) Standards for New Student Orientation provide a meaningful guide for program development and assessment. These CAS Standards were utilized over a two-year period to assess Tulane University’s new student orientation program. The results of the assessments have prompted the pro- fessional staff at Tulane to examine the rationale behind each of the orientation pro- gram’s activities and intentions. Orientation programs are an important mechanism for the transition of new students to the collegiate setting, and often are relied upon to serve as a means for the institution to convey common expectations to new students (Gardner & Hansen, 1993). Orientation programs also function as key tools to assimilate students to a new environment, provide needed technical knowledge for life in a new setting, operate as a form of emersion for the new friends and surroundings which can often be difficult for new students to accept, and provide a means for students to begin taking responsibility for themselves (Mullendore & Biller, 1993). On a cognitive level, orientation is a means for the institution to convey academic and social expectations, as well as provide a representation of the institutions’ commit- ment to students. Through the importance placed on an orientation program, new stu- dents begin to develop a value system about what is important to the campus (Mullendore & Biller, 1993). Whether directly or indirectly addressed through sessions and activities, issues such as personal safety, academic success, and even job placement can become the cornerstone for the attitudes and beliefs of new students (Kramer & Washburn, 1983). Orientation programs play a pivotal role in student success, and yet little research has been done to determine the effectiveness of a specific orientation over a long-range period. The current study is part of an extended examination of Tulane University’s New Wave student orientation program (Nadler & Miller, 1997). Daniel P. Nadler, Ph.D., is Assistant Dean of Students at Tulane University; Michael T. Miller, Ed.D. is an Associate Professor of Higher Education Administration, The University of Alabama; and Jennifer Casebere, M.S., is Formerly Director of Orientation at Tulane University. 28 The Journal of College Orientation and Transition Background and Research Methods Tulane University’s new student orientation program follows a pre-semester seminar format in which various academic and social sessions are presented over a four-day time period. This program, noted for its reliance on student involvement, is particularly unique in that Tulane draws a large number of students from geographically diverse areas. Additionally, the university is a highly selective research university, and is locat- ed in the urban setting of New Orleans, Louisiana. The current study is one in a series of related research projects employing the Standards for New Student Orientation Programs developed by the Council for the Advancement of Standards (CAS) (1988) as an assessment tool and guide for orientation programs. Originally funded through the American College Personnel Association Commission II mini-grant program in 1994, the study of the CAS Standards was initiated with an investigation of senior student affairs officers (SASO) support of the CAS Standards. The results of that study indicated a general agreement among SSAOs that the CAS Standards can and should be used as a template for orientation and transition programs. These perceptions did not differ based on either institutional size or mission (Miller & Nadler, 1997; Miller & Nadler, 1995). A similar research project revealed that the standards, with minor revisions, were also appropriate for use in faculty orientation programs (Miller & Nadler, 1994). A pilot survey was given in 1995, and with validity confirmed, subsequent studies were conducted in 1996 and 1997. The surveys consisted of the 20 CAS Standards, in which students were asked to rate, on a 1-to-5 Likert-type scale, their level of agreement that the New Wave program achieved each of the 20 CAS Standards. The final section of the survey requested demographic data including gender, ethnic background, age, class status, academic major, and transfer status. In fall 1996, the study was continued with an examination of the population of stu- dents participating in the New Wave orientation program (Nadler & Miller, 1997). With 903 students completing the survey, students agreed that the orientation succeeded in providing information on academic policies and procedures (mean 4.07), safety and security (4.07), and opportunities for nonclassroom involvement (4.00). To develop additional baseline data on the use of the CAS Standards as an assess- ment tool and to provide comparative data for changes in the 1997 New Wave orienta- tion program, the survey was again distributed to all new students completing the orien- tation program. Approximately 75% of all new students in 1997 completed the survey, yielding a response of 1,043. Students were given the survey to complete during the final session of the orientation program; completion of surveys was voluntary and anony- mous. For the 20-items comprising the CAS Standards, an internal reliability correlation analysis yielded an alpha level of .92 and a standardized alpha of .92, indicating a strong level of internal reliability. 29Fall 1998 • Volume 6, Number 1 Results of the 1997 Survey Demographic Data As shown in Table 2, the majority of respondents were female (53%), between the ages of 17 and 19 (97%), Caucasian (81%), non-transfer (92%) freshmen (96%), with majors in one of Tulane’s undergraduate liberal arts colleges (77%). Age Students of all age groups shared the same opinions about all of the activities and services in the orientation program except for residence hall move-in, where a significant difference existed between the group aged 17-19 and that of the 20-24 age group. The 17-19 age group rated the residence hall move-in service significantly higher (mean 4.12) than the 20-24 age group (mean 3.17; p=. 017). Major The findings indicated significant differences in registration and university services among students of different majors. For the activity of registration, students of Newcomb College (the female undergraduate college) (mean 3.37) rated it significantly lower than students in Tulane College (the undergraduate male college) (mean 3.65), the Engineering College (mean 3.91), the College of Architecture (mean 3.91), and the University College (the college primarily for part-time students) (mean 4.00; p=.0000). For university services (p=.0012), the female students of Newcomb College (mean 3.74) and University College (mean 4.16) were significantly more satisfied than the male stu- dents of Tulane College (mean 3.51). Class Standing Class standing made a significant difference in students’ rating of the activities and services of the orientation program. Freshmen rated residence hall move-in as mean 4.13, which was significantly higher (p=.001) than sophomore participants mean rating of 3.00. Freshmen also rated the welcoming convocation as good (mean 3.71), while sophomores scored it relatively low (mean 2.83). The difference in class standing was statistically significant at the .012 alpha level. Transfer Status There were significant differences for the few transfer students and non-transfer stu- dents for the activities of residence hall move-in (p=.0001) and university services (p=.021). For both of these activities, non-transfer students (mean 4.14 for residence hall move-in; mean 3.72 for university services overview) rated them significantly higher than transfer students (mean 3.05; 3.22). 30 The Journal of College Orientation and Transition Gender Gender realized the most differences among students in their opinions about the activities and services of the orientation program. Male students (mean 3.51) rated acad- emic advisement significantly higher (p=.022) than did female students (mean 3.34); and male students had a mean of 3.69 for registration (p=.022) yielding a significantly higher mean than female students did (3.48). However, female students rated university ser- vices (mean 3.80) and the hypnotist (mean 4.46) significantly higher than did male stu- dents (mean 3.57; 4.29). Race An analysis of variance test indicated some statistically significant differences on academic advisement (p=.015) and registration (p=.015). The post hoc tests (Tukey- HSD and Shefee) revealed no significant differences among or between students of dif- ferent racial backgrounds. The ANOVA finding was subsequently related to the differ- ences in cell sizes. 1996 and 1997 Orientation Comparison A comparison between 1996 and 1997 was conducted to identify differences between the entering classes for satisfaction with the orientation program and for gather- ing selected demographic information. There were no significant differences between Orientation 1996 and Orientation 1997 in attitudes toward the following six goals: positive relationships with staff (f=.9479); positive relationships with community (f=.2214); understanding Tulane’s expectations (f=.1102); providing information concerning academic policies, procedures, requirements, and programs (f=.3966); explaining the process for class schedules and registration (f=.2945); and providing appropriate information on personal safety and security (f=.2482). The remaining 14 goals were rated significantly higher in 1997 than 1996, perhaps indicating an improved or more effective orientation program. Discussion Higher education administrators must become more accountable and reflective of their stakeholders’ expectations. State governing bodies, constituent boards, the general tax paying public, tuition paying students and their parents, and accrediting agencies are all currently calling on colleges and universities to develop performance indicators which have the potential to rate the effectiveness or viability of the institution. This trend, an indication of a larger philosophical concern related to the pragmatic value and use of a college experience, relates broadly to divisions of student affairs and specifically to the programming available to college students. Orientation and transitional programs, often a new students first impression of an institution, have traditionally been built and structured around the functional attribute of 31Fall 1998 • Volume 6, Number 1 what academic or study skills can be transferred to new students and generally what is expected of new students in terms of performance, behavior, achievement, and so forth. These programs, often lauded for their creativity and thematic structure, rarely articulate specific learning outcomes and intentions, such as those developed by the Council for the Advancement of Standards (CAS). The CAS Standards offer 20 specific recommendations for what a new student or transitional program should provide. Within the context of the New Wave orientation program, traditional first-year stu- dents agreed that the intervention, as measured by CAS, was effective in fulfilling the objectives for an orientation program. These students did identify several areas where they had neutral feelings about program activities, and the trends identify some consistency in program strengths (e.g., developing relationships with new students, providing information on safety and security, and promoting non-classroom opportunities) and areas for improve- ment (e.g., developing relationships with faculty, developing relationships with individuals in the community, and identifying the personal and financial costs of attending Tulane). Perhaps most importantly, this identification allows the professional student affairs staff to restructure and emphasize problem areas, while enhancing activities which are currently identified as strengths. Specifically, the CAS Standards can be measured as a data set in the evaluation of orientation programs, identifying the purpose, strengths, and weaknesses of an orientation program. As colleges and universities struggle with the justification of programming, particular- ly in areas considered “academic support,” tools such as the CAS Standards for New Student Orientation will become increasingly important. There must be an effort on the part of student affairs professionals to begin building both a tradition and rationale for the use of assessment tools in student affairs programming. At the center of these assessments must be the central theme of responsiveness to student concerns. The CAS Standards pro- vide an excellent introductory effort at generalizable evaluative criteria for use in student affairs, and continued efforts must be made to refine these criteria across institutional boundaries. TABLE 1 Levels of Agreement for Orientation Goals Goal 1996 1997 Mean Mean (SD) (SD) n=903 n=1043 The Orientation Program assisted me in: ...developing positive 4.14 4.23 relationships with (.853) (.78) other new students. 32 The Journal of College Orientation and Transition ...in developing 3.87 4.04 familiarity with the (.855) (.78) physical surroundings. ...understanding 3.83 3.90 Tulane’s expectations (.819) (.79) of me. ...understanding the 3.79 3.87 purpose of Tulane. (.791) (.77) ...developing positive 3.66 3.67 relationships with (.904) (.92) Tulane staff. ...understanding the 3.63 3.72 mission of Tulane. (.832) (.84) ...developing positive 3.56 3.53 relationships with (1.01) (1.0) individuals from my community. ...determining my 3.54 3.66 purpose in attending (.906) (.86) Tulane. ...identifying costs 3.45 3.62 of attending Tulane, (1.00) (.98) both in terms of dollars and personal commitment. ...developing positive 3.41 3.57 relationships with (1.03) (.99) Tulane faculty. The Orientation Program: Provided information 4.07 4.09 concerning academic (.707) (.70) policies, procedures, requirements, and programs. Provided appropriate 4.07 4.10 information on personal (.780) (.73) safety and security. 33Fall 1998 • Volume 6, Number 1 Promoted an awareness 4.00 4.12 of nonclassroom (.768) (.74) opportunities. Created an atmosphere 3.81 3.99 that minimized anxiety, (.897) (.85) promoted positive attitudes, and stimulated an excitement for learning. Explained the 3.78 3.83 process for class (.900) (.89) scheduling and registration. Provided information 3.78 3.90 and exposure to (.794) (.75) available institutional services. Provided an atmosphere 3.74 3.96 and sufficient (.803) (.74) information that enabled me to makereasoned and well-informed decisions. Provided opportunities 3.61 3.80 to discuss expectations (.941) (.87) and perceptions with continuing students. Provided referrals 3.55 3.67 to qualified (.947) (.91) advisors and counselors. Provided information 3.44 3.62 about opportunities (.871) (.98) for self-assessment. 34 The Journal of College Orientation and Transition TABLE 2 Demographic Profile of Respondents for 1995-1997 Characteristic 1996 1997 n=903 n=1043 Gender Male 424 (46%) 469 (45%) Female 463 (51%) 557 (53%) Age 17-19 874 (97) 1,016 (97) 20-24 14 (2) 8 (0.8) 25+ 1 (0.1) 1 (0.1) Race Caucasian 736 (83) 845 (81) Asian 55 (6) 68 (6) Hispanic 31 (3) 40 (4) Black 36 (4) 22 (2) American Indian 4 (0.5) 7 (0.7) Other 17 (.2) 28 (. 3) Class Standing Freshman 864 (96) 1,001 (96) Sophomore 17 ( 2) 13 ( 1) Junior 4 (0.4) 6 (0.6) Senior 1 (0.1) 0 (0) Major Newcomb College 393 (43) 494 (47) Tulane College 311 (34) 316 (30) Engineering 110 (12) 138 (13) Architecture 45 ( 5) 54 ( 5) University College 26 ( 3) 24 ( 2) Subculture Identification Academic 240 (26) 262 (25) Collegiate 362 (40) 428 (41) Vocational 99 (11) 112 (12) Nonconformist 59 (7) 34 (3) 35Fall 1998 • Volume 6, Number 1 References Council for the Advancement of Standards for Student Services/Development Programs. (1988). CAS standards for student services/development programs: Student orientation programs self-assessment guide. Washington: Author. Gardner, J. N., & Hansen, D. A. (1993). Perspectives on the future of orientation. In M. L. Upcraft (Ed.). Designing Successful Transitions: A Guide for Orienting Students to College, pp. 183-194. Columbia, SC: Freshman Year Experience. Kramer, G., & Washburn, R. (1983). The perceived orientation needs of new students. Journal of College Student Personnel, 24, 311-319. Miller, M. T., & Nadler, D. P. (1994). Orientation program considerations for new community college faculty. Community College Journal of Research and Practice, 18(5), 441-448. Miller, M. T., & Nadler, D. P. (1995). Orientation priorities based on institutional type: The perspective of chief student affairs officers. Journal of Student Affairs, 4, 53-61. Miller, M. T., & Nadler, D. P. (1997). Consensus of chief student affairs officers toward the CAS orientation standards. Resources in Education, 32(6), 13 pp. ERIC Clearinghouse on Counseling and Student Services Reproduction Service Number CG 027 801. Mullendore, R. H., & Biller, G. M. (1993). Orientation standards, evaluation, and assessment. In M. L. Upcraft (Ed.) Designing Successful Transitions: A guide for orienting students to college, pp. 169-182. Columbia, SC: Freshman Year Experience. Nadler, D. P., & Miller, M. T. (1997). Student satisfaction with orientation: A program assessment and cultural stratification. Journal of College Orientation and Transition, 5(1), 7-14.