AISW Vol. 5, No. 1 33 Effectiveness of Admission Criteria on Student Performance in Classroom and Field Instruction M. Thomas Roseanna McCleary Patricia Henry Abstract: This study examines the effectiveness of admission criteria on graduate student performance in classroom and field instruction in a new MSW program. Graduate applicants’ undergraduate GPA, GRE, and total weighted admission score consisting of four items were gathered. These were correlated with their classroom and field instruction performance. Findings reveal that GRE, undergraduate GPA, and total weighted admission scores are significantly correlated with their class- room performance. End of first year cumulative GPA and human service experience were identified as significant predictors of field performance. Implications of these findings for social work educators and graduate school programs are discussed. Keywords: Admission criteria, classroom performance, field instruction performance Social work educators and administrators shoulder the paramount responsi- bility of selecting the most suitable candidates for graduate social work pro- grams and providing students with the best education and training. In this process, they play a vital role as gatekeepers of the profession, thereby, assuring the public and the professional community of the highest standard in profession- al practice. Admittedly, recruiting appropriate graduate applicants who demon- strate an aptitude and commitment to professional values is élan vital for social work programs. However, this remains an area that has not been adequately researched (GlenMaye & Oakes, 2002; Miller & Koerin, 1998). Social work educators struggle to design effective admission criteria. Although there is no consensus regarding admission criteria in graduate schools across the nation, a combination of factors are utilized by most schools in the admissions decision-making process. Undergraduate grade point average (GPA), scores from the Graduate Record Examination (GRE), work experience, personal statements, reference letters, and academic potential are some of the factors considered by social work schools when admitting students into programs. Some scholars have M. Thomas, Ph.D. is Assistant Professor, Roseanna McCleary, Ph.D. is Assistant Professor, and Patricia Henry, MSW, LMSW is the Field Education Coordinator at the Department of Social Work at California State University Bakersfield, Bakersfield, California, 93311-1099. Copyright© 2004 Advances in SocialWorkVol. 5 No. 1 (Spring 2004) 33-46. Indiana University School of SocialWork. 34 ADVANCES IN SOCIALWORK argued that the selection criteria used to recruit social work students is often invalid given the kind of tasks they are called upon to perform later in their prac- tice.These critics point out that the admission procedures in early social work pro- grams are modeled after the selection process of math, psychology, sociology, medicine, law, and nursing (Wodarski, 1979). Furthermore, the use of the GPA and GRE as predictors of academic success has also been questioned by some researchers (Donahue & Thyer, 1992; Duehn & Mayadas, 1977; Sampson & Boyer, 2001) on the grounds that their relationship to professional performance has not been empirically established. Critics also argue that personal statements, inter- views, and reference letters that lack standardization and predictive powers often reflect what students think social work schools are looking for. These criteria sel- dom predict the skill level and competency required for professional social work practice. Against this backdrop, Duehn andMayadas (1977) have proposed a com- petency-based program with specific entrance and exit requirements for social work programs. Social work literature on the effectiveness of admission criteria on student per- formance in classroom and field instruction or the predictive power of these crite- ria on the graduate students’ success in later practice has not historically attracted adequate research interest. However, there has been some effort in the 1970s and 1990s as evidenced by the review of literature. Some of these studies have focused their attention on the suitability and reliability of using certain admission criteria and its impact on student performance (Constable, 1977; Donahue &Thyer, 1992; GlenMaye & Oakes, 2002; Miller & Koerin, 1998). The link between pre-admission data, classroom, and field instruction (Pelech, Stalker, Regehr & Jacobs, 1999), problems with using undergraduate performance alone as admission criteria (Moxley, Moxely & Najor-Durack, & Dumbrigue, 2000), and the need for establish- ing suitability criteria (Cole, 1991) have also been examined. A review of literature revealed that there have been some efforts particularly in the 1970s and 1990s in understanding the influence of the admissions process on graduate students’ success later in their practice.While some studies have focused on predictive factors that lead to success in the classroom, others have focused on the relationship between those factors and field performance. Very few studies exist that examined the relationship between the admissions process and class- room and field performance particularly of foundation year graduate students. This study highlights the significance of this area and contributes to the existing knowledge in the area of admission criteria and foundation year graduate student performance.The purpose of this studywas to examine the effectiveness of admis- sions criteria on student performance in classroom instruction and field practicum in a relatively new graduate program in a public university in California. This research focused on the following three research questions: 1. Do students who are rated high in admission criteria items such as under- graduate GPA and GRE and overall weighted average, also score high in their first year MSW courses and field practicum? 2. Do students who perform better in classroom instruction also perform better in field instruction? 35Thomas et al./EFFECTIVENESS OF ADMISSION CRITRIA ON STUDENT PERFORMANCE 3. Are these outcomes the same for different cohorts such as full-time, part- time, first year and second year MSW students? REVIEW OF LITERATURE GlenMaye and Oakes (2002) describe the program admissions process in Departments and Schools of Social Work as “... one of the least studied compo- nents of social work education” (p. 67). Yet, Dunlap, Henley, and Fraser (1998) state, “admissions decisions are among the most important decisions made in schools of social work” (p. 455). A review of existing literature reveals an inconsis- tent and fragmented relationship between admissions criteria and student per- formance in social work programs. Findings among studies done over the past 30 years are mixed, even when the same or similar variables and methods are used (Pelech, Stalker, Regehr, & Jacobs, 1999). The following are examples that illustrate the current state of the admissions process in social work education. Pfouts and Henley (1977) describe a multivariate predictive index of student field performance that can be used as an admission tool in graduate social work programs. These researchers attempted to construct an index that would enhance the ability of the admissions process to select candidates who would perform well as practitioners. The results of a factor analysis identified four fac- tors, which were then used as independent variables in a stepwise regression analysis. These factors were: the student’s potential for graduate school, experi- ence in post-college paid employment, the student’s gender, and the quality of undergraduate school. These background variables used as predictors together explained 38.6% of the variation in the field performance. Of these, student potential for graduate school contributed the most to the prediction of student field performance followed by paid work experience. In a similar study, Dunlap (1979) reviewed predictors of student performance using the following criteria: undergraduate grade point average (GPA), graduate record examination (GRE) score, length of prior social work or related experi- ences, andwhether or not the student had an undergraduate degree in social wel- fare. A student interview and letters of reference were also quantified and used as predictor variables. The student’s graduate school GPA and a quantitative faculty rating on his/her “professional potential” were used as the dependent variables. Usingmultiple discriminant analysis, Dunlap found that the best predictor of stu- dent performance was the faculty interview, with undergraduate GPA being a moderate predictor, and GRE score and letters of reference serving as weak pre- dictors. TheDunlap’s study had been called into question byGlisson andHudson (1981) primarily on statistical and methodological grounds. These critics have argued that there is a serious statistical flaw in trying to predict academic performance and professional potential of social work students with faculty interviews and undergraduate grade point averages. Despite these limitations, this study has been applauded by these critics as a groundbreaking endeavor in understanding the effectiveness of admissions criteria in predicting the potential of student applicants. 36 ADVANCES IN SOCIALWORK Using yet another dependent variable to define student performance, Dunlap, Henley and Fraser (1998) looked at the relationship between admissions criteria (previous social work experience, undergraduate GPA, and GRE score) and MSW students’ scores on a pre-graduation comprehensive examination. The authors ran a blockwise multiple regression of exam scores using the background vari- ables of age, gender, and race as covariates. Total GRE score and undergraduate GPA emerged as useful predictors of good performance on the comprehensive examination. This conflicts with Dunlap’s (1979) finding of GRE score being a weak predictor, yet supports the finding of undergraduateGPA as amoderate pre- dictor. Donahue and Thyer (1992) focused on the GRE as part of the admissions process for MSW students. Predictive validity of the GRE has been studied in a variety of graduate programs (e.g., House & Johnson, 2002) with positive results. However, studies that focus on this topic using minority graduate students’ aca- demic performance report negative results (Milner, McNeil & King, 1984; Sampson & Boyer, 2001). Donahue and Thyer (1992) challenged the validity of a portion of the GRE—the reading comprehension sections. Narratives for a set of multiple-choice questions were blanked out and students were asked to guess an answer from the five choices. It was expected that students would be correct 20% of the time due to chance. Results showed an average correct response rate of 36%, significantly higher than what was expected. The authors then calculated a correlation between these scores and the verbal component of the students’ actu- al GRE score and found a statistically significant correlation. This poses questions regarding the strength of the GRE’s content validity and supports Dunlap’s (1979) and Milner, McNeil and King’s (1984) findings. Pelech, Stalker, Regehr and Jacobs (1999) took a different approach to identify- ing predictors in the admissions process. Faculty members teaching in an MSW programwere asked to identify a group of students who experienced problems in four areas (placement breakdown, extended practicum, poor academic perform- ance, and problems with interpersonal relationships) and admissions data for these students were examined. Multiple regression analysis was done using this group of students and a group of randomly chosen students from those whowere not identified. This study supported the finding that undergraduate GPA predicts subsequent academic achievement in an MSW program. An additional finding revealed a negative relationship between age and prior social service experience and academic performance. The authors noted that students who had extensive social work experience might return to school with a false sense of competence. The facultymembers involved in the admission decision-making process are as important as the applicants’ background information. Dailey (1979) examined the validity of admission predictions, including the role of faculty members as admission decisionmakers. The study compared the faculty’s admission decision to student outcome measures of classroom performance and field performance separately. Interestingly enough, classroom faculties were able to predict both classroom performance and field performance at statistically significant levels. On the other hand, field instructors were only able to predict classroom perform- ance and not field performance. 37Thomas et al./EFFECTIVENESS OF ADMISSION CRITRIA ON STUDENT PERFORMANCE Miller and Koerin (1998) and Moxley, Najor-Durack and Dumbrigue (2000) emphasized the importance of using nonacademic criteria in the admissions process. In an exploratory study that looked at the use of nonacademic admis- sions criteria, Miller and Koerin (1998) mailed surveys to accredited MSW pro- grams. Qualitative questions were included in the survey to explore respondents’ views on what constituted appropriate indicators of applicant suitability for a social work program. They concluded that while the majority of study respon- dents did use nonacademic admissions criteria in their programs, they reported similar problems in screening that had already been identified in previous articles that focused on admissions criteria and gate-keeping. Improvement in assess- ment of student suitability and a model for this process was suggested. Moxley, Najor-Durack and Dumbrigue (2000) focused on nontraditional stu- dents and the need to develop alternative strategies to account for differences among this group. Using adult learning theory, the authors support the need to devise strategies for nontraditional students who may not have the level of aca- demic credentials to meet admissions criteria to an MSW program. These strate- gies involved institutional commitment and planning, support for applicants during the admissions process, and helping them with the transition from appli- cant to student. METHODS Data for this study were collected from several sources. These sources included a survey questionnaire for graduate students admitted to the program in 2000 and 2001, pre-admission student information from admission files, quarterly field instruction evaluation files maintained by the field education coordinator, end of first year GPAs of full-time and part-time students, and the current cumulative graduate GPA of all students, including advanced standing students from the admissions and records office of the university. The study was conducted among graduate MSW students in a California public university. The program began admitting students in 2000 and was recently accredited by the CSWE. Currently, there are 87 full time and part time students in the program. Graduate students (N=68) admitted to the program in Fall 2000 and 2001 were asked to voluntarily complete a survey questionnaire that consist- ed of demographic information.The pre-admission information gathered includ- ed such admission criteria as the undergraduate GPA (UGPA), GRE score, and total weighted admission score of four items as rated by two faculty members. These items included: Intellectual and academic potential (UGPA, GRE, and con- ceptual ability, problem-solving ability, writing skills, creativity, and academic skills as demonstrated through personal statement), relevant human service experience (length, demonstrated success and quality of the work experience), leadership potential (social work values, communication skills, initiative, and interpersonal skills), and quality of reference letters (the appropriateness and nature of endorsement). Two faculty members rated each application on the above four items. These items were rated on a scale of 1 to 10, with 1 being “unac- ceptable applicant” and 10 being “outstanding applicant.” Thus, each applicant can get a maximum of 40 points from each rater, totaling 80 maximum points. 38 ADVANCES IN SOCIALWORK Although the points for all items were the same, the relative weight for these items in the overall score were different. Based on experience, knowledge, and information from the social work literature, faculty raters assigned weights for each of the four criteria. Intellectual and academic potential were rated 50%, human service experience and leadership 20% each, and quality of reference let- ters was rated 10%. The item-scores on each student by two faculty raters were then transferred to a percentage scale based on the above relative weight. Thus, the score obtained is the “total weighted admission score.” The scoring instru- ment has neither been pre-tested nor has it been tested for reliability and validity. The cumulative GPA of full time students admitted in 2000 and 2001, the cur- rent cumulative GPA of third year part time students, and second year part time students were also collected. To assure the accuracy of comparisons between cohorts andwithin each cohort, the end of first year GPAwas also calculated. This does not include five advanced standing students and, hence, the sample size for this variable was reduced to 63. The research team decided to use both the end of first year cumulative GPA and current cumulative GPA, as it represents a student grade reflecting their performance in the classroom. The evaluation of field instruction was carried out every quarter, and the con- tent of the evaluation forms were different for foundation field practicum and concentration practicum. The field instructor and the faculty field liaison con- duct the evaluation jointly. Because of this collaborative evaluation process, the possibility of increasing subjectivity in the evaluation needs to be recognized and, therefore, findings should be interpreted with caution. For the foundation field practicum, the evaluation contains the following major areas: (a) Illustration of development of professional self, (b) Social work values and ethics, (c) Micro- level assessment skills, (d) Micro-level intervention skills, (e) Communication skills, (f) Macro-level assessment and intervention skills. Each of these areas has several items as subcategories. The illustration of the development of profession- al self was one of the categories that was defined as: (a) demonstration of profes- sional self demeanor, (b) demonstration of independence and initiative, (c) self- evaluation of performance, (d) effective use of supervision, and (e) commitment to life-long learning. Each of these subcategory items is structured on a progres- sive skill assessment grading rubric ranging from 1 to 5, much like a Likert scale where one indicates the lowest performance in that area and five the highest per- formance in the area. Table 1 illustrates an example of the grading rubric. Students need to earn a minimum of 75 points out of a possible 90 points to pass the foundation field practicum. In the second year, students chose an area of concentration that included chil- dren and family services and health and mental health. For concentration field practicum, the students are evaluated quarterly in the following areas: relation- ship skills, communication skills, assessment skills, collaboration skills, interven- tion skills, social work values and ethics, and development of professional self. Each of these areas has several items as subcategories or indicators that define the area of skill and competence. For example, intervention skill has been opera- tionalized as: (a) effectively contracts with client regarding their stated goals; (b) assists reluctant client in the change process; (c) demonstrates flexibility andwill- 39Thomas et al./EFFECTIVENESS OF ADMISSION CRITRIA ON STUDENT PERFORMANCE ingness to utilize different roles, methods, and techniques; (d) demonstrates abil- ity to proceed from engagement, assessment, problem identification, goal set- ting, and intervention to termination and evaluation; (e) demonstrates ability to monitor ongoing intervention process, accepting feedback from clients, signifi- cant others, and other professionals; (f) demonstrates case management skills, accepting realities of agency constraints, and serving as client advocate; and (g) intervenes in crisis and professionally helps client to stabilize and links themwith resources. Each of these subcategories is structured on a scale ranging from NA (Not Applicable) and a Likert scale ranging from 1-5. Table 2 provides an example of this. Students can earn a maximum of 190 points, and they need 152 points (80%) to pass the field practicum. The actual points they earned were converted into percentages to facilitate comparisons. A sample rating scale in shown inTable 2. The percentage points students gained during foundation and concentration practicum years are considered representative of their performance in field instruction. Since there were items in the Likert-type scale not rated by field eval- uators due to the not-applicable nature of those areas in their learning opportu- nity, the entire field practicum performance points were recalculated based on items actually rated jointly by the evaluators. Accordingly, percentages were cal- culated based on items actually rated. This procedure was adopted in order to assure the accuracy of the field instruction outcome. Furthermore, there were three academic quarters of field practicum in the foundation year and similarly three academic quarters in the concentration year. Since some data weremissing in the fall quarter, which happens to be the beginning field practicum quarter for full time students, the winter and spring quarterly evaluation were selected for analysis. Additionally, mean field instruction scores for winter and spring quar- ters were calculated for foundation-year students and concentration-year stu- dents. Thus, both foundation and concentration year percentage point scores were gathered. Study Participants A description of the demographic profile of the students is important to better understand their commitment and performance in the social work program. Professional Self- Progressive Rating Scale Rubrics development 1 2 3 4 5 1.Demonstra- Appears Occasionally Appears Usually Consis- tion of pro- bored, appears interested completes tently fessional passive, interested, and the respon- demeanor fails to assertive motivated. assign- sible, and maintain and calm Calm and ments poised poise, only under poised with with poise even under unable to routine exceptions and control. extreme plan work. conditions. to high Very conditions. Easily over- stress motivated. Highly whelmed. situations. motivated. Table 1: Sample Rating Scale From Quarterly Foundation Field Instruction Evaluation 40 ADVANCES IN SOCIALWORK Insights into their demographic profiles may help us understand who they are and how they perform in the classroom and field instruction. The study partici- pants’ age ranged from 23 to 64 years and themean age was 39.5 years. More than four-fifths (84%) of the students were females and the rest were males (16%). Regarding their ethnicity, the data further indicated that 57% of the students were White, 18% African Americans, 22% Hispanic/Latinos and the remaining were Pacific Islanders and others. Furthermore, the study participants came from var- ious types of family backgrounds: 41% were married, 38% were single, 16% were divorced and the remaining 5%were separated or had other types of family struc- ture. More than half (52%) were working in full time jobs, 19% in part time jobs, and 31%did notwork and had full time commitments to the program.Themajor- ity (54%) worked in public/governmental agencies. Data Analysis The data were analyzed using SPSS® Version 10. Descriptive data analysis and inferential data analysis were performed in order to answer the research ques- tions under study. The study examined the demographic characteristics of the students in order to gain insight into students’ broader social context. For this, descriptive analyses of selected variables such as age, gender, ethnicity, employ- ment and family background, were performed. The study also tested for possible correlations between admission criteria (GRE, UGPA, total weighted admission score), student performance in the classroom (using current cumulative GPA and end of first year GPA) and field instruction performance (using two-quarter field instruction mean percentage scores). It was assumed that the graduate cumula- tive GPA and the field instruction score would be good indicators of student per- Intervention Skills Rating Scale 1. Effectively contracts with clients NA No opportunity to judge regarding their stated goals 1 Fails to meet expectations 2 Meets expectations some of the time 3 Consistently meets expectations 4 Exceeds expectations some of the time 5 Consistently exceeds expectations 2. Demonstrates ability to monitor NA No opportunity to judge ongoing intervention process, accepting feedback from clients and other professionals. 1 Fails to meet expectations 2 Meets expectations some of the time 3 Consistently meets expectations 4 Exceeds expectations some of the time 5 Consistently exceeds expectations Table 2: Sample Rating Scale From Quarterly Concentration Field Instruction Evaluation 41Thomas et al./EFFECTIVENESS OF ADMISSION CRITRIA ON STUDENT PERFORMANCE formance in the program. In order to ascertain the relationship between admis- sion criteria and students’ performance in the classroom and the field, Pearson’s correlation test was used. Further analysis was performed to examine the differ- ence in outcome for part time students and full time students as well as for stu- dents admitted in 2000 and 2001. In order to compare group differences, t-Tests were used. A step-wise linear regression analysis was used to identify the predic- tors of student performance in field instruction. FINDINGS Admission Criteria and Classroom Performance The total weighted admission score was an important criterion taken into con- sideration in admitting students to the program.Two facultymembers rated each applicant on four items. The total admission score, which was representative of admission criteria, consisted of intellectual and academic potential, relevant human service experience, and leadership potential and quality of reference let- ters. The mean score of these items were calculated based on both the raters' evaluation and a two-tailed Pearson’s correlation test was run. Among these four items, only intellectual and academic potential significantly correlated with end of first year cumulative GPA (see Table 3). The study further found that the GRE score correlated with total weighted admission score and the relationship was statistically significant (r=.353; p<.003). Similarly, higher undergraduate GPA was significantly associated with the total weighted admission score (r=.404; p<.001). It is important to note here that GRE and undergraduate GPA were factors considered under “academic potential” whichwas one of the four items in calculating the total weighted admission score. The correlation between GRE and undergraduate GPAwas statistically significant but relatively weak (r=.293). The current cumulative graduate GPA includes GPA of students graduated in 2002, current full time and part time second year, current part time third year and advanced standing students who were at various stages of study in their MSW program. Due to this variation, correlation of this variable with other variables needs to be considered with caution when making comparisons within and between groups. Hence, the end of first year cumulative GPA for full time and equivalent cumulative GPA for part time students was calculated. This was based on one-year full time equivalent of courses taken by part time students. The data showed that end of first year cumulative GPA significantly correlated with GRE (r=.266; p<.03) and UGPA (r=.411; p<.001). Admission Criteria and Field Instruction Performance Themean field instruction performance score for all graduate students was 88.44 with a standard deviation of 9.50. The admission criteria and field instruction performance were also tested for possible correlation using Pearson's two-tailed test. The admission criteria items (academic potential, work experience, leadership and references) and the field instruction performance were tested for possible relationships using a Pearson's correlation. As can be seen from Table 3, both 42 ADVANCES IN SOCIALWORK human service experience (p<.002) and quality of references (p<.002) were signif- icantly correlated with field instruction performance. The data show that the total admission score significantly correlated with stu- dent performance in field instruction (r=.333; p<.008). Similarly, we found a rela- tively strong and significant correlation between the current cumulative graduate GPA and student performance in field instruction (r=.542; p<.0001). Similarly, the end of first year GPA had a significant correlation with first year field instruction performance (r=.573; p<.0001). However, undergraduate GPAwas not significant- ly related with field performance. The GRE yielded a weak correlation with field performance (r=.224) that was not significant at .05 level. It should be noted here that the GRE and undergraduate GPA were factors in calculating the total weight- ed admission score. Although individually these were not correlated with field instruction performance, the total weighted admission score was associated with student performance in the field. Several factors considered in the admission process and the end of first year cumulative GPA were positively correlated with the field instruction perform- ance. Linear regression analysis using a stepwise method was used to identify the predictors of field instruction performance. Undergraduate GPA, GRE, academic potential, leadership potential, human service experience, quality of reference letters, and end of first year cumulative GPA were used as independent variables. The first year field instruction performance score was treated as the dependent variable. Only the end of first year cumulative GPA (β=.502***) and human service experience (β=.329**) emerged as significant predictors. These variables com- bined accounted for 47% of the variance in field instruction performance (Adjusted R2=.407). Group Comparisons The differences in admission criteria mean scores (such as GRE, UGPA, and total weighted admission score) of full time and part time students admitted in 2000 and 2001 were compared as shown in Table 4. It is important to note that mean GRE score dropped for full time students in 2000 from 1433.85 to 1226.19 for full time students in 2001. Similar drops in mean scores for full time students admit- ted in 2001 were observed in total admission scores, field instruction scores, and current cumulative graduate GPA. Variables End of First Year Field Instruction Cumulative GPA Performance Intellectual/Academic potential .573*** .209 Human service experience .122 .377*** Leadership potential .005 .189 Quality of reference letters .222 .376*** ***p<.001 Table 3: Bivariate Correlation of Admission Criteria Items with End of First Year Cumulative GPA and Field Instruction Performance (n=68) 43Thomas et al./EFFECTIVENESS OF ADMISSION CRITRIA ON STUDENT PERFORMANCE In order to test whether full time students differ significantly frompart time stu- dents on any of the above items, a comparative analysis of group mean differ- ences of GRE scores, UGPA scores, total weighted admission score, and current cumulative GPA was performed. The results did not yield any significant group differences. Similarly, t-tests were run on these items for students admitted in 2000 and 2001. Table 5 summarizes the results. The data show that these two cohorts differ significantly on current cumulative graduate GPA (t=2.59; p=<.01). FINDINGS Do students who were rated high in admissions criteria items also score high in their first year MSW courses?With regard to this first research question the study found that out of the four items that constituted the total weighted admission score, only academic potential significantly correlated with the end of first year GPA. The other three items did not translate into classroom performance of grad- uate students. Although these results cannot be generalized due to the small sam- ple size, these findings suggest that social work programs and admission proce- dures may want to pay more attention to intellectual and academic potential. Admission Year 2000 Admission Year 2001 Areas Full Time Part Time Full Time Part Time GRE mean score 1433.85 1233.24 1226.19 1246.50 UGPA mean score 3.24 3.19 3.24 2.95 Total weighted 87.30 86.58 84.78 89.33 admission score Field instruction 91.15 89.94 85.05 89.33 Score Current cumulative 3.71 3.69 3.58 3.40 graduate GPA Table 4: Mean Comparison of GRE, UGPA, Total Admission Score and Field Instruction Score (n=68) Items 2000 2001 Mean t-Test t-Test Admission Admission Difference Significance Values Mean Mean GRE 1316.71 1222.65 94.06 .171 1.38 Total weighted 86.87 86.27 .60 .76 .29 admission score Undergraduate 3.22 3.1 .10 .33 .96 GPA Current cumulative 3.7 3.4 .23 .01** 2.59 Graduate GPA Field instruction 90.47 86.61 3.86 .10 1.65 performance score **p<.01; ***p<.001 Table 5: Admission Criteria and Performance Differences Between Students Admitted in 2000 and 2001 (n=68) 44 ADVANCES IN SOCIALWORK The study further found a significant relationship between GRE, UGPA, total weighted admission score, and end of first year GPA. Clearly, students’ perform- ance in the GRE and their undergraduate success translates into academic per- formance in the first year graduate program. As evidenced by the correlation sta- tistics, the undergraduate GPA showed the strongest relationship with graduate students’ performance in the classroom. This is also consistent with the findings of Dunlap, Henley and Fraser (1998), where GRE and UGPA were predictors of graduate performance. With regard to the correlation between undergraduate GPA and end of first year graduate GPA, our study also confirms similar findings by Pelech, Stalker, Regehr and Jacobs (1999). Admittedly, these findings may offer some support for including GRE scores and undergraduate grades as part of admission criteria. In spite of the scholarly criticisms leveled against using GRE and UGPA as predictors of academic success, evidently, there is merit in using themas part of an effective admissions process.While the support for GRE is clear based on this study, the applicants’ success in undergraduate programsmay have accounted for their success in the GRE. The second question asked: Do students who perform better in classroom instruction also perform better in field instruction? Of the four admission crite- ria, only relevant human service experience and quality of reference letters had a significant, positive relationship with the field performance of graduate students. Academic potential did not translate into their field performance. However, the total weighted admission score (combination of four admission items scores) sig- nificantly correlated with field instruction performance. This is indicative of the need to give more weight to applicants’ relevant work experience and the testi- mony of referees in the admission criteria. This raises an important issue con- cerning the integration of classroom performance with field performance as the profession not only requires social workers to demonstrate skills in empathy, warmth, and establishing relationship with clients, but also necessitates social workers as having an adequate knowledge base as a springboard from which these skills can develop. It is important to exercise caution, once again, regarding the small sample size of the study in generalizing these findings. These findings show that both the end of first year cumulative GPA and current cumulative GPAs had a significant and positive relationship with the field instruction performance of students. This suggests that students are able to apply the knowledge and skills acquired in the classroom to their field situations. Along similar lines, Dailey (1979) found a significant correlation between classroom performance and field performance, indicating some commonalities that are essential for success in both these areas. It is these common factors that should form the basis of any sound admission process in social work. Another noteworthy point that does not correspond to common expectations is the absence of a significant relationship between undergraduate GPA and field instruction performance. Are classroom and field instruction outcomes the same for different cohorts, such as full-time, part-time, first year, and second year MSW students? With respect to this third research question, a significant difference was observed only between cohorts admitted in 2000 and 2001 in terms of current cumulative GPA. It is important to exercise caution in interpreting the results, as there was a larg- er pool of students in 2001 compared to 2000. Moreover, some students who could not cope with the increasing demands of the program dropped out in the first year. This may have left behind a more resilient group of students during 2000, as compared to 2001. Interestingly, there was no significant difference between full time and part time students in terms of their GRE scores, UGPA scores, total weighted admission score, and current cumulative GPA. Results from the regression analysis identified two significant predictors of field instruction performance of graduate students, namely, end of first year cumula- tive GPA and relevant human service experience. This underscores the signifi- cance of considering work experience as an important component in the criteria for admitting students into graduate programs. This finding is consistent with Pfouts and Hanley’s study (1977) that identified paid work experience as a signif- icant predictor of field performance. Demographic data revealed that the mean age of the population studied was 39.5 years. This is reflective of the growing number of non-traditional students seeking MSW degrees and suggests that social work programs need to modify their admission criteria and/or provide additional help for the non-traditional applicants. This was also consistent with the findings of Moxley, Najor-Durack, and Dumbrigue (2000). With respect to ethnicity, the majority of students were Caucasian, although close to a fifth were African-American, and a littlemore than a fifth were Hispanic. Consistent with the current distributive pattern in other programs, most of the students in this program were women. Although the findings of this study add to the existing literature, there are sev- eral limitations in this study that need to be considered. It should be noted that it is primarily a baseline study using a small sample size in a relatively new gradu- ate program in social work. Hence, it only provides an empirical base for future theoretical formulations by identifying amatrix of important pre-admission vari- ables associated with graduate student performance in classroom performance and field performance. The relationship between demographic variables and stu- dent performance was not explored, as it was not within the scope of this paper. However, it is important to note that demographicsmay also influence successful student learning outcomes. Furthermore, in rating student applications, faculty raters have relied on their experience, knowledge, and information from the social work literature in weighting the problem solving ability, leadership poten- tial, and quality of the references. These factors serve as external threats to the generalizability of the study’s findings to other sub-populations. In conclusion, the study reiterates the significance of adopting appropriate admission criteria in selecting suitable students for graduate social work pro- grams. Although these findings cannot be generalized for all programs mainly due to the small sample size, these results identify indicators and predictors that are related to the success of graduate student performance in classroom and field instruction. Similar studies using large sample size need to be replicated in other settings in order to validate and support these findings. This is an important area of research that would contribute to the establishment of effective admission cri- teria for graduate social work education. The findings in this short study prompt- ed the Admission Committee to revise its methodology by adapting the old crite- 45Thomas et al./EFFECTIVENESS OF ADMISSION CRITRIA ON STUDENT PERFORMANCE rion and reassigning the numerical weight within each of the variable categories, e.g., GPA, GRE, and work experience. Undoubtedly, the gate-keeping function of the profession begins with a reliable and valid admissions process, which, in turn, will contribute to the highest standards for the graduates later in their profes- sional practice. References Cole, B.S. (1991). Legal issues related to social work programadmissions. Journal of SocialWork Education, 27, 18-24. Constable, R.T. (1977). A study of admissions policies in undergraduate education. Journal of Education for SocialWork, 13(3), 19-246. Dailey, D.M. (1979). The validity of admissions predictions: A replication study and implications for the future. Journal of Education for SocialWork, 15(2). Donahue, B., & Thyer, B.A. (1992). Should the GRE be used as an admissions requirement by schools of social work? Journal of Teaching in SocialWork, 6(2), 33-40. Duehn,W.D., & Mayadas, S.N. (1977). Entrance and edit requirements of professional social work educa- tion. Journal of SocialWork Education, 13(2), 22-29. Dunlap, W.R. (1979). How effective are graduate social work admission criteria? Journal of Education for SocialWork, 15, 96-102. Dunlap, K.M., Henley, H.C., & Fraser, M.W. (1998). The relationship between admissions criteria and aca- demic performance in anMSW program. Journal of SocialWork Education, 34, 455-462. GlenMaye, L., & Oakes, M. (2002). Assessing suitability of MSW applicants through objective scoring of personal statements. Journal of SocialWork Education, 38, 67-82. Glisson, C.A. &Hudson,W.W. (1981). Applied statisticalmisuse in educational research: An admissions cri- teria example. Journal of Education for SocialWork, 18(2), 35-44. House, J.D., & Johnson, J.J. (2002). Predictive validity of the graduate record examination advanced psy- chology test for grade performance in graduate psychology courses. College Student Journal, 36, 32-37. Miller, J., & Koerin, B. (1998). Can we assess suitability at admission? A review of MSW application proce- dures. Journal of SocialWork Education, 34, 437-453. Milner, M., McNeil, J.S., & King, S.W. (1984). The GRE: A question of validity in predicting performance in professional schools of social work. Educational and Psychological Measurement, 44, 945-950. Moxley, D.P., Najor-Durack, A., & Dumbrigue, C.Y. (2000). Seven strategies for facilitating access of nontra- ditional students to graduate education in social work. SocialWork Education, 19, 335-348. Pelech,W., Stalker, C.A., Regehr, C., & Jacobs, M. (1999). Making the grade: The quest for validity in admis- sions decisions. Journal of SocialWork Education, 35,c215-226. Pfouts, J.H., & Henley, H.C. (1977). Admissions roulette: Predictive factors for success in practice. Journal of Education for SocialWork, 13, 56-63. Sampson, C., & Boyer, P.G. (2001). GRE scores as predictors of minority students’ success in graduate study: An argument for change. College Student Journal, 35, 271-280. Wodarski, J.S. (1979). Critical issues in social work education. Journal of Education for SocialWork, 15(2) 5-13. Author’s Note: Address correspondence to: M. Thomas, Ph.D., Department of Social Work at California State University Bakersfield, Bakersfield, California, 93311-1099, USA. E-mail: mthomas5@csub.edu Note: An early version of this paper was presented at the 49th Annual ProgramMeeting of the Council on Social Work Education, February 27–March 2, 2003, Atlanta, Georgia. 46 ADVANCES IN SOCIALWORK