AISW Vol. 5, No. 1 Using Portfolios: Integrating Learning and Promoting for SocialWork Students Mona C.S. Schatz Abstract: Portfolios are a valuable educational tool to aid in the integrative experi- ence for graduate social work students. Forty-one graduate students were asked to evaluate their portfolio experience. A Pearson correlation shows that graduate stu- dents find the experience of developing a portfolio to be reflective of their second year MSW program (r=.511; p<.01), reflective of their competence as a social worker (r=.587; p<.01), and reflective of their personal uniqueness (r=.526; p<.01). All stu- dents demonstrated generalist social work practice through the inclusion of materi- als reflecting practice with individuals, families, groups, organizations, agencies, and communities.Students also report that the portfoliowas a valuable tool to foster inte- gration of class and field learning (N=24 or 58.5%).Findings reveal that two-thirds of the students, 68.3%, applied a “medium level of effort” in the development of their portfolios, yet were able to create a final product that adequately reflected their uniqueness, their integration of learning, and their competence as a second year stu- dent. Keywords: Portfolio, reflection, integration, social work education Integration of learning implies that there is a process wherein students takediscreet ideas, thoughts, and knowledge and move to a level of synthesis, incorporating different types of information and thought processes to cre- ate their own personal perspective. Lowy, Bloksberg and Walberg (1971) describe the need to integrate learning as more than a process of aggregating components. They posit that this process of integration is meant to suggest “organic unity, creative synthesis, psychological Gestalt” [author’s italics] (p. 13). Achieving integration for social work students is vital (Lowy, Bloksberg & Walberg, 1971) and must merge the basic foundation concepts and skills in the educational curricula, and later integrate specialized and advanced knowledge and practice. Problems in reaching adequate integration exist. For example, the prevailing theoretical and practice base of generalist social work is so broad that some students are unable to grasp its extensiveness without student-centered educational approaches that aid in the integrative process. 105 Mona C. S. Schatz, DSW is Professor, School of SocialWork; Director, Education and Research Institute for Fostering Families, Colorado State University; and Coordinator and faculty for the Great Plains-IDEA Youth Development Graduate Online Program, Colorado State University, Ft. Collins, CO 80523. Copyright© 2004 Advances in SocialWorkVol. 5 No. 1 (Spring 2004) 105-123. Indiana University School of SocialWork. Furthermore, the experiential learning component in social work, the field placement, varies widely for students. Thus, in some cases, social work pro- grams do not succeed in providing adequate exposure and practice compe- tence related to central curricular areas. A portfolio is a collection of materials assembled in a manner that demon- strates either a prescribed outcome, such as a financial portfolio, or a self-ini- tiated professional outcome, such as an artist uses to bring his or her “best” work to a potential employer. Portfolios have been used in some schools of social work at graduate and undergraduate levels. The value of the portfolio among undergraduate social work students was examined several years ago by Simon and Schatz (1999). In this study, field supervisors identified six ben- efits for students, highlighting its value for self-directed learning. Benefits included: (1) its practicality as a self-directed learning approach; (2) a more focused approach for student learning; (3) greater ability for students to gain an understanding of social work in the social agency; (4) a way of getting stu- dents more organized “in what is often a somewhat chaotic experience;” (5) being able to combine school and field learning through a collection of mate- rials; and, (6) encouragement of student’s own creativity, thus promoting the uniqueness of each learner. (p. 104) No published articles were found that addressed the use of portfolios at the graduate level. In addition, no articles appear in publications that examine either the reflective processes inherent in portfolio develop or the integrative process that is achieved from the use of a portfolio approach. The key question that motivated this study was whether portfolio projects can serve as a valid assessment tool for the graduate learning experience. Does this tool foster a strong process for student integration of the field and classroom educational experiences? A second question sought more under- standing about the role of self-reflection in this learning approach. It asked what role self-reflection plays as students formulate their portfolios. Before moving into the study design, a brief overview about portfolios is provided. Background on Portfolios The use of portfolios is extensive. Discussions span fields as diverse as occu- pational therapy (Kramer & Stern, 1994), health information management (Barron & Sartori, 1994), higher education—graduate level (Condon, 1994; Palmer, Holahan & Johnstone, 1996), field-based experiential higher educa- tion (Lewis &Williams, 1994), social work (Risler, 1999; Simon & Schatz, 1998), and doctoral candidacy (Heiges, 1994). Portfolios also are used to prepare new teachers (Weiser, 1994), foster literacy (Irwin-DeVitis, 1996; Standerford, 1996), teach pre-service English teachers (Yagelski, 1994), English methods (Yancey, 1994; Condon, 1994), creative writing (Fischer, 1994), and develop- mental writing (Rich, 1994). Generally, what students are asked to include in portfolio projects is unique. The portfolio development related to this specific study asked second year graduate social work students to examine their experiences in the field expe- rience and select examples of practice that illustrated their generalist and 106 ADVANCES IN SOCIALWORK advanced generalist practice competencies. These competencies were broad- ly presented at the beginning of the academic year, thus allowing students wide latitude in organizing their notebooks. Portfolios that ask students to collect their class papers and class projects serve more as a “mini-filing” expe- rience and may actually limit some of the real educational benefits that port- folios can offer related to self-discovery, creativity, and self-expression. Strengths frequently associated with portfolio use include the self-directed nature of the assignment, the shared responsibility for assessing the learning experience, and the ongoing developmental nature of the product (ERIC Trends and Issues Alerts, 1993). Portfolios allow students to better portray their learning experience within its unique context (e.g., Yancey, 1994). They help students’ capture personal meaning (Barnett & Lee, 1994) from the learn- ing process. White (1994) states that the portfolio brings teaching, learning, and assessment together as mutually supportive activities in the educational milieu (p. 27). Toward confirming that premise, proponents also suggest that portfolios offer the world of [educational] assessment a view of student learn- ing that is active, engaged, and dynamic, as opposed to the overwhelmingly passive concept that still dominates the educational assessment movement (White, 1994, p. 27). Yet, many of these proponents (e.g., Paulson & Paulson, 1990; White, 1999; Elbow, 1994) are clear that educators must produce evi- dence beyond personal testimony that this educational assessment tool is effective, credible, beneficial, and capable of achieving its intended pur- pose(s) as an assessment tool for the learner(s). Portfolios in the University Milieu During the last two decades, universities have been scrutinizing the quality of education, seeking to better define and articulate learning outcomes. Schon (1987) states that educators worry about the gap between a school’s percep- tion of professional knowledge and the actual competencies required of prac- titioners in the field (p. 10). Ashelman and Lenhoff (1994) suggest that main- taining portfolios for graduate and undergraduate students serve three pri- mary departmental goals. First, they allow departments an assessment strat- egy congruent with the department’s need; second, instruction and assess- ment are based on a constructivist approach; and third, the assessment, when using the portfolio, involves the faculty in a similar process of self-reflection and individual change—a collegial process (p. 66) Use of Portfolios in Social Work Portfolios have been used in social work education programs for several decades at least. At national conferences and workshops over the last 10 or 20 years, social work educators have presented their perspectives and experi- ences using portfolios. Risler (1999) suggested that the portfolio invites diver- gent thinking. This provides valuable illustrations of collaboration in the learning process which occurs between students and instructors during field experiences. For social work education, Chambers and Spano (1982; also see Knox, 1986) believe that integration implies a synthesis, but that synthesis only occurs if the student is made aware of how the elements of the learning are interrelated. They further inform us that the learning process must be 107Schatz/INTEGRATING LEARNING AND PROMOTING REFLECTION THROUGH PORTFOLIOS “reflective, rather than a reflex” (p. 229). They do not mention educational tools which would advance this reflection, however. A portfolio can accom- plish this integration while also allowing students the opportunity for self- direction and self-reflection. Evaluation, Assessment, and Grading of Portfolios In the educational field, student portfolios have been assessed by Knight and Gallaro (1994) to benefit: (1) curriculum, (2) student learning in the class- room, and (3) improving student satisfaction in the learning experience. Standerford (1996) suggests that this is the ultimate goal of educational assessment. Others (e.g., Ashelman & Lenhoff, 1994) suggest that the process of engaging students in self-assessment and reflection for the purpose of making judgments about their own work is highly individual and personal, requiring high order thinking of critical awareness and non-defensiveness of one's evaluative strategies (p. 75). The 1990s provided valuable discussions among among educational spe- cialists, particularly related to the evaluation, assessment, and grading of portfolios (see Paulson & Paulson, 1990; Yancey, 1994; Weiser, 1994). Other issues that have been discussed when assessing portfolios include the wide variability of material included in portfolios (Ashelman & Lenhoff, 1994) and the validity of the portfolio to measure what it is intended to measure (Yancey, 1994). An evaluation study of portfolios by undergraduate social work students found positive support among students and field instructors. Simon and Schatz (1998) used an evaluative survey instrument to assess how well stu- dents perceived their portfolio process as an integrative of the field and class- room learning. In a second study using a survey instrument, Schatz and Simon (1999) found that both field instructors (N=14) and students (N=39) believed that the portfolio supported the integrative aspects of learning need- ed for students in an undergraduate generalist program. Students and field instructors saw the portfolio project as extremely valuable, indicating that the portfolio brought more depth to learning in the field placement experience. This study did not examine how students made decisions related to their final portfolio presentation. PURPOSE ANDMETHODOLOGY This study explored graduate social work students’ decisions regarding how they organized their portfolios and the role self-reflection played in this learning strat- egy. Questions developed to guide this study included the following: 1. Does the portfolio process used for second year graduate social work students promote reflective thinking? 2. Do students achieve an integration of class learning and field experience through the use of a portfolio process? 3. Do students see this tool as helpful for both academic and profes- sional endeavors? 108 ADVANCES IN SOCIALWORK 4. Do students view their portfolio process as promoting their com- petence? 5. Doportfolios adequately illustrate generalist and advancedpractice? RESEARCHDESIGN A research approach that was both qualitative and quantitative was constructed to respond to the research questions. A student survey instrument gathered qual- itative and quantitative responses about the portfolio development experience (see Appendix A). The questionnaire instrument served as a tool for the evalua- tion of the student’s actual portfolio notebook, which was handed in at the end of a year-long seminar and field placement experience. The instructor made the decision to use the portfolio with second year concentration students, in part, because she was the instructor of the seminar class, and in part, because this instructor views the portfolio as an effective self-assessment instrument that can be used throughout one’s social work career. A second research tool (Appendix B) was used by a team of three researchers: Two were graduate research assistants and the third was the author. The team providedmore objectivity and consistency when examining the quality of portfo- lios, because the author also served as the instructor for the student participants. The evaluation of portfolios done by the team was conducted after the grading period so that there would be no inherent conflict or bias in the team’s evaluation of portfolios. Design of Two Instruments: The Survey and Portfolio Review Instrument Students were asked to complete a written survey instrument that inquired into the creation of their portfolio. This survey was completed at the end of the con- centration year graduate experience. To answer the research questions, the sur- vey looked as several focuses. First, some questions explored what types of items students included in their portfolio (Questions 1, 5, 6, and 7) and informed research questions 3, 4, and 5. These items, in aggregate, represented a generalist social work orientation (Schatz, Jenkins & Sheafor, 1990). Second, some questions asked about attitudes related to the portfolio process and the student’s reflection about that process (Questions 2, 3, 4, 5, 11, and 12) and informed research ques- tions 1, 2, and 4. For example, some questions sought to capture student deci- sions about what they chose to include in their portfolio and what they excluded, if any (Questions 6b, 7b). Third, some questions examined areas such as the stu- dent’s level of effort (Questions 10), their view of the strengths of this assignment (Question 9), if they had shared their portfolio with others (Question 8), and if they would continue to use this tool in the future (Question 13). Question 12 asked students about grading and evaluating portfolios. These questions informed research questions 1, 2, and 3. The survey also asked students their age, gender, and social work field setting. The second tool (see Appendix B) was designed in order to use a team score that judged the quality of the portfolio and whether it demonstrated aspects such as: a) the portfolio’s ability to be seen as a highly professional tool, b) the quality of the organization, c) the level of effort, d) the demonstration of social work com- 109Schatz/INTEGRATING LEARNING AND PROMOTING REFLECTION THROUGH PORTFOLIOS petency, e) the demonstration of integration of class and field experience, f) demonstration of advanced generalist practice orientation, and g) reflective qual- ity used by the student in organizing their portfolio. The three research team members did a“trial run”with a portfolio fromanundergraduate student in order to discuss the rating process towards providing consistency on what each item in the review instrument sought to evaluate. Student Sample Student participation was voluntary. Forty-one of the 42 students in the sample completed the written survey (97.6% of the population). Sixteen of the 42 stu- dents, (38.1% of the population) agreed to allow the research team to review their portfolio. This lower participation rate was due to timing: this portfolio assign- ment was due at the end of the graduate coursework and students just wanted to leave and take their work with them. Reviewing these portfolios by the team required an extra week for review purposes. Respondent demographic information shows that the mean age of graduate social work student respondents was 32.69 (standard deviation=7.85); 75.6% (N=31) were women and 24.4% were men (N=10). The field placement setting of students varied. Thirteen students worked in family and children's programs, 11 students were in mental health programs, 7 were in community and education programs, 6 were in medical settings, and 4 were in corrections. RESULTS The portfolios were unique. Illustrative of the reflection process in portfolio development, the students were able to indicate which items in their portfolio illustrated their “best practice,” which item(s) were excluded from their portfolio, whether they were ambivalent about having included certain items in their port- folio, and what decisions influenced them in these decisions. The first research question asked: “Does the portfolio process used for concen- tration graduate social work students promote reflective thinking?” The process of reflecting on one’s competence and uniqueness: When asked if the portfolio served as a reflective tool, students indicated their level of agreement using a four-point Likert scale. A Pearson correlation coefficient was calculated to explore the relationship of these variables. A strong positive relationship was found among these three (r=.526; r=.511, and r=.587; 39 df; p<.001). Table 1 shows these correlations, indicating a reliable relationship between the variables. Demonstrating their competence: Demonstrating their competence: Each stu- dent was able to provide specific examples of his or her: a) emerging practice competence and b) “best” practice(s). Thus, students presented their portfolio in an integrated way and expressed their own sense of competence. A wide range of examples of practice was evident due to the many types of field placement agen- cies that students were involved in. From a pre-constructed list of nine (9) items, respondents were asked to indi- cate whether any of these nine items were included in their portfolio. Students were also asked if they included items such as class activities and assignments. 110 ADVANCES IN SOCIALWORK Table 2 shows the types of items included by students. Demonstrating their competence: Demonstrating their competence: Each stu- dent was able to provide specific examples of his or her: a) emerging practice competence and b) “best” practice(s). Thus, students presented their portfolio in an integrated way and expressed their own sense of competence. A wide range of examples of practice was evident due to the many types of field placement agen- cies that students were involved in. From a pre-constructed list of nine (9) items, respondents were asked to indi- cate whether any of these nine items were included in their portfolio1. Students were also asked if they included items such as class activities and assignments. Table 2 shows the types of items included by students. Respondents listed items in their portfolio that represented experiences beyond the field and classroom if they believed these items represented their social work competence. Twenty-four respondents (58.5%) included non-academic/field items. The lower section of Table 3 lists the most frequent items included which were of this nature, such as copies of grant submissions, certificates, certification awards, licenses, and personnel evaluations. To explore the constructivist process of the portfolio notebook, students were asked to identify two examples from their portfolio that demonstrated their social work competence. A second question asked them to indicate one item in their portfolio that represented their “best” practice. Table 2 illustrates that “practice materials” were listed as most representative of their competence (38.73%) and best work (38.89%), credentials (21.92%) and field materials (15.07%) gained sec- ond in position, professional materials (10.96%) created by the student, and class work materials (8.22%) followed. Furthermore, it appears that students value their work with clients and their recognition for this work. 111Schatz/INTEGRATING LEARNING AND PROMOTING REFLECTION THROUGH PORTFOLIOS Reflects Reflects Reflects 2nd Personal SocialWork Year Field Survey Statements Uniqueness Competence Experience Reflects personal uniqueness Pearson correlation 1.000 .587** .526** Significance** . .000 .000 Reflects social work competence Pearson correlation .587** 1.000 .511** Significance** .000 . .001 Reflects 2nd year field experience Pearson correlation .526** .511** 1.000 Significance** .000 .001 . *Degrees of freedom=39; **Correlation is significant at the 0.01 level (2-tailed). Table 1: Pearson Correlation Demonstrating Reliability of Student Responses on Perception of Competence and Uniqueness (n=41)* Respondents identified certain materials which they had ambivalence about including in their portfolios. For example, some respondents were unsure or uncomfortable including “class work” (25.93%), certain “field materials,” particu- larly if the materials might divulge someone’s identity (18.52%), and “credentials” (14.81%). “Non-client specific practice materials” and “materials developed by the students” were items students’ were less ambivalent about (7.41%). Respondentswere asked if they excluded items from their completed portfolios. Twenty-five respondents (61.0%) indicated that they excluded items when final- izing their portfolio. Most frequently, students excluded items that were “not reflective of me or my best practice” (29.63%) or were “left out because of confi- dentiality.” For a small number of students, “space constraints” (14.81%) and items from their “distant work experiences” (14.81%) were reasons for exclusion. The Process of Reflecting on One’s Competence and Uniqueness When asked if the portfolio reflected the respondent's sense of competence as a second yearMSW student and as a social worker, as well as their personal unique- ness, students indicated their level of agreement using a four-point Likert scale. A Pearson correlation coefficient was calculated to explore the relationship of these variables. A strong positive relationship was found among these three correla- tions (r=.526; r=.511, and r=.587; 39 df; p<.001). Table 3 shows these correlations, indicating a reliable relationship between the variables. 112 ADVANCES IN SOCIALWORK Items That Reflected TheirWork Number Percentage in Field Placement Work with individuals 33 80.5% Work with clients in groups 24 58.5% Work with families 18 43.9% Community work 35 85.4% Training and seminars 39 95.1% Funding activities 11 26.8% Administrative activities 30 73.2 Public relations/media activities 12 29.3% Class assignments and activities 35 85.4% Items from Other Professional Experiences Other training 3 * Certifications and licenses 3 * College transcripts 1 * Grants 2 * Public relations/media materials 3 * Personnel evaluations 2 * *Individual items not tabulated, however, 24 respondents (58.5%) did include some type of non-field items. Table 2: Portfolio Items That Reflect Educational Experience and Items That Reflect Practice Competence From Other Professional Experiences Respondents identified certain materials with which they had ambivalence including in their portfolios. For example, some respondents were unsure or uncomfortable including “classwork” (25.93%), certain “field materials,” particu- larly if the materials might divulge someone’s identify (18.52%), and “credentials” (14.81%). “Non-client specific practice materials” and "materials developed by the students" were items students were less ambivalent about (7.41%). Respondentswere asked if they excluded items from their completed portfolios. Twenty-five respondents, or 61.0%, indicated that they excluded items when finalizing their portfolios.Most frequently, students excluded items that were “not reflective of me or my best practice” (29.63%) or were “left out because of confi- dentiality.” For a small number of students, “space constraints” (14.81%) and items from their “distant work experiences” (14.81%) were reasons for exclusion. The second research question asked: “Do students achieve an integration of class learning and field experience through the use of a portfolio process?” Respondents were asked to evaluate whether “The portfolio had been a valuable 113Schatz/INTEGRATING LEARNING AND PROMOTING REFLECTION THROUGH PORTFOLIOS Portfolio Items That Portfolio Items That Represent Represent SocialWork “Best” SocialWork Practice Portfolio Item Competence N* % of Total N** % of Total Responses Responses Practice materials: assessments/ 29 39.73% 14 38.89% interventions Credentials, e.g., resume, certifications, 16 21.92% 2 5.56% license Field materials, e.g., learning plans, 11 15.07% 5 13.89% evaluations Professional materials 8 10.96% 6 16.67% developed by student Class work materials, 6 8.22% 4 11.11% e.g., organizational scans, research projects Other 6 8.22% 6 16.67% e.g., thank you’s, letter of recommendation, syllabi Total 73 100.00% 36 100.00% *Respondents were asked to provide two examples of portfolio items that represented their social work competence. Some respondents provided more than two responses while others did not respond at all. **Respondents were asked to provide one “best” item that represented their social work competency. Some respondents provided more than one response while some did not provide any response. Table 3: Portfolio Items Respondents Identified as Demonstrating Social Work Competence (N=41) way to integrate class and field learning.” Twenty-four students, or 58.5%, indi- cated their “agreement” or “strong agreement” with this statement; only 9.8% of students “strongly disagree” (Mean=2.44; SD=.78). Reviewers determined whether, in their assessment, the portfolios demonstrat- ed the integration of class and field experience. Using an 8-point Likert scale, the mean score from this review was 4.56 (standard deviation=2.42) slightly higher thanmid-point. This is similar to the mean of respondents (Mean=2.44, standard deviation=.78). “Do students see this tool as helpful for both academic and professional endeavors?” To assess this third question, whether the portfolio served as an edu- cational or professional tool during the past year, students were asked whether they shared their portfolios with others, such as faculty, supervisors, co-workers, family, and fellow students. Thirty-eight (38) students responded (92.7%) affirma- tively to this question.Thirty-five respondents, or 85.4%, sharedportfolioswithother students. Faculty were shown portfolios by 18 students or 43.9% of the respon- dents. Thirteen or 31.7% of the students shared their portfolios with their families. Question four asked: “Do students view their process of portfolio development as promoting competence?” The students were asked about the best way to eval- uate their portfolio. Results show that only one student believed that the instruc- tor should solely assess the portfolio. Most frequently, respondents thought that the assessment process should be mutual, the student meeting with the instruc- tor to discuss the development process and the final submission (14 or 41.18%). Twelve (12) students, or 35.29%, suggested that self-grading the portfolio was the best way to accomplish the evaluative process. Finally, “Do portfolios adequately illustrate generalist and advanced practice?” Because the graduate educational experience rests in a specialized area of study, the research sought to examine whether students’ portfolios illustrated the gen- eralist and advanced generalist social work specialization. Table 4 shows a strong validation for the generalist social work approach. As shown, students have included materials that illustrate the multiple level practice experience (micro- practice,mezzo practice, andmacro practice) in field and themulti-method prac- tice process. The advanced level assessment was only possible to ascertain when reviewers looked at the nature and extent of the practicematerials included. As an example, reviewers were looking for the complexity of practice presented by students. The reviewers used an 8-point Likert scale, “1” representing “very high” and “8” indi- cating “very low.” Reviewers might judge, for example, an advanced generalist representation in the portfolio, whether examples of treatment plans were included, or grant projects or other activities requiring application of competen- cies generally viewed as “graduate level.” The mean score of reviewers for this demonstration of advanced generalist social work practice was 3.5 (standard deviation=2.68), representing a “high” score in this area. Examination of Students’ Level of Effort in Portfolio Development Though no one research question specifically asked about the level of effort, the idea that different student levels of effort might influence the quality of the work 114 ADVANCES IN SOCIALWORK was surmised.When the concentration year began, students were given the port- folio assignment that this research examined. It was suggested at that time that students use their portfolios as an on-going tool where they could collect items that would be indicative of their field and class learning experiences. Since the survey was given at year’s end, the students were asked to select from a three- point scale of “high,” “medium,” and“low” to indicate their level of effort in devel- oping the portfolio. To better guide the response process, these three response choices had a short descriptive narrative as follows: “High–worked on [the port- folio] regularly during the year;” “Medium–worked on intermittently throughout the year;” and “Low–worked on just the last few weeks of each semester.” Table 5 shows that two-thirds of the respondent students (68.3% or 24) put a medium level of effort into development of the portfolio. Nine students or 22% put in a low level of effort; less than 10% (8 students or 9.8%) of the sample population placed high effort into this project. Significancewas foundwith the level of effort and correlationswith (1) the port- folio’s value as a tool for class and field integration (r=.394; p<.005 (one-tailed)), (2) the ability of the portfolio to reflection the second year MSW experience (r=.302; p<.028), and (3) the use of the portfolio as a tool that reflects the student's uniqueness as a social worker (r=.383; p<.007(one-tailed)). No significance was found when correlated with the variable that assessed whether the portfolio reflected the student's competence as a social worker (r=.228; p<.076). The review team that examined each portfolio also assessed the level of effort. This review of portfolios revealed that the level of effort for the sample as com- pared with the population as a whole is significant (Mean=3.5; Sig. .000, p<.001). Having used an 8-point Likert scale, the mean of 3.5, between “high” and “medi- um” on the rating descriptors is quite similar to the respondents themselves, whose mean is 2.12, a middle-rating on a 4-point Likert scale. 115Schatz/INTEGRATING LEARNING AND PROMOTING REFLECTION THROUGH PORTFOLIOS Yes, Included Not Included Not Sure Portfolio Items N % N % N % Micro practice: Work with individuals 33 80.5% 7 17.0% 1 2.4% Mezzo practice: Work with groups 24 58.5% 14 34.1% 3 7.3% Work families 18 43.9% 21 51.2% 2 4.9% Macro practice: Community work 35 85.4% 4 9.8% 2 4.9% Training and seminars 39 95.1% 1 2.4% 1 2.4% Administrative activities 30 73.2% 6 14.6% 5 12.2% Funding development 11 26.8% 26 63.4% 4 9.8% Class activities/ 12 29.3% 28 68.3% 1 2.4% assignments Table 4: Items Included in Portfolio That Represent Multiple Systems Level Practice and Multi-method Practice (N=41) DISCUSSION The study found significant support for the idea that development of a portfolio can serve as a tool that reflects social work competence for graduate students who are nearing the end of their academic experience. Their ability to identify items that represent their competency is evident. The respondents could also identify what items in their portfolio reflected their best work. The items that stu- dents included in their portfolios illustrate their multi-level practice with individ- uals, agencies, and communities, and their class assignments and related school activities. Training and seminar information accumulated during the advanced concentration year were included in student portfolios and were viewed as important. This may be the case, because these external learning opportunities actually enhanced the person's learning, e.g., field experiences, and probably provided more advanced knowledge and skill development. Students’ work with clients was included in all portfolios. For some, issues of confidentiality brought ambivalence for them regarding whether to include this kind of information in the portfolio. These results indicate that students were ambivalent about how or whether they should put client work into their portfo- lio, demonstrating their ability to reflect on value and ethical issues related to practice and professionalization. It is interesting to learn that a medium amount of effort was adequate to build quality individual portfolios. Among some opponents of portfolios there has been expressed concern that they are too time consuming (Simon & Schatz, 1998) and, therefore, not a valuable learning tool. From this respondent group, it may be more realistic to suggest that a medium level of effort is sufficient for students to create a professional/educational tool that demonstrates social work compe- tence. It is worthy of mention that the assessments made by students regarding their level of effort was based on a set of definitions presented in the survey tool. It is possible that the descriptors served to create amid-range response, since stu- dents were comfortable stating that their work was actually intermittent through- out the year, versus, regularly worked on, which qualified for the “high” descrip- tor. This honesty gave the researcher the ability to conclude that the construction of the portfolio, as intended, does not have to be overwhelmingly time-consuming. The time needed by the instructor to examine the portfolios is often a concern. For this author and for those who have joined in adopting portfolios as assess- ment tools, the time and effort is worth the outcome. Students are so pleased to 116 Responses Choices N* 1% High 8 9.8% Medium 24 68.3% Low 9 22.0% Total 41 100.0% *Mean=2.12; sd=.56 Table 5: Level of Effort in Portfolio Development (N=41) ADVANCES IN SOCIALWORK have a tangible product that illustrates the many tasks and responsibilities they undertake in their education and field experience. Faculty should not shy away from this teaching tool because of time requirements. The respondents demonstrated their generalist orientation and advanced gen- eralist practice. The team assessed that more than 80% of the students included materials that reflected advanced practice throughout the continuum of inter- vention levels—individuals, groups, families, organizations, and communities. This demonstration also reflects the specialized, advanced social work curricu- lum at this university program. A number of factors have become apparent through the analysis of data that suggest that there are considerations made about the construction of the portfo- lio and, in turn, influence what is included and not included in the portfolio. When students were asked what two items in the portfolio represent their social work competence, all students had a response. Yet, no two responses were alike. Second, when asked if they had included items in their portfolio that they were ambivalent about, 27 responded in the affirmative. Third, the students were then asked if there were items they had excluded when submitting their portfolio at year's end. Twenty-seven individuals had excluded items. While the items that respondents were ambivalent about or what they excluded is important, what may be more valuable is the awareness that these three questions, considered altogether, capture a process of personal decision-making and reflection that responds to one of the research questions initially posed. A series of five statements were used to uncover whether the portfolio served reflective purposes that included educational integration, social work compe- tence, and personal uniqueness. It was found that the perception of the portfolio’s value as an integrative tool was highly associated with its perceived value in determining social work competence. These correlations show the magnitude and direction of the linear correlations between each set of variables (Craft, 1990). The r scores show positive correlation with strong significance. This is not sur- prising since portfolio work reflects many things for students. Because the students were able to articulate what they did and did not include in their portfolio, the research uncovers a process of decision-making used by stu- dents and a process of reflection about their construction of their portfolio. It might be possible to imply from portions of this analysis that students amass a large quantity of materials that may be worthy of inclusion in their portfolio. Then, closer to the time when the portfolio is to be submitted for review, the stu- dent makes a series of determinations about the final set of materials that will actually be used as representations of practice and educational performance. The reflective process promotes aspects of the person's professional endeavors and their educational experience. Materials that are either too old or questionable on ethic grounds may be excluded, as well as other considerations such as the sheer size and magnitude of the volume. IMPLICATIONS With stronger demands to evaluate educational outcomes in academic programs throughout the nation, this study offers an educational strategy that could serve 117Schatz/INTEGRATING LEARNING AND PROMOTING REFLECTION THROUGH PORTFOLIOS as an outcome assessment tool. Schools of social work or individual social work faculty who consider using a portfolio project may aptly demonstrate what stu- dents are learning and how they take their learning and create integration for themselves. To achieve a process of integration, the educator could consider how class work and fieldwork combine to support that process for students.With fur- ther exploration, we could learn how a portfolio could become more useful for students. Another implication of this study found that students were able to achieve the portfolio experience by merely using a medium amount of work level. This find- ing may be helpful to students who are given the portfolio assignment in the future, because it allows them to realize that they can accomplish it without the assignment being overly time-consuming. LIMITATIONS OF STUDY The survey population comes from the same graduate program as the research team. There are biases when subjects know the researcher. Measures were taken to redirect some potential bias, however, the bias could not be eliminated entire- ly. The respondents represent 97.6% of the population under study. Therefore, the results confidently reflect the population under study. It is, however, not plausi- ble to generalize these findings to other schools of social work that use portfolios. For example, the school of social work used in the study has one specialization in a generalist perspective, where other schools of social workmay have several spe- cializations and, therefore, the survey may not be as useful. Consequently, the results may not meet the needs of other schools of social work that have multiple specializations. FUTURE CONSIDERATIONS Limited research has been able to capture student learning and reflective aspects related to integration in social work educational curricula. This study only initi- ates a possiblemethod, e.g., portfolios, to consider in this area. Though this study found some insights into a process of reflection related to building a portfolio, examining more closely how reflection services the educational experience should build from this initial effort. This study was intended to explore how students reflect and cognitively process construction of the portfolio. Five research questions were posed at the start of this study and have since been successfully answered. This study found the port- folio to be reflective of the students thinking around their competence in the sec- ond year of graduate school. It found that students are able, through the use of portfolios, to express their integration of class, professional uniqueness, and field learning. Respondents confirmed the generalist perspective and students work- ing in multisystem areas of practice. Students also believe that assessment of the portfolio is best done in a collaborative process with the instructor. Most impor- tant, the findings promote the idea that schools of social work can benefit from adopting a portfolio approach. 118 ADVANCES IN SOCIALWORK Endnote 1 These items represented major areas of social work practice in the field placement experience such as work with individuals, groups, families, and communities, as well as items that represented work with or involvement in training, seminars, administrative activities, and public relations/media activities. References. Ashelman, P., & Lenhoff, R. (1994). Early childhood education. InM. Knight (Ed.), Portfolio assessment: An application of portfolio analysis (pp. 65-76). Lanham, MD: University Press of America. Barnett, B.G., & Lee, P. (1994). Assessment processes and outcomes: Building a folio. In L. Jackson & R.S. Cafffarella, (Ed.), Experiential learning: A new approach 62(Summer) (pp. 55-62). CA: Jossey Bass. Barron, M., & Sartori, N. (1994). Planning a portfolio: Medical record administration/Health information management. In M. Knight (Ed.), Portfolio assessment: An application of portfolio analysis (pp. 9-19). Lanham, MD: University Press of America. Chambers, D.E., & Spano, R. (1982). Integration of learning in field instruction. In Sheafor, B.W. & Jenkins, L.E.,Quality field instruction in social work: Program development and maintenance (pp. 226-234). NY: Longman. Condon,W. (1994). Building bridges, closing gaps: Using portfolios to reconstruct the academic commu- nity. In L. Jackson & R.S. Cafffarella, (Eds.), Experiential learning: A new approach 62(Summer) (pp. 197-213). CA: Jossey Bass. Craft, J.L. (1990). Statistics and data analysis for social worker. Itasca, IL: F.E. Peacock Publishers. Elbow, P. (1994).Will the virtues of portfolios blind us to their potential dangers? In Black, L., Daiker, D.A., Sommers, J. & Stygall, G. (1994). New directions in portfolio assessment: Reflective practice, critical the- ory, and large-scale scoring (pp. 40-55). Portsmouth, NH: Boynton/Cook Publishers. ERIC Trends and Issues Alerts (1993). Portfolio assessment in adult, career, and vocational education. Columbus, Ohio: ERIC Clearinghouse on Adult, Career, andVocational Education. Fischer, K.M. (1994). Down the yellow chip road: Hypertext portfolios in Oz. In L. Jackson & R.S. Cafffarella, (Eds.), Experiential learning: A new approach 62(Summer) (pp. 338-356). CA: Jossey Bass. Heiges, J.M. (1994). Portfolio for doctoral candidacy. In L. Jackson & R.S. Cafffarella, (Eds.), Experiential learning: A new approach 62(Summer) (pp. 125-137). CA: Jossey Bass. Irwin-DeVitis, L. (1996). Literacy portfolios: The myth and the reality. In M.D. Collins and B.G. Moss (Eds.), Literacy assessment for today’s schools (pp. 135-144). VA: College Reading Association. Knight, M., & Gallaro, D. (1994). Summary. In M. Knight (Ed.), Portfolio assessment: An application of portfolio analysis (pp. 135-139). Lanham, MD: University Press of America. Knox, A.B. (1986). Helping adults learn. San Francisco: Jossey Bass. Kramer, P., & Stern, K. (1994). Portfolio assessment in occupational therapy. In M. Knight (Ed.), Portfolio assessment:An application of portfolio analysis (pp. 121-134). Lanham,MD:University Press of America. Lewis L.H., &Williams C.J. (1994). Experiential learning: Past and present. In L. Jackson & R.S. Cafffarella, (Eds.), Experiential learning: A new approach 62(Summer) (pp. 5-16). CA: Jossey Bass. Lowy, L., Bloksberg, L.M., & Walberg, H.J. (1971). Integrative learning and teaching in schools of social work. NY: Association Press. Palmer, B.M., Holahan, M.E., & Johnstone, J.R. (1996). The challenge of change: The M.Ed. portfolio. In M.D. Collins, and B.G. Moss (Eds.), Literacy assessment for today’s schools (pp. 123-133). VA: College Reading Association. Paulson, P.R., & Paulson, F.L. (1994). A different understanding. In L. Jackson & R.S. Cafffarella, (Eds.), Experiential learning: A new approach 62(Summer) (pp. 278-292). CA: Jossey Bass. Paulson, L., & Paulson, P. (1990, August). How do portfolios measure up? Paper presented at the Annual Meeting of the Northwest Evaluation Association, Union,WA. (ERIC Document Reproduction Service No. ED 324 329). 119Schatz/INTEGRATING LEARNING AND PROMOTING REFLECTION THROUGH PORTFOLIOS Rich, S. (1994). Test me, Test me knot: The portfolio alternative for developmental writers. In M. Knight (Ed.), Portfolio assessment: An application of portfolio analysis (pp. 47-63). Lanham, MD: University Press of America. Risler, E. (1999). Student practice portfolios: Integrating diversity and learning in the field experience. Aretê 23(1), 89-96. Schatz, M., & Simon, S. (1999). The portfolio approach for generalist social work practice: A successful tool for students in field education. Journal of Baccalaureate SocialWork 5(1) pp. 99-107. Schatz, M.S., Jenkins, L., & Sheafor, B. (1990). Milford redefined: A model of initial and advanced gener- alist social work, Journal of Social Work Education, 26(3). Schon, D. (1987). Educating the reflective practitioner. San Francisco: Jossey-Bass. Schon, D. (1987). Educating the reflective practitioner. San Francisco: Jossey-Bass.Schon, D.A. (1983). The reflective practitioner: How professionals think in action. NY: Basic Books. Simon, S., & Schatz, M. (1998). The portfolio approach for BSW generalist social work students. The New SocialWorker 5(1), 12-15. Standerford, N.S. (1996). Rethinking the role and practice in assessment in teacher education: Learning to assess authentically on multiple levels. In M.D. Collins & B.G. Moss (Eds.), Literacy assessment for today’s schools (pp. 161-183. VA: College Reading Association. Weiser, I. (1994). Revising our practices: How portfolios help teachers learn. In L. Jackson & R.S. Cafffarella, (Eds.), Experiential learning: A new approach 62(Summer) (pp. 293-301). CA: Jossey Bass. White, E.M (1994). Issues and problems in writing assessment. AssessingWriting I(1): 11-28. Yagelski, R.P. (1994). Portfolios as away to encourage reflective practice among pre-service English teach- ers. In L. Jackson & R.S. Cafffarella, (Eds.), Experiential learning: A new approach 62(Summer) (pp. 225- 243). CA: Jossey Bass. Yancey, K.B., &Weiser, I. (1997). Situating portfolios: Four perspectives. Logan, Utah: Utah State University Press. Yancey, K.B. (1994). Teacher portfolios: Lessons in resistance, readiness, and reflection. In L. Jackson & R.S. Cafffarella, (Eds.), Experiential learning: A new approach 62(Summer) (pp. 244-262). CA: Jossey Bass. Author’s Note: Address correspondence to: Mona C.S. Schatz, DSW, School of SocialWork, Colorado State University, Ft. Collins, CO 80523, USA. E-mail: schatz@cahs.colostate.edu 120 ADVANCES IN SOCIALWORK 121Schatz/INTEGRATING LEARNING AND PROMOTING REFLECTION THROUGH PORTFOLIOS Appendix A Survey Instrument Instructions: Please take a few minutes and complete this questionnaire. The more specific your responses the more helpful you will be in aiding this study. Your are wel- come to use the back of the survey to continue your written comments. Return to: SocialWork Dept./Schatz’ mailbox/CSU. 1. Please check below if you have items in your portfolio that represent your work with or involvement in: Yes No Not Sure Individual clients worked with ____ ____ ____ Clients in groups worked with ____ ____ ____ Family clients worked with ____ ____ ____ Communities’ members/organizations ____ ____ ____ Training and seminars ____ ____ ____ Administrative activities ____ ____ ____ Funding development activities ____ ____ ____ Public relations/media activities ____ ____ ____ Class activities/assignments ____ ____ ____ 2. Having completed your portfolio please respond to the following statements: SA A D SD a) The portfolio has been a valuable way to 1 2 3 4 class learning and the field agency placement. b) The portfolio reflects my competence as a social 1 2 3 4 worker. c) The portfolio reflects my second year field 1 2 3 4 experience. d) The portfolio helped me identify areas for field. 1 2 3 4 e) The portfolio reflects my uniqueness as a social 1 2 3 4 worker. 3. Describe two items in your portfolio that reflect your practice competence? a) ______________________________________________________________________ b) ______________________________________________________________________ 4. Describe one item you included in your portfolio that reflects your best social work practice. 5. Describe any item(s) that you were ambivalent about including in your portfolio? 6. Did you include items beyond field placement that were related to past or pres- ent work situations? a) Yes ___ No ___ b) Why did you include these items? Appendix A Survey Instrument (cont.) 7. Were there items you excluded from this portfolio submission? a) Yes ___ No ___ b)What thoughts or considerations led you to exclude these items from your portfolio? 8. Please check below if you shared your portfolio (at whatever stage of comple- tion) at any time during the year with: Yes No a. your supervisor ____ ____ b. co-workers ____ ____ c. other students ____ ____ d. family members ____ ____ e. friends ____ ____ f. university faculty ____ ____ 9. What are the strengths of using a portfolio assignment for the second year field experience, if any? 10.What level of effort did you put into the development of this portfolio? High Medium Low (Worked on regularly (Worked on (Worked on just the last throughout the year) throughout the year) few weeks of each semester) 11.What two conclusions would you make about yourself as a social worker by reflecting on what you included in your portfolio? 12.What do you believe is the best way to evaluate the portfolio? For example, should the instructor meet individually with each student? Have students self- grade their notebook? Other approaches? 13.Will you continue to use and update your portfolio? Yes __ No __ Not sure __ Comments? _________________________ Age:___ Gender: Female __ Male __ Social work setting: Mental Health ___ Medical ___ Family/children ___ Community/Education ___ Hospice ___ Corrections/Probation ___ Have you ever done a portfolio before? Yes ___ No ___ Not Sure ___ 122 ADVANCES IN SOCIALWORK Appendix B Instrument: Review of the Portfolio Reviewer: __________________________ Date of the Review: __________________ Student Code ____ 1. Items in the portfolio reflect work in any of the following situations. Yes No Not Sure Individual clients you have worked with ____ ____ ____ Clients in groups you have worked with ____ ____ ____ Family clients you have worked with ____ ____ ____ Communities members/organizations ____ ____ ____ Training and seminars ____ ____ ____ Administrative activities ____ ____ ____ Funding development activities ____ ____ ____ Public relations/media activities ____ ____ ____ Class activities/assignments ____ ____ ____ 2. Are there items in the portfolio that represent efforts outside of the field experience? Yes __ No __ List items: _____________________________________________ 3. Portfolio presentation is professional. 1 2 3 4 5 6 7 8 Can’t judge Very High High Medium Low Very low 4. There is a quality of organization to the portfolio. 1 2 3 4 5 6 7 8 Can’t judge Very High High Medium Low Very low 5. There is an apparent level of effort by the student. 1 2 3 4 5 6 7 8 Can’t judge Very High High Medium Low Very low 6. Portfolio demonstrates social work competency. 1 2 3 4 5 6 7 8 Can’t judge Very High High Medium Low Very low 7. Portfolio demonstrates integration of class and field experience. 1 2 3 4 5 6 7 8 Can’t judge Very High High Medium Low Very low 8. Portfolio demonstrates advanced generalist practice orientation. 1 2 3 4 5 6 7 8 Can’t judge Very High High Medium Low Very low 9. Portfolio resonates a reflective quality by student. 1 2 3 4 5 6 7 8 Can’t judge Very High High Medium Low Very low 10. General comments 123Schatz/INTEGRATING LEARNING AND PROMOTING REFLECTION THROUGH PORTFOLIOS