SCIENTIFIC PUBLICATIONS International Journal of Psychology: Biopsychosocial Approach 2016 / 18 ISSN 1941-7233 (Print), ISSN 2345-024X (Online) http://dx.doi.org/10.7220/2345-024X.18.2 ASSESSING GLOBAL SERVICE-LEARNING: A MIXED-ME THODS APPROACH TO E VALUATING STUDENTS’ INTERCULTUR AL DE VELOPMENT Stephen W. Jones Crown College, USA David D. Hof, Douglas R. Tillman University of Nebraska at Kearney, USA Abstract. Research problem: Global Service-Learning (GSL), as service-learning that takes place outside of an institution’s home culture, has long been touted as an important contributor to student’s intercultural development. However, GSL practitioners have not yet established effective and consistent methods for evaluating the growth that students experience as a result of GSL experiences. Purpose. The purpose of this study is to evaluate the effectiveness of cultural immersion as a service-learning pro- ject and to use the Intercultural Development Inventory (IDI) to better understand the changes in student development as a result of this project. Methods. A mixed method approach was used to assess students’ intercultural development, utilizing a functionalist instrument (quantitative), alongside critical-reflection guided student journals and participant observation (qualitative). Results. The results showed that the program accomplished some of its goals, however, the structure of the program was not facilitative (at least short term) of allowing all students to access the learn- ing and transformation which was available for them. Conclusions. In conclusion, the authors note emphasizing participation, critical reflection, and intercultural growth, while also using the interplay between functionalist instruments, student generated reports, and the researcher-as-an-instrument through ethnography seem to be a nat- ural and helpful contribution to service-learning. Keywords: Advocacy, Service Learning, Multiculturalism. INTRODUC TION Crabtree (2008) captured one of the most difficult paradoxes in Global Service-Learning (GSL). On one hand, GSL “is a multifaceted endeavor and should be informed by multiple disciplinary and interdis- ciplinary literatures” (p. 19). On the other hand, “because we are working 29 Stephen W. Jones, David D. Hof, Douglas R. Tillman 30 across many disciplines, it can be difficult to find each other’s work” (p. 19). The present study represents an attempt to bring models from divergent fields together, in this case especially, to address the issue of assessment of GSL. It has often been contended (Berry, 1990; Green & Johnson, 2014; Fitch, 2004; Kiely, 2004; Pusch, 2005; Crabtree, 2008) that service-learn- ing can promote intercultural growth, but at the same time, “evaluat- ing international service-learning is not easy” (Tonkin & Quiroga, 2004, p. 146). Furthermore, empirical research has not backed the assumption that “intercultural contact would itself produce increased cross-cultural awareness and reduced ethnocentrism” (Crabtree, 2008, p. 21; also see Pettigrew, 1998). Yet “Pusch 2004 … found a relationship between ISL [International Service-Learning] experiences and increased self aware- ness, cross-cultural skills, and intercultural learning” (Crabtree, 2008, p. 21). The question remains, how should practitioners approach the complicated arena of the development and measurement of intercul- tural competence? Certainly this endeavor requires “a basic proficiency in cross-cultural psychology and communication” (Crabtree, 2008, p. 21). There have been several proposals for methods to evaluate Global Service-Learning, including post experience interview and focus groups (Tonkin & Quiroga, 2004), a longitudinal study using “document analysis, on-site participant observation, focus groups, and semistructured and unstructured interviews” (p. 8) which resulted in Kiely’s (2005) Trans- formational Service-Learning Process Model, the qualitative Character Education Model and its related course assessment reflection guide pro- posed by Balas (2006). However, “there is limited research on the impact of international service-learning programs on students’ learning and development” (Kiely, 2004, p. 5). Kiely notes that “in comparison with studies examin- ing the impact of domestic service-learning, research ‘on the effects of international service-learning is limited and anecdotal in nature’ (Kraft, 2004, p. 303)” (p. 6). Even though “empirical studies have found that participation in international service-learning increases students’ intercultural compe- tence” (Kiely, 2004, p. 5), only a few studies (Westrick, 2004; Fitch, 2004) have employed the psychometrically valid Intercultural Development Inventory (IDI). This article presents an assessment technique which uses 2016, 18, 29–50 p.International Journal of Psychology: A Biopsychosocial Approach 31 a functionalist tool – the IDI – in concert with other methodologies, such as qualitative journaling and ethnography. This technique particularly sought to respond to Kiely and Hartman’s (2011) call to understanding the researcher as an instrument. They explain: A s the instrument, the qualitative researcher has the benefit of adjusting and responding more immediately to changes in the environment, unpredic table and evolving program conditions, par ticipant needs, as well as the abilit y to docu- ment verbal and non-verbal human perceptions, meanings and relations through document analysis, obser vations and inter views. (p. 293) Two research questions of the present study were 1) to evaluate the effectiveness of this particular service-learning project; 2) using the Intercultural Development Inventory (IDI), attempt to better understand the change in the students’ individual development as a result of this experience. BRIEF OVERVIE W OF THE PROJEC T & IDENTIFIC ATION AS GLOBAL SERVICE-LEARNING BACKGROUND OF THE RESEARCH PROJEC T The service-learning project in question grew out of a short-term immersion program offered by a chapter of the international graduate counseling honor society Chi Sigma Iota for students at a small Midwest university. During this experience, the students participate in a four-day experience in which they travel to the Pine Ridge Indian Reservation, the present home of the Oglala Lakota (Sioux) people. While Interna- tional Partnership for Service-Learning and Leadership has “traditionally been skeptical about the relative effectiveness of short-term programs” (Tonkin & Quiroga, 2004, p. 132; see also Medina-Lopez-Portillo, 2004), the students that have participated in the past reported a transforma- tive experience. The authors met and agreed that a research component could be added as part of the trip. As one of the authors is a certified administrator of the Intercultural Development Inventory, this was the instrument chosen as part of the research process. Learning Goals As most of the graduate counseling students at this Midwest uni- versity are both employed and required to earn practicum hours, it is difficult to schedule a cultural immersion trip longer than an extended Stephen W. Jones, David D. Hof, Douglas R. Tillman 32 weekend, although this would be preferred (Medina-Lopez-Portillo, 2004). The learning goals of the trip are two-fold: 1) to enable the stu- dents to experience another culture first-hand and at a greater depth than they previously had, thus allowing for growth in their intercultural skills and awareness; 2) to familiarize the students with a people group which needs advocacy in greater society to allow for a contextualized expression of social advocacy, which is a key component to community counseling. This second point is worth elucidating further as it is central to the service component of the experience. Watson, Collins, Collins, and Coer- reia (2004) note, within clinical mental health counseling, “it becomes clear that the traditional focus on helping the client adapt to his or her environment is untenable when that environment is unjust or otherwise flawed in its treatment of these clients” (p. 291). In such cases, not only does the counselor need to take the environment into account but the environment itself “becomes a primary target for change as a means of improving the mental health of the clients” (p. 291). Lewis, Lewis, Daniels, and D’Andrea (2011) indicate there has been a movement within clinical mental health which signifies 1) it is crucial for counselors to address larger systemic issues in addition to personal and micro-environment issues such as the family; 2) counselors must understand the cultural realities of their clients; 3) advocacy and other forms of intervention should reflect the cultural realities of the clients. This trip represents an effort toward helping the counselors-in-training become students of culture in a very real way. This is an alternative to classroom-only advocacy efforts that service-learning proponents should appreciate: A s educators like Humphrey Tonk in and Melvin Mason realize that cross- cul- tural appreciation and sk ills are best learned by a combination of classroom study and direc t encounter with the people of another culture, so educators around the globe are increasingly finding ways to credit experiential learning and ser vice - learning in par ticular for the mastering of a variet y of sk ills (Berr y & Chisholm, 1999, p. 18). Service Goals The service goals of the immersion experience flow, then, out of the desire for students to be actively engaged with a group, which desires advocacy. Counseling students at this Midwest College are required to develop an advocacy plan as part of their studies, and some of the 2016, 18, 29–50 p.International Journal of Psychology: A Biopsychosocial Approach 33 students who have attended this experience have chosen to advocate for the Lakota people. It is true that the service of advocacy is largely future, and it is also true that advocacy on behalf of another people is not without its own ethical concerns. Those issues, however, are not the focus of this article. The advocacy component is not, in itself, the main active service accomplished by the students. Wade (1997) notes “service is, of course the foundation of service-learning, yet it is an illusive concept to define” (p. 63). Recalling that “service may mean different things in different societies” (Merrill, 2005, p. 182), it is important to consider what service means to those being served. In this case, when asked how the students could serve, the hosts explained that the students could best serve through a three-sided experience of coming to their home; being part of the experience, and then educating others about the Lakota ways and traditions. The relational act of the students seeking to learn from, and listen to, the Lakota hosts was interpreted as service. This has been more recently confirmed by service-learning colleagues at Oglala Lakota Col- lege in preparation for a revised version of this same project. Although different from the more ‘normal’ “direct… [and] indirect… advocacy” (Wade, 1997, p. 68–69) approaches to service, service by rela- tionship is consistent with the sentiment that “service is also an attitude, a relationship, and a way of being in the world” (p. 67). Though the defi- nition of service for the Lakota certainly extends beyond listening and learning, these seem to be important elements. Merrill (2005) reminds us that, often, “one needs to return to earlier centuries to understand fully the sources of present attitudes regarding service” (p. 183). This experience meets the basic definition of GSL by combining “academic instruction and community based service in an international context” (Crabtree, 2008, p. 18). ME THOD This study utilizes the interpretive frame rather than the social sci- entific frame for two reasons. First, the sample size (N=6) is too small to generate significant generalizable data. Second, qualitative data was intentionally incorporated into the research following the recommen- dation of Medina-Lopez-Portillo (2004), who cautioned “future research Stephen W. Jones, David D. Hof, Douglas R. Tillman 34 on the development of intercultural sensitivity should also rely on mul- tiple methods of collecting data” (p. 192). Studies using the Intercultural Development Inventory (IDI) should supplement the quantitative data through “triangulating its results with the collection of qualitative data” (p. 193). In addition to providing for sufficient triangulation, “the data from these multiple sources provides a more complete understanding of thought processes and changes in perceptions about a student’s own culture and target culture, even when students stay in the same [devel- opmental] stage” (p. 193). The IDI yields functionalist data, which can be triangulated through student journals. The journals served as one half of the qualitative piece in the present study. The other portion of the qualitative research came from an attempt to engage in the Kiely and Hartman’s (2011) observation of the role of “Researcher as Instrument.” They explain that in “shifting from detached social scientist who designs sophisticated non-human instruments and (quasi) experimental methods, a fundamental distinguishing charac- teristic for all types of qualitative research is that the researcher is the primary instrument for data gathering, analysis, interpretation and rep- resentation” (p. 293). This is felt perhaps most clearly in studies utilizing ethnography – especially where the principle method of ethnography is participant observation. Kiely and Hartman noted that it is surprising that to date, there are relatively few researchers who have used ethnog- raphy as a research approach in ISL [International Service-Learning] con- texts. Given that a major distinguishing factor of the ISL experience is that participants are situated and often immersed in a dramatically different culture, ethnography provides a particularly useful qualitative research design in study abroad and ISL contexts (pp. 303–305). Thus the research- ers intentionally engaged in the experience as participant observers. Energy was therefore focused on the understanding and explication of the events and data presented, rather than toward an attempt at pre- dicting future results. This is especially important given the extremely complex variables involved in this project, which would be impossible to reproduce with precision. For instance, the group dynamics may play an important role (Kiely and Hartman, 2011) in the impact an immersion like this has on intercultural competency development – particularly if such development is tied to the affective domain, as suggested by Burk- holder (2003) and Deardorff (2004). 2016, 18, 29–50 p.International Journal of Psychology: A Biopsychosocial Approach 35 Sample The sample of students studied in this research was the entire popu- lation of eligible participants. As only six participants have self-selected for the intercultural immersion program, it was easy to do a whole popu- lation sample. The subjects were one male and five female UNK gradu- ate students. Four of the students self-identified as European Americans, and two identified as biracial White-Hispanic/Latino Americans. This research project was reviewed and approved by IRB. All participants were invited to participate in the research project in addition to the im- mersion experience, and all six consented. Relevant Data Quantitative data were collected via the Intercultural Development Inventory, which was administered prior to and following the experi- ence. Qualitative data were collected via student responses to guided journaling questions. Four different sections of questions were provided, and students had the option of making their responses to certain ques- tions private. All participants did self-report that they had answered the questions, and it may be assumed that they did so as there was no incen- tive (such as a grade) to false-report. Intercultural Development Inventory Hammer (2009) explains that “the IDI is a 50-item paper-and-pencil (and online) questionnaire… that can be completed in about 15-20 min- utes… [T]he IDI analytic structure generates an individual (or group) graphic profile of the respondent’s overall position on the intercultural development continuum” (p. 246). Bennett and Hammer created the first version of the Intercultural Development Inventory (IDI) based on Bennett’s (1986, 1993) Develop- mental Model of Intercultural Sensitivity (DMIS). This version was sub- jected to testing by Paige, Jacobs-Cassuto, Yershova, and De Jaeghere (1999), who suggested refinements to the instrument. Following these refinements, they concluded (2003, p. 485) that “Hammer and Bennett’s Intercultural Development Inventory is a sound instrument, a satisfactory way of measuring intercultural sensitivity as defined by Bennett (1993) in his developmental model.” These refinements resulted in the develop- ment of Version 2 of the IDI, the version used in the present study. Stephen W. Jones, David D. Hof, Douglas R. Tillman 36 The process of the development of Version 1 and Version 2 of the IDI are explicitly detailed in Hammer, Bennett, and Wiseman (2003). Reliabil- ity, content validity, and construct validity were all shown to be satisfac- tory in the research associated with the IDI development. Hammer (2007) recently released Version 3, which has the same items as Version 2, but has undergone considerably more testing in general and back-translation for non-English versions. Hammer (2009) “administered the 50-item IDI to a significantly larger, cross-cultural sam- ple of 4,673 individuals from 11 distinct, cross-cultural sample groups” (p. 252). These individuals were from diverse sectors, from colleges and universities to non-governmental organizations to businesses, and par- ticipants “completed the IDI in their native language using rigorously back-translated versions of the IDI unless English was the language of the organization” (p. 252). Hammer continued, “overall, these results from testing IDI v.3 persuasively demonstrate the generalizability of the IDI across cultural groups. Additional analysis of the data by distinct sam- ple groups also clearly demonstrated the culture-specific applicability of the IDI v.3” (p. 252). This recent research seems to override concerns raised by Greenholtz (2005), which appear to be the only such concerns brought against the IDI to date. Hammer concludes: “The Intercultural Development Inventory (v.3…) is the premier cross-culturally valid and reliable measure of intercultural competence” (2009, p. 246). Developmental Model of Intercultural Sensitivity As mentioned previously, “the IDI is an empirical measure of the the- oretical concepts defined by the DMIS” (Hammer, 2007, p. 26). However, “the IDI should not be confused with the developmental model itself ” (p. 26). The Developmental Model of Intercultural Sensitivity has six stages which are divided into two groups. The first three, Denial, De- fense, and Minimization, are Ethnocentric stages. The remaining three, Acceptance, Adaptation, and Integration, are Ethnorelative stages. In Denial, a person is “unable to experience differences in other than extremely simple ways” (Bennett, 2007, p. 19), and has a worldview struc- ture which has “no categories or only broad categories for construing cultural difference.” Furthermore, “in some cases, people with this orien- tation may dehumanize others, assuming that different behavior is a de- ficiency in intelligence or personality” (p. 19). 2016, 18, 29–50 p.International Journal of Psychology: A Biopsychosocial Approach 37 The following stage, Defense, is characterized as an experience of difference “in a polarized way – ‘us and them,’” with stereotyped under- standings of other cultures, which are “now perceived as threatening” (Bennett, 2007, p. 20). The three main variations of Defense are as follows: Defense/Denigration, wherein any other culture (and its values and be- haviors) are judged negatively; Defense/Superiority, wherein positive elements of one’s own culture are exaggerated; and Defense/Reversal, wherein the “poles are reversed” and one views “another culture as supe- rior while maligning one’s own” (p. 20). Minimization often leads to an inflated sense of “having ‘arrived’ at intercultural sensitivity” (Bennett, 2007, p. 21). In Minimization, at- tention to differences is diminished, while physical and transcendental similarities are emphasized. While this stage appears to be very sensi- tive compared to the polarization of Defense, “the assumed commonal- ity with others is typically defined in ethnocentric terms: since everyone is essentially like us, it is sufficient in cross-cultural situations to ‘just be yourself ’” (p. 21). The first ethnorelative stage of the DMIS is Acceptance. A person in acceptance “perceive[s] that all behaviors and values, including [his/her] own, exist in distinctive cultural contexts and that patterns of behaviors and values can be discerned within each context” (Bennett, 2007, p. 22). This stage indicates “acceptance of the distinctive reality of the other cul- ture’s worldview” (p. 22). The second ethnorelative stage of the DMIS is Adaptation. This stage emphasizes both cognitive frame-shifting and behavioral code-shifting. Bennett emphasizes the role of empathy in Adaptation, which “describes a shift in perspective away from our own to an acknowledgement of the other person’s different experience” (1998, p. 208). He expounds, “the ability to empathize with another worldview in turn allows modified be- havior to flow naturally from that experience” (2007, p. 23). The IDI has five scales which combine to measure movement along the DMIS stages detailed above. Figure 1 demonstrates the relationship between the DMIS stages and the IDI scales. The IDI also has a scale, which measures Encapsulated Marginality (EM), which was originally conceptualized as part of the Integration phase. Encapsulated Margin- ality indicates whether a person feels as though they are isolated from their own culture (Hammer & Bennett, 2007). Stephen W. Jones, David D. Hof, Douglas R. Tillman 38 Experience of Difference DMIS Stage Denial Defense Minimization Acceptance Adaptation Integration DMIS & v.2 Stages Ethnocentric Stages Ethnorelative Stages V.2. IDI Scale Denial/Defense (DD) Minimization (M) Acceptance/Adaptation (AA) Encapsulated Marginality (EM)Reversal (RR) v.3 Intercultural Development Continuum Stages Denial Polarization Minimization Acceptance Adaptation Cultural Disengagement (Defense) (Reversal) Intercultural Development Continuum Monocultural Intercultural Mindset Mindset Measured on a separate plane, not considered part of the continuum Figure 1. New directions in the IDI (version 2 compared to version). Intercultural Development Continuum Since the present study began, the first annual IDI conference was held in St. Paul, Minnesota. At this conference, Hammer indicated that there were several changes that could be made in the conceptualization (and practice) of the IDI. The most far-reaching shift is from understand- ing the IDI as only measuring sensitivity to also measuring competence. The ability to shift frames of reference from one’s own cultural pattern to that of a cultural other is viewed now as a measurable skill. Hammer wrote, “the IDI provides key insights on the capabilities of managers and employees for dealing with cultural differences” (2009, p. 253). Hammer further explained that “the underlying intercultural development contin- uum that is assessed by the IDI posits that individuals (and groups) have 2016, 18, 29–50 p.International Journal of Psychology: A Biopsychosocial Approach 39 a greater or lesser capability to perceive [culturally grounded] differ- ences between themselves and others” (2009, p. 247). Other changes include the introduction of Polarization as a major category for Defense and Reversal, and the introduction of the idea of Cultural Disengagement. IDI Guided Development In addition to the natural use of the IDI to conduct a pre-post evalu- ation of a program such as this, it has been suggested (Hammer, 2009; Hammer 2007) that the IDI can be used in a developmental way. The initial scores are thus used as a “needs assessment” (p. 33) for the group. Out of this assessment, a development plan can be tailored for the group. The researchers used the IDI in this way, allowing the initial group results to especially guide the reflection questions. Journaling Guided journaling was used to serve multiple purposes in this expe- rience, in line with Clayton’s (2008) notion that critical reflection serves to generate, deepen, and document learning. As a documentation tool, it provided a window into the students’ thought processes over the sev- eral days of the experience. Furthermore, “formal reflection activities” have been shown to “help students ‘go deeper’ in their understanding of the service experience as well as their own beliefs, including identify- ing and exploring changes in their beliefs as a result of the experience” (Crabtree, 2008, p. 28). IDI guided development through journaling. As a tool to generate and deepen learning, the guided reflection prompts were intentionally developed in response to the pre-assess- ment IDI data. The authors developed four sets of questions in collabo- ration with an IDI administrator. Questions for the beginning of immer- sion, morning reflections, evening reflections, and the end of the immer- sion were developed following Clayton’s (2008) DEAL Model. According to the DEAL Model, students are to D-Describe their experience, E-Ex- amine their experience, and AL-Articulate the Learning which comes out of the experience. Stephen W. Jones, David D. Hof, Douglas R. Tillman 40 The initial round of IDI results indicated that although the group had a primary orientation in Minimization, they had trailing issues in Reversal (R scale). The researchers believed that these issues in Reversal might be holding the group back from further development. Of additional con- cern was the reality that the students were going to be visiting a site where great injustices (Pine Ridge Indian Reservation, and Wounded Knee in particular) had been perpetrated by the US dominant culture, with which all of the students identified. Since reversal is associated with “demonstrating concern for global and domestic inequity” (Hammer, 2007, p. 6), it seemed especially important to help the students process the experience without entrenching deeper into a cultural self-depreca- tion. Thus the IDI guided development plan, and the reflection questions in particular, focused on the development in the area of Reversal. Participant Observation In addition to the IDI and the student journals, the authors decided to insert themselves into the research process, as discussed previously in the comments on the Researcher as Instrument and the comments on Ethnography. In this case, it seemed best to employ Participant Observa- tion. As participants with the group, the researchers were watching for cultural interactions that might help explain any changes in intercultural development in the students. This involved “seeking insight into multi- ple dimensions of the … experience including its impact on students” and paying attention to “roles, power, relationships, critical incidents, responses, language, emotions, conversations, interactions, rituals, group dynamics … community mapping, environmental conditions, physical characteristics, and spaces” (Kiely & Hartman, 2011 p. 307). Overview of the Immersion Experience The experience involved 1) an evening training session one week prior to departure; 2) four days at the Pine Ridge Indian Reservation; 3) and a follow-up debrief. The training session was preceded by the ini- tial IDI assessment which was followed by an overview of the Develop- mental Model of Intercultural Sensitivity and an introduction to the cul- ture of the Lakota of Pine Ridge. The session lasted approximately four hours. 2016, 18, 29–50 p.International Journal of Psychology: A Biopsychosocial Approach 41 Day one involved arriving at the Reservation and settling into the house where the researchers stayed. The students were led through an introduction to Ting-Toomey’s (1999) Observe, Describe, Interpret, Sus- pend Evaluation Model, including an extended period of generating self- awareness. The beginning-of-trip reflection questions were distributed and students reflected in their journals prior to retiring that evening. Day two started with the students journaling responses to the morn- ing questions. Trip hosts led the group to the site of the Wounded Knee Massacre, and shared her family’s connection to the Massacre. Black Elk’s grave was visited followed by a debriefing in the afternoon. That even- ing the group participated in an inipi or sweat lodge. Few students com- pleted the evening reflections on day two, citing extreme tiredness. Day three started more slowly, which allowed the students to com- plete the evening reflections from the night before as well as to journal their responses to the morning questions. Red Cloud Indian School and Red Cloud’s grave were visited this next day. Following this was a driv- ing tour of Pine Ridge, including a brief shopping stop at the grocery/ hardware store. That evening trip hosts shared dinner with the group and explained the Sun Dance, a sacred cultural event in which the group participated in following day. The students ended the day by completing their reflections. Day four started with an opportunity for guided journaling, fol- lowed by packing up and leaving the house where the researchers and students had been staying. The group had been invited to participate in the fourth and final day of a Sun Dance. For those familiar with the Sun Dance, the group stayed in the outer circle, as supporters. This dance had begun before the researchers and students arrived, and continued until sundown. Following this, the group participated in the one element of direct service that this experience included: Previous to our arrival, the researchers and students had been invited to serve the food in the com- munal feast that follows the Sun Dance. The group helped with cleanup and then served the food. Finally, the group returned home at about 3:00 AM. After returning, everyone took the IDI about three weeks later. Stu- dents were given the end of trip journaling questions, and at least half of the students did respond to these questions. Stephen W. Jones, David D. Hof, Douglas R. Tillman 42 RESULTS Research Participant Group Overall Pre-experience IDI assessment The Group’s pre-assessment indicated an overall profile in early- to-mid-Minimization (DS Scale 98.32), with trailing issues in Reversal (R Scale 3.28). The Minimization Scale score was itself at a kind of ad- vanced transitional phase (M Scale, 3.41), and the Similarity Cluster (3.63) was nearly resolved while the Universalism Cluster (3.13) was in the mid- dle of the transitional phase. It appeared that an important develop- mental task was to resolve the Reversal issue, which would then allow for development through Minimization. The Encapsulated Marginality (EM) Scale appeared to be resolved (3.83), and there were no issues on the Denial Defense (DD) Scale (4.54). Post-experience IDI assessment The Group’s post-assessment indicated an overall profile in mid- Minimization (DS Scale 97.76), with trailing issues in Reversal (R Scale 3.52). The Minimization Scale itself is at a mid-point (3.09), with transi- tional issues in both the Similarity (2.97) and Universalism (3.25) clusters. The Reversal scale is nearly out of transition, but there are still a few is- sues, which must be resolved. Within Minimization, both aspects need attention. Comments The Group’s score is very interesting, and perhaps a bit misleading because of the small size of the group. Although half of the students had significant issues along the EM Scale, the read of this scale was as “re- solved” both before and after, which obscured some of the issues facing the students. The group profile does indicate an increase in issues within Minimization, particularly the Similarity Cluster, which was viewed in some of the individual profiles. Encouragingly, the Reversal issues ap- pear to have shown some overall improvement (from 3.28 to 3.52), being now nearly complete with the transitional phase. Individual Students Two students’ data has been selected to be shown as examples. They are noted to demonstrate the kind of data that are available regarding students via this assessment technique. 2016, 18, 29–50 p.International Journal of Psychology: A Biopsychosocial Approach 43 Student 1. Pre This white female student’s pre-assessment indicated an overall profile in late Minimization (105.76 out of 145 on the DS scale), without major trailing issues in Denial/Defense (4.38 on the DD scale) or Rever- sal (4.22 on the R scale). However, she is in transition in Encapsulated Marginality (3.4 on the EM scale). Issues that this student would likely be working through include: A sense of understanding what her cultural identity is (EM scale, 3.4); Recognizing that there are significant differ- ences across cultures (Similarity Cluster, 3.2); and Recognizing that her own values are not always appropriate in different cultural situations (Universalism Cluster, 3.5). Post This student’s post-assessment indicates an overall profile in early Minimization (93.48 out of 145 on the DS scale), without major trailing is- sues in Denial/Defense (DD scale, 4.31) or Reversal (R scale, 3.89). Encap- sulated Marginality is resolved, being well past the “In transition phase” (EM Scale, 4.6). Issues that this student would likely be working through include: Recognizing that there are significant differences across cul- tures (Similarity Cluster, 2.2); and recognizing that her own values are not always appropriate in different cultural situations (Universalism Cluster, 2.75). Comments Overall, this student’s profile indicates a significant retreat into Mini- mization. While this is notable, it is also very important to notice that she was able to resolve the issues related to Encapsulated Marginal- ity. Now, with a more solid sense of her own cultural identity, she has laid the framework for dealing with the issues related to Minimization. It seems possible that the experience at Pine Ridge exposed her to enough cultural difference that she was able to recognize that she was from an outside culture – perhaps aiding her in resolving the Encapsu- lated Marginality. At the same time, it appears that Minimization was her response to what otherwise might have been overwhelming cultural differences. The thought that “we’re all really the same” can be a cop- ing strategy for dealing with the discomfort she felt at various times like while watching the Sun-Dance. This student had begun the “Multicul- tural Counseling” class one week before taking the post-assessment. Stephen W. Jones, David D. Hof, Douglas R. Tillman 44 Student 5. Pre This white female student’s pre-assessment indicated an overall pro- file right on the beginning cusp of Minimization (92.61 out of 145 on the DS Scale), with fairly significant trailing issues in Reversal (2.78 on the R Scale). Issues that this student would likely be working through in- clude: A strong feeling that her home culture has major flaws as com- pared with other cultures (R Scale, 2.78); and a sense that while people from other cultures are not basically like her in their approach to daily life, they often share the same values (Universalism Cluster, 2.5). Post This student’s post-assessment indicates an overall profile right at the end of Minimization (108.79 out of 145 on the DS Scale). The R Scale (3.78) and the M Scale (3.67) are both at just the beginning of the “re- solved” portion of those scales, which begins at 3.66. However, the stu- dent now shows that the EM Scale (2.8) is in significant transition. Issues that this student would likely be working through include: Understand- ing a sense of who she is ethnically speaking, what cultural group she belongs to, how to navigate between two or more cultures, and a sense that other cultures should be interpreted according to her own value system (Universalism Cluster, 3.25). Comments This student has shown the most “forward” development of any of the six participants. Encouragingly, she has resolved most of the issues in Reversal and Minimization, although there is still a trailing issue within the Universalism cluster. It seems that this rapid development has gen- erated some confusion for the student regarding her cultural identity: Her initial EM scale was in the resolved phase at 3.8 but moved to 2.8, into the “in transition” phase on the post-assessment. When she received her final IDI scores, she indicated that, in part because of this experi- ence, and in part because of a class she just started (Multicultural Issues in Counseling), she is really beginning the struggle of understanding her “White” ethnicity. 2016, 18, 29–50 p.International Journal of Psychology: A Biopsychosocial Approach 45 DISCUSSION One student saw important development (full stage), but four out of the six students regressed overall. Three students showed negative movement within the overall Minimization scale, which was not the de- sired effect. Additionally, there were two individuals who demonstrated new issues in Encapsulated Marginality, and a third who showed no de- velopment in this regard. Encouragingly, there was positive movement along the Reversal scale, which was specifically targeted. Also encourag- ing was the development of student 5, who nearly completed a whole stage of development. Several items are worth considering here: What caused the forward progress within the targeted “R” (reversal) scale? What caused the nega- tive trend in the “M” (minimization) scale? Why did one student develop issues, and another resolve issues within the “EM” (encapsulated margin- ality) scale, and why was there no development for yet another individ- ual who started with issues in EM? What role did intensity of dissonance (Kiely, 2005) and intensity factors (Paige, 1993) play in the growth and lack of growth experienced by the students? What ethical issues arise when such changes are linked with this kind of program? Finally, to re- spond to these issues, what changes can be made in future iterations of this program? On the Evaluation Technique The questions listed above are here to demonstrate what kind of quandaries arose based on this blended assessment technique. The IDI generated the majority of the questions, and the journals and ethno- graphic observations aided us in discovering initial answers to them. The IDI, in particular, allowed the authors to explore the realm of cultural transformation noted by Crabtree (2008). This is an important follow-up to Kiely’s (2004) finding that “students who experience cultural transformation begin to see the way their cultural baggage shapes and also distorts their frame of reference” (p. 13). The authors now have a tool to view just how that self/other awareness is playing out. Furthermore, it was demonstrated that the IDI could be used to guide development to impact change in a targeted area. Whether the authors targeted the best area, or should have tried to do a combination Stephen W. Jones, David D. Hof, Douglas R. Tillman 46 of Reversal and Minimization is not particularly of concern for this argu- ment. The important point is that even though the circumstance had the potential to push students toward Reversal, the authors were apparently able to mitigate this effect through intentional, critical-reflection-driven intervention. Additionally, although the program was accomplishing some of its goals, the structure was not facilitative (at least short term) of allowing all students to access the learning and transformation, which was avail- able for them. The program model has been substantially revised for the upcoming year, and was able to be revised based on empirical data. Ethical issues One key element that this assessment technique brought to the at- tention of the authors is that certain ethical issues accompany Global Service-Learning endeavors. Two issues are particularly salient: First, although this experience was brief, there were issues on the EM scale (associated with a person’s sense of cultural identity) for several stu- dents. Kiely (2004) warned, “service-learning educators who have ‘trans- formative intentions’ need to recognize the long-term struggle inherent in the nature of transformation learning” (p. 18). Crabtree (2008) con- curs thusly, “the nature of the cross-cultural encounter, awakening of global awareness, powerful cognitive dissonance that often results and immense personal growth that becomes possible are each phenomena with enormous disruptive as well as transformative power” (p. 28). Certainly the finding that cultural identity questions can be initiated in a short experience such as this should at least give us pause. Particu- larly of concern is how service-learning practitioners can support stu- dents in their long-term transformative learning. Crabtree concludes, “It would be unethical for us to be unprepared to manage these changes in/for ourselves in addition to helping our students process them” (p. 28). Another ethical concern is this: although as a group the students were less culturally competent after returning, they expressed verbally and on the IDI Perceived Scale (a component of the IDI which measures the self-perception of a person or group’s intercultural sensitivity) that they felt they were at least as or more culturally competent than they were when they left. Programs that rely solely on qualitative measures to determine the culturally transformative experience of their students 2016, 18, 29–50 p.International Journal of Psychology: A Biopsychosocial Approach 47 thus may run the risk of overestimating the positive impact and under- estimating the negative impact of their programs. Future directions The greatest lack in the present study is longevity. No service-learn- ing programs exist in isolation. This experience occurs within the context of a degree program, and it is important to consider how further work in that program factors into a student’s ability to process their service- learning experience. Ideally, students would take the IDI again after making presentations (a continued form of reflection and advocacy) in the spring following their immersion experience. Additionally, and importantly, is the necessary task of blending eval- uation techniques such as the one propose here with other evaluation models, like Kiely’s (2005) Transformative Learning Model, Balas’s (2006) Character Education Model, and Clayton’s (2008) ever-developing DEAL Model for Critical Reflection. If GSL assessment “should attend to the participatory dimensions of ISL, including the quality of information sharing” (Crabtree, 2008, p. 29), and the authors can expect a gradual move toward more “strin- gent assessment protocols,” then this three-pronged assessment method is at least an important method to consider. Emphasizing participation, critical reflection, and intercultural growth, while also using the interplay between functionalist instruments, student generated reports, and the researcher-as-an-instrument through ethnography seems to be a natu- ral and helpful contribution. References Balas, G. R. (2006). The Lessons of Anapra: International Service-Learning and Character Education. Journal of College & Character, VII (7), 1–10. Bennett, M. J. (1986). Towards ethnorelativism: A developmental model of intercultural sensitivity. In R. M. Paige (Ed.), Cross-cultural orientation: New conceptualizations and applications. New York: University Press of America. Bennett, M. J. (1993). Towards ethnorelativism: A developmental model of intercultural sensitivity. In R. M. Paige (Ed.), Education for the intercultural experience. Yarmouth, ME: Intercultural Press. Bennett, M. J. (1998). Intercultural communication: A current perspective. In M. J. Bennett (Ed.), Basic concepts of intercultural communication: Selected Readings. Yarmouth, ME: Intercultural Press. Stephen W. Jones, David D. Hof, Douglas R. Tillman 48 Bennett, M. J. (2007). A developmental model of intercultural sensitivity. In R. M. Hammer, The Intercultural Development Inventory (IDI) manual. Ocean Pines, MD: IDI, LLC. Berry, H. A. (1990). Service-learning in international and intercultural settings. In J. C. Kendall & Associates (Eds.), Combining service and learning: Vol. 1. A resource book for community and public service (pp. 311–313). Raleigh, NC: National Society for Internships and Experiential Education. Berry, H. A. & Chisholm, L. A. (1999). Service-learning in higher education around the world: An initial look. New York: The International Partnership for Service- Learning. Burkholder, J. B. (2003). An evaluation of Grace University’s 1997, six-month, missions training program in Mali, West Africa. Unpublished doctoral dissertation, Trinity International University. Clayton, P. (2008, February). Generating, deepening, and documenting learning: The power of critical reflection. Paper presented at the 3rd Annual Conference on Applied Learning in Higher Education, St. Joseph, MO. Crabtree, R. D. (2008). Theoretical foundations for international service-learning. Michigan Journal of Community Service-Learning, 15(1), 18–36. Deardorf, D. K. (2004). The identification and assessment of intercultural competence as a student outcome of internationalization at institutions of higher education in the United States. Unpublished doctoral dissertation, North Carolina State University, Raleigh, North Carolina. Fitch, P. (2004). Effects of intercultural service-learning experiences on intellectual development and intercultural sensitivity. In M. Welch & S. Billig (Eds.), New perspectives in service-learning: research to advance the field. Greenwich, CT: Information Age Publishing. Green, P. & Johnson, M. (2014). Crossing boundaries: Tension and transformation in international service-learning. Sterling, VA: Stylus. Greenholtz, J. F. (2005). Does intercultural sensitivity cross-cultures? Validity issues in porting instruments across languages and cultures. International Journal of Intercultural Relations, 29, 73–89. Hammer, R. M. (2007). The Intercultural Development Inventory (IDI) manual. Ocean Pines, MD: IDI, LLC. Hammer, R. M. (2009). The Intercultural Development Inventory (IDI): An approach for assessing and building intercultural competence. In M. A. Moodian (Ed.), Contemporary leadership and intercultural competence: Understanding and utilizing cultural diversity to build successful organizations.USA: SAGE Publications, Inc. Hammer, M. R., Bennett, M. J., & Wiseman, R. (2003). Measuring intercultural sensitivity: The Intercultural Development Inventory. In R. M. Paige (Guest Ed.), Special issue on the Intercultural Development Inventory. International Journal of Intercultural Relations, 27(4), 421–443. Kiely, R. (2004). A Chameleon with a Complex. Michigan Journal of Community Service-Learning, 10(2), 5–20. 2016, 18, 29–50 p.International Journal of Psychology: A Biopsychosocial Approach 49 Kiely, R. (2005). A Transformative Learning Model for Service-Learning. Michigan Journal of Community Service-Learning, 12(1). Kiely, R., & Hartman, E. (2011). Qualitative research in international service- learning. In R. Bringle, J. Hatcher, & S. Jones (Eds.), Research perspectives in international service-learning (Vol. 1) (pp. 291–318). Sterling, VA: Stylus. Lewis, J. A., Lewis, M. D., Daniels, J. A., D’Andrea, M. J. (2011). Community counseling: A Multicultural-Social Justice Perspective (4rd. ed.). Belmont, CA: Brooks/Cole. Medina-Lopez-Portillo, A. (2004 Fall). Intercultural Learning Assessment. Frontiers: The Interdisciplinary Journal of Study Abroad, X, 179–200. Merrill, M. C. (2005). The cultural and intercultural contexts of service-learning. In L. A. Chisholm (Ed.), Knowing and doing: The theory and practice of service- learning (pp. 177–202). New York: The International Partnership for Service- Learning and Leadership. Paige, R. M. (1993). Education for the intercultural experience. Yarmouth, Me: Intercultural Press. Paige, R. M., Jacobs-Cassuto, M., Yershova, Y. A., & DeJaeghere, J. (1999). Assessing intercultural sensitivity: A validation study of the Hammer and Bennett (1998) Intercultural Development Inventory. Paper presented at the International Academy of Intercultural Research conference, Kent State University, Kent, Ohio. Paige, R. M., Jacobs-Cassuto, M., Yershova, Y. A., & DeJaeghere, J. (2003). Assess- ing intercultural sensitivity: An empirical analysis of the Intercultural Development Inventory. In R. M. Paige (Guest Ed.), Special issues on the Intercultural Development Inventory. International Journal of Intercultural Relations, 27(4), 467-486. Pettigrew, T. F. (1998). Intergroup contact theory. Annual Review of Psychology, 49, 65–85. Pusch, M. D. (2005). Teaching intercultural skills and developing the global soul. In L. A. Chisholm (Ed.), Knowing and doing: The theory and practice of service- learning (pp. 145–162). New York: The International Partnership for Service- Learning and Leadership. Ting-Toomey, S. (1999). Communicating Across Cultures. New York: The Guilford Press. Tonkin, H. & Quiroga, D. (2004, Fall). A qualitative approach to the assessment of international service-learning. Frontiers: The Interdisciplinary Journal of Study Abroad, X (Fall), 131–149. Wade, R.C. (1997). Community service-learning. Albany, NY: State University of New York Press. Watson, A. L., Collins, R. L., & Collins Coerreia, F. (2004). Advocacy and social action in the context of ecological counseling. Ecological Counseling. Robert Conyne & Ellen Cook, Eds. Alexandria, VA: American Counseling Association Westrick, J. (2004). The influence of service-learning on intercultural sensitivity: A quantitative study. Journal of Research in International Education, 3, 277–299. Stephen W. Jones, David D. Hof, Douglas R. Tillman 50 PASAULINIŲ KOOPERUOTŲJŲ STUDIJŲ VERTINIMAS: MIŠRIŲ ME TODŲ BŪDAS VERTINANT STUDENTŲ TARPKULTŪRINĘ R AIDĄ Stephen W. Jones Crown koledžas, JAV David D. Hof, D ouglas R. Tillman Nebraskos universitetas Kearnyje, JAV Santrauka. Mokslinė problema. Pasaulinės kooperuotosios studijos (PKS) – kooperuoto- sios studijos, vykstančios už organizacijos vidinės kultūros ribų – ilgą laiką buvo giriamos už svarbų įnašą į tarpkultūrinę studentų raidą. Vis dėlto, pasaulinių koope- ruotųjų studijų specialistai dar nėra nustatę efektyvių ir nuoseklių metodų, galinčių įvertinti studentų raidą visuotinių kooperuotųjų studijų patirčių metu. Tikslas. Šio tyrimo tikslas yra įvertinti kultūrinės imersijos, kaip pasaulinių kooperuotųjų studijų programos, efektyvumą ir naudojantis tarpkultūrino ugdymo aprašu (Intercultural Development Inventory) geriau suprasti studentų raidos pokyčius po šio projekto. Metodai. Studentų tarpkultūrinė raida vertinta mišrių metodų būdu naudojant funkcionalų įrankį (kiekybinį), kritinio atspindžio principu pagrįstus studentų žurnalus ir dalyvių stebėjimą (kokybinis). Rezultatai. Rezultatai atskleidė, kad programos metu pasiekti keletas tikslų, tačiau programos struktūra nepadėjo (bent jau remiantis trumpalaikiais rezultatais) studentams įsisavinti jiems prieinamos informacijos apie mokymąsi ir transformaciją. Išvados. Autoriai pabrėžia, kad dalyvavimas, kri- tinis atspindys, tarpkultūrinis augimas, kartu naudojant sąveiką tarp funkcionalių instrumentų, studentų ataskaitų, ir tyrėjas-kaip-instrumentas etnografijoje metodų yra suprantama ir naudinga pagalba kooperuotosioms studijoms. Pagrindiniai žodžiai: advokatavimas, kooperuotosios studijos, tarpkultūriškumas. Received: December 8, 2015 Accepted: August 24, 2016