Microsoft Word - Anderson et al_final.docx Journal of Effective Teaching in Higher Education, vol. 5, no. 1 https://doi.org/10.36021/jethe.v5i1.252 Assessing a Modified Jigsaw Technique with Theoretical Triangulation Bill Anderson, Illinois State University, jander2@ilstu.edu Rachel Rymer, MetroHealth Jennifer Versaskas, Aurora University Abigail Bueter, Adler University Mahalia Masood, Illinois State University Abstract. The jigsaw technique has been successfully used in classrooms for decades, though less in higher education. Groups are formed with each student having a precise piece of information necessary to complete the assignment. Next, expert groups of students with the same material meet to sharpen their understanding before reforming their initial group to share their new insights. This study sought to assess a modified jigsaw format where students received their material two days before the class, began with the expert groups, and closed with mixed groups where students taught their material and completed the assignment. A random 30% sample of assignments (n = 110) from three classes was analyzed using a pattern-matching technique (Yin, 2013) and coded utilizing the six levels of Bloom’s revised taxonomy (Anderson & Krathwohl, 2001) and six significant learning types (Fink, 2013) as predetermined codes. One-way ANOVA found no significant difference between classes, F (2, 540) = 1.244, p. = 0.289, when coded with the taxonomy. Results were positive, indicating thinking well above rote learning (M = 2.9, SD = 1.233) with most (39.9%) students effectively comparing material. Coding was similarly positive for learning types (Fink, 2013), again, with no significant difference between classes, F (2, 501) = 3.036, p. = 0.084. Integration, making connections between varied information, was the primary type of learning (31.9%) used. Evidence of students learning about themselves was also noted (31.3%). The modified jigsaw was a well-received addition to the class and effective in teaching this material. Keywords: jigsaw technique; collaborative learning; cooperative learning; critical thinking Collaborative learning is an overarching term for a many educational approaches that require group efforts by students, promote critical thinking, and emphasize learning and personal responsibility (Laal & Ghodsi, 2012). Cooperative learning is a specific kind of collaborative learning that involves purposeful interaction to enable students to learn together by teaching each other (Nusrath et al., 2019; Yoshida, 2018) and to produce a shared result (Turkman & Buyukaltay, 2015). Although all learners are responsible for the success of the group (Maden, 2011; Timayi et al., 2015), Turkman and Buyukaltay (2015) assert that group work only becomes actual cooperative learning when students make intentional effort to bring the learning of all group members to the topmost level. Assessing a Modified Jigsaw Technique 54 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 One of the most effective and commonly used forms of cooperative learning is the jigsaw technique (Costouros, 2020; Lalit & Piplani, 2020; Yoshida, 2018) that purposefully breaks down a larger concept into smaller, but critical, pieces of information. Studies have determined that jigsaw is highly effective because it engages students twice (in their original groups and in their expert groups), encourages learning from one another, increases self-esteem, makes students responsible for teaching in their groups, improves social/relationship skills, increases student engagement, short-term and long-term retention of the subject matter. (Yoshida, 2018, p. 209) This study involved modifying the traditional jigsaw and evaluating success based on the level of student thinking, or processing, and the type(s) of learning necessary to complete the assignment. Literature Review The jigsaw classroom was initially implemented in 1971 to specifically address the tension of the recent desegregation of American schools by lessening competition between students and motivate learners to see each other as resources (Aronson et al., 1978). Aronson et al. (1978) explained, “It would be valuable if the basic [classroom] process could be changed so that children could learn to like and trust each other not as an extracurricular activity but in the course of learning…” (p. 23). Initially, the instructor introduces the assignment and the overall topic. Next, home groups are formed, and each member is assigned an essential part of the information to further develop. Students then briefly discuss the assignment and familiarize themselves with their material. “The home groups then break apart, like pieces of a jigsaw puzzle, and the students move into jigsaw groups consisting of members from the other home groups who have been assigned the same portion of the material” (Maden, 2011, p. 913). These expert groups primarily involve sharing insights about the specific subtopic and addressing related questions (Aronson, 2021; Aronson & Patnoe, 2011). Together they seek to develop mastery of their assigned material (Barkley, 2010; Costouros, 2020), with each student contributing from their perspective and skills. Next, the original home groups are reassembled with students taking responsibility for selecting what needs to be taught and how to teach it (Costouros, 2020), then sharing their new knowledge with their team members (Aronson, 1971, 2021). By making each student part of the solution, the jigsaw technique blurs the distinction between students who know and students who do not yet know, requires all students to make active responses, and moves away from the experience of learning as a solitary activity that is detached from the social context. (Maden, 2011, p. 915) Assessing a Modified Jigsaw Technique 55 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 “Once each member of the home group has taught their piece of the puzzle to the other members of the group, the whole puzzle is formed” (Costouros, 2020, p. 156). Afterwards, students will then take the quiz or complete the assignment. Although, until recently, the jigsaw technique had been minimally used in higher education (Crone & Portillo, 2013; Weidman & Bishop, 2009), studies have indicated that the technique has proven to be quite adaptable to college classrooms (Costouros, 2020; Dhull & Verma, 2019; Turkmen & Buyukaltay, 2015). As a result, current applications in the college classroom are not only increasing but continuing to evolve (e.g., Lalit & Piplani, 2019; Nusrath et al., 2019; Yoshida, 2018). The original design (Aronson et al., 1978) previously described remains the most often used, but several variations are in common use as well. Described briefly, these variations include the following: • Jigsaw II assigns all material to all students, tests learning after the expert groups, and rewards the team with the highest score (Doymus et al., 2010; Yoshida, 2018). • Jigsaw III is similar to Jigsaw II but with the competition between groups removed. Also, there is intentional consideration of language proficiencies in bilingual classrooms (Doymus et al., 2010; Hedeen, 2003). • Jigsaw IV is like II and III but reteaches material as necessary after the quiz on expert group material (Timayi et al., 2015; Turkmen & Buyukaltay, 2015). • A reverse jigsaw focuses on perception and on the understanding of a range of interpretations, with each expert group then teaching the class as a whole (Hedeen, 2003; Samuel, 2018). • The subject jigsaw involves developing problem solving in the subject matter and utilizes a second home group mixing both material and students (Haviz & Lufri, 2019). • Communication jigsaws opens access of all information to all students and seeks to extend interaction and to expand connections between students (Yoshida, 2018). All the variations listed incorporate the structured interaction of the jigsaw and share the basic parts of the strategy (Doymus et al., 2010). All variations include students taking the responsibility for their own and other students’ learning and groups cooperating toward a common educational goal. Current research with jigsaw learning is largely positive, finding more active participation (Lalit & Piplani, 2020) and evidence of improved academic performance (Dewati et al., 2019; Nusrath et al., 2019; Turkmen, & Buyukaltay, 2015). Also, and in agreement with Aronson’s (1978) original work, studies have reported the jigsaw method can increase college students’ oral communication skills and their confidence in themselves as students and scholars (Crone & Portillo, 2013; Lalit & Piplani, 2019; Nusrath et al., 2019). But perhaps of greatest relevance here are reports of higher order thinking and analysis (Dewati et al., Assessing a Modified Jigsaw Technique 56 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 2019; Nusrath et al., 2019) and findings of more frequent usage of critical thinking (Yatimah et al., 2019). There are, of course, recognized difficulties associated with cooperative learning, primarily as some students simply do not enjoy group work. Many perceive it as more difficult than working alone, and group work typically does require more concentrated effort (Weimer, 2017). Common negative observations also include ineffective communication (Weimer, 2020) and lack of preparation (Weimer, 2017). The latter has also been identified as problematic using the jigsaw technique (Benton, 2016; Costouros, 2020; Nusrath et al., 2019). Although most studies specific to jigsaw work have reported positive student responses (Calkins & Rivnay, 2021; Lalit & Piplani, 2020; Maden, 2011), others found no significant difference in student satisfaction between lecture based and jigsaw learning (Costouros, 2020; Nusrath et al., 2019). Indeed, students who do not enjoy the method may not put forth their best effort (Costouros, 2020; Nusrath et al., 2019), and this is likely detrimental to the jigsaw process for all students. Still, “higher student satisfaction is not necessarily indicative of learning” (Costouros, 2020, p. 159). Modified Jigsaw The jigsaw assignment considered here was to compare common parenting practices of Americans of African, Asian, European, Indigenous, Latinx, and Middle Eastern descent. This topic, and others like it, are generally considered in distinct textbook chapters separated by ethnicity. However, the goal here was not only to learn unique distinctions but to purposefully consider similarities as well. The material divided easily and intuitively with each part critical to the final jigsaw assignment. Also, with classroom discussions acutely addressing race or racism being difficult for some (Kaplowitz et al., 2019), the jigsaw technique’s history of specific and intentional awareness of racial tension in education (Aronson et al., 1978) was considered relevant and valued. Interestingly, this element was still noted 25 years later (e.g., Crone & Portillo, 2013) and contained the added possibility that students could learn to be more tolerant, understanding, and accepting of others and their differences (Costouros, 2020). Therefore, the jigsaw was an immediate and logical choice when incorporating multiple subtopics with a strong, common theme and attempting to positively effect racial understanding (e.g., Aronson & Patnoe, 2011). The actual jigsaw assignment initially followed the progression of the subject jigsaw technique (Doymus et al., 2010; Haviz & Lufri, 2019). The general topic and assignment were introduced in class, and students received a packet of information on one of the six groups. Because, as previously noted, preparation is critical, students were given two days to prepare for the actual in-class jigsaw. Following recommendations to assess discussion readiness prior to the jigsaw (e.g., Barkley, 2010; Benton, 2016), students were required to submit, prior to class, a brief review of their material, 3-4 bulleted statements of information they found unfamiliar or significant and why, and 2 pertinent, upper-level questions prepared for their first group. Assessing a Modified Jigsaw Technique 57 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 In class, the expert groups (students with the same topic) met first. This change was based on a general characteristic of the subject-jigsaw model (Haviz & Lufri, 2019) and the successful use of this variation by Crone and Portillo (2013). Students initially discussed their questions, specifically noted similarities and differences in parenting styles and priorities, and determined information to teach in the groups to follow. Thirty minutes was allowed for this. Next, they moved to jigsaw groups consisting of four to six experts, depending on class size and attendance; taught their material to the group; and were questioned by others. Following this, the conversation turned to determining similarities between groups and points that were unique to one, or some, of the groups. Thirty-minutes was also allowed here. Following this home group, a 15-minute class debriefing was used. After a quick review of the concept of stereotypes, the class was asked if any prevailing stereotypes were challenged by the material or in their conversation. Students were then given three days to complete the final assignment that consisted of a brief description of their expert group, a description of the jigsaw group using all materials, a detailed conclusion based on comparing/contrasting, and questions they were left with. This differs from the traditional final group assignment or quiz but still involves accountability and cooperating toward a common educational goal (e.g., Yoshida, 2018). Purpose of This Study This study first sought to evaluate student satisfaction with the activity and to assess the efficacy of this jigsaw variation to establish higher levels of critical thinking (Anderson & Krathwohl, 2001) and to determine what type of learning (Fink, 2013) was most prevalent. Because satisfaction with the jigsaw technique may be predictive of the success of the modified jigsaw here (Costouros, 2020; Nusrath et al., 2019), it will be considered first, using confidential student feedback provided through the Individual Development and Educational Assessment (IDEA, 2021a). Were they able to learn from others and did they perceive it as truly cooperative and meaningful to their learning? It is likely that if student satisfaction with the jigsaw, and cooperative learning in general, is low, the student engagement will be low as well, and vice versa. Next, did the assignment necessitate the higher levels of critical thinking and reasoning necessary for meaningful learning (e.g., Anderson & Krathwohl, 2001, p. 65)? It was hoped that this jigsaw experience would require less simple reporting and greater instances of comparing, contrasting, and inferring as described by Anderson & Krathwohl’s (2001) revision of Bloom’s Taxonomy of Learning. This is a well-established hierarchical continuum with each progressively higher level indicating increasing cognitive complexity and processing. Briefly described from least complex to most complex, the levels can be described as follows: 1. Remembering: exhibiting memory of previously learned information 2. Understanding: demonstrating previous learning by connecting Assessing a Modified Jigsaw Technique 58 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 3. Applying: using existing knowledge, or creating new skills, to solve problems 4. Analyzing: breaking down existing knowledge and examining specific parts 5. Evaluating: presenting informed judgments and justifying a position 6. Creating: elaboration used to put information together in new ways It was expected that the coding would reveal an average level of reasoning used by students to complete the assignment significantly beyond Anderson and Krathwohl’s (2001) remembering level. Though remembering is indeed crucial for meaningful and complex learning to occur, it is viewed as insufficient for completing the assignment here. Accurately noting both similarities and differences would certainly require comparing from the understanding level. However, it was hoped that the median response will be at the analyzing level identified by discrimination and insight (Anderson & Krathwohl, 2001). Lastly, Fink’s (2013) model of significant learning was selected to determine the types of learning that were most prevalent. Although Fink (2013) acknowledged Bloom’s Taxonomy as having great value, adding “any model that commands this kind of respect half a century later is extraordinary” (p. 34), he was more interested in important types of learning in higher education that are not readily described by the taxonomy. His model goes, “beyond the cognitive domain of Bloom’s taxonomy and beyond cognitive learning itself to determine what kinds of learning are resulting from the assignment” (Fink, 2013, p. 34). A critical distinction here is that Fink’s (2013) types of learning are interdependent as opposed to linear and hierarchical like the Taxonomy. Therefore, “Achieving any one kind of learning simultaneously enhances the possibility of achieving the other kinds of learning as well” (Fink, 2013, p. 37). Types include the following: 1. Foundational knowledge concerns valid remembering and understanding of information and ideas. 2. Application involves using and developing skills and when to appropriately apply them. 3. Integration is noticing, identifying, and understanding the connections between things 4. Human dimension describes learning about self and others to increase effective interaction. 5. Caring focuses on developing new interests and feelings. 6. Learning how to learn results in a deeper understating of learning itself and becoming a more effective learner. To best accomplish the level of comparing, contrasting, and thinking mentioned earlier, this assignment was designed as integration learning, requiring the perceiving of similarities and differences, and connecting, “one body of knowledge Assessing a Modified Jigsaw Technique 59 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 with other ideas and bodies of knowledge” (Fink, 2013, p. 86). Certainly, integration bears an intuitive and clear connection to jigsaw learning. Not only to the structure but also to the goal of fostering critical thinking that can help bring about a deeper comprehension of material (Ghaith & El-Malak, 2004). Methods Participants were female students majoring in Human Development and Family Science. Most were juniors (54.6%), followed by 40% seniors and 5.4% sophomores. Most were European American (78.2%), 9.1% were Latinx, 6.4% were African American, 3.7% were of Middle Eastern descent, and 2.6% were Asian American. The jigsaw assignment was applied to three consecutive semesters in an undergraduate multicultural studies class, n= 37, n = 38, and n = 43, and was completed by 110 of 128 students. A random sample of 30% (n = 34) of the assignments was selected for analysis. Case Study and Theoretical Triangulation A case study approach was utilized as it is suitable for examining educational processes (Hamilton & Corbett-Whittier, 2013) and understanding the meaning for students involved (Merriam, 1998). Case study is defined as an in-depth study of a particular instance (Hyde, 2000; Merriam, 1998) and is therefore “not a methodological choice but a choice of what is to be studied” (Stake, 2005, p. 443). Because case studies in education commonly focus on a single assignment or course and results are sometimes difficult to perceive objectively (Bishop-Clark & Dietz-Uhler, 2012; Hamilton & Corbett-Whittier, 2013), triangulation is more necessary in case-study research than other designs (Divan et al., 2017) and involves using multiple perspectives in the study of the same phenomenon (Hopper & Hoque, 2000; van Drie & Dekker, 2013; Stake, 2005). This multimethod- multimeasure process ensures that the findings are not dependent on any single measure or method of measurement (Krathwohl, 2009), provide a more complete perspective (van Drie & Dekker, 2013), and improve an educator’s understanding of the findings (Bishop-Clark & Dietz-Uhler, 2012). Krathwohl’s (2009) multimethod requirement will include student assignments from three classes, replicated across time, to examine the same learning outcome and independent student feedback from each class, “a redundancy of data gathering” (Stake, 2005, p. 454). Concerning the latter, all three course sections were evaluated using the Individual Development and Educational Assessment instrument (IDEA, 2021a) designed to assess progress on teaching objectives with results reported on a 5-point scale as follows: 1 = No apparent progress; 2 = Slight progress; 3 = Moderate progress; 4 = Substantial progress; 5 = Exceptional progress. The three, out of 32, IDEA objectives most relevant for considering student satisfaction with the jigsaw technique are grouped within the instrument under the title, Fostering Student Collaboration, and include the following (IDEA, 2021b): • #5: Formed teams or discussion groups to facilitate learning Assessing a Modified Jigsaw Technique 60 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 • #16: Asked students to share ideas and experiences with others whose backgrounds and viewpoints differ from their own • #18: Asked students to help each other understand ideas or concepts Krathwohl’s (2009) multimeasures will include theoretical triangulation, the use of multiple theoretical perspectives simultaneously to study the same aspect of the research (Hopper & Hoque, 2006), to validate findings by showing that independent measures agree (van Drie & Dekker, 2013; Yin, 2003). Although the role of theory in the Scholarship of Teaching and Learning (SoTL) research has received little attention (Divan et al., 2017), there is evidence that case studies in education can indeed be supported by educational theory (Anderson, 2019; Yin, 2014). For this study, two established educational theories were utilized to understand student learning outcomes by identifying levels of cognitive processing (Anderson & Krathwohl, 2001) and types of learning (Fink, 2013). This was accomplished using pattern-matching analysis (Yin, 2003, 2014), designed to compare a predicted pattern with the empirical findings of the study (e.g., Anderson, 2019) and recommended for hypothesis testing in case studies (Pearse, 2019). Although Yin (2003) originally prescribed generating an initial theory from existing research and using an opposing alternative theory for comparison, he later explicitly recommended that analyzing data with the theoretical model(s) that led to the study (e.g., Fink, 2014) could produce dependable analysis and results. The technique used here reflects this later view by utilizing both previously mentioned theories as predetermined codes (Pearse, 2019). A one-way ANOVA was used to determine difference between the three classes and the acceptability of combining the classes for analysis. An average for each objective was reported. Coding First, two independent coders utilized the propositions of Anderson and Krathwohl (2001) to identify levels of cognitive processing. Descriptors for each level of the taxonomy were selected from the revised taxonomy (Anderson & Krathwohl, 2001) in a pilot study of six assignments. The most relevant are listed in Table 1 below. Table 1 Student Essay Coding Scheme for Bloom’s Revised Taxonomy Next, independent coders utilized the propositions of Fink (2013) to identify types of learning. Brief definitions of each type were developed and selected in a pilot study as well. These are listed in Table 2. Code Taxonomy level Example descriptors 1 Remembering Telling, recalling, recognizing 2 Understanding Comparing, contrasting, interpreting 3 Applying Implementing, utilizing, making use of 4 Analyzing Attributing, identifying motives, distinguishing 5 Evaluating Interpreting, prioritizing, critiquing, supporting 6 Creating Elaborating, predicting, imagining, proposing Assessing a Modified Jigsaw Technique 61 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 Table 2 Student Essay Coding Scheme for Types of Learning Code Learning type Example descriptors 1 Foundational Basic facts and understanding major ideas 2 Application Applying or learning a new skill or way of learning 3 Integration Connecting new ideas, experiences, or information 4 Human Dimension Learning about self and others; societal implications 5 Caring Developing interest or seeing value in the subject 6 Learning to Learn Understanding the process of learning; motivated At completion, a kappa score of intercoder reliability was determined for both using the intra-class correlation coefficient function of SPSS v.26. A score of >0.61 was considered substantial (McHugh, 2012). A one-way ANOVA was used to determine difference between the three classes. Results Student Satisfaction All students responding voluntarily with the IDEA (IDEA, 2021b) were quite positive concerning group work, rating each objective as a 4 or 5. A one-way ANOVA indicated a statistically significant difference between the three classes, F (2, 6) = 9.000, p. = 0.16. Post hoc analysis using a Scheffé test indicated the difference was in the pairwise comparison of class 1 and class 3, with no significant difference between 1 and 2 nor 2 and 3. The effect is possibly attributable to unequal student response sizes. Classes were considered together and an average for each is stated below and all indicate noticeable student satisfaction. • #5 Formed teams or discussion groups to facilitate learning (M = 4.8/5.0, SD = .1000) • #16 Asked students to share ideas and experiences with others whose backgrounds and viewpoints differ from their own (M = 4.9/5.0, SD = .000) • #18 Asked students to help each other understand ideas or concepts (M = 4.7/5.0, SD = .046) Level of Thinking Coding results using the six levels of Bloom’s Revised Taxonomy (Anderson & Krathwohl, 2001) yielded a kappa score of interrater reliability of 0.701. Means were very similar across all three classes: class 1 (M = 2.88, SD = 1.219), class 2 (M = 2.80, SD = 1.248), and class 3 (M = 3.01, SD =1.242), and a one-way ANOVA indicated no statistically significant difference between the three, F (2, 540) = 1.244, p. = 0.289. Considering the three classes together yielded M = 2.9 (SD = 1.233), indicating the applying level. However, this only accounted for 19.2% of the Assessing a Modified Jigsaw Technique 62 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 responses. Considering a mode of 2 and examining the resulting rank order comparisons indicated the greatest number of responses (60.4%) at the understanding level (39.9%) and the analyzing level (20.5). Other results included remembering at 8.4%, evaluating at 9.7%, and creating at 2.2%. Type of Learning Coding with Fink’s (2013) six types of learning produced a kappa score of interrater reliability of 0.677. Means were again very similar across all three classes: class 1 (M = 2.99, SD = 0.991), class 2 (M = 3.03, SD = 1.237), and class 3 (M = 3.24, SD =1.080). A one-way ANOVA indicated a no statistical difference between the three, F (2, 501) = 3.036, p. = 0.084. Considering the classes together yielded M = 3.09 (SD = 1.109) and a mode of 3, indicating the greatest response at integration (31.9%). However, Human Dimension (31.3%) produced a near identical measure. Percentages for other types of learning were much less common and included application at 19.2%, functional at 9.7%, caring at 7.1%, and learning to learn at 6%. Discussion and Limitations In agreement with Maden (2011) and, more recently, Lalit and Piplani (2020) most students in all three classes appeared to be satisfied with jigsaw group work. Each IDEA (IDEA, 2021a) objective relevant to group work was rated favorably by the instrument, and all three were listed as a Strength to retain, defined as using the technique more often than classes of similar size and comparable student motivation. Although written for general classroom group work, objective #16: “Asked students to help each other understand ideas or concepts” (M = 4.9, SD = 0.0) was particularly interesting as it essentially described students teaching each other (e.g., Barkley, 2010; Costouros, 2020; Nusrath et al., 2019; Yoshida, 2018). This is certainly a critical element without which, according to Benton (2016), “the [jigsaw] process falls apart” (p. 42). As discontent or frustration with the technique could have had a negative influence on other group members (Benton, 2016; Costouros, 2020; Nusrath et al., 2019), it is intuitively likely that high student satisfaction with the technique influenced the positive coding outcomes discussed next. Although pattern-matching analysis (Yin, 2014) with the revised taxonomy (Anderson & Krathwohl, 2001) did not achieve a mean at the anticipated analyzing level, (4.0/6.0), it did show cognitive processing well above the remembering level (1.0/6.0) of rote learning and appropriate to the assignment requirements. Results indicated an average response at the applying level (M = 2.9, SD =1.233), indicating students making apt use of existing knowledge and available skills to prepare for the jigsaw and complete the assignment. However, applying did not account for most responses. Almost two-thirds (60.4%) exhibited a combination of comparing and contrasting at the understanding level (2.0/6.0) and attributing and distinguishing at the analyzing level (4.0/6.0). According to Anderson and Krathwohl (2001), this is not unusual because “most authentic academic Assessing a Modified Jigsaw Technique 63 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 tasks require the coordinated use of several cognitive processes” (p. 89) to create meaningful knowledge, and the two here aligned well with the assignment requirements. Most of the statements coded at the understanding level reflected statements of comparing multiple sources to build a better appreciation of all. These included the following examples: We found that all parents wanted their children to succeed and get the best education possible. However, we also found that children of Native American and Latinx culture may not have the same opportunities...This is definitely contrary to existing stereotypes. Many of these populations have immigrated to America and have been disadvantaged most of their time here. Analyzing responses were somewhat more diverse in their application. All related to breaking information into parts for closer inspection, but most statements coded as analyzing here involved “determining a point of view, bias, values, or intent underlying presented material” (Anderson & Krathwohl, 2001, p. 68) or students making attributions based on these. Many were indicative of statements focusing on specific critical points of information with students attempting to identify and make sense of causes or motives, a very specific form of analysis. Examples included the following: I have heard the term “tiger moms” in reference to Asian American mothers. It is a stereotype that the parents push their children to extremes to be successful by any means. When discussing the article, that stereotype proved false. What mainstream society might view as pushing their children too hard should be viewed as fostering a love of learning and expecting their children to work hard. Overall, parents in all these cultures raise their children similarly when it comes to education and family. Parents want to see their children succeed and to be the best individuals they can be… it left me thinking how much influence socio-economic status has on children’s futures. Another aspect of analyzing noted here was distinguishing. This can be seen in the examples below as attempts to discern relevant from irrelevant parts but more often to distinguish fact from fiction: One of the more important factors that was discussed was that the Latina/o community come from many different countries making the group actually very diverse, though they share many of the same values. I feel people often assume the Latinx community is all from the same country with the same traditions because they speak Spanish. I was aware of a few stereotypes about the [Asian American] culture, such as “all Asians are good at math” or the idea that parenting is more of a Assessing a Modified Jigsaw Technique 64 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 dictatorship. Fortunately, the material I was given not only challenged these stereotypes but provided me with a whole new perspective... As indicated by the mean, student thinking here is less related to retention and more related to creating new understanding and transferring learning to other areas. This meets Anderson and Krathwohl's (2001) definition of meaningful learning and “provides students with the knowledge and cognitive processes they need for successful problem solving” (p. 65). Results here also specifically confirmed Anderson and Krathwohl’s (2001) observation that “learning to analyze may be viewed as an end in itself, [but] it is probably more defensible educationally to consider analysis as an extension of understanding” (p. 79). Applying pattern-matching analysis (Yin, 2014) with Fink’s (2013) significant learning experiences found positive results as well. As predicted, an average response of integration (M = 3.09, SD = 1.109) was found. Findings here demonstrated that the cognitive process of understanding and learning to integrate are perhaps relatable, though not identical, in this collaborative setting. Integration certainly involves seeking connections, but it is not limited to the assigned material. For instance, the example below connects information from the jigsaw with ideas introduced weeks earlier in the class: The emphasis that African Americans place on balancing individual needs and the family as well as respecting elders is also seen in the Latinx and Native American culture. We discussed how the balance between the individual and family needs can be the outcome of parents trying to keep their family subsystems and culture flourishing. With the integration of a variety of cultures, we thought that emphasizing the importance of their culture was a way to preserve their history and legacy. A culture’s uniqueness gives people the guidance of knowing who they are and where they come from. This reminded us the discussion on cultural competence during the first week of class. Likewise, integrating can facilitate an interdisciplinary awareness and learning (Fink, 2013). The statement below indicates a possible connection to coursework in human development: …Puerto Rican mothers stressed interdependence and the European American mothers pushed for independence. After learning about this, I began to think about how humans innately need others to survive and that we need interdependence for relationships to be considered relationships. I believe the way Latina/o culture teaches children is more developmentally appropriate… Integrative learning can also assist students in learning to link different views and methods of analysis and begin to develop a more inclusive perspective of a problem (Fink, 2013). This is comparable to what Fink (2013) described when stating, The theme that occurs repeatedly in the discussion of these ventures is the desirability of breaking down walls and overcoming the isolation of students Assessing a Modified Jigsaw Technique 65 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 and subjects from each other that is they are seeking to create hitherto absent connections and integration among different people and different ideas. (p. 50) Interestingly, coding with Fink (2013) again resulted in two distinct but related elements, integration (31.9%) and human dimension (31.3%), again accounting for approximately two thirds (62.2%) of the responses. Examples of learning about others included this observation: In my [expert] group, we all agreed that Native American parenting is similar to typical American parenting, but better. We said this because these parents have goals for their kids such as having them become more independent, having a good work ethic, and having them care for others. The goals are the same for Native American parents as well, but they seem to follow through with them. Learning about the self can be seen below. I had always heard that many Latina/o students received more financial aid than most others. But they have access to less aid for higher education…I’m a little embarrassed but I can now correct my thinking. …it made me realize how we tend to focus on every existing difference, when we share a variety of similarities. Like the results with the taxonomy, these two types of learning are also relatable. Fink (2013) stated confidently that integrative learning can aid students in building connections between what they are learning in class and their daily lives, the human dimension. Lastly, it should be noted again that all participants here were female. Although our Family and Consumer Sciences department always includes a smaller percentage of male students, all female classes are typical in the Human Development and Family Science major. Therefore, care should be taken in generalizing these results to male students or more diverse classes. Future Directions Costouros (2020) noted that the jigsaw technique could vary with the instructor. Certainly, various jigsaw models have a range of strengths, applications, and goals. All involve a search to better understand an existing identified whole by considering the pieces, as with the parenting practices in America used here. However, it is also conceivable that this modified form could have usefulness beyond analyzing distinct elements of a single previously identified concept or topic. Considering that integrative learning increases the possibility of other types of learning (Fink, 2013), it seems likely that broader, more open-ended topics could be explored in the classroom with this jigsaw as well. For instance, contemporary and historical views of the Voting Rights Act could be compared and contrasted. Similarly, varied Assessing a Modified Jigsaw Technique 66 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 arguments concerning climate change could be explored using credible news articles. Perhaps this more investigative open-ended search could be used with such important current topics to identify unanswered questions or separate fact from opinion. Although there may be no specific completed puzzle as a goal, practicing analysis and problem-solving has positive educational value (Anderson & Krathwohl, 2001). Also, though theory in teaching and learning design and SoTL research has been somewhat overlooked (Divan et al., 2017), educational theory played a significant and necessary role in design and analysis here (e.g., Merriam, 1998). More specifically, Yin’s (2014) observation that theory used in design could play an authentic role in analysis was confirmed in coding and triangulation. Perhaps it is time for theory, theorizing, and theory testing to play a greater role in SoTL work. Conclusion Collaborative learning has consistently been shown to encourage critical thinking (Dewati et al., 2019; Nusrath et al., 2019) to enable students to teach others (Crone & Portillo, 2013) and to result in meaningful learning experiences. Likewise, the jigsaw technique (Aronson, 2021; Aronson & Patnoe, 2011) has long proven useful in fostering higher order thinking and processing (Crone & Portillo, 2013; Nusrath et al., 2019) across multiple variations. That was the case here as well as the analysis revealed two primary levels of processing, understanding, and analyzing, both significantly beyond rote learning and strongly related to each other. Although the comparing aspects of each were perhaps an inevitable part of the assignment and therefore not unexpected. The same could be said for the integration type of learning (Fink, 2013) as considering seemingly dissimilar information to create new connections was also a part of the assignment. However, other findings, also positive, were unexpected as both thinking and learning went deeper as aspects of analyzing began to challenge known stereotypes held by some students. Not only did this confirm Anderson and Krathwohl’s (2001) probable link between processing at understanding and analyzing, for some this became an opportunity to learn and to change their mind on some previously held beliefs. It seems possible that these new insights could also be understood as learning about self and others, or Fink’s (2013) human dimension. Like understanding and analyzing above, results here appear in agreement with Fink’s (2013) observation that significant learning “is not hierarchical but rather relational and even interactive” (p. 37). These results indicated that the changes made here maintained the educational value of the jigsaw technique. This modified form was indeed successful in producing consistent, worthwhile learning and more critical thinking for most students. Maden (2011) and Costouros (2020) have described the jigsaw as having potentially limitless variations. Findings here certainly serve to confirm such flexibility. Assessing a Modified Jigsaw Technique 67 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 Conflict of Interest Statement The authors declare that there are no conflicts of interest regarding the publication of this article. References Anderson, B. (2019). Teaching developmental theory with interrupted video case studies: Multi-measure evidence. Journal of Scholarship of Teaching and Learning, 19(5), 123–136. doi:10.14434/josotl.v19i5.25385 Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Blooms taxonomy of educational objectives. Longman. Aronson, E. (2021). Jigsaw in 10 easy steps. http://jigsaw.org/#steps Aronson, E., Blaney, N., Stephan, C., Sikes, J., & Snapp, M. (1978). The jigsaw classroom. Sage Publications. Aronson, E., & Patnoe, S. (2011). Cooperation in the classroom: The jigsaw method. Pinter and Martin. Barkley, E. F. (2010). Student engagement techniques. Jossey-Bass. Benton, R. (2016). Put students in charge: A variation on the jigsaw discussion. College Teaching, 64(1), 40–45. https://doi.org/10.1080/87567555.2015.1069725 Bishop-Clark, C., & Dietz-Uhler, B. (2012). Engaging in the Scholarship of Teaching and Learning: A guide to the process, and how to develop a project from start to finish. Stylus. Calkins, S. C., & Rivnay, J. (2021). The jigsaw design challenge. Journal of Effective Teaching in Higher Education, 4(3), 19–35. https://doi.org/10.36021/jethe.v4i3.249 Costouros, T. (2020). Jigsaw cooperative learning versus traditional lectures: Impact on student grades and learning experience. Teaching & Learning Inquiry, 8(1), 154–172. doi:10.20343/teachlearninqu.8.1.11 Crone, T. S., & Portillo, M. C. (2013). Jigsaw variations and attitudes about and the self in cognitive psychology. Teaching of Psychology, 40(3), 246–251. doi: 10.1177/0098628313487451 Dewati, K. T. P. S., Basori, B., & Efendi, A. (2019). Comparison of application of Jigsaw IV and Reverse Jigsaw methods based on students’ participation and achievement. Journal of Informatics and Vocational Education, 2(2), 75–81. doi:10.20961/joive.v2i2.35750 Assessing a Modified Jigsaw Technique 68 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 Divan, A., Ludwig, L., Matthews, K., Motley, P., & Tomljenovic-Berube, A. (2017). Survey of research approaches utilized in the scholarship of teaching and learning publications. Teaching & Learning Inquiry, 5(2). https://doi.org/10.20343/teachlearninqu.5.2.3 Doymus, K., Karacop, A., & Simsek, U. (2010). Effects of jigsaw and animation techniques on students’ understanding of concepts and subjects in electrochemistry. Educational Technology Research and Development, 58, 671–691. https://doi.org/10.1007/s11423-010-9157-2 Dhull, O., & Verma, G. (2019). Jigsaw teaching technique and teaching science. International Journal of Research and Analytical Reviews, 6(2), 809–815. Fink, L. D. (2013). Creating Significant Learning Experiences. Jossey-Bass. Ghaith, G., & El-Malak, M.A. (2004). Effect of jigsaw ii on literal and higher order EFL reading comprehension. Educational Research and Evaluation, 10, 105– 115. Hamilton, L., & Corbett-Whittier, C. (2013). Using Case study in education research. Sage. Haviz, M., & Lufri, L. (2019). Implementing of subject jigsaw learning model and its impact on students’ achievement in Embryology course. Jurnal Pendidikan Biologi Indonesia, 5(3), 435–442. https://doi.org/10.22219/jpbi.v5i3.9864 Hedeen, T. (2003). The reverse jigsaw: A process of cooperative learning and discussion. Teaching Sociology, 31, 325–332. doi: 10.2307/3211330 Hopper. T., & Hoque, Z. (2006). Triangulation approaches to accounting research. In Z. Hogue (Ed.), Methodological issues in accounting research: Theories and methods (pp. 477–486). Spiramus Press. Hyde, K. F. (2000). Recognizing deductive processes in qualitative research. Qualitative market research: An International Journal, 3, 279–292. doi: 10.1108/13522750010322089 IDEA. (2021a). About IDEA. https://www.ideaedu.org/about-idea/ IDEA. (2021b). IDEA Notes of Instruction. https://www.ideaedu.org/idea-notes-on- instruction/ Kaplowitz, D. R., Griffin, S. R., & Seyka, S. (2019). Race dialogues: A facilitator's guide to tackling the elephant in the classroom. Teacher’s College Press. Krathwohl, D.R. (2009) Methods of educational and social science research: The logic of methods (3rd ed.). Waveland Press. Assessing a Modified Jigsaw Technique 69 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 Laal, M., & Ghodsi, S.M. (2012). Benefits of collaborative learning. Social and Behavioral Sciences, 31, 486–490. doi: 10.1016/j.sbspro.2011.12.091 Lalit, M., & Piplani, S. (2019). Active learning technology—Jigsaw method: An innovative method of learning anatomy. Journal of the Anatomical Society of India, 68(2), 147–152. doi: 10.4103/JASI.JASI_57_19 Maden, S. (2011). Effect of Jigsaw I technique on achievement in written expression skill. Educational Sciences: Theory & Practice, 11(2), 911–917. McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. https://pubmed.ncbi.nlm.nih.gov/23092060/ Merriam, S. B. (1998). Qualitative research and case study applications in education. Josey-Bass. Nusrath, A., Dhananjaya, S. Y., Dyavegowda, N., Arasegowda, R., Ningappa, A., & Begum, R. (2019). Jigsaw classroom: Is it an effective method of teaching and learning? Student’s opinions and experience, 13(2): JC01–JC04. doi: 10.7860/JCDR/2019/39613.12540 Pearse, N. (2019). An illustration of deductive pattern-matching procedure in qualitative leadership research. The Electronic Journal of Business Research Methods, 17(3), 143–154. doi:10. 34190/JBRM.17.3.004 Samuel, I. R. (2018). Effects of jigsaw IV, group investigation and reversed jigsaw cooperative instructional strategies on basic science students’ achievement and retention. International Journal of Innovative Education Research, 6(2), 54–62. Stake, R. E. (2005) Qualitative case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), The sage handbook of qualitative research (pp. 443–466). Sage. Timayi, J., Bolaji, C., & Kajuru, Y. (2015). Effects of Jigsaw IV cooperative learning strategy (J4CLS) on academic performance of secondary school students in geometry. International Journal of Mathematics Trends and Technology, 28, 12–18: https://www.ijmttjournal.org/archive/ijmtt-v28p504 Turkman, H., & Buyukaltay, D. (2015). Which one is better? Jigsaw II versus Jigsaw IV on the subject of the building blocks of matter and atom. Journal of Education in Science, Environment and Health, 1(2), 88–94. doi:10.21891/jeseh.43349 van Drie, J., & Dekker, R. (2013). Theoretical triangulation as an approach for revealing the complexity of a classroom discussion. British Educational Research Journal, 39(2), 338–360. https://doi.org/10.1080/01411926.2011.652069 Assessing a Modified Jigsaw Technique 70 Journal of Effective Teaching in Higher Education, vol. 5, no. 1 Weidman, R., & Bishop, M. J. (2009). Using the jigsaw model to facilitate cooperative learning in an online course. The Quarterly Review of Distance Education, 10(1), 51–64. https://www.learntechlib.org/p/103626/ Weimer, M. (2017, July). My students don’t like group work. The Teaching Professor. https://www.teachingprofessor.com/topics/for-those-who- teach/my-students-dont-like-group-work/ Weimer, M. (2020, March). Student attitudes about group work. The Teaching Professor. https://www.teachingprofessor.com/topics/for-those-who- teach/student-attitudes-about-group-work/ Yatimah, D., Solihin, S., Adman, A., & Syah, R. (2019). Jigsaw learning model based on cooperative instructional strategies to improve academic discussion and adult education on environmental concepts. Journal of Physics: Conference Series, 1402(3), 1–4. doi:10.1088/1742-6596/1402/3/033039 Yin, R. K. (2003). Case study research: Design and methods (2nd ed.). Sage. Yin, R. K. (2014). Case study research: Design and methods (5th ed.). Sage. Yoshida, M. (2018). Communication jigsaw: A teaching method that promotes scholarly communication. International Journal of Emerging Technologies in Learning, 13 (10), 208–224. doi.org/10.3991/ijet.v13i10.8850