Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 2 An Emerging Critical Lens: Elementary Teacher Candidates’ Developing Evaluation of Social Studies Resources on Online Sites of Curriculum Sharing Jennifer Gallagher East Carolina University Abstract: This article shares findings from a qualitative inquiry that explores preservice teachers’ evaluation of social studies curriculum resources found on Pinterest and Teachers Pay Teachers. Both at the beginning and the end of their elementary social studies methods course at a large university in the southeastern United States, the teacher candidates were asked to identify good and bad examples of social studies resources and justify their choices. Their choices and the justification they provided were analyzed using qualitative coding. Findings indicate that while teacher candidates’ choices and justifications were sometimes further developed by the end of their social studies methods course, their critical evaluation of shared online curriculum resources was incomplete or limited to near-exact examples from the class. The author poses a number of on-going considerations regarding teacher candidate equity literacy skills as they relate to evaluation and usage of online curriculum sharing sites. Key words: Online curriculum literacy, Pinterest, Teachers Pay Teachers Introduction Often, pedagogy is defined as the “how” of education while curriculum is defined as the “what”. For decades, commercially published curriculum materials have been the primary resource for instructional materials utilized by teachers to determine the “what” of curriculum. These resources then have had considerable impact on “how” children are taught in schools (Lowenberg Ball & Cohen, 1996). But at a time when state and local governments are continually underfunding resources for teachers (The Century Foundation, 2020) and at a time when the mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 3 influence of social media on professional and personal life continues to increase (Greenwood et al., 2016) and merge (Sihi & Lawson, 2018), there could be little doubt that our society would see a convergence of curriculum resources and social media. Shelton et al. (2020) have termed sites related to this convergence “the online marketplace of ideas” because they differ from more traditionally vetted resource sites such as those from professional teaching organizations. Ultimately, the field of teacher education is interested and motivated to improve the pedagogical practices of teachers. In this work it is necessary to attend to the current conditions under which teachers make their pedagogical choices (Cornbleth, 2001). Given the rate of teachers and teacher candidates are utilizing the online marketplace of ideas for curriculum resources (Greene, 2016; Hunter & Hall, 2018), there is perhaps no more pressing condition to understand than that of how teacher candidates are learning to evaluate the pedagogical resources within these digital spaces. In their study of 158 elementary pre-service teachers’ lesson plans, Sawyer et al. (2020) found that Pinterest and Teachers Pay Teachers were the most frequent site-specific internet resources utilized. Therefore, specific attention to teacher candidate evaluation of resources on these two sites is warranted. This article shares findings from a qualitative inquiry that explores preservice teachers’ evaluation of social studies curriculum resources found on Pinterest and Teachers Pay Teachers. Both at the beginning and the end of their elementary social studies methods courses, the teacher candidates were asked to identify good and bad examples of social studies resources on Pinterest or Teacher Pay Teachers and also justify their choices. Their choices and the justification they provided were analyzed using qualitative coding in order to understand the changes in their criticality of resources. Findings indicate that while teacher candidates’ evaluations were sometimes further developed by the end of their social studies methods course, their tools for critically consuming shared online curriculum resources were not fully developed or limited to near-exact examples from the class. This empirical evidence is important because it illustrates the continued importance of cultivating critical curriculum literacy skills within teacher education programs. Literature Review Teachers’ Curricular Choices Pedagogic choices can be defined as “the complex, practical-oriented set of understandings which they [teachers] use (to) actively shape and direct the work of teaching” (Elbaz, 1983, p. 3). While evaluating and choosing curricular materials is just one part of the many pedagogical mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 4 choices that teachers make, these decisions (like other pedagogical choices) are influenced by teachers past experiences and their beliefs (McCutcheon, 1995). In social studies education, pre-service teachers and in-service teachers’ curricular choices have been found to be influenced by technical and philosophical reasons (Swalwell & Pellegrino, 2015). Their choices have also been found to reflect more sanitized or “master narrative” versions of the subject (Swalwell & Pellegrino, 2015). Additionally, despite being drawn to online resources for the teaching of history, teacher candidates are not very adept at distinguishing good resources from bad ones in digital spaces (Salinas, Bellows, & Liaw, 2011). In particular, social studies teachers and teacher candidates are often drawn to curricular resources and activities that are “cute” (Bauml, 2016) and “fun” (DiCamillo, 2010). Teachers’ Use of Social Media Educators have been found to use social media for a number of professional purposes (Hunter & Hall, 2018). For example, they use Twitter and Pinterest for provide ongoing professional learning (Holmes et al., 2013; Krutka & Carpenter, 2016) and Facebook for peer support networks (Kelly & Antonio, 2016). But the use of social media by educators also poses a number of problems and challenges. Carpenter and Harvey (2019) found these challenges fit into five levels of context: the intrapersonal, interpersonal, local school community, online educator community and broadly, culture and society. For example, educators may experience intrapersonal challenges that include internal conflicts of how to represent themselves online to broader online educator community challenges or interpersonal challenges of engaging in echo chambers because specific online communities become insular by members who “overwhelmingly sharing the same opinions and philosophies” (p. 7). Despite these challenges, many teachers turn to social media for curricular resources in particular. Pinterest According to Hertel and Wessman-Enzinger (2017) curricular resources found on Pinterest may be low quality. In their analysis of math resources, they found approximately 33% of those analyzed had mathematical errors. Despite this, many teachers still turn to Pinterest for curricular materials and they have varying strategies for navigating and using the resources found there. According to Schroeder, Curcio and Shelton (2019), elementary and secondary teachers engage with Pinterest in a variety of ways and there are differences in how pre-service teachers and in- service teachers navigate and evaluate curriculum resources on Pinterest. mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 5 Teachers Pay Teachers While not always included in research on social media in education, Teachers Pay Teachers should be considered a platform of social media because it allows users to create and share content (often for purchase, in which Teachers Pay Teachers earns a portion). Teachers Pay Teachers boasts that five million teachers use Teachers Pay Teachers for curriculum resources and that two out of every three teachers in the U.S. is a member (Teachers Pay Teachers, 2020). Teachers selling lesson plans on Teachers Pay Teachers, also known as teacherpreneurs, are largely white and female, as well as experienced practitioners who engage in Teachers Pay Teachers in a variety of ways (Shelton & Archambault, 2019). They earn supplemental income (some have become millionaires) through the marketplace and some have left the teaching profession to sell their lesson plans full time (Greene, 2016; Kardoza, 2018). However, despite the financial success of the platform, curriculum expert reviewers evaluated the curriculum resources sold on the site as mostly mediocre or poor (Polikoff & Dean, 2019). The Context of Elementary Social Studies Education The context of elementary social studies education is important to consider when attempting to understand teacher candidates’ use of the questionable social studies curriculum resources on social media. The National Council for Social Studies identifies five characteristics of powerful social studies: meaningful, integrative, challenging, value-based and active (MICVA). Instead of utilizing time in the elementary classroom to provide powerful social studies education, the subject has been largely marginalized within elementary classrooms since the standards-based movement began in the 1990’s (Fitchett & Heafner, 2010; Rock et al., 2006; Zhao & Hoge, 2005). Additionally, elementary teachers and teacher candidates often report low levels of confidence in their social studies content knowledge (e.g., Bolick et al., 2010). In their study, Bolick et al. (2010) found that elementary teacher candidates struggled to apply the content from their social studies-related general education courses to elementary teaching and that Praxis content knowledge testing may exacerbate these low levels of confidence (p. 7). These contextual factors are important as they may influence the reasons why teachers rely on online curriculum sharing sites and the frequency with which they use them within specific curriculum subjects. Additionally, within any study that attempts to understand criticality in evaluating social studies resources, it is important to acknowledge that social studies curriculum has long been problematic. Social studies education and curriculum has long been found to be frequently rooted in materials and practices that are dominated by whiteness (e.g. Hawkman & Shear, 2020) mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 6 as well as sexist (e.g. Engebretson, 2014), Islamophobic (e.g. Sensoy, 2014), colonial settler (e.g. Masta, 2018; Shear et al., 2015), heteronormative and cisnormative (e.g. Mayo, 2017). Bias, prejudice and narrow “master narratives” (see more: Demoiny & Ferraras-Stone, 2018; Salinas, Blevins & Sullivan, 2012) are not a new phenomenon in social studies education since the influence of social media, but it is important to understand how these problems are addressed (or ignored) within the more contemporary practices of evaluating curriculum on Pinterest and Teachers Pay Teacher. Rodriguez et al. (2020) found that given a particular Black History topic and critical curriculum literacy scaffolding, teacher candidates were limited in their ability to critically evaluate curriculum resources and they “generally defaulted to popular, highly visible resources and made evaluations about credibility, reliability, and utility based on the metrics that for-profit platforms are designed to reinforce” (p. 516). Thus, they were greatly influenced by the mechanisms of the platform itself and limited in their ability to navigate those mechanisms in ways that addressed race or racism within their curriculum analysis. Equity Literacy in Social Studies Teacher Education The Equity Literacy Framework (Gorski and Swalwell, 2015) is a set of four abilities that enable educators to be “a threat to inequity in all of their spheres of influence” (Equity Literacy Institute, 2020). The four abilities include: “1. Recognize even subtle forms of bias, discrimination, and inequity. 2. Respond to bias, discrimination, and inequity in a thoughtful and equitable manner. 3. Redress bias, discrimination, and inequity, not only by responding to interpersonal bias, but also by studying the ways in which bigger social change happens. 4. Cultivate and sustain bias- free and discrimination-free communities, which requires an understanding that doing so is a basic responsibility for everyone in a civil society.” (p. 35). According to Gorski and Swalwell (2015) the abilities of equity literacy are applicable to all subject areas and are most effective when they are interdisciplinary and integrated. In a number of different contexts, research in teacher education has found that teacher candidates struggle with even the first ability of the equity literacy framework - “recognizing even subtle forms of bias, discrimination and inequity” (Gorski & Swalwell, 2015). This is especially true of teacher candidates ability to recognize bias, discrimination and inequity in regard to race. Crowley and Smith (2015) found that some teacher candidates rejected ideas of racial privilege when discussing race and whiteness in education. The researchers attributed limited experience with structural thinking as an obstacle that prevented some teacher candidates to recognize systemic (rather than interpersonal) forms of racism. mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 7 Methodology This research was framed by the following two guiding questions: (a) How did teachers candidates justify their choices of good and bad social studies resources on sites of curriculum sharing at the beginning and end of their social studies methods course? (b) Was there a change in the justifications from the beginning to the end of the social studies methods course? Conceptual Framework In this work, I draw from the vast fields of social studies education and social justice education. I particularly employ two frameworks, one from each of these fields, for understanding the phenomenon under study. The first is the MICVA framework of Powerful and Purposeful Elementary Social Studies Teaching explicated by the National Council for Social Studies (2017) which proclaims valued characteristics of social studies education: Meaningful, Integrative, Challenging, Value-Based, and Active. This framework was used to make meaning of the teacher candidates evaluation of the curriculum resources in context of accepted characteristics of powerful social studies within the field. The second is the equity literacy framework of Gorski and Swalwell (2015) reviewed above. Within Gorski and Swalwell’s (2015) equity literacy framework, recognizing “even subtle forms of bias, discrimination and inequity” is an important first ability for teachers in order for them to also be able to subsequently respond to and redress bias and inequity and ultimately be able to “cultivate and sustain bias-free and discrimination-free communities” (p. 35). This framework grounded the research by assuming the value of the equity literacy skill to recognize bias, discrimination and inequity (the first ability of the equity literacy framework) – and applying it to teacher candidates’ evaluation of resources from sites of curriculum sharing. Research Design This research project is a qualitative naturalistic inquiry (Lincoln & Guba, 1985) because it does not attempt to isolate variables to make causal claims. Rather, the research is designed to better understand a phenomenon as it exists bound by context. The work also aligns with the conceptions of the qualitative researcher as bricoleur, or quilt maker, who uses the tools and strategies necessary and available to understand the research question within a context (Denzin & Lincoln, 2008). Participants and Context. The participants of the study were fourteen teacher candidates who were students in an elementary social studies methods courses at a large public university in the mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 8 southeastern United States. Thirteen participants were female, and one was male. Three participants identified as African American, one participant identified as Latina and ten participants were white. The teacher candidates were part of classes in two different semesters of the same elementary social studies methods course. All were taught by the principal investigator/author. All of the students in both sections of the course were offered the opportunity to have their pre- and post-data analyzed, only six from the first section and eight from the second semester section consented to have their work analyzed. Both courses took place in the second semester of the students’ junior year in their undergraduate teacher education program. While the teacher education program does not wholly ground itself in inquiry or social justice education, this particular elementary social studies methods course centers on inquiry, master and counter narratives of the social sciences, disciplinary literacy, as well as justice-oriented and MICVA characteristics of social studies education. Procedure and Data Collection. At the beginning of the course, as part of a course pre- assessment, students were asked to browse Teachers Pay Teachers and Pinterest for the following (a) a good example of a social studies curriculum resource and (b) a bad example of a social studies curriculum resource. This pre-assessment did not offer additional criterion in order to understand the prior concepts of “good” and “bad” social studies curriculum students were entering the course with. For each answer, they were asked to provide a link where they found the resource and give a brief explanation of why they chose it as either “bad social studies” or “good social studies”. During the course of the semester, the instructor modeled evaluations of curriculum resources and students (multiple modeled examples were from Pinterest and Teachers Pay Teachers). The students were also provided multiple opportunities to evaluate social studies curriculum and resources particularly for the narratives and perspectives they were illuminating about the social world. They were scaffolded with readings, activities and discussions that compared “master” and “counter” narratives (e.g. See Demoiny, & Ferraras-Stone, 2018). At the end of the semester, the students completed the same google form again as part of their semester post-assessment asking them to identify a “good” example of social studies curriculum resource and a “bad” example of social studies curriculum resource with short justifications for each. This assessment attempted to evaluate how their conceptions of “good” and “bad” social studies had grown or changed after their course experiences. Data Analysis. In order to answer the research questions, the teacher candidates’ short written responses of their justifications were analyzed. The links of resources that the students provided were also analyzed to provide context for the students justifications and to evaluate the mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 9 resources the students were choosing. The written data were analyzed in two rounds of qualitative coding. The first semester participants’ data were organized into a spreadsheet and coded through emergent ideas directly from the data. This process was done by drawing from ideas of grounded theory (Strauss & Corbin, 1998) which allowed for recurring ideas to emerge directly through the data in a grounded theory approach while also acknowledging that all knowledge is theory-laden. Initial themes emerged from this first set of data. After the second round of data collection, with the new semester of participants, their data was organized and initally coded first using the a priori codes from the first cycle - but nuances within the first set of codes emerged through the second set of data. Therefore, all the data was re-coded through a second cycle of focused coding (Saldaña, 2016) with the further refined coding structure. Then, the coding was compared both within and between cases to understand the differences in teacher candidate evaluation of online resources from the beginning of the course to the end and the commonalities among teacher candidates’ growth and limitations. The coded data that represented commonalities in the teacher candidates’ growth and limitations were developed into themes as the research findings. Findings At the beginning of the course, participants largely evaluated online resources with vague reasoning about the substance of the content and potential level of engagement. At the end of the course, they more often identified additional characteristics, including notions of equity, and more sophisticated evaluations of intellectual demand. However, despite one focus of the course being anti-bias curriculum, some students were still choosing problematic resources. By the end of the course, some students had developed partially developed lenses of criticality when choosing resources but were often referring to near exact examples of problematic resources from class. While the intention of this research was not to evaluate the efficacy of the course in developing students’ critical online curriculum literacy skills, rather the data was collected to understand commonalities and conceptual changes among pre-service teachers during particular points in their teacher education program. The findings are organized starting with the most salient theme. Moving from a Vague Focus of Engagement to More Sophisticated Lenses of Intellectual Demand At the beginning of the course, teacher candidates were largely evaluating resources based on their perceived potential for engagement from students. However, the justifications the teacher mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 10 candidates used to make this evaluation were limited in their specificity of what engagement meant. Their justification often only used the buzzwords of “engaging”, “interactive” or they only included the general actions that would be required of students, such as “draw” or “write”. One example of a limited evaluation criteria of engagement initially used by a Teacher Candidate D was “This lesson is a great way for the students to learn about different Christmas traditions in other countries. It is fun and engaging yet informative lesson for the students to learn.” Similarly, Teacher Candidate D used a limited conception of engagement to negatively evaluate a resource. “Although this seems to be an informative lesson, it is not very engaging for the students. The students will only be reading and answer questions in this lesson which is not engaging.” Teacher Candidate C also used a limited conception of engagement to evaluate a resource as bad. “I don't think this lesson is typically a "good" lesson plan because it looks like a lot of worksheets for the students and I have always been told to try and steer away from worksheets as much as possible. From flipping through the lesson preview I saw multiple worksheets and I didn't like the worksheets that they had posted.” With the simplistic notion of “worksheets are bad” and “answering questions is not engaging”, the teacher candidates didn’t specify what characteristics or intellectual demand requirements of the worksheets or questions made them “bad”. These pre-assessment justifications don’t provide any specific or sophisticated ideas of what engagement means or the differences between questions that require lower levels of engagement, versus questions that require more intellectual demand. In the post-assessment, teacher candidates were able to identify more specific ideas of intellectual demands required of students within the resource that made the resources more engaging. Rather than just referring to a resource as “interactive” or “engaging”, many students were able to identify that resources would require students to “create”, “reflect”, “analyze”, “research”, “investigate”, “make an educated opinion”, “discuss” and “make decisions”. Teacher candidate H was able to evaluate a resource as “bad” during the post-assessment because he was able to specify the low-level intellectual demand that the resource required of students. He wrote, “This just has students remembering information about the Bill of Rights verbatim.” This evidence supports a claim that students were growing in their ability to evaluate resources based on intellectual demand. In sum, the participants displayed evidence of first starting with vague notions of engagement and then growing to more specific ideas about intellectual demands when choosing sources at the end of the semester. This finding relates to the NCSS MICVA framework, as it shows that teacher candidates were developing more sophisticated ways of evaluating the MICVA characteristic of “challenging” by the end of their social studies methods course. mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 11 Shifting Focus from Content to Processes of Learning The second most repeated code when analyzing the pre-assessment data was the focus on content. Many teacher candidates justified their evaluation of a good or bad curriculum resource based on if they found the content knowledge embedded in the resource “important” or “deep” or just related to social studies. For example, during the pre-assessment Teacher Candidate J evaluated a Constitution Unit as good because “the US Constitution is a huge piece of history that is still used today.” Similarly, Teacher Candidate N evaluated a resource as “good social studies” because “it teaches children about real world social issues.” The content was a primary lens utilized for these evaluations and justifications. During the post assessment the teacher candidates were more focused on the processes of learning in the curriculum resources than the actual topics of the content. Teacher candidates frequently used terms like “inquiry”, For example, Teacher Candidate J, who previously focused the whole of her evaluation on the content of the topic, focused on the potential for inquiry and the topic when evaluating resources at the end of the course. She wrote, “This is a good example of social studies because it starts off with a question. Then students with or without the help of the teacher answer the question below in the chart. You can make this assignment into an inquiry-based lesson. It has the students learning about communities, which is social studies.” Likewise, Teacher Candidate K also prioritized the process of learning in her post-assessment evaluation of a “good social studies curriculum resource”. She explained, “This is an inquiry-based lesson that has the students learning about a topic enough to where they can make an educated opinion. This not only works on history but also plays on civics and government.” This finding illuminates how teacher candidates were growing in their ability to evaluate the characteristic of active from the NCSS MICVA framework - as active implies that students are actively, rather than passively, constructing new knowledge within processes of learning. A Developing But Incomplete Lens of Equity Literacy A comparison of the coding from the pre-assessment to the post-assessment also provides evidence that the group of teacher candidates were developing a lens of equity literacy in their evaluation of online curriculum resources. In the pre-assessment, only two teacher candidates referred to issues of perspective within a curriculum resource, one to evaluate the resource as bad because it came from a home-school parent and the second to justify a resource that vaguely explained the strategy of “hot seating” a historical figure as bad because it could lead to bias. mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 12 In the post assessment, six out of the fourteen participants evaluated one or both of their resources using multicultural perspectives as an evaluation criterion. Teacher Candidates F and G considered perspectives missing from the resource that they evaluated as bad. Teacher F wrote, “This is a bad example of social studies curriculum because it teaches a single story. It teaches only that it was our country's ‘destiny’ to take over the West but teaches nothing about the Native American or the Mexican perspective.” And in her evaluation of a display board of student artwork of American Symbols, Teacher G explained, “I believe this activity of American symbols is bad social studies because yes these are American symbols and they should be taught, but not all the students will be able to see themselves in these symbols. I think this is trying to conform everyone to the same things that overall represent ‘America’, but it isn't all of America.” As a group, the teacher candidates moved from exhibiting no lens towards multicultural perspectives to several evaluations based on the inclusion of multiple perspectives in the curriculum at the end of the semester. In an even more developed lens of equity literacy, three teacher candidates expressed concern over how multicultural perspectives were being represented in a curriculum resource. Both were concerned with how Native Americans were being represented in resources on Teachers Pay Teachers. Teacher Candidate B justified her chosen Thanksgiving resource as bad in the following way, “This is a lesson on Native Americans. It only shows the stereotypical aspect of Native Americans rather than the critical narrative or different perspectives.” and Teacher Candidate C evaluating a different resource on Native Americans explained, “This lesson looks like a bad example because already on the very front, they are portraying Native Americans in a very stereotypical way. This is how the whole lesson kind of seems and it looks like it describes them in one way and not different perspectives.” Teacher Candidate L used perspective to evaluate both of her resources in the post-assessment. But in her evaluation of a good resource, she included ideas about how diversity was being represented in her justification. She evaluated an interactive social studies notebook resource as good because, “It doesn't just go through history, it gets students involved and engaged. It also goes through Native Americans and that not all of them use totem poles that is only a certain area of American Indians that do that. It is diverse!” All six of the teacher candidates who used the inclusion of multicultural perspectives as part of their evaluation of resources exhibited a more developed equity literacy skill of recognizing inequity and bias than they did previously at the start of the semester. mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 13 Despite this evidence of developing lenses of multicultural perspectives, some students were still choosing resources that might be expertly evaluated as problematic. Teacher Candidate F chose a resource she determined good at the end of the semester because it had students analyzing multiple resources. However, the “pioneer journal” project had a singular view of history in that students were to make pioneer journals from the perspectives of pioneers who migrated west (http://www.literacylovescompany.com/2016/05/classroom-diy-pioneer-journals.html?m=1). This project ignores the plight of indigenous groups who lost land and resources to pioneers. Additionally, Teacher Candidate L’s resource, which she evaluated as good in part because it shows that “not all Native Americans used totem poles”, was problematic in that it was still grouping together diverse tribes into monolithic categories and displaying inauthentic cartoon images of indigenous people (https://www.teacherspayteachers.com/Product/Interactive- Notebook-Social-Studies-Bundle-1-Grades-3-5-1575617). So, while these teacher candidates were exhibiting partially developed abilities to consider what curriculum was representing multicultural perspectives, the decision by some to still evaluate problematic resources as “good” shows only a very beginning development of the first equity literacy skill of Gorski and Swalwell’s (2015) framework. Discussion Value of the Findings In one respect, the teacher candidates in this study were given an innately difficult task. Previous research has shown that there are not a lot of high-quality resources on curriculum sharing sites and in fact there may be an over-abundance of low-quality resources (Hertel & Wessman- Enzinger, 2017). And beginning research particular to social studies (e.g., Rodriguez et al., 2020) would suggest that there are few curriculum resources on these sites that include many of the MICVA characteristics or address race and racism in meaningful ways. Despite the problematic nature of the task, the justifications that teacher candidates wrote provide a glimpse into the teacher candidates’ developing criticality of curriculum on the already potentially problematic sites. Interestingly, the finding that teacher candidates were initially using a lot of vague buzzwords of “engaging” and “interactive” were some of the same words often used by “sellers” and “pinners” of the curriculum resource. Therefore, teacher candidates were initially evaluating the resources in ways that mirrored how the resources were being “advertised”. This further supports Rodriguez et al.’s (2020) claim of the overwhelming effect of racial platform capitalism as the mailto:gallagherj17@ecu.edu http://www.iajiss.org/ http://www.literacylovescompany.com/2016/05/classroom-diy-pioneer-journals.html?m=1 https://www.teacherspayteachers.com/Product/Interactive-Notebook-Social-Studies-Bundle-1-Grades-3-5-1575617 https://www.teacherspayteachers.com/Product/Interactive-Notebook-Social-Studies-Bundle-1-Grades-3-5-1575617 Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 14 teacher candidates were obviously influenced by the internal evaluation mechanisms of the platform that do not address race or racism (or even broader conceptions of equity). However, at the end of the course, the teacher candidates were able to employ additional criteria when evaluating online curriculum resources after their social studies methods course – many of which included intellectual demand and some of which included ideas of equity. These research findings relate heavily to the work of Michelle Bauml (2016) in social studies education who prompted her teacher candidates to determine if an activity or resource was worthy of teaching by asking them to consider, “Is it cute or does it count?” In similar ways, the teacher candidate participants in this study were able to move in their criticality of resources over time. This study shows that teacher candidates can grow in their evaluation of what “counted” in specific ways during their teacher education program and the findings reflect that their lens developed in ways that focused on higher intellectual demand, inquiry learning processes and multicultural perspectives – although their evaluation was not always critical enough to deeply unpack remaining problems in the resources they chose, particularly regarding race. Focus and Examples from the Course Influenced Evaluation While the purpose of this study was not to evaluate the impact of the social studies methods course, the changes in the teacher candidates’ criticality from the beginning of the semester to the end of the semester clearly relate to topics addressed in the course. The methods course focused on inquiry - a learning process that teacher candidates heavily relied upon when evaluating resources at the end of the semester. The methods course also modeled critically evaluating curriculum resources for bias and multicultural perspectives. In fact, some of the examples students found of bad resources - such as those displaying stereotypical or “single story” images of Native Americans were near mirror examples as the ones modeled in class. Therefore, while some students were able to find similar “bad examples” of bias in curriculum resources, they weren’t necessarily transferring that equity literacy ability of recognizing inequity to new content topics. The Development of Equity Literacy Takes Time One participant, Teacher Candidate L was able to explain an important aspect of recognizing bias in her justification - making sure indigenous people are not represented in inaccurate ways - when she explained she didn’t want students to only think of Native Americans as only living in teepees. But she was unable to recognize that her “good” resource was still exhibiting a lot of mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 15 racist stereotypes of native people through cartoon representations and inauthentic grouping of distinct tribes. Therefore, her equity literacy capability of recognizing bias (Gorski & Swalwell, 2015) was present, but it was not necessarily fully developed. This substantiates other studies that have found that developing critical lenses to recognize bias as well as equity and inequity is not without obstacles and such critical lenses take time to fully develop (King & Ladson-Billings, 1990) - like so many aspects of social change and human development. And similar to Rodriguez et. al (2020), this study has found that teacher candidates were unable to fully contextualize the problematic, racialized social studies curriculum resources found on Pinterest and Teacher Pay Teachers and further validates the need for specific tools of critical race media literacy in social studies teacher education. Infusing critical perspectives of social media into teacher education, as employed, examined and advocated in Schroeder & Curcio (2022), may further teacher candidates’ equity literacy skills within online curriculum resource evaluation because it could help them to contextualize the resource within an “ecosystem” of profit, consumerism and capitalism that often runs counter to equity (p. 138). Conclusion This article shared findings that support the claim that teacher candidates can grow in their ability to critically evaluate online social studies curriculum resources during their teacher education program. While this research provides an important glimpse into teacher candidates evaluation and potential use of online teaching resources, it also provides for further calls for work and research in the field. The first is a call to understand more longitudinal progressions of how teachers change in their ability evaluate online social studies resources as they grow from novice to expert classroom teacher. If teacher candidates are able to develop more critical online curriculum skills during their teacher education program, how can these skills be sustained and further developed in their in-service practice? This would be an important line of research to help improve continuing professional development for teachers. It will also be important to study specific interventions to cultivating teacher candidates’ and in- service teachers’ critical online curriculum skills. Are there best practices in cultivating teacher candidates or in-service teachers who can critically evaluate curriculum resources on sites of curriculum sharing? Rodriguez et. al (2020) found that one tool developed within the field (Gallagher et al., 2019) although initially helpful, did not fully support teacher candidates’ critical evaluation of curriculum resources. Nor did the regular course instruction in this study. However, given the apparent impact of the course focus and course examples on the ways in which the teacher candidates did develop in their evaluation of online resources in this study, it’s clear that mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 16 what we as teacher educators are doing in our methods courses matters tremendously to the ways in which our students are developing equity literacy habits. As the course instructor as well as the principal investigator in this study, the findings brought me great pause in considering how to make every single example from class an example that teacher candidates could use as a model for the equity literacy abilities they will need to practice throughout their career. Given the current context of increasing influence of online curriculum sharing sites, it will be important for the field to continue to create and evaluate focused strategies moving forward - especially given the restraint of time that both teacher educators and classroom teachers face in pedagogical planning and instruction. How can teacher education programs prepare students to evaluate online resources in ways that are conducive to the limited time available in preservice courses and provide strategies that are reasonable given the limited time for curricular planning they will have as in-service teachers? This article shares empirical evidence that teacher candidates can begin to develop these skills within their initial teacher education program. However, it is important to consider that their critical online curriculum skills may not be fully developed and will need continued professional development given the rapidly changing context of online curriculum sharing. mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 17 References Bauml, M. (2016). Is it cute or does it count? Learning to teach for meaningful social studies in elementary grades. The Journal of Social Studies Research, 40(1), 55-69. Bolick, C. M., Adams, R., & Willox, L. (2010). The marginalization of elementary social studies in teacher education. Social Studies Research & Practice, 5(2), 1-22. Cardoza, K. (2018). Why teachers selling lesson plans has sparked debate. PBS Newshour. Retrieved from: https://www.pbs.org/newshour/show/why-teachers-selling-lesson- plans-have-sparked-debate Carpenter, J. P., & Harvey, S. (2019). “There's no referee on social media”: Challenges in educator professional social media use. Teaching and Teacher Education, 86 (November 2019), 102904. The Century Foundation. (2020, July 22). TCF Study Finds U.S. Schools Underfunded by Nearly $150 Billion Annually. Retrieved January 08, 2021, from https://tcf.org/content/about- tcf/tcf-study-finds-u-s-schools-underfunded-nearly-150-billion-annually/?agreed=1 Cornbleth, C. (2001). Climates of constraint/restraint of teachers and teaching in W. Stanley (Ed.) Critical issues in social studies research for the 21st century, Greenwich, CT: Information Age Publishing, 73-95. Demoiny, S. B., & Ferraras-Stone, J. (2018). Critical literacy in elementary social studies: Juxtaposing historical master and counter narratives in picture books. The Social Studies, 109(2), 64-73. Denzin, N. K., & Lincoln, Y. S. (2008). Strategies of qualitative inquiry (Vol. 2). Sage. DiCamillo, L. (2010). “Fun” pedagogy curtails intellectual rigor in a U.S. History class. The Journal of Social Studies Research, 34(2), 175–207. Egan, K. (1978). What is curriculum?, Curriculum Inquiry 8(1), 65-72. Elbaz, F. (1983). Teacher thinking: A study of practical knowledge. Croom Helm. mailto:gallagherj17@ecu.edu http://www.iajiss.org/ https://www.pbs.org/newshour/show/why-teachers-selling-lesson-plans-have-sparked-debate https://www.pbs.org/newshour/show/why-teachers-selling-lesson-plans-have-sparked-debate https://tcf.org/content/about- Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 18 Engebretson, K. E. (2014). Another missed ppportunity: Gender in the national curriculum standards for social studies. Social Studies Research & Practice 9(3), 21-34. Equity Literacy Institute. (2020). Educational Equity: Equity Literacy Framework. Retrieved January 09, 2021, from https://www.equityliteracy.org/equity-literacy. Fitchett, P. G., & Heafner, T. L. (2010). A national perspective on the effects of high-stake testing and standardization on elementary social studies marginalization. Theory & Research in Social Education, 38(1), 114-130. Gorski, P. C., & Swalwell, K. (2015). Equity literacy for all. Educational leadership, 72(6), 34-40. Greene, K. (2016). For sale: Your lesson plans. Educational Leadership, 74(2), 28–33. Greenwood, S., Perrin, A., & Duggan, M. (2020, July 31). Demographics of Social Media Users in 2016. Retrieved January 08, 2021, from https://www.pewresearch.org/internet/2016/11/11/social-media-update-2016/ Hawkman, A. M., & Shear, S. B. (Eds.). (2020). Marking the" invisible": Articulating Whiteness in Social Studies Education. IAP. Hertel, J. T., & Wessman-Enzinger, N. M. (2017). Examining Pinterest as a curriculum resource for negative integers: An initial investigation. Education Sciences, 7(2), 45. Holmes, K., Preston, G., Shaw, K., & Buchanan, R. (2013). "Follow" me: Networked professional learning for teachers. Australian Journal of Teacher Education, 38(12), 55-65. Hunter, L. J., & Hall, C. M. (2018). A survey of K-12 teachers’ utilization of social networks as a professional resource. Education and Information Technologies, 23(2), 633-658. Kelly, N., & Antonio, A. (2016). Teacher peer support in social network sites. Teaching and Teacher Education, 56, 138-149. King, J. E., & Ladson-Billings, G. (1990). The teacher education challenge in elite university mailto:gallagherj17@ecu.edu http://www.iajiss.org/ https://www.equityliteracy.org/equity-literacy https://www.pewresearch.org/internet/2016/11/11/social-media-update-2016/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 19 settings: Developing critical perspectives for teaching in a democratic and multicultural society. European Journal of Intercultural Studies, 1(2), 15-30. Krutka, D. G., & Carpenter, J. P. (2016). Participatory learning through social media: How and why social studies educators use Twitter. Contemporary Issues in Technology and Teacher Education, 16(1), 38-59. Lincoln, Y. & Guba, E. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage. Lowenberg Ball, D. L., & Cohen, D. K. (1996). Reform by the book: What is—or might be—the role of curriculum materials in teacher learning and instructional reform?. Educational researcher, 25(9), 6-14. Masta, S. (2018). Settler Colonial Legacies: Indigenous Student Reflections on K-12 Social Studies Curriculum. Intersections: Critical Issues in Education, 2(2), 4. Mayo, J. B. (2017). Sexuality and queer theory in the social studies. The Wiley handbook of social studies research, 254-269. McCutcheon, G. (1995). Developing the curriculum: Solo and group deliberation. Longman Publishers USA. National Council for Social Studies (2017). Powerful, Purposeful Pedagogy for Elementary Social Studies. Polikoff, A., & Dean, J. (2019). The Supplemental Curriculum Bazaar: Is What's Online Any Good? Fordham Institute. Retrieved from https://fordhaminstitute.org/national/research/supplemental-curriculum-bazaar Rock, T. C., Heafner, T., Oldendorf, S., Passe, J., O'Connor, K., Good, A., & Byrd, S. (2006). One state closer to a national crisis: A report on elementary social studies in North Carolina schools. Theory and Research in Social Education, 34(4), 455-483. Rodriguez, N., Brown, M., & Vickery, A. (2020). Pinning for profit? Examining elementary preservice teachers’ critical analysis of online social studies resources about Black history. Contemporary Issues in Technology and Teacher Education (CITE Journal), 20(3), n3. mailto:gallagherj17@ecu.edu http://www.iajiss.org/ https://fordhaminstitute.org/national/research/supplemental-curriculum-bazaar Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 20 Saldaña, J. (2016). The coding manual for qualitative researchers (3rd ed.). Sage. Salinas, C., Bellows, M. E., & Liaw, H. L. (2011). Preservice social studies teachers’ historical thinking and digitized primary sources: What they use and why. Contemporary Issues in Technology and Teacher Education, 11(2), 184-204. Salinas, C., Blevins, B., & Sullivan, C. C. (2012). Critical historical thinking: When official narratives collide with other narratives. Multicultural Perspectives, 14(1), 18-27. Sawyer, A. G., Dredger, K., Myers, J., Barnes, S., Wilson, R., Sullivan, J., & Sawyer, D. (2020). Developing teachers as critical curators: Investigating elementary preservice teachers’ inspirations for lesson planning. Journal of Teacher Education, 71(5), 518–536. Schroeder, S., Curcio, R., & Lundgren, L. (2019). Expanding the learning network: How teachers use Pinterest. Journal of Research on Technology in Education, 51(2), 166-186. Schroeder, S., & Curcio, R. (2022). Critiquing, curating, and adapting: Cultivating 21st-century critical curriculum literacy with teacher candidates. Journal of Teacher Education, 73(2), 129-144. Sensoy, Ö. (2014). Beyond fearing the savage: Responding to islamophobia in the classroom. In E. W. Ross (Ed.), The social studies curriculum: Purposes, problems, and possibilities (4th ed., pp. 289-312). State University of New York Press. Shear, S. B., Knowles, R. T., Soden, G. J., & Castro, A. J. (2015). Manifesting destiny: Re/presentations of indigenous peoples in K–12 US history standards. Theory & Research in Social Education, 43(1), 68-101. Shelton, C., Schroeder, S., & Curcio, R. (2020). Instagramming their hearts out: What do edu- Influencers share on Instagram? Contemporary Issues in Technology & Teacher Education 20(3), 529–554. Shelton, C. C., & Archambault, L. M. (2019). Who are online teacherpreneurs and what do they do? A survey of content creators on TeachersPayTeachers.com. Journal of Research on Technology in Education, 51(4), 398-414. Sihi, D., & Lawson, K. (2018). Marketing leaders and social media: blending personal and mailto:gallagherj17@ecu.edu http://www.iajiss.org/ Journal of International Social Studies, v. 12, n. 1, 2022, pp. 2-21 Corresponding author: gallagherj17@ecu.edu ©2012/2023 National Council for Social Studies International Assembly http://www.iajiss.org ISSN: 2327-3585 Page 21 professional identities. Journal of Marketing Theory and Practice, 26(1-2), 38-54. Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. London: Sage. Swalwell, K., & Pellegrino, A. M. (2015). Teachers׳ curricular choices when teaching histories of oppressed people: Capturing the US Civil Rights Movement. The Journal of Social Studies Research, 39(2), 79-94. Teachers Pay Teachers (2020). About Us. Retrieved from https://www.teacherspayteachers.com/About-Us Zhao, Y., & Hoge, J. D. (2005). What elementary students and teachers say about social studies. The Social Studies, 96(5), 216-221. About the Author: Jennifer Gallagher - is an assistant professor in the Literacy Studies, English Education and History Education Department in the College of Education at East Carolina University where she teaches social studies education courses and serves as the program coordinator for the undergraduate and graduate history education programs. Gallagher taught social studies for several years in Colorado before earning her PhD at Iowa State University. Her research and her professional efforts focus on supporting teacher candidates, in service teachers, and P-20 students in critical inquiry of the social world. Her work has been presented and published in state, national, and international outlets including Social Education, Social Studies and the Young Learner, Social Studies Research and Practice, Education, Citizenship and Social Justice and Journal of Social Studies Research. She can be reached at gallagherj17@ecu.edu or on Twitter at @DrGSocialEd mailto:gallagherj17@ecu.edu http://www.iajiss.org/ https://www.teacherspayteachers.com/About-Us mailto:gallagherj17@ecu.edu