Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 51 Rethinking Reliability after Students Evaluate a Facebook Page about Health Care in Singapore James S. Damico, Indiana University, U.S.A. Mark Baildon, National Institute of Education, Singapore ___________________________________________________________________________________________ Abstract: This article considers what happened when a group of secondary level Social Studies students in Singapore evaluated the reliability of an opposition politician’s Facebook page on affordable healthcare. The students in the study found the politician’s Facebook page to be unreliable because of its purpose, motive, or agenda. Findings suggest that even though students were aware of certain factors used to determine source reliability – provenance, purpose, source content, cross-referencing – they seemed unsure about how these factors might be weighed and used together to determine the extent to which the source was reliable. This study illustrates the need for a more comprehensive conceptual understanding of reliability, especially when working with complex sources (e.g., Facebook pages) and complex topics, such as healthcare. The authors outline what this conceptual understanding entails and offer suggestions for promoting discussion among students and teachers in order to help cultivate a conceptual understanding of reliability. Key words: source reliability, reliability of resources Health care is a complex topic and what intensifies this complexity is that an overwhelming amount of information about health care is but a keystroke, mouse click or finger swipe away. So, how do we make decisions about what information sources to engage with to help us understand health care and to take timely, responsive action when it comes to specific health care issues and decisions? We, of course, want and need sources we can trust. We want reliable information to ensure we understand the topic in order to help us make judicious decisions that impact our personal lives as well as to participate in discussions about ways to best develop and deliver health care across society. Discerning the trustworthiness of information, of course, is no straightforward task. In the internet age, we need to contend with the sheer vastness of information available to us (or coming at us) in an ever increasing variety of formats, including YouTube videos, Facebook posts, tweets, emailed links, etc. Our decision making is also shaped by our prior knowledge and experiences, political allegiances, and diverse belief systems. Moreover, information sources come with agendas, reflecting particular purposes and values of the author or agency responsible for creating or disseminating the source. In this article, we draw on an example of what happened when a group of secondary level social studies students in Singapore (14–15 year olds) evaluated the reliability of a Facebook page as part of an activity about Singapore’s health care system. We highlight how the work of these students compelled us to rethink how to best guide students to evaluate the reliability of information sources, which, in turn, led us toward a more comprehensive and nuanced understanding of reliability, especially for working with 21st century sources like Facebook pages and complex topics, such as health care. Toward this end, we outline what a conceptual understanding of reliability might include and we make a case that the teaching of the specific skill, evaluating reliability, be rooted in classroom discussions about what reliability as a concept means and how to leverage this conceptual understanding to evaluate sources. Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 52 Reliability of Information Sources When someone is reliable, s/he is able to be trusted due to past performance. An information source is reliable when we can trust the author(s), agency, or sponsor of the source has taken appropriate measures to ensure the information is accurate and the claims made are well supported with sound, relevant evidence. In this sense, reliability and credibility are companion concepts. We, in fact, use them interchangeably in our work and in this article. Flanagin and Metzger (2008) made a similar case, contending that notions of credibility, reliability, trust, reputation, quality, authority, expertise and competence are closely affiliated. They organize these concepts into two dimensions: trustworthiness (e.g., reputation, reliability, trust) and expertise (quality, accuracy, authority, competence). They also outline three components of credibility that pertain especially to web-based information sources. Source credibility refers to the accuracy, comprehensiveness, professionalism and sponsor credentials of a website. Message credibility requires understanding the ways source structure, language, delivery of information and its currency, use of evidence and citations affect credibility assessment. Media credibility is the relative credibility assigned to the media channels through which messages are sent. One aspect of teaching students to assess the reliability of information sources is to emphasize source attribution – identifying the author of a source, her or his motivations, how the source came into being and the intended audience. Some have argued that this is a core goal of social studies education (e.g., Beck, McKeown, & Worthy, 1995; McKeown & Beck, 1994;VanSledright, 2010; Wineburg, 2001). For Wineburg (1991), students should learn to apply the “sourcing heuristic” by “looking first to the source of the document before reading the body of the text” (p. 77). This involves reading for the subtext of documents to reconstruct authors’ purposes, intentions and goals (Wineburg, 2001). And we do know that students can become more skilled in evaluating information sources with appropriate heuristics. For example, Britt and Aglinskas (2002) used a computer application called Sourcer’s Apprentice to help students effectively learn to use the sourcing heuristic to evaluate sources. Nokes, Dole, and Hacker (2007) also found that explicit teaching of the sourcing heuristic in high school history classes had value in helping students learn and apply the heuristic in their work. Elementary students taught to use criteria to evaluate the trustworthiness of online sources were also found to improve their assessments of source reliability (Baildon & Baildon, 2008). However, other studies highlight the range of challenges students have evaluating source reliability. After teaching his young students to critically analyze source reliability, VanSledright (2002) noted how they jumped from an initial trust of accounts to the view that all sources were inherently unreliable. Much to his dismay, his upper elementary students moved from a naïve trust of sources to an over- generalized suspicion that authors regularly and intentionally distort the truth. Even older students often use superficial or irrelevant criteria to evaluate sources (Braten, Stromso, & Britt, 2009; Brem, Russell, & Weems, 2001). Britt and Aglinskas (2002) found that neither secondary students nor undergraduate students were able to consistently and effectively evaluate the reliability of historical sources. Online information sources present particular challenges for adolescents. In a comprehensive survey- based study, Flanagin and Metzger (2010) found that most children (11–18 years old) take seriously the issue of the credibility of online information, yet many do not employ a rigorous approach to evaluate online information (p. xiii). Their findings also show that children and adults both demonstrate an optimistic bias in their own abilities to discern credible information better than typical or average users. Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 53 Similarly, the work of literacy scholar, Don Leu, and his colleagues have shown, for example, that groups of middle school students (13–14 year olds) were unable to discern an internet hoax about a fictitious endangered species called the Pacific Northwest Tree Octopus (Krane, 2006). Fortunately, there is no shortage of instructional resources to guide educators and parents in helping students become more careful and critical evaluators of source reliability. These include lists of questions and suggestions from college or university libraries (e.g., http://www.lib.umd.edu/ues/guides/evaluating-checklist), guides specific to academic disciplines, such as social studies (History Matters (http://historymatters.gmu.edu/browse/makesense/) and other tools, e.g., Schrock guide for information literacy (http://www.schrockguide.net/information-literacy.html)). Coiro (2005) outlined a process with a set of questions and tips to guide students to more skillfully and successfully answer the question, “How do I know this [information source] is true?” The questions and corresponding tips were: Does this information make sense? Be skeptical and ask around. Where else can I look? Search the internet using keywords in quotation marks, or look in a book. Who created the website and why? Explore the About Us link with a critical eye. Who is the author? Search the internet using the author's name in quotation marks. Who is linking to the site? Type Link: followed by the URL of the website in question into the search box of the Google home page (Coiro, 2005, p. 34). Coiro went on to describe how a student used this question asking process to determine another online hoax site about California’s Velcro crop was invalid. While these resources are useful, there is no evidence to suggest that teachers across the United States are systematically engaging students in this type of work. No state in the United States assesses students’ ability to critically evaluate the reliability of online information (Leu, O’Byrne, Zawilinski, McVerry, & Everett-Cacopardo, 2009) and there is no explicit emphasis on online source evaluation with the Common Core State Standards (Council of Chief State School Officers, 2010). The education context in Singapore, however, is somewhat different. The evaluation of source reliability has long been a staple of the Singapore social studies curriculum. The emphasis on evaluating the reliability of online sources, however, represents a new frontier. Social Studies Education in Singapore At the center of the Singapore social studies curriculum for secondary level students are O-Level examinations (part of the General Certificate of Education, or GCE, requirements in the United Kingdom and many Commonwealth nations). In O-Level History and Social Studies examinations, students are asked to do source work – to perform six source-based thinking skills (make inferences, evaluate reliability, evaluate utility, evaluate claims, compare and contrast sources, and construct explanations) with black and white visual sources (e.g., images, political cartoons) and 50–100 word print sources. The use of online sources for the classroom teaching of these source-based skills has not been a focus. In terms of the social studies curriculum, the following questions, taken from the Upper Secondary Social Studies 3 Express Activity Book for 9th and 10th grade students (Ministry of Education[MOE], 2007), are prompts to help students determine whether a source is reliable: Who produced the source? Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 54 When was it produced? Was the creator of the source an eyewitness? Where did he get his information from? Is there any contradiction within the source? Is the source biased in any way? Is there consistency when cross referred to other sources? Does the source agree with background knowledge? With this set of questions as a guide, teachers help students by using different tools, such as graphic organizers, diagrams or charts that visually present steps or procedures to follow. While students work through this process and consider the questions outlined above, there is no explicit guidance on how a final determination of reliability might be made based on answers to these questions. Methods The guiding question for this qualitative inquiry was: What happened when two secondary level Singapore social studies teachers asked students to evaluate the reliability of a complex digital text, the Facebook page of an opposition party politician, as the students investigated the affordability of health care in Singapore? The students did this work with the Critical Web Reader or CWR (http://cwr.indiana.edu/), a technology tool that allows teachers to create curriculum activities using online information sources, whether it be websites, YouTube videos, blogs, etc. (Baildon & Damico, 2011). The CWR takes any online information source, places it within a frame that teachers can customize to provide guiding questions, models, and suggestions that readers use as they engage with the information source. A writing tool is also embedded within the frame where students document their thinking and answer questions with each online source (i.e., their analyses, interpretations, and questions). The CWR infrastructure saves all teacher and student work to facilitate data collection and management. The two teachers in this study created a CWR activity called: “How Affordable is Health Care in Singapore?” and implemented it with their Secondary 3 students (14–15 year olds). This is an issue- based question because it emphasizes an “authentic and contemporary” policy question (Hess, 2009, p. 41). Thirty seven students worked on the activity on September 15, 2011, but only 15 completed the entire activity primarily due to time limitations. This was the first time the students had studied health care, which is part of the textbook unit on “Understanding Governance” that includes a comparative study of Singapore’s and Britain’s health care systems. Students worked individually on the CWR activity in a computer laboratory during two 60-minute class periods. A learning outcome in the unit is to “evaluate the effectiveness of the policies in meeting the health care needs of the people and nation” (MOE, 2008, p. 10). Our interest here is what happened when students were asked to evaluate the reliability of one source in this activity, a Facebook posting titled “Is Health Care Affordable in Singapore? Here is My Take” (http://www.facebook.com/note.php?note_id=139911682746578) from Nicole Seah, a 26-year old member of the opposition National Solidarity Party (NSP). Description of Web Source Nicole Seah was the youngest person (24 at the time) to run for office in the May 7, 2011 General Election in Singapore. Her party’s goals are to build a multi-party democracy and a “more open, dynamic, Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 55 vibrant and inclusive society through consensus and the democratic process” (“National Solidarity Party (Singapore)”, n.d.). No member of the NSP has won an election to gain a seat in Singapore’s Parliament. Although the 2011 election was heavily contested and political opposition parties garnered more votes than any other time in Singapore’s history, the People’s Action Party (PAP) has dominated Singapore’s government since independence in 1965. Seah’s Facebook post (Seah, 2011) on the affordability of health care in Singapore was dated April 21, 2011. There is a link to her Facebook home page where she is listed as a “public figure” and the About page notes that she is a member of the NSP, an honors graduate of the National University of Singapore, and that she “practices yoga, reads extensively and enjoys good music.” Although Seah did not win a seat in Parliament, she was a popular figure who was able to use social media effectively during her campaign. Although criticized for her lack of political experience, during the campaign Seah capably defended her positions. She also noted her volunteer work and community involvement since secondary school and described her political awakening as happening when she delivered food to poor elderly people whose basic needs were not being met. While at the National University of Singapore, she was managing editor of an independent online journal, the Campus Observer. She continually made a case for representing young people in Singapore to ensure they have a greater stake in society and a voice in government policy decisions. Her Facebook post outlines her position on the affordability of Singapore’s health care. Here is a main claim from the post: Whether health care is affordable really depends on which group of people you are asking. The reality of the situation is that people who are less well-off or people who belong to the lower- income group will have a problem managing health care costs. (Seah, 2011, para. 2) She went on to argue that the government needs to more aggressively step in by “re-prioritizing the allocation of resources” (N. Seah, 2011, para. 10) to ensure health care is affordable for all groups of people in Singapore. She concludes her post by saying “it is really a matter of who you ask and whether there is more that can be done for those who need it badly” (para. 12). Classroom Activity The Singapore students in their classroom were asked to answer four questions to evaluate the reliability of this Facebook post: 1. What is the provenance of the source? (time, place, and person responsible for the source) 2. What is the purpose of the source? 3. Is the content reliable? 4. To what extent is this source reliable? Explain. Of note, the questions did not explicitly ask students to corroborate or cross-reference this source. The students also worked independently to answer these questions. The two teachers believed the students could readily work with this source because the students had some background knowledge about both the content and the provenance of the source (i.e., they knew about Seah and opposition parties). Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 56 So, what happened when the group of 15 students was asked to evaluate the reliability of this Facebook post from a young opposition party politician? Findings When prompted to answer the four questions—1. What is the provenance of the source? 2. What is the purpose of the source? 3. Is the content reliable? 4. To what extent is this source reliable? Explain.—all 15 students deemed the Facebook page unreliable because of its purpose, but there was some overlap as well as differences with how they arrived at this conclusion. (Note: some students’ responses fall into more than one of the categories below.) Five students did not dedicate much time to analyzing source content and deemed the source unreliable because its provenance (the author’s affiliation with an opposition party) suggested “motive” or an “agenda.” These responses tended to be shorter with a more immediate determination of reliability. For example, one student noted the author had a “hidden agenda, which was to gain more votes for her party in the Singapore General Election. . . . Hence this source is unreliable as it has motive.” Another student cited bias as the problem. “The source is biased, in that it was made by a member of the NSP and hence partial to the NSP.” Many (10 of the 15) students provided more extensive content analysis in their responses (e.g., 375–500 words) to support the idea that the source was unreliable because of the author’s purpose. For example, one student combined an analysis of provenance and source content to conclude, Source B is unreliable in term [sic] of its purpose as Nicole Seah is one of the National Solidarity Party (NSP) candidates for the General Election, her aim is to convince the Singaporeans that the Singapore’s health care system is still unaffordable for some of the people. This can be seen from “The reality of the situation is that people who are less well-off or people who belong to the lower-income group will have a problem managing health care costs.” This is so that the Singaporeans will start to support her instead of the current government and believe that she can help to improve the health care system. Interestingly, three students analyzed the source content and found it reliable, but still deemed the source unreliable because of its purpose. Here is one example: This source is reliable in saying that Singaporeans that do not earn much will suffer more in paying for health care in Singapore. This can be seen from, “The reality of the situation is that people who are less well-off or people who belong to the lower-income group will have a problem managing health care costs.” This shows that the most disadvantaged group of people in Singapore is the lower-income group as they would have more issues in struggling with the payment of the health care fees. This source, however, is not reliable because it has a purpose. This source was created to convince Singaporeans that she was concerned for their welfare. This can be seen from, “It is really a matter of who you ask and whether there is more that can be done for those who need it badly.” By saying this, it shows that she is empathizing with the citizens so that they will respect and hold her in high regards. More citizens then will then vote for her. This source was created on Wednesday, April 20, 2011 at 9:13am, Singapore’s General Election 2011 was coming Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 57 up and candidates wanted to get into the good books of Singaporeans so that they can garner more votes and be elected. Therefore by creating this source, it was to help themselves gain seats in the General Elections. Hence, this source is unreliable. Six of the 15 students used cross referencing to help them determine the reliability of the source, even though they were not prompted explicitly to do this. This group of students referred to other sources to both refute and support the source content. They used a Singapore Ministry of Health video to refute Seah’s claims about the affordability of health care (this source provided evidence that health care was affordable) and a blog post called “High Cost of Medical Care in Singapore,” (which was located on a news site called The Online Citizen) to support Seah’s views (since this source similarly argued that health care was not affordable for low income groups). Although these students performed cross referencing, they all concluded that the Facebook source was unreliable because of its purpose. One student even viewed the source content as reliable, cited evidence from the source, and used cross referencing with blog post from The Online Citizen to provide further evidence in support of the claim that Singapore’s health care did not meet the needs of low income groups. However, in her final evaluation, this student deemed the source unreliable because Nicole Seah was an opposition party candidate and wanted votes. In sum, not a single student determined the Facebook page to be reliable as a source of information about the affordability of health care in Singapore. Determining provenance merely pointed to the author having a purpose (to win the election); students analyzed source content but even when students viewed the content to be reliable, their ultimate evaluation settled on problems with the author’s purpose; students cross referenced to other sources but this seemingly did not affect their overall evaluation in any way. Discussion Reflecting on the students’ responses compelled us to try to better understand what they were really asked to do in responding to the four questions (1. What is the provenance of the source? 2. What is the purpose of the source? 3. Is the content reliable? 4. To what extent is this source reliable? Explain). What did they need to know and understand to answer these questions and, ultimately, evaluate the reliability of this source? This pushed us to examine the three core concepts embedded in these questions – provenance, purpose, and content; to ask: what conceptual understanding of each does a reader need to have to evaluate information sources in general and this Facebook post in particular? For starters, we realized that determining the reliability of this source was not necessarily simple and straightforward. Nichole Seah discloses who she is and communicates that she cares about the issue and Singapore. Although young and seemingly inexperienced, she outlines some important issues related to affordable health care in Singapore. Moreover, while she is obviously trying to persuade readers (and is a member of the opposition party), she presents her views in a straightforward, seemingly honest way. While Seah outlines her specific purpose in the title “Here is My Take” (about whether health care is affordable in Singapore), the purposes of social media sites like Facebook can operate at multiple levels of meaning and thus, have different purposes than traditional information sources (Greenhow & Robelia, 2009). Facebook tends to be a form of self-publishing where the purpose is to primarily garner emotional support, maintain and mobilize relationships and enhance self-presentation. Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 58 Seah is explicit in her purpose and respectful to readers. She also presents a reasonable argument, which is consistent with findings based on others’ work, such as Barr’s (2001) study of Singapore’s health care system which found certain procedures and conditions, such as congenital abnormalities, cosmetic surgery, abortion, infertility procedures, mental illness, and treatment for drug addiction were prohibitive for those at lower income levels. Moreover, several studies highlight the success of the Singapore system, yet point out that “Singapore provides a ‘floor’ of basic health care that is accessible to all the people, but advanced and high-tech care is not equitably distributed, nor was it ever intended to be” (Hsiao, 2007). Thus, the claims or arguments Seah makes are not inaccurate. This source, however, is also limited. It does not provide clear and substantive evidence to support her case, and it is up to readers to draw from other sources to corroborate the claims. She also seemingly takes for granted that readers will come to the source with sufficient background knowledge to understand key aspects of the context, which are central to understanding the source (e.g., the relationships between rising defense spending, public infrastructure and health care expenditures). With the complexities of this Facebook page in mind, Table 1 outlines our attempt to outline key procedures and associated conceptual understandings for evaluating the reliability of sources. The left hand column lists the three components of reliability; the right hand column describes what we believe are the core understandings required for each component. When considering source provenance it is important for students to know that all authors are in some ways biased, yet that is no excuse to dismiss all sources as equally biased; authors’ backgrounds, experiences, and willingness to be up front about their interests count as well. When considering purpose, it is important to understand that all sources are written with a purpose, sources vary in how up front these purposes are, and that some purposes are more trustworthy than others. In terms of content, the careful evaluation of claims and evidence and corroboration (cross referencing) is key to gauging the reliability of a source. These three concepts also interact dynamically, so an ultimate decision about the reliability of a source needs to be holistic, a careful assessment of the interdependence of these core concepts. Table 1. Key Procedures and Conceptual Understandings for Evaluating Reliability Procedure Key Conceptual Understandings Evaluate provenance 1. Author background, expertise, experience affect their competence to speak about particular issue (depends on issue they discuss and their experience with it). 2. All authors are biased or have limited views but we must determine if the bias is acceptable or if it should disqualify them. 3. Disclosure which states one’s background, interests and positions is important. If not stated, can do background check. Assess purpose 1. All sources written/created with some purpose in mind. 2. Some purposes explicitly stated while others may be vague, implied, hidden or not easily discerned. 3. Some purposes more trustworthy than others (e.g., inclusive, sincere, respectful, balanced purposes more trustworthy than efforts to mislead, deceive, or solely promote self- Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 59 interest). Analyze content and cross reference 1. Claims and evidence must be evaluated for accuracy. This requires evaluating reasonableness, if content fits with what is already known, what other sources say, etc. 2. Need to check content for errors, bias, and tone (e.g., if emotive, one sided, advocacy focused, etc.) to determine if reliable as a source of information. 3. Need to corroborate information, check to see if it is consistent with other sources or refuted by other sources. Make determination of reliability Each of the above factors must be weighed together to make an overall judgment of reliability because informed conclusions and decisions require reliable information. The students in the Singapore classroom focused exclusively on the author’s purpose and deemed the source unreliable because it had a motive, intention or agenda. We, instead, would want students to demonstrate that they knew that Nicole Seah, like all authors, had a purpose (or multiple purposes) when she created this information source, but that this should not automatically make the source unreliable. We would also want students to understand that authors have different purposes with some being more trustworthy than others. We would want students to note, for example, that Seah had a purpose, but there is nothing necessarily troubling about her agenda; she’s trying to inform a public about problems related to affordable health care and she does this in ways that are honest with examples that seemingly would resonate with Singaporean people. So, while there are limitations of this source (i.e., limited evidence), we have difficulty declaring that, overall, this Facebook post is unequivocally unreliable. Reliability in History Education, Reliability for Citizenship Education In history education, Barton (2005) made a case to consider reliability in terms of the evidentiary potential of a source; for each time we encounter a source to ask questions like, “Is there evidence in this source that I can use to help answer my questions?” Barton argued that use of the sourcing heuristic, which includes identifying the author and her/his purposes (Wineburg, 1991) has resulted in a myth that historians use sourcing to evaluate source bias and reliability. According to Barton, the bias contained in a source may actually be what makes the source reliable because it provides an accurate view of a particular perspective at a particular time or place. Other than for testimonial accounts, reliability rarely needs to be considered because “it is the very fact of the existence of the source that constitutes historical evidence” (Barton, 2005, p. 747). Thus, if a goal is for students to do the work of historians in the ways Barton points to, going through a process of determining the reliability of sources would be mostly unnecessary, or even misguided. We agree that evaluating reliability is not an end in itself and is, in most cases, best viewed and understood as part of an analytical process that emphasizes the extraction of useful, relevant evidence to answer questions. Yet, we find it necessary to make two key distinctions: one pertains to working in the realm of historical inquiry; the other is to distinguish investigations in history to more general purposes and practices of inquiry in social studies, which includes an emphasis on citizenship education, and in education more generally. Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 60 While historians are most interested in working with the evidence, a source provides about the particular historical event or issue they are investigating, at times they still need to determine source provenance and authenticate sources to make sure they are not working with forgeries or counterfeit documents (and therefore faulty evidence). This might be especially the case for older sources. For example, one of our colleagues, an historian who studies China's international relations during the Tang dynasty consistently authenticates sources in his own work (Wang, personal interview, 2012). Chinese and Japanese traders and politicians during this period regularly dealt in the exchange of state letters and insignia for economic and diplomatic purposes and on occasion these were forged to gain court favor and geopolitical advantage in East Asia (Wang, 2005). Although these forged materials do provide evidence about the importance of official documentation in Chinese-Japanese relations, it is also necessary to authenticate sources to determine which Japanese officials, for example, “carried the prestige of the Chinese court” (Wang, 2005, p. 20). Moving outside of historical inquiry (and attempts by middle school or secondary school teachers to help students mirror or approximate the practices of an historian) and into the more general and all purpose roles and responsibilities of being a citizen, (what we see as part of social studies education and education more broadly) we argue it is crucial to have reliable information to reach informed conclusions or to make good decisions about issues, such as affordable health care. As citizens, our first line of defense against making faulty decisions is reliable information. This is highly important because of the misinformation, scams, hoaxes, and unvetted, fake or doctored information sources that circulate in online spaces. Identifying purposes, weighing bias and determining the reliability of information, then, is first order intellectual work to ensure useful, relevant and sound information is being consulted. If our goal is to best understand an issue, we do not want to waste our time with sources that mislead, misinform, or are outright deceptive. Rethinking Source Work in 21st Century Contexts As teachers in Singapore, the United States and around the world include more complex online sources of information, such as Facebook pages, YouTube videos, blogs and websites, as part of classroom instruction, it is essential to help students carefully and critically evaluate the reliability of these sources. But to do this work well requires a conceptual understanding of this core skill. Table 1 is intended to help teachers and students understand the nature of reliable sources as they perform key steps to evaluate source reliability. And surely one way to cultivate this conceptual understanding is to provide students with consistent opportunities to use questions like these in Table 1 to work with sources in a systematic way. However, we think this approach is limited if students primarily engage in this work individually, which was the case for this group of Singapore students. It is important for students to discuss their sense making. Walter Parker’s (2006) description of two different types of classroom discussion, seminar and deliberation, offers a way forward. The purpose of a seminar discussion is for participants “to plumb the world deeply” toward an “enlarged understanding of the text and one another” (2006, p. 12). A deliberation is focused on decision making, as participants identify and evaluate alternatives and choose a course of action. We see attention to both of these discussion types as pivotal next steps. With a seminar discussion, teachers would guide students to delve into the concept of reliability, posing and pursuing questions like: What does reliability mean to us? Is the reliability of some sources more difficult to discern than others? Why might we have different views about the reliability of a source? In what Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 61 ways has the concept of reliability stayed the same or changed over time? An exploration of the affordances and limitations of different reliability frameworks, formulas and assorted checklists could also be included to enrich a classroom inquiry. In terms of deliberation, teachers could narrow the discussion on a more specific issue, to decide, for example, whether or not a particular source is reliable enough to use in a research report. Students would need to ultimately decide whether a source should be deemed appropriate to be included in the report or not. We could, for example, imagine the group of Singapore students having a generative deliberation about the Nicole Seah Facebook post. Conclusion Digital information sources can be complex in several ways: provenance or authorship and sponsorship are sometimes difficult to discern; the purposes of sources can be vague or be intended to deceive; the structure, language features and knowledge demands of sources can make it challenging to evaluate the content’s accuracy; and it is often relatively easy to corroborate the view of one source by locating another source with the same perspective, argument or set of facts. Moreover, there is no shortage of pressing personal and public issues—e.g., related to health care, climate change, water shortages, war, terrorism, etc.—to better understand and address. By including the Facebook page in the classroom activity, the teachers and students in this study began to consider the challenges of critically analyzing and evaluating this type of information source. Our findings indicate the need for conceptual scaffolding to accompany the procedural scaffolding that is often emphasized in Singapore’s social studies classrooms. Both forms of scaffolding with an emphasis on how students can do this work in online environments with complex online sources deserve greater attention. We want to make sure students are not just answering questions about the provenance, purpose and content of information sources or merely following procedural steps; instead we want to see the development of deeper understandings about how each of these interact in ways that shape a final determination of reliability. To help ensure this happens, students need a range of opportunities for dialogue, for seminar and deliberation discussions, to help develop sophisticated understandings about concepts like reliability and to apply these understandings to an ever dizzying array of information sources. References Baildon, R., & Baildon, M. (2008). Guiding independence: Developing a research tool to support student decision-making in selecting sources of information. The Reading Teacher, 61(8), 636–647. Baildon, M., & Damico, J. S. (2011). Social studies as new literacies in a global society: Relational cosmopolitanism in the classroom. New York: Routledge/Taylor & Francis. Barr, M. (2001). Medical savings accounts in Singapore: A critical inquiry. Journal of Health Politics, Policy and Law, 26(4), 709–726. Barton, K. C. (2005). Primary sources in history: Breaking through the myths. Phi Delta Kappan, 86(10), 745–753. Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 62 Beck, I., McKeown, M., & Worthy, J. (1995). Giving text a voice can improve students' understanding. Reading Research Quarterly, 30(2), 220–238. Braten, I., Stromso, H. I., & Britt, M. A. (2009). Trust matters: Examining the role of source evaluation in students’ construction of meaning within and across multiple texts. Reading Research Quarterly, 44(1), 6–28. Brem, S. K., Russell, J., & Weems, L. (2001). Science on the Web: Student evaluations of scientific arguments. Discourse Practices, 32(2–3), 191–213. Britt, M. A., & Anglinskas, C. (2002). Improving students’ ability to identify and use source information. Cognition and Instruction, 20(4), 485–522. Coiro, J. (2005). Making sense of online text. Educational Leadership, 63(2), 30–35. Council of Chief State School Officers. (2010). Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects. Retrieved from http://www.corestandards.org/wp-content/uploads/ELA_Standards.pdf Flanagin, A. J., & Metzger, M. J. (2010). Kids and credibility: An empirical examination of youth, digital media use, and information credibility. Cambridge, MA: The MIT Press. Flanagin, A. J., & Metzger, M. (2008). Digital media and youth: Unparalleled opportunity and unprecedented responsibility. In M. J. Metzger & A. J. Flanagin (Eds.), Digital media, youth, and credibility (pp. 5–28). Cambridge, MA: The MIT Press. Greenhow, C., & Robelia, B. (2009). Old communication, new literacies: Social network sites as social learning resources. Journal of Computer-Mediated Communication, 14, 1130–1161. Hess, D. E. (2009). Controversy in the classroom: The democratic power of discussion. New York and London: Routledge. Hsiao, W. (2007). Talking health care at Harvard: An interview with Professor William Hsiao. SMA News, 39(9), 1–7. Retrieved from http://news.sma.org.sg/3909/Feature.pdf Krane, B. (2006). Researchers find kids need better online academic skills. Advance, 25(12). Retrieved from http://advance.uconn.edu/2006/061113/06111308.htm Leu, D., O’Byrne, I., Zawilinski, L., McVerry, J. G., & Everett-Cacopardo, H. (2009). Comments on Greenhow, Robelia, and Hughes: Expanding the New Literacies Conversation. Educational Researcher, 38(4), 264–269. McKeown, M. G., & Beck, I. L. (1994). Making sense of accounts of history: Why young students don't and how they might. In G. Leinhardt, I. Beck, & C. Stainton (Eds.), Teaching and learning in history (pp. 1–26). Hillsdale, NJ: Erlbaum. Ministry of Education [MOE]. (2007). Upper secondary social studies express 3 activity book. Singapore: Curriculum Planning & Development Division. http://news.sma.org.sg/3909/Feature.pdf http://advance.uconn.edu/2006/061113/06111308.htm Journal of International Social Studies, Vol. 5, No. 1, 2015, 51-63. Corresponding author email: damico@indiana.edu ©2012/2018 International Assembly Journal of International Social Studies Website: http://www.iajiss.org ISSN 2327-3585 P a g e | 63 Ministry of Education (Singapore) (2008). 2192 Combined Humanities GCE Ordinary Level Social Studies Syllabus. Curriculum Planning and Development Division. National Solidarity Party (Singapore). (n.d.) In Wikipedia. Retrieved June 24, 2015) from https://en.wikipedia.org/wiki/National_Solidarity_Party_(Singapore) Nokes, J. D., Dole, J. A., & Hacker, D. J. (2007). Teaching high school students to use heuristics while reading historical texts. Journal of Educational Psychology, 99(3), 492–504. Parker, W. C. (2006). Public discourses in schools: Purposes, problems, possibilities. Educational Researcher, 35(8), 11-18. Seah, N. (2011). (2011, April 21). Is healthcare affordable in Singapore? Here is my take [Facebook note]. Retrieved from https://www.facebook.com/notes/nicole-seah- %E4%BD%98%E9%9B%AA%E7%8E%B2/is-healthcare-affordable-in-singapore-here-is-my- take/139911682746578?comment_id=966524&offset=0&total_comments=108 VanSledright, B. A. (2002). Confronting history's interpretive paradox while teaching fifth graders to investigate the past. American Educational Research Journal, 39(4), 1089–1115. VanSledright, B. (2010). What does it mean to think historically . . . and how do you teach it? In W. C. Parker (Ed.), Social studies today: Research and practice (pp. 113–120). New York: Routledge. Wang, Z. (2005). Ambassadors from the islands of immortals: China-Japan relations in the Han-Tang period. Honolulu, HI: University of Hawai’i Press. Wineburg, S. (1991). Historical problem solving: A study of the cognitive processes used in the evaluation of documentary and pictorial evidence. Journal of Educational Psychology, 83(1), 73– 87. Wineburg, S. (2001). Historical thinking and other unnatural acts: Charting the future of teaching the past. Philadelphia, PA: Temple University Press. Reliability of Information Sources Social Studies Education in Singapore Methods Description of Web Source Classroom Activity Findings Discussion Reliability in History Education, Reliability for Citizenship Education Rethinking Source Work in 21st Century Contexts Conclusion References