Microsoft Word - Dandotkar_final.docx Journal of Effective Teaching in Higher Education, vol. 5, no. 2 https://doi.org/10.36021/jethe.v5i2.323 Knowing, Thinking, and Learning: Fostering Critical Thinking in Undergraduate Psychology Classes Srikanth Dandotkar, University of Southern Indiana, sri.dandotkar@usi.edu Laura Cruz, The Pennsylvania State University M. Anne Britt, Northern Illinois University Abstract. We examined the relationship between the levels of sophistication (high- sophisticated and low-sophisticated) of students’ domain general epistemic beliefs and an important component of students’ critical thinking skills—their ability to evaluate arguments. Participants evaluated arguments and took an epistemic belief survey before recalling arguments in a Surprise Recall task. Our findings suggest that students’ general beliefs about the speed of knowledge acquisition predicted how well they evaluated arguments and their memory for critical argument elements (i.e., claim-predicates). Implications of this connection between argument analysis and epistemic beliefs in the context of improving students’ critical thinking skills are discussed. Keywords: critical thinking; epistemic beliefs; argument evaluation; argumentation; psychology class Introduction Critical thinking remains one of the primary learning outcomes ascribed to higher education; a position the skill has held for decades. As the basic argument goes, the content knowledge students may gain in college is likely to either dissipate or become surpassed in a rapidly evolving post-graduation world. Critical thinking, on the other hand, is frequently framed by both academics and employers as a persistent skill that can be applied across multiple contexts to solve a multitude of future problems (Bellaera, 2021). While the preceding statement about the value of critical thinking is shared by many, the articulation of precisely which skills, dispositions, and other attributes comprise the skill of “critical thinking” is highly contested and the subject of a robust line of scholarly inquiry (Davies, 2015). Similarly, communication scholars have identified a number of frameworks for argument analysis and development that, when provided to students, may influence their beliefs about the complexity of knowledge (Gaipa, 2004; Seiter & Gas, 2007; Walton, 2006; Weston, 2018). The shaky construct validity of the term “critical thinking” has contributed to assessment challenges that have constrained the development of shared, evidence- based pedagogical practices designed to turn college students into proficient critical thinkers (Behar-Horenstein & Niu, 2011). Despite these limitations, some progress has been made in developing consensus (Bellaera, 2021). In 2009, for example, the American Association of Colleges and Universities (AAC&U) issued a cross- institutional rubric for critical thinking as one of sixteen fundamental learning outcomes in higher education. This widely used VALUE rubric identified five essential components of critical thinking, including the evaluation of evidence and Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 124 the analysis of argument(s) and position(s) (AAC&U, 2009). Similarly, in 2021, British scholar Lauren Bellaera interviewed 176 faculty in the humanities and social sciences and determined that analysis and evaluation were consistently identified as prominent components of critical thinking (Bellaera et al., 2021). While the considerable scholarly debates surrounding critical thinking have produced limited consensus, with areas of shared agreement, such as valuing “the skill of recognizing and constructing arguments—i.e., critical thinking as reasoning skills,” and a host of others that are more disputed (Davies, 2015). The present study seeks to gain insight into factors that contribute to students’ abilities to engage in critical thinking, with particular emphasis on the mitigating role of epistemic beliefs. Literature Review As the example in the previous paragraph illustrates, many of the areas of consensus related to critical thinking focus on the construct as a cognitive skill or set of skills. Others have noted that cognitive processes do not occur in vacuum, and successful critical thinkers also possess an identifiable set of dispositions, or frames of mind, about a subject, including attributes such as curiosity, respect for the viewpoints of others, open-mindedness, fairness (ethical), and the desire to engage in critical thinking (Davies, 2015). A 2010 Australian study comparing students (n = 26) and faculty (n = 21) members’ beliefs indicated high levels of agreement on the significance of both skills and dispositions in critical thinking, though there was less agreement on which specific skills and dispositions were the most salient (Bahr, 2010). A review of the broader literature leads to similar conclusions. Many of the previously mentioned studies and practices operate under the assumption that critical thinking is a generalized, even generalizable, set of skills and dispositions, which can be taught both within and independent of the disciplinary context. That said, there remains considerable scholarly debate related to the influence of disciplinary context and the degree to which specific disciplinary domains may produce stronger critical thinkers than others (Moore, 2004; Moore 2011; Wang, 2017). Part of the argument in favor of discipline-specific critical thinking points to differences in epistemology (e.g., theories of knowing) across disciplinary domains, but this distinction is complicated by research on student epistemic beliefs (e.g., ways of knowing). A 2005 study by Schommer-Aikins et al. (2005), for example, found that higher levels of academic performance could be similarly predicted by both domain-general and domain-specific epistemic beliefs. In other words, what students believe about how knowledge is acquired and constructed (i.e., epistemic beliefs) is a significant factor in determining their ability to engage in critical thinking, regardless of discipline. Schommer et al. (1997) further advanced the concept of epistemic beliefs to identify distinct domains (e.g., Structure of knowledge, Speed of Knowledge Acquisition) and levels (sophisticated and unsophisticated). Educational psychologists have recognized that dispositions and epistemic beliefs, while certainly not the same, often have a symbiotic relationship, and that they demonstrate “similar patterns of relationships to other constructs” (Kardash & Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 125 Sinatra, 2003; Schommer-Aikins & Easter, 2006). Those “other constructs” include critical thinking. Several seminal studies established an early link between sophisticated, or advanced, epistemic beliefs and success with critical thinking (Kurfiss, 1988), though the exact nature of that link remains disputed across disciplinary contexts (Jones & Merritt, 1999). It even seems likely that these linkages persist across cultural contexts which affect epistemic beliefs. In 2011, for example, two linked experiments affirmed that both epistemic belief and cognitive disposition strongly influenced the ability of Chinese students (N = 111 and 138, respectively) to engage broadly in critical thinking (Chan et al., 2011). The first study established a base line for this population using pre-existing surveys, but for the second study, the researchers used an argument evaluation task, in this case, participants were given a passage describing a controversial subject, as a proxy for critical thinking. Psychologists have studied this skill independently from critical thinking, noting that a student’s ability to analyze an argument may be sensitive to differences in reading ability, vocabulary, and a host of other factors (Britt et al., 2016). To date, however, there has been comparatively little research that has explored the relationship between students’ epistemic beliefs and argument evaluation, irrespective of whether the latter stands as a proxy for critical thinking. Given that the link between critical thinking and epistemic belief has been well-established in prior literature, there may be a similar link between epistemic belief and argument evaluation. If so, then the existence of such a link may provide deeper insights into how instructors can strengthen the critical thinking of their students. The present study was designed to test this hypothesis. Before providing the details of the current study, we would like to define important constructs specific to our study, namely, epistemic beliefs and argument evaluation. Epistemic beliefs refer to one’s beliefs about knowledge (Hofer & Pintrich, 1997; Schommer, 1990) and knowing (Schommer, 1990). While epistemic beliefs could be about any topic (e.g., climate change) or domain (e.g., science), we are particularly interested in students’ domain-general epistemic beliefs as conceptualized by Schommer (1990). According to Schommer (1990), epistemic belief is a system of relatively independent beliefs (of an individual) pertinent to knowledge and how it is acquired. Schommer has identified five such independent beliefs (listed in Table 1) that comprise the epistemic belief system, including one’s knowledge and knowing- specific beliefs about Speed (i.e., how quickly or slowly one acquires knowledge), Structure (i.e., how complex, or simple is the structure of knowledge), Construction and Modification (i.e., how one creates new knowledge or modifies the existing one), Successful Student (what makes one successful in acquiring knowledge and if such characteristics can be acquired or not), and Objectivity (whether the truth value of the knowledge is certain or probable). Students’ beliefs specific to each of these components, according to Schommer, exist on a continuum of sophistication with unsophisticated views on one end and sophisticated views on the other. Table 1 shows what constitutes these extreme views specific to five independent beliefs as conceptualized by Schommer (see Table 1, which is copied from Dandotkar et al., 2022). Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 126 Table 1 General Epistemic Beliefs on a Continuum of Sophistication Beliefs Unsophisticated View Sophisticated View B1. Speed of Knowledge- acquisition-KSP Quick learning Slow learning B2. Structure of Knowledge-KST Simple knowledge Complex-knowledge B3. Knowledge Construction & Modification-KCM Passive-learning Active learning by questioning B4. Meaning of Successful Students-SS Innate Fixed ability Acquired Incremental ability B5. Obtaining Objective Truth-OT Certainty-knowledge Probabilistic knowledge Note. Based on Schommer (1990) as presented in Dandotkar et al. (2022). Schommer (1990) has developed a questionnaire, the Epistemic Belief Survey, to measure individuals’ levels of sophistication pertinent to each of these five epistemic belief components. In our study, we have utilized a condensed version of this Epistemic Belief Survey (Wood & Kardash, 2002), the details of which are provided in the Method section. In this study, we have conceptualized the ability of students to engage in Argument Evaluation as a proxy for Critical Thinking skill (Bellaera et al., 2021). Argument evaluation is an individual’s ability to evaluate simple two-clause arguments (see Arguments 1 and 2 below labeled as A1 and A2). Simple arguments (e.g., Arguments 1 and 2) have a claim (i.e., banks shouldn’t charge ATM fees) and at least a reason (i.e., because the fees make their customers unhappy in A1) that is relevant and minimally sufficient in supporting the claim for the argument to be acceptable or warranted (Johnson & Blair, 1977; Toulmin, 1958; Voss & Means, 1991). Whether or not a reason is relevant to the claim depends on how the main verb or verb-phrase (i.e., predicate) of the claim (e.g., “shouldn’t charge ATM fees” in A1 & A2) is related to the reason. The predicate for A1 and A2 is the same, “shouldn’t charge ATM fees,” and it is related to the reason in A1 (because the fees make their customers unhappy) but not the one in A2 (because banks are financial institutions). Therefore, A1 is a minimally acceptable argument whereas A2 is not. A1. Banks shouldn’t charge ATM fees because the fees make their customers unhappy. A2. Banks shouldn’t charge ATM fees because banks are financial institutions. Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 127 Evaluating arguments such as A1 and A2 requires students to examine whether the reasons are relevant and minimally sufficient to support the claim (Angell, 1964; Johnson & Blair, 1977; Freeman, 1991; Salmon, 1984; Toulmin, 1958; Voss & Means, 1991). This ability is contingent on students’ memory for the precise nature of the predicate (e.g., “shouldn’t charge ATM fees” in A1 & A2). In fact, research has found that students who remember the predicates of argument claims also evaluate arguments better than those whose memory for the predicates is poor. This was found both when students recalled the argument immediately after they evaluated it (Britt et al., 2007) and after a delay (Dandotkar et al., 2016). Our Study The present study examined the relationship between students’ epistemic beliefs and their ability to evaluate simple arguments. For the study, participants completed three sequential tasks: an on-line argument evaluation task (i.e., flawed judgment task), in which students identified flawed arguments; an electronic survey related to their epistemic beliefs; and a surprise recall task, where participants recalled the arguments they had evaluated earlier. The students’ scores on the first task (argument evaluation as measured by the flawed judgement task) served as the dependent measure while their scores on the third task (surprise recall task of the argument predicates) served as a covariate. The level of participants’ epistemic sophistication, drawn from the second task (Epistemic Belief Survey), served as the critical independent variable. Of the five epistemic dimensions that the epistemic belief survey is designed to capture, some dimensions are considered to be knowledge-related and some knowing-related (Hofer & Pintrich, 1997; Schommer et al., 1997). For instance, epistemic dimension pertinent to the speed of knowledge acquisition-KSP can be viewed as knowing-related while the dimension specific to the structure of knowledge-KST can be viewed as knowledge-related. As per the Knowledge-only hypothesis, we predicted that students with high-sophisticated views about the structure of knowledge-KST beliefs would evaluate arguments more accurately than students with low-sophisticated structure of knowledge beliefs. As per the Knowing- only hypothesis, on the other hand, we predicted that students with high- sophisticated views about the speed of knowledge acquisition-KSP beliefs would evaluate arguments more accurately than those with low-sophisticated beliefs. These hypotheses are based on the previous findings pertinent to KSP and KST beliefs (Schommer, 1993; Schommer et al., 1997). Methods Participants After receiving approval from the institutional review board, 127 undergraduate students were recruited from lower (Intro: n = 75) and upper level (Cognition: n = 44) psychology classes at a Midwestern university. Students received course credit for their voluntary participation. As per the IRB rules, students were not penalized for their non-participation or for refraining from continuing their participation. Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 128 Because the data were collected online, students were given clear instructions about exiting the study. Data from 12 participants were dropped because of incomplete responses, making data from 115 participants included in the final analysis. As requested in the approved IRB application, students’ demographic information was only used to present aggregates, without identifying individuals. Participants included 71 females and 44 males with an average age of 20.14 years. Students’ year in school was as follows: 51 first-year, 24 second-year, 30 third- year, and 10 fourth-year. Tasks and Instruments Flawed Judgment Task (FJ Task). This task involved students evaluating simple two-clause arguments (e.g., A1 and A2) (Britt et al., 2007). To measure students’ argument evaluation skills, we deployed a widely used argumentation task, the flawed judgment task (Britt et al., 2007; Britt et al., 2016; Dandotkar et al., 2016; Larson et al., 2009). In this task, participants read 36 arguments presented one-at- a-time on the computer and rated whether each argument was flawed (F) or acceptable (OK) based only on its structure. Students’ performance on the flawed- judgment task, as measured by argument evaluation scores, served as the dependent measure. Surprise Recall task. To control for memory for claim-predicates specific to the arguments that students had evaluated as part of the flawed-judgment task, a surprise recall task was devised where students—after taking the epistemic belief survey that we will elucidate next—were asked to recall the arguments they had evaluated before based on a topic prompt (e.g., “Banks” for A1 and A2). Epistemic Belief Survey (EBS). As stated earlier, we measured participants’ level of epistemic sophistication pertinent to Schommer’s (1990) five components with the help of a condensed version of Schommer’s (1990) Epistemic Belief Survey (Wood & Kardash, 2002). Please see Dandotkar et al. (2022) for a basic but elaborative account of the survey, how it is measured, and how a dimension-reducing technique (i.e., Principal Component Analysis) is used to identify items from the survey that sufficiently measure the epistemic dimensions unique to the sample. The condensed version of the epistemic belief survey itself is widely viewed as reliable, and it has consistently demonstrated sound psychometric properties (Wood & Kardash, 2002). The 38 statements about knowledge and its acquisition (e.g., “You can believe most things you read”) in the survey that students read had agreement rating options. Students rated their agreement with each statement on a 5-point Likert-type scale (1: strongly disagree; 5: strongly agree). These survey items captured students’ beliefs about knowledge and knowing along five dimensions listed in Table 1. A complete list of the survey items is presented in Appendix A. Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 129 Procedure The study was conducted online using the Qualtrics platform. Participants received a link when they signed up for the study (either through SONA-platform or from their instructors) and performed the following tasks in that order after they voluntarily consented using the online consent form that they saw first when they clicked on the study link. Participants first read the 36 arguments (as a part of the Flawed Judgment Task) one-at-a-time on the computer and rated whether each argument was flawed (F) or acceptable (OK) based only on its structure and not on what they believed about it. Then, participants completed the domain-general epistemic belief survey. As stated earlier, participants indicated the extent to which they agreed with each item on a 5-point Likert-scale (1: strongly disagree; 5: strongly agree). Finally, participants received cues from the first task (Flawed judgment task) and were asked to recall the arguments they had evaluated earlier as a part of the surprise task; they were not informed about the need to remember the arguments ahead of time. Participants were instructed to recall the arguments as completely and accurately as possible. Furthermore, participants were asked not to leave any item blank, essentially forcing them to make the best possible guess. Measures, Preliminary Data Analyses, and Design Dependent Measure. Students’ performance on the Flawed Judgment Task was used to compute this measure. For each participant and condition, average proportion of accurately evaluated arguments was computed, which were arcsine transformed to get participants’ flawed-judgment scores (Kirk, 1982) that served as the critical dependent measure. Covariate Measure. Participants’ responses from the surprise recall task were used to calculate this measure. Towards the end of the study, participants were prompted with key words from each argument (e.g., “banks” for A1 and A2) that they had evaluated in the flawed judgment task and were asked to recall the arguments in as close to the exact wording as possible. Participants’ responses were coded for accuracy of claim-predicates. Only an exact matching predicate response (e.g., “should be abolished”) or its synonym (e.g., “must be abolished” or “should be removed” or “must be removed”) was coded as accurate, and the rest were coded as inaccurate. For each participant and condition, proportions of accurately recalled predicates were computed and were arcsine transformed (Kirk, 1982) to obtain each participant’s predicate-recall score, which served as a covariate in the analyses reported in the primary findings part of the results section. It may appear as if the recall measure used does not directly address the relationship between epistemic beliefs and argument evaluation. It is important to note that the relationship between memory for claim-predicates and an individual’s argument evaluation ability has been established (Britt et al., 2007; Dandotkar et al., 2016). The current study attempts to control the mediating role that participants’ memory for claim-predicates plays in how they evaluate arguments. Accordingly, participants’ performance specific to memory for claim-predicates, Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 130 which is elucidated in the methods section, is used as a covariate. Using participants’ memory performance pertinent to claim-predicates allows us to isolate its effects and test the unique effect that epistemic belief has on a participant’s argument evaluation performance, over and above the established effect of memory for claim-predicates on argument evaluation. Independent Measure. Students’ responses to items on the Epistemic Belief Survey were utilized to identify participants as high or low sophisticated students pertinent to each of the identified epistemic dimensions. In other words, each participant was identified either as having a high-sophisticated or low-sophisticated view related to each epistemic dimension. Although the survey itself was meant to capture five epistemic dimensions (i.e., Structure of Knowledge-KST, Speed of Knowledge Acquisition-KSP, Knowledge Construction and Modification-KCM, Meaning of Successful Students-SS, & Attainability of Objective Truth-OT), preliminary analysis usually yields fewer dimensions than the original number (i.e., five). This is because we used stringent measures to validate the items in the survey by conducting a preliminary Principal Component Analysis (PCA). Using such dimension reducing analysis as PCA allows researchers to systematically identify the survey items that uniquely measure a particular epistemic dimension. Further, PCA analysis allows researchers to weed out the items that do not uniquely contribute to measuring an epistemic dimension. Consequently, only those epistemic dimensions that are unique to the sample are captured by the survey. In other words, PCA usually results in capturing fewer than expected epistemic dimensions, which makes the selected dimensions (because of the selected items) uniquely reliable and valid to the study sample. Details of this preliminary data analysis are presented next. For each of the identified epistemic dimensions, participants would be grouped (or identified) into sophisticated or unsophisticated believers using a tertiary split. Only those participants’ data for a given epistemic dimension that fell in the upper and lower one-third of the epistemic sophistication score would be included in the final analysis (reported in the results section). Participants’ level of sophistication (high or low) specific to each of the identified epistemic-dimensions would serve as the between-subjects independent measure. Preliminary Data Analysis. The epistemic belief survey had 38 statements about knowledge and knowing that students rated their agreement with on a 5-point Likert scale. Out of these 38 items, we removed items that were unrelated (<1.5) or negatively related to other items in a given dimension, as recommended by Ferguson et al. (2013). This resulted in 20 remaining items. An initial principal component analysis with oblique rotation yielded three factors—Speed of Knowledge Acquisition-KSP, Structure of Knowledge-KST, and Knowledge Construction and Modification-KCM—that met the Kaisser-Guttman retention criteria of eigenvalues greater than unity. Further, we removed one item because it loaded on two factors significantly. The final 12 items loaded on to three factors (Speed of Knowledge Acquisition-KSP, Structure of Knowledge-KST, & Knowledge Construction & Modification-KCM) which met the Kaisser-Guttman retention criteria of eigenvalues greater than unity and explained 48.49% of sample-variation. Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 131 Because the PCA yielded only three epistemic dimensions that are unique to our sample, the current study is restricted to examining the relationship between students’ domain-general epistemic beliefs pertinent only to these three epistemic dimensions (i.e., Speed of Knowledge Acquisition-KSP, Structure of Knowledge- KST, & Knowledge Construction & Modification-KCM) with students’ critical thinking skills as measured by the Flawed-Judgment Task. Design. We adopted a between-participants design with the level of epistemic sophistication (high-sophisticated or low-sophisticated) as the between-subjects factor and predicate-recall score as the between-subjects covariate. Results Primary Findings Three ANCOVAs—one each for the Speed of Knowledge Acquisition-KSP, Structure of Knowledge-KST, & Knowledge Construction and Modification-KCM—were conducted on participants’ flawed-judgment scores with Epistemic-sophistication level (high or low) as a between-subject factor and predicate-recall score as a covariate. For each dimension, high and low scored participants were identified based on a tertiary split. There was a main effect of Speed of Knowledge Acquisition-KSP, F(1,77) = 5.9, p = .017, ƞp2 = .07. No other findings were significant (All Fs<2). Table 2 shows participants’ argument evaluation scores for each of the three identified dimensions as a function of participants’ level of epistemic sophistication (high-sophisticated or low-sophisticated). These results suggest that students who had high-sophisticated views about the speed of knowledge acquisition-KSP evaluated arguments significantly better than those who had low-sophisticated views about it. This effect was found after controlling for students’ memory for claim-predicates. Table 2 Proportion of Accurately Evaluated Arguments (Arscined) with Standard Errors as a Function of Epistemic Belief Dimension and its level (Low or High) Level of Epistemic Sophistication Epistemic Belief Dimension Speed of Knowledge Acquisition Structure of Knowledge Knowledge Construction & Modification High-sophisticated .77 (.02) .75 (.03) .71 (.02) Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 132 Low-sophisticated) .66 (.02) .70 (.02) .73 (.03) Additional Findings To explore which epistemic dimension predicted students’ memory for claim- predicates, we ran three separate independent-samples t-tests on participants’ predicate-recall scores, one for each of the identified epistemic dimensions (i.e., Speed of Knowledge Acquisition-KSP, Structure of Knowledge-KST, and Knowledge Construction and Modification-KCM). The effect of epistemic sophistication on participants’ memory for claim-predicates was found for the Speed of Knowledge Acquisition-KSP dimension alone, t(78) = 2.78, p = .007. Students who had high- sophisticated views about the speed of knowledge acquisition also recalled claim- predicates more often than those who had low-sophisticated views. This finding is important because it corroborates the importance of students’ views about Speed of Knowledge Acquisition. Students’ views about the speed of knowledge acquisition- KSP not only predicted students’ argument-evaluation performance (as reported in the main results), but it also significantly predicted students’ memory for claim- predicates (as reported here). This shows the potentially important role students’ beliefs about speed of knowledge acquisition-KSP plays in determining students’ critical thinking skills. Epistemic sophistication was not significant in predicting participants’ memory for claim-predicate when it comes to the Structure of Knowledge-KST or Knowledge Construction and Modification-KCM (t<2) dimensions. Table 3 shows average predicate-recall scores as a function of epistemic sophistication related to three epistemic dimensions. Table 3 Predicate Recall Score (Arscined) with Standard Errors as a Function of Epistemic Belief Dimension and Epistemic Sophistication Level of Epistemic Sophistication Epistemic Belief Dimension Speed of Knowledge Acquisition Structure of Knowledge Knowledge Construction & Modification High-Sophisticated .82 (.02) .84 (.04) .78 (.03) Low-Sophisticated .68 (.05) .72 (.04) .83 (.03) Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 133 Discussion In this study, we examined the effects of domain-general epistemic beliefs—Speed of knowledge acquisition-KSP, Knowledge Construction and Modification-KCM, and Structure of Knowledge-KST—on students’ evaluation of informal arguments. Knowledge-only hypothesis predicted that students with a sophisticated belief about the Structure of Knowledge-KST as complex would also evaluate arguments more accurately than those held the more basic belief that it is a simple list of facts. In contrast, the knowing-only hypothesis predicted that students with complex belief about the Speed of Knowledge Acquisition-KSP as a slow, time-consuming process would evaluate arguments more accurately than those who held a simpler belief that acquiring knowledge is a quick process. The current study findings support the Knowing-only hypothesis. As predicted, students with complex views about speed of knowledge acquisition-KSP as a slow and time-consuming process also evaluated arguments more accurately than their counterparts who held simpler views. This was found after controlling for participants’ predicate recall skill. On the other hand, the Knowledge-only hypothesis was not supported. There was no significant difference in the first task performance between students with complex and simpler views about the structure of knowledge. Even though no specific predictions were made pertinent to Construction and Modification of Knowledge-KCM beliefs, we examined its effect and found that students with simpler and complex views about Construction and Modification of Knowledge-KCM did not differ in their argument-evaluation performance. The current study findings extend previous findings (Schommer-Aikins et al., 2005; Schommer et al., 1997) about the importance of the Speed of Knowledge Acquisition-KSP beliefs in academic performance. Schommer et al. (1997), for instance, found that the GPAs of students with complex beliefs about the Speed of Knowledge Acquisition-KSP was higher than those who had simpler views. The current study found that having complex views about the Speed of Knowledge Acquisition-KSP also facilitates students’ evaluation of arguments. In fact, these findings confirm the notion that individuals’ beliefs about knowing play an important role in argument evaluative tasks and, by extension, critical thinking. Implications for teaching and learning practice The present study does not assess the impact of a particular teaching intervention. In other words, we do not seek to answer a “what works?” question from Pat Hutchings’s well-known taxonomy of research questions in the Scholarship of Teaching and Learning (2000). Rather, we ask a “what is” question, intended to shed insight into a known bottleneck in student learning, in this case the development of critical thinking skills, so that future researchers and practitioners may develop effective interventions to address it. Our findings suggest that epistemic beliefs, especially about the complexity of knowledge and the speed by which it is acquired, may have a strong influence on how students engage in key parts of critical thinking, especially argument analysis. Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 134 Given the insights gained from this study, instructors may wish to pay increased attention to epistemic beliefs as a critical component of how their students engage (or do not engage) in critical thinking. There is some evidence that relatively short- term interventions, such as directed reflective writing, may bolster the sophistication of students’ epistemic beliefs, at least in the social sciences (Dandotkar et al., 2022). Further, emerging research on undergraduate research practices suggests that participation in the creation of knowledge through mentored research, may also positively influence, and be influenced by, epistemic beliefs, especially in a STEM context (Lopatto et al., 2022; Reed, 2022). While comparatively less work has been conducted on epistemic beliefs in other domains, ongoing psychology research suggests that epistemic beliefs may be sensitive to both disciplinary and broader, culturally-generated beliefs about knowledge (Merk et al., 2018). In other words, instructors will need to recognize that students will hold a variety of epistemic beliefs, which necessitates teaching strategies that are inclusive of this range. From a transdisciplinary perspective, scholars posit epistemic beliefs as a companion to the larger process of metacognitive awareness, or how students navigate the pathway towards becoming self-directed learners (Mason & Bromme, 2010; Muis, 2007) and for which a wide range of validated interventions have been identified. These may include practices such as exam or assignment wrappers (e.g., structured reflections on performance), concept mapping as a study strategy (e.g., connecting course concepts visually), or teach-backs (e.g., paired discussions of how problems are solved). These practices may influence beliefs about how knowledge is acquired, perhaps especially if students are prompted to make these connections (Ku & Ho, 2010). The effects of those beliefs may not be limited to the students. A recent line of inquiry focuses on the epistemic beliefs of instructors, for example, and how these beliefs may influence how their students learn skills such as critical thinking (Fuesting et al., 2019; Maggioni & Parkinson, 2008; Muenks et al., 2021). These insights suggest that instructors may wish to engage in reflective and reflexive practice regarding their own epistemic beliefs and the influence these may have on the learning that takes place in their classrooms. It is possible that short-term interventions, whether a single assignment or even a single semester, may be insufficient to change some of the fundamental beliefs about knowledge held by students. Scholars of critical thinking have suggested that the associated skills and dispositions take multiple iterations of practice, over considerable periods of time, to promote measurable changes in a student's abilities. Epistemic beliefs may take even longer to develop, which means that potential interventions would not be undertaken at the level of the single classroom but across the curriculum, perhaps even extending to encompass primary and secondary instruction, in one direction, and graduate education, the other. This longer time period may be necessary to hone the integration of an increasing number of components that scholars keep identifying as integral to the critical Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 135 thinking process. Research on critical thinking has suggested that the process includes first skills, then dispositions, and finally actions. This study suggests that there may even be a fourth component to take into consideration i.e., epistemic beliefs. If a student believes that you can learn everything there is to learn about a subject in a single setting, that student will not see the value in taking the time to learn critical thinking skills. If a student believes in singular truths, that student will not see the value in being open to multiple points of view, a key disposition in the critical thinking process. Indeed, if a reader is not willing to believe that the acquisition of learning is a super-complex process to be illuminated rather than solved, then that reader may not be willing to consider the multitude of ways in which what we know, as both learners and teachers, how we think, and why we learn are intertwined. Conflicts of Interest The authors declare that there are no conflicts of interest regarding the publication of this article. References Angell, R. B. (1964). Reasoning and logic. Appleton-Century-Crofts. Association of American Colleges and Universities (AAC&U). (2009). Critical Thinking VALUE rubric. https://www.aacu.org/initiatives/value- initiative/value-rubrics/value-rubrics-critical-thinking Bahr, N. (2010). Thinking critically about critical thinking in higher education. International Journal for the Scholarship of Teaching and Learning, 4(2). https://doi.org/10.20429/ijsotl.2010.040209 Behar-Horenstein, L. S., & Niu, L. (2011). Teaching critical thinking skills in higher education: A review of the literature. Journal of College Teaching & Learning, 8(2). https://doi.org/10.19030/tlc.v8i2.3554 Bellaera, L. (2021, June 18). Making connections that matter. Liberal Education. https://www.aacu.org/liberaleducation/articles/making-connections-that- matter-critical-thinking-in-theory-and-practice Bellaera, L., Weinstein-Jones, Y., Ilie, S., & Baker, S. T. (2021). Critical thinking in practice: The priorities and practices of instructors teaching in higher education. Thinking Skills and Creativity, 41, 100856. https://doi.org/10.1016/j.tsc.2021.100856 Britt, M. A., Kopp, K. J., Durik, A. M., Blaum, D., & Hastings, P. (2016). Identifying general cognitive abilities involved in argument comprehension and evaluation. Zeitschrift für Pädagogische Psychologie, 30(2–3), 79–95. https://doi.org/10.1024/1010-0652/a000173 Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 136 Britt, M. A., Kurby, C. A., Dandotkar, S., & Wolfe, C. R. (2007). I agreed with what? Memory for simple argument claims. Discourse Processes, 45(1), 52–84. https://doi.org/10.1080/01638530701739207 Chan, N. M., Ho, I. T., & Ku, K. Y. (2011). Epistemic beliefs and critical thinking of Chinese students. Learning and Individual Differences, 21(1), 67–77. https://doi.org/10.1016/j.lindif.2010.11.001 Dandotkar, S., Cruz, L. E., Stowell, J. R., & Britt, M. A. (2022). Knowing about knowing: Assessing how reflective writing influences undergraduate students’ epistemic beliefs. Journal of Effective Teaching in Higher Education, 5(1), 1– 17. https://doi.org/10.36021/jethe.v5i1.251 Dandotkar, S., Magliano, J. P., & Britt, M. A. (2016). Effect logical relatedness and semantic overlap on argument evaluation. Discourse Processes, 53(7), 581– 602. https://doi.org/10.1080/0163853X.2015.1087295 Davies, M. (2015). A model of critical thinking in higher education. In Paulsen, M. (Eds.), Higher education: Handbook of theory and research ( Vol. 30). Springer. https://doi.org/10.1007/978-3-319-12835-1_2 Ferguson, L. E., Bråten, I., Strømsø, H. I., & Anmarkrud, Ø. (2013). Epistemic beliefs and comprehension in the context of reading multiple documents: Examining the role of conflict. International Journal of Educational Research, 62, 100–114. https://doi.org/10.1016/j.ijer.2013.07.001 Freeman, J. B. (1991). Dialectics and the macrostructure of arguments: A theory of argument structure. Foris. Fuesting, M. A., Diekman, A. B., Boucher, K. L., Murphy, M. C., Manson, D. L., & Safer, B. L. (2019). Growing STEM: Perceived faculty mindset as an indicator of communal affordances in STEM. Journal of Personality and Social Psychology, 117(2), 260–281. https://doi.org/10.1037/pspa0000154 Gaipa, M. (2004). Breaking into the conversation: How students can acquire authority for their writing. Pedagogy, 4(3), 419–437. Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemic theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67(1), 88–140. https://doi.org/10.3102/00346543067001088 Hutchings, P. (2000). Approaching the scholarship of teaching and learning. In Huber and Hutchings, ed. Opening lines: Approaches to the scholarship of teaching and learning, 1–10. Carnegie Foundation for the Advancement of Learning. https://doi.org/10.3102/00346543067001088 Johnson, R. H., & Blair, J. A. (1977). Logical self-defense. McGraw-Hill Ryerson. Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 137 Jones, P. C., & Merritt, J. Q. (1999). Critical thinking and interdisciplinarity in environmental higher education: The case for epistemological and values awareness. Journal of Geography in Higher Education, 23(3), 349–357. https://doi.org/10.1080/03098269985290 Kardash, C. M., & Sinatra, G. M. (2003). Epistemological beliefs and dispositions: Are we measuring the same construct? Poster paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. Kirk, R. (1982). Experimental design: Procedures for the behavioral sciences (2nd ed). Brooks/Cole. Ku, K. Y., & Ho, I. T. (2010). Metacognitive strategies that enhance critical thinking. Metacognition and Learning, 5(3), 251–267. https://doi.org/10.1007/s11409-010-9060-6 Kurfiss, J. G. (1988). Critical thinking: theory, research, practice, and possibilities. ASHE-ERIC Higher Education Report No. 2. ASHE-ERIC Higher Education Reports, The George Washington University, One Dupont Circle, Suite 630, Dept. RC, Washington, DC 20036–1183. Larson, A. A., Britt, M. A., & Kurby, C. A. (2009). Improving students’ evaluation of informal arguments. The Journal of Experimental Education, 77(4), 339–366. https://doi.org/10.3200/JEXE.77.4.339-366 Lopatto, D., Rosenwald, A. G., Burgess, R. C., Silver Key, C., Van Stry, M., Wawersik, M., ... & Reed, L. K. (2022). Student attitudes contribute to the effectiveness of a genomics CURE. Journal of Microbiology & Biology Education, e00208–21. https://doi.org/10.1128/jmbe.00208-21 Maggioni, L., & Parkinson, M. M. (2008). The role of teacher epistemic cognition, epistemic beliefs, and calibration in instruction. Educational Psychology Review, 20(4), 445–461. https://doi.org/10.1007/s10648-008-9081-8 Mason, L., & Bromme, R. (2010). Situating and relating epistemological beliefs into metacognition: Studies on beliefs about knowledge and knowing. Metacognition and Learning, 5(1), 1–6. https://doi.org/10.1007/s11409-009- 9050-8 Merk, S., Rosman, T., Muis, K. R., Kelava, A., & Bohl, T. (2018). Topic specific epistemic beliefs: Extending the theory of integrated domains in personal epistemology. Learning and Instruction, 56, 84–97. https://doi.org/10.1016/j.learninstruc.2018.04.008 Moore, T. (2004). The critical thinking debate: How general are general thinking skills? Higher Education Research & Development, 23(1), 3–18. https://doi.org/10.1080/0729436032000168469 Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 138 Moore, T. J. (2011). Critical thinking and disciplinary thinking: A continuing debate. Higher Education Research & Development, 30(3), 261–274. https://doi.org/10.1080/07294360.2010.501328 Muenks, K., Yan, V. X., Woodward, N. R., & Frey, S. E. (2021). Elaborative learning practices are associated with perceived faculty growth mindset in undergraduate science classrooms. Learning and Individual Differences, 92, 102088. https://doi.org/10.1016/j.lindif.2021.102088 Muis, K. R. (2007). The role of epistemic beliefs in self-regulated learning. Educational Psychologist, 42(3), 173–190. https://doi.org/10.1080/00461520701416306 Reed, L. K. (2022). Student attitudes contribute to the effectiveness of a genomics CURE. Journal of Microbiology & Biology Education, e00208–21. https://doi.org/10.1128/jmbe.00208-21 Salmon, M. H. (1984). Logic and critical thinking. Harcourt Brace Jovanovich. Schommer-Aikins, M., Duell, O. K., & Hutter, R. (2005). Epistemological beliefs, mathematical problem-solving beliefs, and academic performance of medical school students. The Elementary School Journal, 105 (3), 89–304. https://doi.org/10.1086/428745 Schommer-Aikins, M., & Easter, M. (2006). Ways of knowing and epistemological beliefs: Combined effect on academic performance. Educational Psychology, 26(3), 411–423. https://doi.org/10.1080/01443410500341304 Schommer, M. (1990). Effects of beliefs about the nature of knowledge on comprehension. Journal of Educational Psychology, 82(3), 498–504. https://doi.org/10.1037/0022-0663.82.3.498 Schommer, M. (1994). An emerging conceptualization of epistemological beliefs and their role in learning. In R. Garner & P. A. Alexander (Eds.), Beliefs about text and instruction with text (pp. 25–40). Lawrence Erlbaum Associates. Schommer, M., Calvert, C., Gariglietti, G., & Bajaj, A. (1997). The development of epistemological beliefs among secondary students: A longitudinal study. Journal of Educational Psychology, 89(1), 37. https://doi.org/10.1037/0022-0663.89.1.37 Seiter, J. S., & Gass, R. H. (2007). Teaching students how to analyze and adapt to audiences. Communication Teacher, 21(2), 45–48. https://doi.org/10.1080/17404620701529456 Toulmin, S. E. (1958). The uses of argument. Cambridge University Press. Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 139 Voss, J. F., & Means, M. L. (1991). Learning to reason via instruction in argumentation. Learning and Instruction, 1, 337–350. https://doi.org/10.1016/0959-4752(91)90013-X Walton, D. (2006). Examination dialogue: An argumentation framework for critically questioning an expert opinion. Journal of Pragmatics, 38(5), 745–777. https://doi.org/10.1016/j.pragma.2005.01.016 Wang, S. (2017). An exploration into research on critical thinking and its cultivation: An overview. Theory & Practice in Language Studies, 7(12), 1266–1280. http://dx.doi.org/10.17507/tpls.0712.14 Weston, A. (2018). A rulebook for arguments. Hackett Publishing. Wood, P. K., & Kardash, C. A. (2002). Critical elements in the design and analysis of studies of epistemology. In B. K. Hofer & P. R. Pintrich (Eds.) Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 231–260). Erlbaum. Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 140 Appendix A Items From the Epistemic Belief Survey Listed Under the Intended Epistemic Belief Category Epistemic Belief (Latent Factor) Item # Item Speed of Knowledge Acquisition (KSP) 3 If something can be learned, it will be learned immediately. 7 Almost all the information you can understand form a textbook you will get during the first reading. 11 You will just get confused, if you try to integrate new ideas in a textbook with knowledge you already have about a topic. 16 Working on a difficult problem for an extended period of time only pays off for really smart students. 18 Usually, if you are ever going to understand something, it will make sense to you the first time. 24 If I can't understand something quickly, it usually means I will never understand it. 34 Most words have one clear meaning. 38 The information we learn in school is certain and unchanging. Structure of Knowledge (KST) 4 I like information to be presented in a straightforward fashion; I don't like having to read between the lines. 5 It is difficult to learn from textbook unless you start at the beginning and master one section at a time. 12 When I study, I look for specific facts. 13 If professors would stick more to the facts and do less theorizing, one could get more out of college. 21 I really appreciate instructors who organize their lectures carefully and then stick to their plan. 26 I don't like movies that don't have a clear-cut ending. 28 It's waste of time to work on problems that have no possibility of coming out with a clear-cut answer. 30 It is annoying to listen to lecturers who cannot seem to make their mind up as to what they really believe. 31 A good teacher's job is to keep students from wandering from the right track. Knowing, Thinking, and Learning Journal of Effective Teaching in Higher Education, vol. 5, no. 2 141 33 The best thing about science courses is that most problems have only one right answer. 36 When I learn, I prefer to make things, as simple as possible. Knowledge Construction & Modification (KCM) 2 The only thing that is certain is uncertainty itself. 6 Forming you own ideas is more important than learning what the textbooks say. 8 A really good way to understand a textbook is to reorganize the information according to your own personal scheme. 10 You should evaluate the accuracy of information in textbooks if you are familiar with the topic. 15 Wisdom is not knowing the answers but knowing how to find answers. 20 Today's facts may be tomorrow's fiction. 22 The most important part of scientific work is original thinking. 23 Even advice from experts should be questioned. 25 I try my best to combine information across chapters or even classes. 32 A sentence has little meaning unless you know the situation in which it was spoken. 37 I find it refreshing to think about issues that experts can't agree on. Characteristi cs of Successful Students (SS) 14 Being a good student generally involves memorizing a lot of facts. 17 Some people are born good learners; others are just stuck with a limited ability. 19 Successful students understand things quickly. 29 Understanding main ideas is easy for good students. 35 The really smart students don't have to work hard to do well in school. Attainment of Objective Truth (OT) 1 You can believe most things you read. 9 If scientists try hard enough, they can find the answer to almost every question. 27 Scientists can ultimately get to the truth.