Georgia Journal of Literacy 2024, Vol. 46(1), pp. 20–33 https://doi.org/10.56887/galiteracy.136 ISSN: 2833-7611 Research Article CONTACT Dr. Susan Green, District Literacy Specialist, Rock Hill Schools, Rock Hill, SC; email sgreen@rhmail.org. Dr. Maryann Mraz, Professor, Reading and Elementary Education Department, University of North Carolina at Charlotte, Charlotte, NC; email memraz@uncc.edu. The Effects of Question-Answer-Relationship (QAR) Instruction on Standardized Reading Comprehension Test Scores of Third-Grade Students Susan Green District Literacy Specialist, Rock Hill Schools, Rock Hill, SC Maryann Mraz Reading and Elementary Education Department, University of North Carolina at Charlotte, Charlotte, NC ABSTRACT Despite many school reform initiatives designed to ensure reading proficiency for all students, recent reports from the National Assessment of Educational Progress (NAEP) reveal that only 37 percent of fourth-grade students and only 34 percent of eighth-grade students performed at or above the proficiency levels measured in reading (NAEP, 2017). This quantitative study used a non-equivalent control group design to examine the impact of direct instruction of the Question-Answer- Relationship (QAR) strategy on standardized reading test scores of third-grade students. It also specifically examined the impact of direct instruction of the QAR strategy on traditionally underserved students' standardized test scores. A two-way Analysis of Covariance (ANCOVA) was conducted. Adjusted marginal mean post- test scores in the group receiving treatment of the QAR strategy (64.580) were higher than the mean post-test scores of students who did not receive QAR instruction (56.382). Adjusted marginal mean post-test scores of students who did not receive QAR instruction (56.382). Adjusted marginal mean post-test scores of historically underserved students showed no significant differences (60.05 and 60.90, respectively). In addition, adjusted marginal mean post-test scores of historically underserved and non-historically underserved students who received QAR instruction showed no significant differences (54.48 and 58.29). KEYWORDS reading comprehension; reading proficiency; metacognition; QAR (Question- Answer- Relationship); historically underserved students; standardized reading tests; high-stakes standardized testing espite many school reform initiatives designed to ensure reading proficiency for all students, recent reports from the National Assessment of Educational Progress (NAEP) reveal that only 32 percent of fourth grade students and only 31 percent of eight grade students performed at or above the proficiency levels measured in reading (NAEP, 2022). Poor performance on standardized reading tests can result in negative consequences for students, including limiting student learning, tracking, creating negative self-perceptions, and contributing to increased drop-out rates (National Council of Teachers of English, 2014). Although the negative impacts of standardized testing are felt by all students, the impact is especially severe for historically underserved students and students of low socio-economic status (SES). Differences between the scores of students with different backgrounds, including ethnic, racial, gender, disability, and income, are marked on standardized tests. The National Center for Education D mailto:sgreen@rhmail.org mailto:memraz@uncc.edu Green & Mraz 21 Statistics reported that by the end of fourth grade, African American, Latino, and poor students of all races are two years behind their wealthier, predominantly white peers in reading and math; by eighth grade, they have slipped three years behind, and by the twelfth grade they are four years behind (National Center for Education Statistics, 2022). Demonstration of reading proficiency on standardized reading tests involves answering questions. Classroom reading instruction, however, does not always provide direct instruction in questioning skills. Raphael and Pearson (1985) contend that direct instruction in strategies for answering specific types of questions can advance comprehension. The Question-Answer Relationship (QAR) taxonomy was developed as a way for students to learn where information may be found to answer comprehension questions (Ezell et al., 1996). QAR is a way for students to understand that the answer to a question is directly related to the type of question asked. QAR categorizes questions according to where the answers can be found. In the Book questions will be literal because the answer will be contained in the text. In my Head questions will be inferential because the answer will require information that is not contained in the text (Cummins et al., 2012). This method of categorizing questions according to their answer source is intended to support comprehension (Kinninburg & Prew, 2010). QAR is a valuable, well-known strategy that can be used to transport students to think deeply and analytically about text and prepare them for standardized testing while still focusing on higher-level thinking (Raphael & Au, 2005). Educators work to guide students to become more tactical thinkers by helping them understand their metacognitive processes. In the QAR framework, students analyze the question-answer relationship while becoming more aware of their metacognitive strategies as a step toward better reading comprehension (Raphael & Pearson, 1985). Research has shown that by grade three, children scoring significantly below the norm on achievement tests will continue to experience failure throughout their academic years (Ezell et al., 1996). Today, standardized testing in literacy education is common. For test-takers, the ability to locate and recall information in the text is crucial for success on standardized tests. The primary deficiency of students’ responses on high-stakes tests is the failure to support answers (Gunning, 2006). Students need to know how to return to a passage to locate details, verify information, and find text evidence to support their answers. Lower-level questioning involves students locating information directly stated in the passage. Higher-level questioning involves students drawing inferences and making conclusions from information in the passage. QAR can be adapted to test- taking in assisting students in locating sources of information and differentiating questions based upon question-answer classification (Gunning, 2006). Students can learn which questions are textually explicit and know they can go to the passage and find the information they need. Students can learn which questions are textually implicit and know they must make inferences based on information contained in the passage. Research shows that one of the most effective ways to improve students’ achievement and to reduce the literacy achievement gap is to promote metacognition and higher-level thinking skills; however, historically underserved students and students of low SES are more likely to be instructed in basic skills rather than higher-level thinking processes (Gunning, 2006). QAR can serve as a vehicle to teach higher-level thinking skills while preparing students for high-stakes tests without sacrificing high-quality instruction (Raphael & Au, 2005). Studies have shown that direct instruction in metacognitive strategies, like QAR, assists students in reaching high levels of literacy. In this era of high-stakes standardized testing, it is imperative that teachers not only focus on skills to promote a high level of literacy development but also provide support for minority students as they navigate standardized assessments. QAR instruction, when used effectively in the 22 Georgia Journal of Literacy 46(1) context of high-quality literacy instruction, can increase metacognition and provide students with a high-level strategy to increase reading comprehension and reading proficiency. QAR instruction may also serve as a strategy to increase scores on standardized reading tests, which would promote increased educational opportunities for all. Purpose and Theoretical Framework The purpose of this study was to examine the impact of direct instruction of the Question-Answer Relationship (QAR) strategy on standardized reading test scores of third-grade students. Findings were intended to provide analysis of standardized test scores and the impact of QAR as an intervention that will serve not only to increase standardized test scores but also increase educational opportunities for all students, including historically underserved students, whose educational opportunities have been greatly limited by poor performance on standardized tests. The following questions guided this study: Research Question 1: Do students who received QAR instruction do better on standardized reading assessments than students who do not receive QAR instruction? Research Question 2: Is the impact of the QAR treatment the same for historically underserved and non-historically underserved students? Prior research has focused primarily on QAR as a strategy to improve reading comprehension. There is little research directly linking QAR to improved standardized test scores (Cummins et al., 2012; Kinniburg & Baxter, 2012; Raphael & Pearson, 1985). This study examined QAR as a strategy to improve standardized test scores, which is critical in this era of high-stakes testing. Furthermore, there is no research focusing specifically on the effect of QAR on standardized test scores of historically underserved students or students of low socio-economic status (SES). This study used two theoretical frameworks to examine the impact of QAR as an intervention to increase standardized test scores and, in turn, educational opportunities. Piaget’s Cognitive Constructivism provides a strong framework for QAR instruction as an intervention to increase student knowledge and comprehension skills. Constructivism is a theory of learning anchored in the belief that students learn by actively constructing their own knowledge. It is an active process where learners construct meaning through a process of involvement and interaction with their environment. Cognitive constructivism focuses on the importance of the mind in learning and the development of cognitive structures in learners (Schcolnik et al., 2006). Piaget’s terms accommodation and assimilation are used to describe the interaction between mind and environment in the learning process. Learners use their cognitive structures to interpret the environment and assimilate new information into their existing cognitive schemas. Assimilation is limited to the extent of the existing schemas until cognitive structures modify based on new knowledge. Learning is continuous, and cognitive structures are always in process as the mind interacts with the environment (Mohapatra et al., 2015). Michel Foucault’s Power as Knowledge Theory provides a powerful lens through which to examine the effects of QAR as a strategy not only to improve the standardized test scores of third-grade students but also as a method of empowering all students through increased educational opportunities. Michael Foucault, a postmodern theorist, explored the ways in which political power was subtly invested in the mechanisms of knowledge in the modern world (Lemert, 2016). For Green & Mraz 23 Foucault, power and knowledge were not seen as separate entities but as irrevocably connected. Knowledge is always an exercise of power, and power is always a function of knowledge. Foucault believed power was everywhere, dispersed throughout society, and able to shape individuals within society. He believed that power and knowledge were constructed by truths created within societies and that these truths became a standard set of rules accepted by societies, providing power to those who had the status to create these truths. Foucault believed that these truths were reinforced through societal institutions, like schools, but believed that these truths were not static but rather marked by a constant battle where individuals pushed the boundaries of power (Lemert, 2016). Although he believed that the relationship between power and knowledge was constraining and often limited behaviors, he also recognized the potential it also held to open new ways of thinking and behaving. Foucault recognized the potential for power to be a productive force in society, believing that power produces a reality that shapes individuals within societies through the attainment of knowledge (Lemert, 2016). Foucault viewed power as a fully socialized phenomenon and contended that power and the norms it creates are so embedded within our society that they tend to be unthinkingly followed by individuals who fail to fully realize the control it has over their daily actions (Lemert, 2016). He focused not only on the power of institutions and their ability to discipline but also on how these norms often create advantages for some while placing many at a great disadvantage. Relevant Literature Metacognition can be defined as the process of monitoring or regulating cognition or “thinking about thinking” (Wilson & Smetana, 2009, p. 20). It refers to the awareness of the cognitive process involved in thinking. Metacognition in reading is multifaceted. It involves constructing meaning from text, recognizing when comprehension fails, and choosing appropriate strategies to increase comprehension. Metacognition includes the readers’ knowledge of the reading process as well as command of the strategies related to reading (Raphael, 1982). Research studies have been conducted to explore the relationship between metacognition and reading proficiency. Early studies concluded that younger students and emergent readers had metacognitive deficits in reading (Myers & Paris, 1978). Subsequent studies explored the effects of classroom interventions that provided explicit instruction in reading strategies and reading strategy use to improve metacognition and reading comprehension. Students who received metacognitive training showed an increased awareness of effective reading strategies and improved performance on reading tasks (Cross & Paris, 1988). Reading is a metacognitive process. Within text comprehension, readers must not only derive meaning from text content but also integrate new knowledge with background information from the readers’ prior knowledge (Soodla et al., 2016) to engage in higher levels of thinking and understanding. The use of reading strategies plays an important role in reading and reading comprehension. Proficient readers can use a variety of strategies and can apply appropriate reading strategies in various contexts. Strong metacognitive knowledge allows readers to be successful on reading comprehension tasks because students engage in strategic reading. They know how and when to use good reading strategies that are most effective for the task (Soodla et al., 2016). The QAR taxonomy, first described by Pearson and Johnson (1978) and further developed by Raphael and Pearson (1985), is a metacognitive strategy used to improve reading comprehension by identifying questions according to their relationship to two primary sources of information: the reading material and the reader’s background knowledge. It requires students to 24 Georgia Journal of Literacy 46(1) think about the relationship between the text and the questions. Using the QAR strategy, students can understand the question type, which correlates with knowing how to find the information to answer the question. Raphael (1986) categorized QAR questions according to where the answers can be found. In the Book questions are literal because the answer is contained in the text. In My Head questions are inferential because the answer requires information not contained in the text. There are four types of question–answer relationships: • Right There: The answer can be found in one place in the text. • Think and Search: The answer can be found in a few places in the text. • Author and You: The answer cannot be found in the text. The reader must use information in the text and find the answer in their head. • On My Own: The answer cannot be found in the text. The answer is developed from the reader’s background knowledge. (Raphael, 1986) In the QAR framework, students analyze the question-answer relationship while becoming more aware of their metacognitive strategies as a step toward better reading comprehension (Raphael & Pearson, 1985). Research supports the use of QAR in elementary school classrooms to increase the answering abilities and the reading comprehension skills of students (Ezell, 1992; Ezell et al., 1996; Raphael & Au, 2005). The level of awareness that comes with the classification of questions leads to success in answering reading comprehension questions. While many studies explored QAR as a strategy to increase students’ reading comprehension skills, there are few that focus on QAR as a strategy to improve performance on standardized reading tests. Standardized tests require students to answer questions that are both textually explicit and textually implicit in nature, which require students to perform both lower- level and higher-level thinking about text (Wang, 2006). Researchers are just beginning to examine QAR as a framework for comprehension instruction that would not only raise students’ reading comprehension skills but also improve students’ performance on standardized reading tests. In this era of high-stakes standardized testing, it is imperative that teachers not only focus on skills to promote a high level of literacy development but also provide support for all students as they navigate standardized assessments. Methodology A non-equivalent control group design was used for this study. Established third-grade classrooms were assigned to treatment or control groups. Students in both groups were pre-tested with a standardized reading test. Students in the treatment classrooms received six weeks of direct instruction in QAR. At the end of the six-week period, students in both groups were post-tested with a comparable standardized reading test. The research questions guiding this study were: Research Question 1: Do students who received QAR instruction do better on standardized reading assessments than students who did not receive QAR instruction? Research Question 2: Is the impact of the treatment the same for historically underserved and non-historically underserved students? The population of this study was comprised of third-grade students and teachers from eight third-grade classrooms in two elementary schools in a rural community in the south. The choice Green & Mraz 25 of schools in this study was purposeful. The two schools share similar student achievement levels and demographics. Third-grade classrooms were chosen specifically for this study since third grade is the first year of standardized testing in the state. The first elementary school that participated in this study had a school population of 590 students, with an average of 19 students in each third-grade classroom. Overall achievement indicators showed that 68% of students had achieved proficiency on recent past standardized state reading assessments. Ethnicity data for School One reported that the school was comprised of approximately 76% white students and 24% historically underserved students (UCPS Enrollment and Ethnicity Data, 2018). The second elementary school that participated in this study had a school population of 604 students, with an average of 17 students in each third-grade classroom. Overall achievement indicators showed that 70% of students had achieved proficiency on recent past standardized state reading assessments. Ethnicity data for School Two reported that the school is comprised of approximately 74% white students and 26% historically underserved students (UCPS Enrollment and Ethnicity Data, 2018). Four third-grade classrooms from each school participated in this study. All four third- grade classrooms at School One were assigned as treatment classrooms, and all four third-grade classrooms at School Two were assigned as control classrooms. Teachers in treatment classrooms participated in professional development on the use of QAR in the classroom and adhered to a six- week QAR instructional plan created by the researcher. Teachers completed daily rubrics during the six-week instructional period to self-report on their QAR classroom instructional practices. Teachers in treatment classrooms met with the researcher for two hours prior to the beginning of each of the four phases of the six-week intervention period for professional development, which included directives, clarification, and support. Teachers in treatment and control classrooms were not permitted to use QAR as an instructional strategy prior to this intervention. Teachers in control classrooms had agreed not to use QAR during this instructional intervention period. Data Collection and Analysis A non-equivalent control group design was used for this study, and instrumentation included pre- and post-assessments published by Triumph Learning, which included full-length assessments that mirror the format, question type, and rigor of the State End-of-Grade Standardized Reading Assessment. These provided students with grade-level appropriate text to answer standardized reading comprehension questions that require application of both lower and higher-order thinking skills. Test developers used the Common Core State Standards to determine text complexity when selecting reading passages and employed quantitative measures and guidelines for making qualitative decisions about passages and questions included in each standardized reading assessment (Triumph Learning, 2015). Both pre- and post-assessments contained six reading selections, which included fiction, non-fiction, folktale, and poetry passages with corresponding multiple-choice questions for each passage. The forty-four multiple choice questions included questions from three of the QAR question types: Right There, Think and Search, and Author and You. The final QAR question type, On My Own, is not included in pre- or post-assessments since the answer would be developed solely from the reader’s background knowledge and is not included on standardized reading assessments. The pre-assessment contained 44 multiple-choice questions: 19 were text-based (Right There or Think and Search), and 25 were inferential (Author and You). The post-assessment contained 44 multiple-choice questions: 19 were text-based (Right There or Think and Search), and 25 were inferential (Author and You). 26 Georgia Journal of Literacy 46(1) Principals at the two participating schools were provided with copies of both pre-and post- assessments prior to the implementation of this study. Classroom teachers were provided with these pre- and post-assessments immediately prior to both assessments being administered in treatment and control classrooms. The pre-test assessments were administered by all eight third- grade classroom teachers the week prior to the six-week instructional intervention period. The post-test assessments were administered by all eight third-grade classroom teachers the week after the six-week instructional intervention period. Students in all classrooms had to complete pre and post-assessments in one testing session, and no students, including Exceptional Children (EC) or English-Language Learners (ELL), were excluded. Pre- and post-tests were scored by the researcher. Students received credit for correct answers on both assessments, and scores were calculated. Quantitative data analysis involving descriptive and inferential statistics was used to examine student scores and draw comparisons between groups. Research has shown that QAR is a strategy that can be successfully implemented within existing instructional reading frameworks when teachers are provided with adequate professional development in QAR. Teachers in treatment classrooms participated in a two-hour professional development session prior to pre-test measures. During this professional development session, teachers were introduced to the QAR strategy, and each question type was explained and discussed. Teachers made materials to be used in their classrooms during the six-week instructional intervention phase, which included anchor charts and question cards to be categorized according to the QAR taxonomy as part of daily QAR classroom instruction. In addition, teachers were provided with a detailed four-phase plan and materials that they used to guide instruction during the six-week instructional intervention period and a copy of Two for One: Using QAR to Increase Reading Comprehension and Improve Test Scores (Green, 2016), which provided further clarification of each phase. Each phase (Table 1) of instruction took place within current classroom reading instructional frameworks and included a minimum of six days of instruction in the use of QAR. The four teachers in the treatment classrooms met at the beginning of each phase of the six- week instructional intervention period to discuss QAR classroom instructional strategies for each phase. Teachers were also provided with guidance and support from the researcher during these meetings to ensure classroom instruction would align with the instructional plan and at the end of each day of the six-week instructional classroom intervention of, QAR teachers completed a daily rubric where they self-reported on QAR classroom instruction. Teachers in treatment classrooms scored themselves on teaching behaviors for each phase of the instructional intervention. Table 1: Phases Within the Six-Week Instructional Period Phase 1 Phase 2 Phase 3 Phase 4 Introduce and Model QAR Model QAR with Subcategories QAR with Just Right Text QAR with Sample Test Questions Teachers introduced the concept of QAR, explaining that answers to reading comprehension questions can be found in two places: in the text and in the reader’s mind. Teachers began instruction in all four categories of QAR; explaining the two categories of In the Book: Right There and Think and Search and the two categories of In my Head: Author and You and On My Own. Teachers modeled, using a shared text, how to complete the QAR graphic organizer provided by the researcher. Teachers modeled how to classify and answer standardized reading comprehension questions. Green & Mraz 27 Teachers used the QAR anchor chart provided by the researcher to introduce only the two main categories of QAR: In the Book and In my Head. Teachers used a shared text and periodically stopped and asked questions to model classifying questions according to QAR and QAR vocabulary. Teachers had students complete the QAR graphic organizer with questions created around a reading passage provided by the researcher. Teachers provided students with a reading selection and students classified (labeled) and answered questions using the QAR strategy. Teachers used a shared text and periodically stopped and asked questions to model classifying questions according to QAR and QAR vocabulary. Teachers encouraged students to use QAR vocabulary to classify and develop questions on their own for all four categories. Teachers monitored student responses and provided clarification for students who had difficulty correctly using the QAR strategy. Teachers monitored student classifications and answers and provided clarification for students who had difficulty using the QAR strategy to answer standardized reading comprehension questions. Teachers encouraged students to use QAR vocabulary to classify and develop both categories of questions on their own. Quantitative data analysis involving descriptive and inferential statistics was used to examine student scores and draw comparisons between groups. To answer both research questions, a two- way Analysis of Covariance (ANCOVA) was performed. Treatment or control status served as one independent variable, and historically underserved or non-historically underserved status served as the second independent variable. The pre-test served as the covariate. This study used a quasi-experimental control design to examine the impact of QAR on standardized reading test scores of third-grade students. It is important to delimit the boundaries of this investigation to interpret the results and potential future impacts most accurately. The use of a quasi-experimental design limits the generalizability of the findings. Participants included students from schools in the same cluster within a rural county in the southeast. The choice of schools and classrooms was purposeful and sought to strengthen the validity of this study since the two schools share similar student demographics and include a similar number of historically underserved students. However, schools and classrooms were also chosen for convenience. The researcher was employed as a third-grade teacher in the treatment school and served as both a researcher and a participant, implementing the six-week instructional intervention in her own third- grade classroom. Since the researcher in this study was also a participant, the dual role of the researcher must be addressed. As a teacher in one of the treatment classrooms, the researcher provided QAR instruction to her students. One can assume that although the researcher did not use QAR as an instructional strategy prior to the six-week instructional intervention period, her knowledge of the strategy was extensive. While the use of a shared scripted instructional plan and a teaching rubric for self-reporting classroom instruction served to mitigate the inconsistencies resulting from the researchers’ knowledge of the strategy, it is important to note this as a limitation of this study. This study also specifically examined the impact of direct instruction of the QAR strategy on historically underserved students. The findings of this study and implications for historically underserved students may not be generalizable since there was a small sample of historically underserved students included as participants in this study. 28 Georgia Journal of Literacy 46(1) Results One hundred thirty-one students participated in this study between School One and School Two. Descriptive Statistics are included in Table 2. Table 2: Descriptive Statistics of Standardized Reading Assessment Scores of Historically Underserved and Non-Historically Underserved Students in Control and Treatment Groups Pre-Standardized Reading Assessment Post-Standardized Reading Assessment No QAR QAR No QAR QAR Min N-Min Min N-Min Min N-Min Min N-Min M 52.85 61.53 53.14 61.78 49.70 60.37 61.09 65.80 SD 18.63 18.21 18.73 20.68 20.32 17.90 19.78 18.98 N 20 38 22 51 20 38 22 51 Note. Min = Historically underserved; N-Min = Non-Historically underserved Measures were taken to ensure fidelity of implementation. It was established that prior to the study, QAR was not being used as an instructional strategy in any of the eight third-grade classrooms participating in this study. The teachers at School One, who were responsible for providing QAR instruction in their classrooms, participated in professional development prior to providing direct instruction of QAR in the classrooms. All four teachers in the treatment classrooms followed a scripted plan for QAR instruction and used identical materials with their students. Teachers in treatment classrooms were also required to complete a daily teaching rubric to self-report on QAR classroom instruction. All four teachers reported high levels of alignment between the instructional plan and classroom implementation of QAR classroom instruction. Results of the Self-Reporting Teaching Rubric are outlined in Table 3. The numbers in each Likert Scale category show how many teachers self- reported the frequency in which they engaged in the tasks listed on the rubric in each classroom for each teaching segment. Table 3: Teaching Rubric Scores: Self-Reporting on Classroom Instruction for Treatment Classrooms for Each Teaching Segment Never Rarely A Few Times Frequently Often Room 1 0 0 2 3 194 Room 2 1 0 0 0 198 Room 3 4 0 3 6 186 Room 4 2 0 0 0 197 Note. Total number of teaching segments = 199 Green & Mraz 29 There was not a statistically significant two-way interaction between condition and ethnicity on post-test standardized reading comprehension scores while controlling for pre-test standardized reading comprehension scores, F(1,126) = 1.97, p = .163, partial η2 = .015. According to Cohen (1988), this indicates a small effect size. Therefore, an analysis of the main effects for condition (no QAR and QAR) and ethnicity (historically underserved and non-historically underserved) was performed. There was a statistically significant main effect for condition F(1,126) = 15.007, p < .001, partial η2 = .106. Adjusted marginal mean post-test scores in the group receiving treatment of the QAR strategy (64.58) were higher than the mean post-test scores of students who did not receive QAR instruction (56.38). According to Cohen (1988), this indicates a medium effect size. There was not a statistically significant effect for ethnicity, F(1,126) = .153, p = .697, partial η2 = .001. According to Cohen (1988), this indicates a small effect size. Adjusted marginal mean post-test scores of historically underserved and non-historically underserved students showed no significant differences (60.05 and 60.90), respectively. In addition, adjusted marginal mean post-test scores of historically underserved and non-historically underserved students who received QAR instruction showed no significant differences (54.48 and 58.29), respectively. Discussion This quantitative study examined the impact of direct instruction of the QAR strategy on standardized reading test scores of third-grade students. It also specifically examined the impact of direct instruction of the QAR strategy on historically underserved students’ standardized test scores. The results will be discussed around the research questions and hypotheses guiding this study: Research Question 1: Do students who received QAR instruction do better on standardized reading assessments than students who did not receive QAR instruction? In seeking to answer research question one, the results of this study show a significant difference in students’ mean scores on the post-test between third-grade students who received six weeks of QAR instruction and those who did not. Students who were in treatment classrooms had significantly higher mean scores (64.58) on the post-test compared to students in the control group (56.38). These results support the use of QAR to improve standardized reading test scores of third- grade students. Providing students with instruction in the metacognitive skill of classifying questions according to the location of the answer (text-based or inferential) assisted students in navigating standardized-type reading questions and increased student performance on a standardized reading comprehension test. Research Question 2: Is the impact of the treatment the same for historically underserved and non-historically underserved students? In seeking to answer research question two, the results of this study do not show a significant difference in students’ mean scores on the post-test between historically underserved and non-historically underserved students. While results showed that historically underserved students benefitted from QAR instruction, students’ ethnicity did not impact the results of this study. Students benefitted from QAR instruction regardless of their ethnicity status. However, since the findings of this study support the use of QAR to improve standardized test scores of third- grade students, this would include third-grade historically underserved students. QAR can be 30 Georgia Journal of Literacy 46(1) viewed as a viable strategy to increase standardized reading comprehension test scores of both historically underserved and non-historically underserved students. Of utmost importance to the paradigm of best practices in literacy education, this study presented evidence to substantiate the benefits of QAR in improving standardized reading comprehension test scores of third-grade students. Students in the treatment classrooms had substantially higher mean scores on the post-assessment measure after six weeks of direct instruction in the QAR strategy than students in the control classrooms. The results have instructional and theoretical implications for practice. Instructional implications include support for direct instruction in metacognitive skills to increase students’ reading performance and proficiency. In this study, direct instruction in metacognition, involving the awareness of the relationship between reading comprehension questions and the sources of information to answer questions correctly, resulted in increased reading performance on a standardized assessment. Students who received direct instruction in metacognition showed an increase in awareness of effective reading strategies, which resulted in improved performance in reading tasks. As a result of increasing students’ metacognitive knowledge, students employed strategic reading behaviors. The findings of this study align with previous studies concluding that direct instruction and use of metacognitive strategies facilitate students’ understanding of critical cognitive processes and promote students’ reading development. In addition, instructional implications include support for the use of QAR to increase students’ reading comprehension skills. Students in this study who used the QAR strategy were more successful in answering reading comprehension questions than students who did not receive QAR strategy instruction. QAR was easily implemented within established literacy classroom frameworks and a six-week instructional period provided students with the ability to correctly classify and answer text-based and inferential questions within a reading comprehension selection. The findings of this study align with others that have established QAR as a method to increase students’ awareness of sources of information to improve reading comprehension question- answering skills and could be easily implemented within established classroom instructional frameworks to enhance students’ performance on reading comprehension tests. This study also highlights the importance of providing instruction in higher-level critical thinking skills. QAR instruction focuses on the relationship between questions and answers and provides students with procedural knowledge to practice both lower-level and higher-level reading and thinking skills. Students were instructed to employ both lower-level and higher-level reading and thinking skills as part of this instructional intervention. QAR requires students to engage in lower levels of thinking by recalling text when answering text-based questions. However, QAR also requires students to employ higher levels of thinking like application, analysis, synthesis, and evaluation when categorizing and answering inferential questions. In addition to increasing students’ abilities to answer questions, QAR was also used to provide students with an opportunity to generate their own questions. Question generating and discussion around cognitive techniques allowed students to maximize the benefits of QAR as a higher-level metacognitive strategy. The instructional implications for educators working with historically underserved students are clear. While this study showed no statistically significant interaction effect between the QAR strategy and ethnicity, findings did support the use of QAR to improve standardized test scores of all students in this study. These findings support the use of QAR to increase non-historically underserved and historically underserved students’ test scores. There is significant research that shows one of the most effective ways to improve student literacy achievement is through Green & Mraz 31 instruction of metacognition and higher-level thinking skills; however, historically, underserved students are more likely to be instructed in basic skills rather than higher-level thinking processes (Gunning, 2006). QAR can be viewed as a viable strategy to promote higher levels of literacy development for historically underserved students as well as a strategy to increase standardized test scores. Future research must adhere to a narrower focus where researchers examine different aspects of QAR and the effects on students’ development of higher-level literacy skills. Therefore, the next steps should include an examination of QAR in relation to students’ age, length of instructional intervention, longevity of skills, and the impact QAR has on different levels of readers. In addition, since standardized testing has become a seemingly permanent fixture in education, future research should examine the impact QAR instruction has on standardized test scores as well as the impact QAR has on historical groups of students. Future research should also explore the length of instruction required for the QAR strategy students to improve reading comprehension and promote higher levels of literacy development. Throughout the history of education, groups of students have struggled to show proficiency on standardized reading assessments. Differences between the scores of students with different backgrounds, including ethnic, racial, gender, disability, and income, are marked on standardized tests. The consequences associated with low standardized test scores extend far beyond testing, limiting opportunities for higher education and future employment. Future research should also focus on uncovering strategies that would increase the performance of historically underserved groups of students, therefore mitigating the negative consequences associated with poor performance on standardized assessments. In this era of high-stakes standardized testing, it is imperative that researchers not only focus on skills to promote a high level of literacy development but also focus on support for students as they navigate standardized assessments, specifically on skills and strategies that target the learning needs of historically marginalized student groups. This quantitative study examined the impact of direct instruction of the QAR strategy on standardized reading test scores of third-grade students. It also examined the impact of direct instruction of the QAR strategy on historically underserved students’ standardized test scores. The findings support the use of QAR to increase standardized test scores of both historically underserved and non-historically underserved students. The research presented provides educators with an additional strategy to support students’ reading comprehension and improve student scores on standardized reading assessments. This is essential since scores on standardized assessments can either open or close doors to educational opportunities that influence future success. References Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum. Cross, D. R., & Paris, S. (1988). Developmental and instructional analyses of children’s metacognition and reading comprehension. Journal of Educational Psychology, 80(2), 13– 142. https://doi.org/10.1037//0022-0663.80.2.131 Cummins, S., Streiff, M., & Ceprano, M. (2012). Understanding and applying the QAR strategy to improve test scores. Journal of Inquiry & Action in Education, 4(3), 18–25. https:// digitalcommons. buffalostate.edu/jiae/vol4/iss3/2 https://doi.org/10.1037/0022-0663.80.2.131 32 Georgia Journal of Literacy 46(1) Ezell, H. K. (1992). Use of peer-assisted procedures to teach QAR reading comprehension strategies to third-grade children. Education and Treatment of Children, 15(3), 205–227. https://www.jstor.org/stable/42899267 Ezell, H. K., Hunsicker, S. A., Quinque, M. M., & Randolph, E. (1996). Maintenance and generalization of QAR reading comprehension strategies. Reading Research and Instruction, 36(1), 64–81. https://doi.org/10.1080/19388079609558228 Green, S. (2016). Two for one: Using QAR to increase reading comprehension and improve test scores. The Reading Teacher, 70(1), 103–109. https://doi.org/10.1002/trtr.1466 Gunning, T. G. (2006). Closing the literacy gap. Pearson Education. Kinniburg, L. H., & Baxter, A. (2012). Using question-answer relationships in science instruction to increase reading achievement of struggling readers and students with reading disabilities. Current Issues in Education, 15(2), 1–8. https://cie.asu.edu/ojs/index.php/ cieatasu/article/view/915 Kinninburg, L. H., & Prew, S. S. (2010). Question answer relationships (QAR) in the primary grades: Laying the foundation for reading comprehension. International Journal of Early Childhood Special Education, 2(1), 31–44. https://doi.org/10.20489/intjecse.107956 Lemert, C. (2016). Social theory: The multicultural and classic readings (6th ed.). West View Press. Mohapatra, J. K., Mahapatra, M., & Parida, B. K. (2015). Constructivism: The new paradigm from theory to practice. Atlantic Publishers. Myers, M., & Paris, S. (1978). Children’s metacognitive knowledge about reading. Journal of Educational Psychology, 70(5), 680–690. https://doi.org/10.1037//0022-0663.70.5.680 National Center for Education Statistics. (NCES). (2022). Report on the condition of education 2022 (NCES 2022-144). U.S. Department of Education, Institute of Education Sciences. https://nces.ed.gov/pubs2022/2022144.pdf National Council of Teachers of English. (NCTE). (2014). How standardized tests shape—and limit—student learning. James R. Squire Office of Policy Research. Pearson, P. D., & Johnson, D. D. (1978). Teaching reading comprehension. Holt, Rinehart, and Winston. Raphael, T. E. (1986). Teacher question answer relationships, revisited. The Reading Teacher, 39(6), 516–522. http://www.jstor.org/stable/20199149 Raphael, T. E., & Au, K. H. (2005). QAR: Enhancing comprehension and test taking across grades and content areas. International Reading Association, 59(3), 206–221. https://doi.org/ 10.1598/rt.59.3.1 Raphael, T. E. (1982). Question-answering strategies for children. The Reading Teacher, 36(2), 186–190. https://www.jstor.org/stable/20198181 Raphael. T. E., & Pearson, D. (1985). Increasing students’ awareness of sources of information for answering questions. American Educational Research Journal, 22(2), 217–235. https://doi.org/10.3102/00028312022002217 https://www.jstor.org/stable/42899267 https://doi.org/10.1080/19388079609558228 https://doi.org/10.1002/trtr.1466 https://cie.asu.edu/ojs/index.php/%E2%80%8Ccieatasu/article/view/915 https://cie.asu.edu/ojs/index.php/%E2%80%8Ccieatasu/article/view/915 https://doi.org/10.20489/intjecse.107956 https://doi.org/10.1037/0022-0663.70.5.680 https://nces.ed.gov/pubs2022/2022144.pdf http://www.jstor.org/stable/20199149 https://doi.org/%E2%80%8C10.1598/rt.59.3.1 https://doi.org/%E2%80%8C10.1598/rt.59.3.1 https://www.jstor.org/stable/20198181 https://doi.org/10.3102/00028312022002217 Green & Mraz 33 Schcolnik, M., Kol, S., & Abarbanel, J. (2006). Constructivism in theory and in practice. English Teaching Forum, 1(4), 12–20. http://files.eric.ed.gov/fulltext/EJ1107896.pdf Soodla, P., Jogi, A., & Kikas, E. (2016). Relationship between teachers’ metacognitive knowledge and students’ metacognitive knowledge and reading achievement. European Journal of Psychology of Education, 32(1), 201–218. https://doi.org/10.1007/s10212-016-0293-x Triumph Learning. (2015). Using standards-based programs to prepare students for next- generation high stakes assessments: Common core performance coach English language arts [White Paper]. http://www.triumphlearning.com/research.html Union County Public Schools. (UCPS). (2018). Enrollment and ethnicity reports. U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2022 Reading Assessment. https://www.ucps.k12.nc.us/Page/3030 Wang, D. (2006). What can standardized reading tests tell us? Question-answer relationships and students’ performance. Journal of College Reading and Learning, 36(2), 21–37. https:// doi.org/10.1080/10790195.2006.10850185 Wilson, N. S., & Smetana, L. (2009). Questioning as thinking: A metacognitive framework. Middle School Journal, 11(1), 20–28. https://doi.org/10.1080/00940771.2009.11461709 Received: December 29, 2023 | Accepted: March 28, 2024 | Published: May 15, 2024 http://files.eric.ed.gov/fulltext/EJ1107896.pdf https://doi.org/10.1007/s10212-016-0293-x http://www.triumphlearning.com/research.html https://www.ucps.k12.nc.us/Page/3030 https://doi.org/10.1080/10790195.2006.10850185 https://doi.org/10.1080/10790195.2006.10850185 https://doi.org/10.1080/00940771.2009.11461709 Purpose and Theoretical Framework Relevant Literature Methodology Data Collection and Analysis Table 1: Phases Within the Six-Week Instructional Period Results Table 2: Descriptive Statistics of Standardized Reading Assessment Scores of Historically Underserved and Non-Historically Underserved Students in Control and Treatment Groups Table 3: Teaching Rubric Scores: Self-Reporting on Classroom Instruction for Treatment Classrooms for Each Teaching Segment Discussion References