Microsoft Word - 6585.docx 2601 Copyright © 2024 The Author IDEAS is licensed under CC-BY-SA 4.0 License Issued by English study program of IAIN Palopo IDEAS Journal of Language Teaching and Learning, Linguistics and Literature ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) Volume 13, Number 1, June 2025 pp. 2601 - 2622 Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education Neni Nurkhamidah1, Ninuk Lustyantie2, Fathiaty Murtadho 3 1Pendidikan Bahasa Inggris, Universitas Media Nusantara Citra, Jakarta 2,3 Linguistik Terapan, Universitas Negeri Jakarta, Jakarta Corresponding E-Mail: neninurkhamidah@gmail.com Received: 2025-05-10 Accepted: 2025-07-03 DOI: 10.24256/ideas.v13i1.6585 Abstract This study addresses the limitations of existing critical reading assessments in Indonesian higher education, which often focus narrowly on comprehension and recall, neglecting higher-order critical thinking and ecological literacy skills essential for addressing complex environmental challenges. The problem is evident as students frequently achieve near-perfect scores on current tests that inadequately measure their ability to critically analyze academic texts or apply ecological awareness. This research develops a comprehensive assessment specification for a Critical Reading course at Media Nusantara University that integrates ecological literacy alongside traditional reading skills. Using a Design and Development Research (DDR) methodology, the study analyzed course syllabi, learning outcomes, and existing assessments, and incorporated feedback from a Focus Group Discussion (FGD) involving one lecturer and two assessment experts, alongside interviews with five students. The resulting assessment balances multiple-choice questions with open-ended essay tasks and includes ecological case studies to better evaluate higher-order thinking and real-world problem-solving abilities. The revised assessment specification uniquely combines critical reading with ecological literacy to prepare students for both academic success and responsible environmental citizenship. Findings indicate that this approach fosters deeper engagement with texts and environmental issues and offers a practical framework for more valid and reliable assessments in higher education. Keywords: Assessment specification; critical reading; ecology literacy Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2602 Introduction English serves as a global language that provides access to a wealth of information across various fields, from science and technology to literature and the arts. Therefore, mastering reading in English is a critical foundational skill that plays a vital role in a student’s academic (Balan et al., 2019; Blaabæk, 2020; Hicks, 2023)and career development (Dudina, 2024). For non-native speakers, developing strong reading skills goes beyond understanding vocabulary and grammar; it also requires the ability to interpret, analyze, and engage with texts in a meaningful way. While mastering English is crucial, it presents significant challenges for non-native speakers(Bordag et al., 2021; Dardjito et al., 2023; Hibatullah, 2019; Sari, 2023; Srikrai et al., 2016).The process of developing strong reading skills in English requires overcoming linguistic, cultural, and cognitive barriers, making it a demanding but essential skill for navigating both educational and professional environments. Critical reading is a fundamental academic skill that goes beyond basic understanding to include actively questioning, analyzing, and evaluating the content presented in texts. This process is vital in academic and professional settings, where it enables readers to critically engage with arguments, assess the validity of evidence, and identify biases or assumptions. By not merely absorbing information, critical readers are able to develop independent thought, form well- supported conclusions, and recognize gaps or limitations in the material. The importance of this skill is particularly pronounced in fields like literature, social sciences, and natural sciences, where complex concepts and diverse viewpoints require thorough analysis. Research has emphasized the importance of developing critical reading skills in higher education to enhance students' ability to analyze and think critically (Arifin, 2020; Gönen & Kızılay, 2022). However, research shows that critical reading skills remain underdeveloped among Indonesian university student (Hanum et al., 2020; Jumiari, et al., 2024; Sudarwati & Manipuspika, 2021; VinaNurviyani, 2018). Various studies also show that students often fail to utilize advanced critical reading strategies and struggle to move beyond basic comprehension. For example, research has highlighted that student, even at the tertiary level, tend to focus on literal understanding and rarely engage with the deeper implications of a text, such as identifying biases or assessing the validity of arguments (Sultan et al., 2018)). Moreover, studies reveal a gap between students perceived and actual critical reading abilities, highlighting the need for innovative approaches to assessment (Din, 2020; Shamida et al., 2021) Meanwhile, higher education increasingly acknowledges ecological literacy as a vital approach to addressing environmental problems. It goes beyond just understanding how natural systems function; it also focuses on the connections between human activities and the environment. As global challenges like climate change and biodiversity loss become more urgent, it is essential for students to gain IDEAS, Vol. 13, No. 1, June 2025 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 2603 the knowledge and skills necessary to support sustainable practices. By incorporating ecological literacy into higher education, students gain a deeper understanding of ecosystems and the impact of human behavior on these systems (Abduganiev & Abdurakhmanov, 2020; Hartono, 2020). This well-rounded approach encourages students to critically analyze sustainability, ethical resource management, and environmental justice, promoting responsible global citizenship Although both critical reading and ecological literacy are becoming more acknowledged in higher education, existing assessment methods often fail to address their intersection. While critical reading assessments primarily focus on comprehension and analysis, they rarely account for ecological literacy, such as considering the environmental consequences of the material. This oversight means that students may not be adequately prepared to apply their critical reading skills to environmental issues. Furthermore, the lack of an integrated framework that assesses both critical reading and ecological literacy limits the ability of educators to fully evaluate students' readiness to tackle contemporary environmental problems. Given the complexity of today’s global challenges, it is crucial for students to develop both critical reading skills and ecological knowledge to address pressing ecological issues in their academic and professional lives (Chambers & Radbourne, 2014; Häggström & Schmidt, 2020; Silvhiany & Huzaifah, 2020a). Other study has had emphasized the importance of interdisciplinary frameworks that merge ecological and critical literacies to improve sustainability education outcomes (Kerçin & Demir, 2024a). To effectively implement this, it is necessary to develop a comprehensive blueprint for the assessment tool. A blueprint is a detailed plan or structured framework that outlines the key components and processes for creating an assessment. It includes clear specifications such as the learning outcomes to be achieved, the design and structure of the test, the content areas to be covered, and the criteria for scoring and evaluation (Raymond & Grande, 2019). Test blueprints outline the content that will be included in a test, along with other essential elements such as the weight assigned to each topic and the assessment format. This document is also referred to as a test plan, table of specifications, or test specifications (Raymonda & Grande, 2019). Blueprinting serves as an essential step for enhancing the constructive alignment and validity of assessments by aligning content with specific learning objectives (Boland et al., 2015). Additionally, Pawade et al., (2020)suggest that blueprinting in assessments helps improve content validity by ensuring fair distribution of marks across topics, especially in complex fields like medical education. Finally, (Mathur et al., 2023)discuss how blueprinting in curriculum design helps allocate marks proportionately across various topics, ensuring balanced assessments. Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2604 Media Nusantara University (MNC) was selected for this study due to its inclusion of ecological literacy topics in its critical reading course. This integration is designed to raise students' awareness of environmental issues and encourage them to apply critical reading skills in the context of sustainability. Through these readings, students not only improve their ability to read critically but also gain a deeper understanding of ecological challenges and the importance of sustainable practices. This approach prepares students to engage with academic material in a way that is both intellectually rigorous and socially responsible, fostering a well- rounded educational experience that addresses both academic and environmental concerns. The test results reveal that students’ scores were almost perfect, which suggests that the test may not have been challenging enough. This could indicate that the questions were too simple, preventing students from showcasing their critical reading skills and their ability to engage with more complex, nuanced material. As a result, the test may not have encouraged students to apply deeper levels of analysis or think critically about the content. This highlights the need for researchers to develop a detailed blueprint for assessment. A well-designed blueprint ensures that test questions are structured in a way that challenges students to think critically, engage with complex texts, and demonstrate their analytical abilities. By developing a blueprint, researchers can create assessments that not only evaluate students' ability to understand the material but also their capacity for critical reading, encouraging deeper engagement with academic content. Method The method used in this study is Design and Development Research (DDR), which is particularly appropriate for this research because it systematically supports the iterative design, development, and evaluation of educational artifacts such as assessment specifications. DDR enables researchers to produce practical and effective instructional tools while continuously refining them through feedback and analysis (Seels & Richey, 1994, p. 127). According to Creswell (2009:167), qualitative research is "an interpretive form of inquiry where researchers interpret what they observe, hear, and comprehend." This study employs a qualitative content analysis approach, with the researcher acting as the primary instrument for data collection and interpretation. This study follows the Design and Development Research (DDR) procedure outlined by Nunamaker et al., (1990) and (Ellis & Levy, 2010). The method consists of six steps as it is shown in this following figure IDEAS, Vol. 13, No. 1, June 2025 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 2605 Fiqure 1. The 6-Phase Design and Development Research Approach The first three phases identifying the problem, describing the objectives, and designing and developing the artifact remain consistent with the original model to ensure rigorous problem definition and artifact creation. The modification begins with the fourth phase, where a Focus Group Discussion (FGD) replaces the separate steps of testing the artifact, evaluating testing results, and communicating those results. This change streamlines the process by consolidating feedback and evaluation into a single, collaborative phase, allowing for more in-depth expert insight and discussion on the assessment specification. The implication of this modification is a more flexible and efficient evaluation process that still maintains rigor through expert validation. The FGD involved one experienced lecturer and two experts specializing in educational assessment and test development. Their selection was based on their professional expertise and familiarity with higher education assessment practices. Additionally, five students enrolled in the academic reading course were purposively selected for interviews to provide insight into the usability and challenge level of the current assessment. Ethical approval was secured from the university’s research ethics committee, and all participants gave informed consent, ensuring voluntary participation and confidentiality. The data for this study are collected from course materials and FGD that analyzed qualitatively. The course materials, such as syllabi, reading lists, and previous assessments, are analyzed to evaluate how effectively they address critical reading and ecological literacy. The FGD data focus on gathering feedback about the assessment specification developed by the research. This analysis will combine these findings to improve the assessment specification, ensuring it more effectively evaluates both reading skills and ecological knowledge. The process will provide clear recommendations for refining the assessment, making it more effective in preparing students to critically engage with texts and understand environmental issues. Result Identifying the problem The issue with the academic reading course at Media Nusantara University is that almost all students in the class achieve near-perfect scores in a reading test. This was revealed during class interviews with five students, who mentioned that the questions were too easy to solve. Upon further analysis of the exam questions, it was found that most of them were at cognitive levels C2 and C3, focusing on understanding and application. However, these levels did not sufficiently challenge students' critical thinking skills in depth. Furthermore, it was noted that grammatical aspects were still heavily included in the test, which is not aligned Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2606 with the expected B2 proficiency level. This suggests that the test was not fully focused on assessing the students' ability to think critically or engage with academic texts at the required level. Therefore, there is a need to adjust the difficulty of the questions to ensure that students are assessed more comprehensively, encouraging them to engage more deeply in analyzing and evaluating academic texts. This would help develop their critical thinking abilities in alignment with higher academic standards and the expectations for B2 proficiency. Furthermore, according to the learning outcomes (CPL) of the English Department, students must achieve at least a B2 level on the CEFR scale, which means that they should be able to read and understand a wide variety of texts with a large degree of independence. This includes adapting their reading style and speed depending on the text's complexity, while also using appropriate reference tools selectively. At the B2 level, students should be able to analyze and critically engage with texts, recognizing different viewpoints and understanding complex arguments. In addition, they must demonstrate the ability to evaluate and interpret academic material critically, which is essential for their academic development and aligns with the learning objectives of the course. However, the current state of the class's critical reading skills is far below the CEFR B2 level. Describing the Objective The researcher intends to create a thorough assessment blueprint for the critical reading course at Media Nusantara University, with the goal of improving students' ability to critically analyze texts, while also incorporating elements of Eco literacy. Design and Develop the Artifact To conduct this step, researcher follow the steps proposed by (Raymonda & Grande, 2019) as it is shown below: 1. Identify major knowledge and skill domains In order to design an effective assessment tool for the Critical Reading course, the researcher begins by analyzing the course syllabus to identify the major knowledge and skills that students are expected to develop. This analysis helps in defining the learning outcomes and aligning the assessment specifications with the course objectives. Table 1 presents the Critical Reading and Analysis Skills Learning Outcomes as derived from the syllabus, outlining the key abilities students are expected to demonstrate upon completing the course. Table 1. Critical Reading and Analysis Skills Learning Outcomes Course Description Learning Outcome IDEAS, Vol. 13, No. 1, June 2025 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 2607 This subject helps students develop critical thinking skills and encourages them to form reasoned arguments by critically responding to various types of texts they encounter at the university level. The material covers (1) how to evaluate the evidence or arguments presented, (2) how to identify any influences affecting the evidence or arguments, (3) how to assess the limitations or focus of the text, and (4) how to examine the assumptions or implications made by the author. Students will also be required to respond to texts. Classroom activities will involve group work and individual assignments. Able to apply the basic concepts and theories of Linguistics in English to respond to various linguistic phenomena. Able to produce adequate translations and language transfers of both spoken and written texts from English to Indonesian and vice versa. Able to demonstrate English language proficiency, as evidenced by achieving a score that corresponds to at least CEFR level B2. In the Critical Reading course, students need to understand not only the vocabulary and grammar of a text, but also its meaning, including any environmental themes or issues. The researcher then determines how the test is a reading into writing task. (Chan et al., 2015) explain that this task involves integrating information from multiple reading sources into written responses. This is commonly seen in academic settings, where students are asked to synthesize information from various texts and write an organized response that addresses specific issues. 2. Delineate the assessment objectives In the second stage of developing the test specification, the researcher focuses on delineating the assessment objectives, which involves defining the specific learning outcomes and behaviors the test will evaluate. Assessments align with clearly defined learning outcomes help teachers to evaluate students' performance accurately (Bara & Bara, 2023). At this stage, the researcher translates the sub learning outcomes from the course syllabus into measurable indicators for each learning outcome. These indicators serve as a clear and specific guide for assessing whether students have achieved the intended outcomes. For this course, the researcher develops indicators that cover both critical reading skills and ecological literacy, as these are central to the course's learning goals. The indicators are derived from the sub learning outcomes outlined in the syllabus, ensuring that all Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2608 major knowledge and skill domains are addressed. Table 2. Sub Learning Outcomes Sub Learning Outcome 1. Students are able to interpret ideas and messages in the text 2. Students are able to relate texts to current global and local issues 3. Students are able to critique texts by identifying biases, assumptions, or unspoken points in the text 4. Students are able to evaluate arguments in the text 5. Students are able to propose feasible solutions based on the issues in the text 3. Decide on the assessment format This step involves selecting the most appropriate method to assess students' critical reading skills and their ecological literacy, ensuring that the format chosen reflects the knowledge and skills being tested. One important factor in deciding the assessment method is the location of the assessment objective within Bloom’s Taxonomy. Multiple Choice Questions (MCQs) are effective for testing basic comprehension and recall, which are at the lower levels of Bloom’s Taxonomy (Netere et al., 2024). However, for assessing higher order thinking skills for critical reading and ecological literacy, methods such as essays or argument analysis would be more suitable. Another consideration is the reliability of scores produced by the method. While MCQs provide reliable and objective results, they may not be sufficient for evaluating higher order skills such as critical thinking or the application of ecological literacy. In contrast, essays and open-ended questions can provide a deeper assessment of these skills, but they require careful and consistent grading, often using a rubric to maintain fairness. Thus, a combination of MCQs for comprehension and essays for analytical tasks would improve the reliability of the test. The validity of the test also needs to be considered. The method chosen must effectively measure the intended skills. MCQs are good for testing comprehension and knowledge recall but may not capture complex cognitive skills like evaluating an argument or analyzing ecological issues. Therefore, incorporating essays or argument analysis questions would ensure the test remains valid by evaluating students' deeper understanding and ability to critically engage with the content. Practical constraints such as time also play a role in determining the assessment format. While MCQs can be graded quickly and are less resource-intensive, essays or IDEAS, Vol. 13, No. 1, June 2025 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 2609 short-answer questions require more time to grade and may require additional resources. The assessment format should strike a balance between effectively measuring student learning and being feasible within the constraints of the course. Researcher also considers the curricular context—how the assessment fits within the broader course structure. If earlier assessments have already evaluated basic reading comprehension through MCQs, the final assessment could focus more on higher-order thinking skills, such as argument analysis and ecological literacy, ensuring a comprehensive assessment of students’ progress throughout the course. 4. Specify the category weights In the fourth stage of developing a test specification for the Critical Reading course, the focus is on specifying the category weights to ensure the test is balanced and reflective of the course’s objectives. The first step in this process is determine how many assessment tasks students can reasonably complete within the allotted time. This helps ensure that the test is both feasible and fair. The test will be held for 120 minutes, then researcher decides how many multiple-choice questions can be included without sacrificing the quality of assessment for tasks that require deeper analysis, such as essays or case studies. Before receiving feedback from experts, the researcher decides to use 15 multiple-choice questions and 5 essay questions. Once the number of test items is decided, the next step is to weight each major category or domain in the test blueprint based on its overall importance. Lastly researcher verifies that the number of assessment tasks aligns with the assigned category weights. This means ensuring that there are enough tasks in each domain to support the claims you wish to make about students' performance. Researcher decided that test should include enough essay-based tasks. After completing the four stages, an assessment specification was created, adapting the table developed by (Fitriani , 2018 and Nurkhamdiah, 2019) Focus Group Discussion In the FGD with one lecturer and two experts, several key recommendations were made for the product. The lecturer expressed concerns about relying too much on multiple choice questions saying that while they are good for testing basic understanding, they don't challenge students to think critically. The lecturer suggested using fewer multiple-choice questions and adding more open-ended questions to encourage deeper analysis and evaluation of the text. One of the experts agreed and recommended including multiple case studies on ecological issues. This would give students a range of real-world situations to analyze, helping them apply both critical reading and ecological literacy. The expert emphasized that using several case studies would allow students to consider Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2610 different perspectives and better engage with complex environmental problems. Additionally, the group suggested that the cognitive levels in the assessment should be more evenly distributed, rather than focusing too much on lower levels. It was recommended to increase the proportion of higher-level cognitive tasks to better challenge students and promote deeper critical thinking and analysis. This would ensure that the assessment fosters a more comprehensive understanding of the material and encourages students to engage at a higher intellectual level. The expert also suggested developing an assessment rubric where the writing aspect is evaluated, but it should not outweigh the focus on comprehension. Design Revision After receiving valuable feedback and suggestions during the Focus Group Discussion (FGD), the researcher made several revisions to the assessment specification she had developed. The feedback highlighted the need to reduce the reliance on multiple-choice questions and incorporate more open-ended questions to foster deeper analysis and evaluation. Additionally, the suggestion to include multiple case studies related to ecological issues was adopted to provide students with a broader range of real-world scenarios to engage with. Furthermore, based on the feedback regarding cognitive levels, the researcher ensured that the assessment included a more balanced distribution of cognitive levels, with an emphasis on higher-order thinking tasks. This revision aimed to encourage students to engage in critical thinking and apply advanced analytical skills to the material. With these revisions, the researcher finalized the assessment specification, ensuring it was more aligned with the learning objectives of the course and better equipped to evaluate both critical reading and ecological literacy. The final of assessment specification is shown in this following table. Table 3. The Assessment Specification No Sub Learning Outcomes Topic Cognitive Level (Bloom's Taxonomy) Indicator Type of Question Number of Questions Score Time (Minutes) 1 Students are able to interpret ideas and messages in the text Recycling Practices C3 - Synthesis Students are able to determine key information of the text Multiple Choice Question (MCQ) 5 10 10 IDEAS, Vol. 13, No. 1, June 2025 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 2611 Students are able to summarize the message of the text Multiple Choice Question (MCQ) 5 10 10 2 Students are able to relate texts to current global and local issues Deforestation C3 - Synthesis Students are able to relate texts to current global and local issues Essay 2 10 20 3 Students are able to critique texts by identifying biases, assumptions, or unspoken points in the text Chemical Pollution C4 - Analysis Students are able to critique texts Essay 1 10 10 Students are able to identify biases or unspoken assumptions Essay 2 10 20 Students are able to analyze the impact of biases or assumptions on argument validity Essay 1 10 10 4 Students are able to evaluate arguments in the text Marine Life C5 - Evaluation Students are able to evaluate the strength of evidence presented in Case Study 2 20 20 Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2612 the argument 5 Students are able to propose feasible solutions based on the issues in the text Global Warming and Climate Change C6 - Creation Students are able to propose feasible solutions based on the issues in the text Case Study 2 20 20 As suggested by the expert, the researchers developed rubrics for each sub-CPMK in the test as follows: Indicator 1. Students are able to interpret ideas and messages in the text Topic : Recycling practices Question type: Multiple choice question Table 4. Rubric for Indicator 1 Criterion Score 2 Score 0 Accuracy of the answer with the answer key Answer aligns with the key Answer does not align with the key Indicator 2. Students are able to relate texts to current global and local issues Topic : Deforestation Question type: Essay Table 5. Rubric for Indicator 2 Criterion Score 4 (Excellent) Score 3 (Good) Score 2 (Needs Improvement) Score 1 (Poor) The relation of Content & Ideas with current global and local issues Clear, focused, and insightful. All points are well- developed with Clear and relevant ideas, but some points are underdevelope d or lack depth. Ideas are somewhat developed but lack sufficient detail or clarity. Ideas are unclear, underdevelope d, or irrelevant to the topic. IDEAS, Vol. 13, No. 1, June 2025 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 2613 thorough explanation. Writing Quality & Grammar Writing is clear, well- organized, and cohesive. The argument is supported with strong evidence. The grammar, punctuation, and spelling are flawless or nearly flawless. Writing is mostly clear and coherent with minor issues in organization or transitions. Few minor grammar or punctuation errors that do not hinder understanding. Writing has organizational issues, unclear or awkward phrasing, and some noticeable grammar or punctuation errors. The argument is only partially supported by evidence. Writing is unclear, disorganized, and difficult to follow. Numerous grammars or punctuation errors hinder readability. The argument lacks sufficient evidence or is poorly structured. Indicator 3. Students are able to critique texts by identifying biases, assumptions, or unspoken points in the text Topic : Chemical Pollution Question type: Essay Table 6. Rubric for Indicator 3 Criterion Score 4 (Excellent) Score 3 (Good) Score 2 (Needs Improvement) Score 1 (Poor) Critiquing Texts Provides a thorough, thoughtful critique addressing all aspects of the text. Provides a clear critique but may miss minor details or aspects of the text. Critique is basic, addressing surface-level aspects without depth. Critique lacks depth, or is unclear or superficial. Identifying Biases/Unspoken Assumptions Identifies multiple biases or assumptions Identifies some biases or assumptions, but may not Identifies few biases/assump tions with minimal Fails to identify biases or assumptions, or identifies Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2614 with clear evidence. fully explain their impact. explanation. them inaccurately. Analyzing the Impact of Biases/Assumptio ns on Argument Validity Provides a thorough analysis of how biases or assumptions affect argument validity. Analyzes the impact, but some points may lack full development or clarity. Provides a limited analysis, lacking depth in understanding bias/assumptio n impact. Fails to analyze or provides incorrect analysis of the impact. Writing Quality & Grammar Writing is clear, well- organized, and cohesive. The argument is supported with strong evidence. The grammar, punctuation, and spelling are flawless or nearly flawless. Writing is mostly clear and coherent with minor issues in organization or transitions. Few minor grammar or punctuation errors that do not hinder understanding. Writing has organizational issues, unclear or awkward phrasing, and some noticeable grammar or punctuation errors. The argument is only partially supported by evidence. Writing is unclear, disorganized, and difficult to follow. Numerous grammars or punctuation errors hinder readability. The argument lacks sufficient evidence or is poorly structured. Indicator 4. Students are able to evaluate arguments in the text Topic : Marine Life Question type: Essay Table 7. Rubric for Indicator 4 Criterion Score 4 (Excellent) Score 3 (Good) Score 2 (Needs Improvement) Score 1 (Poor) Evaluating the Strength of Evidence in the Argument Effectively evaluates and analyzes multiple Evaluates evidence with some analysis, but the justification of Evaluates some evidence, but the analysis is weak or unclear Fails to evaluate evidence adequately, or the analysis is IDEAS, Vol. 13, No. 1, June 2025 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 2615 pieces of evidence, providing a clear justification of their relevance and strength. relevance and strength is not fully developed. regarding relevance or strength. incorrect or missing. Writing Quality & Grammar Writing is clear, well- organized, and cohesive. The argument is supported with strong evidence. The grammar, punctuation, and spelling are flawless or nearly flawless. Writing is mostly clear and coherent with minor issues in organization or transitions. Few minor grammar or punctuation errors that do not hinder understanding. Writing has organizational issues, unclear or awkward phrasing, and some noticeable grammar or punctuation errors. The argument is only partially supported by evidence. Writing is unclear, disorganized, and difficult to follow. Numerous grammars or punctuation errors hinder readability. The argument lacks sufficient evidence or is poorly structured. Indicator 5. Students are able to propose feasible solutions based on the issues in the text Topic : Global Warming and Climate Change Question type: Essay Table 8. Rubric for Indicator 5 Criterion Score 4 (Excellent) Score 3 (Good) Score 2 (Needs Improvement) Score 1 (Poor) Proposing Feasible Solutions Proposes clear, well- thought-out solutions that are practical, Proposes relevant solutions that address the issues, but some Proposes solutions that are somewhat relevant, but the connections to Proposes solutions that are irrelevant, impractical, or not directly Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2616 feasible, and directly address the issues identified in the text. The solutions are thoroughly explained with strong reasoning and evidence. may lack full development or clarity in terms of feasibility or reasoning. the issues are unclear or the solutions are not fully feasible. related to the issues in the text. Writing Quality & Grammar Writing is clear, well- organized, and cohesive. The argument is supported with strong evidence. The grammar, punctuation, and spelling are flawless or nearly flawless. Writing is mostly clear and coherent with minor issues in organization or transitions. Few minor grammar or punctuation errors that do not hinder understanding. Writing has organizational issues, unclear or awkward phrasing, and some noticeable grammar or punctuation errors. The argument is only partially supported by evidence. Writing is unclear, disorganized, and difficult to follow. Numerous grammars or punctuation errors hinder readability. The argument lacks sufficient evidence or is poorly structured. Discussion The findings reveal that the original academic reading assessment at Media Nusantara University was inadequate in effectively measuring students’ critical reading skills, as indicated by the overwhelmingly high scores that failed to discriminate among varying levels of student ability. This inadequacy aligns with prior research indicating that Indonesian university students often struggle to move beyond literal comprehension to engage in deeper critical analysis (Hanum et al., 2020; Jumiari et al., 2024; Sultan et al., 2018). The dominance of questions at cognitive levels C2 and C3, focusing on understanding and application, fell short of the higher-order thinking skills (C4–C6) that are essential for academic and professional success (Arifin, 2020; Gönen & Kızılay, 2022). Additionally, the emphasis on grammatical components conflicted with the course’s goal of achieving CEFR B2-level proficiency, which requires independent, critical IDEAS, Vol. 13, No. 1, June 2025 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 2617 engagement with complex texts (Shamida et al., 2021) The revision of the assessment specification directly responds to these limitations by rebalancing cognitive demands through the incorporation of more essay and case study questions targeting synthesis, analysis, evaluation, and creation. This revision reflects educational assessment theories advocating for constructive alignment, where test content must closely match intended learning outcomes to ensure validity and reliability (Boland et al., 2015; Raymond & Grande, 2019). Moreover, integrating ecological literacy into the assessment expands the scope of critical literacy, resonating with interdisciplinary frameworks that combine critical reading with environmental awareness to prepare students for contemporary global challenges. (Chambers & Radbourne, 2014; Kerçin & Demir, 2024b; Silvhiany & Huzaifah, 2020).This integration enhances students' ability to critically engage with texts within real-world ecological contexts, an approach rarely addressed in existing assessments (Häggström & Schmidt, 2020). Despite these advancements, several limitations must be acknowledged. The Focus Group Discussion included only three experts, and student feedback was limited to five participants, potentially restricting the representativeness of the findings. The absence of large-scale pilot testing and psychometric analysis means the revised assessment’s effectiveness and reliability remain to be empirically validated in broader settings. Future research should address these gaps by involving a larger, more diverse sample and employing quantitative methods to substantiate the assessment’s validity and impact on student learning outcomes. From a practical standpoint, implementing the revised assessment involves challenges. Essay and case study formats require more grading time and evaluator expertise, necessitating well-developed rubrics and grader calibration to maintain fairness and consistency. The design must also balance cognitive rigor with feasibility, considering course time constraints and resource availability. Furthermore, ecological case studies must remain relevant and updated to sustain student engagement and reflect current environmental issues. Institutional support in faculty training and curriculum integration will be crucial to the sustainable adoption of this interdisciplinary assessment approach. In summary, this study contributes to educational assessment by developing a more comprehensive, cognitively balanced, and ecologically informed critical reading assessment. By addressing documented shortcomings in prior assessments and aligning with contemporary critical literacy frameworks, the revised specification better equips students to meet academic standards and engage meaningfully with pressing ecological issues, fostering both intellectual and social responsibility. Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2618 Conclusion This study successfully developed a comprehensive assessment specification for the Critical Reading course at Media Nusantara University that integrates ecological literacy alongside traditional critical reading skills. The revised assessment addresses the limitations of the previous test by incorporating higher- order cognitive tasks and real-world ecological case studies, fostering deeper critical thinking and environmental awareness among students. For practice, this assessment framework offers educators a practical and balanced tool to evaluate students’ critical reading abilities more effectively, encouraging analytical engagement with academic texts and ecological issues. The inclusion of well- defined rubrics supports consistent and fair grading, which can improve the reliability of assessments across different instructors and cohorts. Institutions seeking to enhance interdisciplinary competencies in their curricula may consider adopting similar integrated assessment approaches. For future research, further empirical validation of this assessment tool through large- scale pilot testing is necessary to confirm its reliability, validity, and impact on student learning outcomes. Investigations into how students’ ecological literacy develops through such integrated assessments could provide valuable insights into optimizing curriculum design. Additionally, exploring the scalability and adaptability of this framework across different disciplines and cultural contexts would contribute to broadening its applicability and relevance. References Abduganiev, O. I., & Abdurakhmanov, G. Z. (2020). Ecological Education For The Purposes Sustainable Development. The American Journal of Social Science and Education Innovations, 02(08), 280–284. https://doi.org/10.37547/tajssei/Volume02Issue08-45 Arifin, S. (2020). The Role of Critical Reading to Promote Students’ Critical Thinking and Reading Comprehension. Jurnal Pendidikan Dan Pengajaran, 53(3), 318– 326. https://doi.org/10.23887/jpp.v53i3.29210 Balan, S., Katenga, J. E., & Simon, A. (2019). Reading Habits and Their Influence on Academic Achievement Among Students at Asia Pacific International University. 11th International Scholars Conference, 7(1), 1490-1516. https://doi.org/10.35974/isc.v7i1.928 Bara, E., & Bara, G. (2023). The connection between assessment and learning outcomes. Balkan Journal of Interdisciplinary Research, 9(2), 51–58. https://doi.org/10.2478/bjir-2023-0006 Blaabæk, E. H. (2020). Reading when the sun does not shine: The effect of reading on children’s academic performance. Research in Social Stratification and Mobility, 67, 100485. https://doi.org/10.1016/j.rssm.2020.100485 Boland, J., Finn, Y., & Geoghegan, R. (2014). Blueprinting assessment to enhance IDEAS, Vol. 13, No. 1, June 2025 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 2619 constructive alignment. In 16th Ottawa Conference, Transforming Healthcare through Excellence in Assessment and Evaluation (pp. 25–29). Ontario, Canada. https://doi.org/10.13140/RG.2.1.1913.0724 Bordag, D., Opitz, A., Polter, M., & Meng, M. (2021). Non-native Readers Are More Sensitive to Changes in Surface Linguistic Information than Native Readers. Bilingualism, 24(4), 599–611. https://doi.org/10.1017/S1366728920000772 Chambers, J. M., & Radbourne, C. (2014). Teaching Critical Literacy Skills Through the Natural Environment as Text. Applied Environmental Education and Communication, 13(2), 120–127. https://doi.org/10.1080/1533015X.2014.950888 Chan, S., Inoue, C., & Taylor, L. (2015). Developing rubrics to assess the reading- into-writing skills: A case study. Assessing Writing, 26, 20–37. https://doi.org/10.1016/j.asw.2015.07.004 Creswell, J. W. (2012). Education research: Planning, Conducting, And Evaluating Quantitative and Qualitative. Pearson Dardjito, H., Rolls, N., Setiawan, A., & Sumekto, D. R. (2023). Challenges in Reading English Academic Texts for Non-English Major Students of an Indonesian University. Studies in English Language and Education, 10(3), 1290–1308. https://doi.org/10.24815/siele.v10i3.29067 Din, M. (2020). Evaluating university students’ critical thinking ability as reflected in their critical reading skill: A study at bachelor level in Pakistan. Thinking Skills and Creativity, 35, 100627. https://doi.org/10.1016/j.tsc.2020.100627 Dudina, O. (2024). Fostering Critical Reading and Research Skills in Medical Students through English for Careers Classes. Revista Romaneasca Pentru Educatie Multidimensionala, 16(1), 172–183. https://doi.org/10.18662/rrem/16.1/817 Ellis, T., & Levy, Y. (2010). A Guide for Novice Researchers: Design and Development Research Methods. https://www.researchgate.net/publication/228411189 Ellis, T., & Levy, Y. (2010). A guide for novice researchers: Design and development research methods. International Journal of Doctoral Studies, 5, 107-118. https://doi.org/10.28945/1237 Fitriani, D. (2018). Development of an assessment framework based on the European Profiling Grid (EPG) for assessment competencies in the Bachelor's degree program of English Education (Master’s thesis, Universitas Negeri Jakarta) http://repository.unj.ac.id/1153/1/Dian%20Fitriani_2236159095.pdf Gönen, S. I. K., & Kızılay, Y. (2022). Reading beyond the lines: Teaching critical reading in higher education. The Reading Matrix: An International Online Journal, 22(1), 73-92. https://www.readingmatrix.com/files/27- Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2620 mb6vs3ty.pdf Häggström, M., & Schmidt, C. (2020). Enhancing children’s literacy and ecological literacy through critical place-based pedagogy. Environmental Education Research, 26(12), 1729–1745. https://doi.org/10.1080/13504622.2020.1812537 Hanum, M., Wahyuni, S., & Maulizan, M. (2020). A DESCRIPTIVE OF STUDENTS CRITICAL READING SKILLS IN NARRATIVE TEXTS. Jurnal Ilmiah Mahasiswa Pendidikan, 1(1). https://jim.bbg.ac.id/pendidikan/article/view/17 Hartono, R. (2020). Evaluating Sustainable Education Using Eco-Literacy. HABITAT, 31(2), 78–85. https://doi.org/10.21776/ub.habitat.2020.031.2.9 Hibatullah, O. F. (2019). The Challenges of International EFL Students to Learn English in a Non-English Speaking Country. Journal of Foreign Languange Teaching and Learning, 4(2). https://doi.org/10.18196/ftl.4240 Hicks, T. (2023). The Effect of Reading Habits on Academic Success. Culminating Experience Projects. 389. https://scholarworks.gvsu.edu/gradprojects/389 Jumiari, J., Nasrullah, Sutiono, C., & Khairin Utomo, M. (2024). A STUDY ON EFL STUDENTS’ LEVELS OF CRITICAL THINKING SKILLS. Celtic: A Journal of Culture, English Language Teaching, Literature and Linguistics, 11(2). https://doi.org/10.22219/celtic.v11i2 Kerçin, D., & Demir, M. Ş. (2024). An Integrative and Interdisciplinary Assessment of Environmental, Ecological and Eco-Literacy. Journal of Computer and Education Research, 12(24), 383–402. https://doi.org/10.18009/jcer.1458854 Netere, A. K., Hughes, T., Babey, A. M., Hawes, M., Mifsud, J., Kelly, J. P., Liang, W., Hernandez, M., Karpa, K., Al-Sallami, H., Fernandes, L. B., Aronsson, P., Restini, C., Caetano Crowley, F., Djouma, E., Hinton, T., Liu, J. J., Mraiche, F., & White, P. J. (2024). Evaluating the quality of multiple-choice question pilot database: A global educator-created tool for concept-based pharmacology learning. Pharmacology Research and Perspectives, 12(5). https://doi.org/10.1002/prp2.70004 Nunamaker Jr, J. F., Chen, M., & Purdin, T. D. (1990). Systems development in information systems research. Journal of management information systems, 7(3), 89-106. https://www.jstor.org/stable/40397957 Nurkhamdiah, N. (2019). Developing Assessment Specifications on the Basis of European Profiling Grid (Epg) Digital Media Enabling Competences for Undergraduate English Education Program. https://journal.uhamka.ac.id/index.php/uicell/article/view/5110 Masruddin, M., Amir, F., Langaji, A., & Rusdiansyah, R. (2023). Conceptualizing linguistic politeness in light of age. International Journal of Society, Culture & Language, 11(3), 41-55. Pawade, Y. R., Mehta, S., Mahajan, A. S., Patil, S. N., Barua, P., Desai, C., & Supe, A. N. IDEAS, Vol. 13, No. 1, June 2025 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 2621 (2020). ‘Blueprinting in assessment’ – an online learning experience. South- East Asian Journal of Medical Education, 13(2), 77. https://doi.org/10.4038/seajme.v13i2.214 Raymond, M. R., & Grande, J. P. (2019). A practical guide to test blueprinting. Medical Teacher, 41(8), 909–915. https://doi.org/10.1080/0142159X.2019.1595556 Sari, D. M. (2023). Analysis Students Difficulty In Reading English Article. Jotika Journal in Education, 3(1), 27-31. https://doi.org/10.56445/jje.v3i1.107 Seels, B. B., & Richey, R. C. (2012). Instructional technology: The definition and domains of the field. Iap. Shamida, A., Sidhu, G. K., & Nawi, S. M. (2021). Postgraduate Students’ Perceived and Actual Performance in Critical Reading Skills. Asian Journal of University Education, 17(3), 76–84. https://doi.org/10.24191/ajue.v17i3.14518 Silvhiany, S., & Huzaifah, S. (2021, January). Developing Online Critical Literacy Assessment for EFL Learners. In 4th Sriwijaya University Learning and Education International Conference (SULE-IC 2020) (pp. 378-386). Atlantis Press. https://doi.org/10.2991/assehr.k.201230.134 Srikrai, P. S., Lin, D., Lakaisone, S., & Sirinthorn, S. (2016). English language difficulties of non-native English postgraduate students in an English for academic purposes at a Thai university. Proceedings of CLaSIC, 301-315. https://fass.nus.edu.sg/cls/wp- content/uploads/sites/32/2020/10/srikrai_suphawat_phiphawin.pdf Sudarwati, E., & Manipuspika, Y. S. (2021). Improving Students’ Critical Reading Ability: Cooperative Learning Strategy. Journal of English Language Teaching and English Linguistics, 6(1). https://journal.upy.ac.id/index.php/eltics/article/download/539/913/321 2 Sultan, Rofiuddin, A., Nurhadi, & Priyatni, E. T. (2018). Development of mass media text-based instructional materials to improve critical reading skills of university students. Pedagogika, 131(3), 26–47. https://doi.org/10.15823/p.2018.32 urviyani, V. (2018). Fostering College Student’s Critical Reading Skill Through Padlet Application. Bahtera: Jurnal Pendidikan Bahasa Dan Sastra, 17(1), 66– 73. https://doi.org/10.21009/BAHTERA.171.5 Usman, N., Hendrik, H., & Madehang, M. (2024). Difficulties in understanding the TOEFL reading test of english language education study program at university. IDEAS: Journal on English Language Teaching and Learning, Linguistics and Literature, 12(1), 755-773. Neni Nurkhamidah, Ninuk Lustyantie, Fathiaty Murtadho Developing a Critical Reading Assessment Specification Infused with Ecological Literacy for Higher Education 2622