ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 10, Issue 2, June 2022 https://journal.uniku.ac.id/index.php/ERJEE 629 INTEGRATING ONLINE READING ACTIVITY IN A CRITICAL READING COURSE: A COURSE EVALUATION Sri Kusuma Ningsih English Education Department, Faculty of Teacher Training and Pedagogy, University of Muhammadiyah Prof. DR. HAMKA, East Jakarta, Indonesia) Email: sri_kusuma@uhamka.ac.id APA Citation: Ningsih, S. K. (2022). Integrating online reading activity in a critical reading course: a course evaluation. English Review: Journal of English Education, 10(2), 629-636. https://doi.org/10.25134/erjee.v10i2.6285. Received: 01-02-2022 Accepted: 29-04-2022 Published: 30-06-2022 INTRODUCTION In the present day, readers can get information easily from digital sources. By accessing digital sources, people have some light benefits. They can enrich their knowledge and skills. Most people use digital sources for their needs everywhere and every day. Manalu (2019) explores the expectations of students in reading digital texts. It has been noticed from the study that the participants felt good reading digital texts. They enjoy reading digital texts because they are inspiring, informative, and in line with the way they are reading. A study by Gilbert (2017) shows that language learners engage in characteristically different reading activities and techniques while reading web text and that digital literacy skills are required in conjunction with the development of traditional target language literacy skills. Online reading for students offers advantages in improving their understanding, and the availability of e-books promotes better reading habits and enhances motivation. Huang (2013) examines the expectations of students about an e-book reading program. The findings indicate that students received positive feedback on the program's strategy resources and learning effects. They pointed out the ability of e-books to promote better reading habits and motivation. They also viewed e- books as more affordable, portable, and environmentally friendly than print-based texts. Additionally, Ciampa (2012) explores and investigates the effect on reading motivation and listening comprehension of online e-books among six grade 1 students aged seven from Ontario, Canada. The study also indicates that, after using the online e-book reading program, all participants improved their understanding scores from pretest to post-test, enjoyed the online reading experience, and regularly read the online e-books at home during their free time. Then, Yeh, Hung, and Chiang (2017) analyze how students use annotations online during their reading processes and how such usage can contribute to their progress. The finding shows that the students strengthened their reading comprehension after the intervention. Previous researcher shows that students of all ages and levels of experience generally prefer online textbooks to paper (Lin, 2019). For example, Aldaihani et al. (2015) analyze English programs taught at the College of Technology Studies (CTS), the Public Authority for Applied Education and Training (PAAET) in Kuwait. The study found that the participants consider the English program effective in improving their English proficiency. However, they also pointed to some inadequacies that appear to be related to the programs. Abstract: Advancement of information and communication technology (ICT) has promoted changes on students’ reading behavior; from conventional reading and online reading. In the current study, online reading activity was integrated in a critical reading course in an English department of a private university. In the course, fifty-eight EFL students were exposed with paper-based reading and online reading. After eight week, an evaluation was performed to the course. All participating students were surveyed and twelve of whom were interviewed. The quantitative data from the survey were analysed using Rasch Model while thematic analysis was performed to the qualitative data. Findings showed students’ preference to online reading activity than paper-based reading, particularly for those who were already familiar with electronic reading devices. Online reading was perceived to provide students with flexibility while reading and helped them navigate the reading sections at their ease. Small screen size and lights from the screen appeared to issue students’ eye irritation. Recommendation thus was offered particularly regarding further curriculum development that integrate online reading activity in a reading course. Keywords: online reading activity; reading course evaluation. Sri Kusuma Ningsih Integrating online reading activity in a critical reading course: a course evaluation 630 Examination of previous studies as discussed above has presented two critical issues, convenient sample size and students' perceptions of using digital text for more specific language learning. A study by Ciampa (2012) states that it would be better if the sample used was larger and varied and checked the use of online e-books for early readers. Also, Manalu (2019) states that to make generalizations clearer, the research must include a greater number of participants from different educational institutions. Examination of previous studies, as discussed previously, has presented two critical issues, convenient sample size and students' perceptions of using digital text for more specific language learning. A study by Ciampa (2012) states that it would be better if the sample used is larger and also varied and check the use of online e-books for early readers. Students need to improve their skills alongside the advancement of technology in all fields, including learning. In this era, reading and learning activity is very easy to do with the many technologies that promote and are readily available to enhance students' reading skills. Moreover, Hazaea and Alzubi (2016) said that the practices of EFL reading can be changed by using mobiles. Mobile phones give students opportunities to practice reading independently and seek feedback. Manalu (2019) indicates that advances in technology that continue to evolve trigger accelerating changes in almost every life. Moreover, technology has changed the way we do things and is no exception in reading. Technology makes knowledge readily available, and we can access information such as online news, Wikipedia, articles, etc., with the Internet. The digital text provides a modern reading experience and enables readers to communicate by displaying document types, fonts, and colours. The transformation of world technology in this era has led to increased use of online information in academic activities. This makes online reading become learning material and teaching material for most institutions (Wu and Chen, 2011). As Taki and Soleimani (2012) investigated online reading, digital text is one of the media used in online reading. Digital text can also be interpreted as a paper-based test that is made into a digital text version that can be accessed through a technology device (Huang, Liaw, & Lai, 2016). Lin (2019) described digital text as the text specially designed to be used through a technological device with various features, enabling it to use mobile phones, help overcome students’ stress over financial issues and encourage them to get to know more about copyright issues. Research conducted by experts on students' perceptions of online reading within the academic environment still causes debate because besides having a positive impact, online reading also gets a negative perception. Pasaribu (2020) investigated 25 students’ responses to online reading classes. The data were collected from online records, reflections, and questionnaires. The results yield that online reading and tasks have promoted their autonomous learning and get in touch in meaningful learning encounters that they were able to plan, carry out, and evaluate their learning. Despite the advantages and importance of online reading mentioned earlier, students expressed the opposite in the study of Mizrachi and Chou. The study of Mizrachi (2014) investigated that students prefer printed text for longer and more extensive readings, whereas for fewer readings, they feel comfortable using both printed and digital text. This is because staring at the screen for too long when reading online will cause the eyes easier to fatigue and also causes a decrease in learning outcomes. Despite the advantages and importance of online reading mentioned earlier, however, in the study of Mizrachi and Chou, students expressed the opposite. In the study of Mizrachi (2014), investigated that students prefer printed text for longer and more extensive readings, whereas for fewer readings, they feel comfortable using both printed and digital text. This is because staring at the screen for too long when reading online will cause the eyes easier to fatigue and also causes a decrease in learning outcomes. The evaluation was needed to give the idea that the study was significant. The definition of evaluation has been explained in(Murphy et al., (2018) study. There have been previous studies on a digital text that influence online reading can increase motivation and vocabulary of foreign languages (Chen et al., 2013). The evaluation of language programs is a way of identifying practical values to be able to explain the unity of interventions of the developer intervention and inhibitory factors, the type of language, and the consequences that will occur. Besides that, he said evaluation is different from measurement (collection of information about personal knowledge, expertise, and ability) and assessment (quantifiable measures of theoretical constructs), specifically in its context, intention, and fundamentally multi-methodological epistemology (Norris, 2016). The latest advancement of the issue is the context of the research and evaluation of the online ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 10, Issue 2, June 2022 https://journal.uniku.ac.id/index.php/ERJEE 631 reading understanding literacy system. The current study could be the potential for social-collaborative learning, where learners can display feedback from other users and access various perspectives and interpretations of the texts. In addition, students can improve their comprehension of the text and critical thinking skills by involving them in making annotations. There is plenty of research on online reading (e.g., Aldaihani et al., 2015; Huang, 2013). However, assessing the effectiveness of online reading activity in the course and the contributing factors to the students’ reading activity is rarely conducted in Indonesia. The current study was aimed to evaluate a Critical Reading course in a private university in Jakarta where an online reading activity was integrated into the course activity. The course evaluation included the effectiveness of online reading activity in the course, students’ behaviour toward online reading activity, and the contributing factors of such behaviour. Thus, the study aims to address the following research questions: (1) How do students perceive the online reading activity integrated into the Critical Reading course? (2) What are the contributing factors influencing students’ online reading activity? This study focused on evaluating the effectiveness of an online reading activity implemented under the Critical Reading Course activity at the English Education Department. The findings of the current study will contribute theoretically and practically. Theoretical signification would support the theories on Curriculum Development, especially about evaluating online reading activity in a reading course. Practically, the result of the study will have a beneficial insight for teachers to reconsider the strategy of teaching academic reading. METHOD Critical Reading is a required course for all 2nd- year students. In this course, students will learn reading strategies to understand and retain information, the organization of reading passages, and strategies for learning and retaining vocabulary. Building on these strategies, students will develop skills to analyse texts critically. In addition, students will practice and develop skills for paraphrasing and summarizing. A Critical Reading course has 14 meetings, and each meeting covers 4 credit hours or 200 minutes per week. During the first seven meetings, students learn conventionally, using printed texts during the course. For the rest of the seven meetings, they read using digital texts. They are allowed to use various online reading devices, such as smartphones, tablets, laptops, computers, and others. In the first seven meetings of reading course, students needed to prepare for class by reading the book entitled “Mindscapes: Critical Reading Skills and Strategies. In the class, they discussed a specific topic. For the rest of the seven meetings, students did online exercises. Doing the online reading, the lecturer provided an account for each student to access the website. On the website, there were several course options. Before starting online reading, the lecturer first gave directions. If students had not reached the minimum score, the lecturer would provide extra time so that students could try other exercises until they got a minimum score. The main aim of this study is to evaluate a Critical Reading course where an online reading activity was integrated into the course activity. Particularly, it addressed two research questions: 1) How do students perceive the online reading activity integrated into the Critical Reading Course? And 2) What factors influence students’ online reading activity? A mixed-methods design was employed to answer the research questions. The quantitative method with survey design and qualitative research with an interview were employed to explore students' perspectives about online reading activity and the contributing factors of such behaviour. The site under study was a private university in Jakarta. The participants for this study consisted of English Education Department students (4 were male, 52 were female) enrolled in the Critical Reading Course. Two CR Classes consist of 26 each class. To obtain the quantitative data regarding students’ perception of the effectiveness of online reading, a 20-items questionnaire was adapted from Tseng (2010) in Taiwan. Some modifications were made to address the research question and for more clarity and precision. The questionnaire consists of three parts: demography information, opinions for the printed text version and the digital text (10 items), perception toward digital text (10 items). The questionnaire was developed online using google form. To gain an automated data collection and provides the easiness for further data analysis. The link will be distributed online through email and WhatsApp. Participants responded to the statement by choosing one out of the five options from strongly agree to disagree strongly. To examine the questionnaire’s reliability, it was tested using Cronbach’s Alpha Test. Perveen (2018) observed that the instrument had provided an appropriate reliability since the Cronbach Alpha value was greater than 0.60 (α = 0.83). Sri Kusuma Ningsih Integrating online reading activity in a critical reading course: a course evaluation 632 Using a 20-item test, the questionnaire was separated into two dimensions (i.e., opinions for the printed text and digital text versions and perceptions towards digital text). The dimensions address students’ perception of the application of the online reading program, along with the individual items as follows: opinions for the printed text version (10 questions) and perceptions towards digital text (10 questions). Both dimensions are coded as A (opinions for the printed and digital text versions) and B (perceptions towards digital text) to ease further data analysis. Prior to actual data collection, there are 58 EFL students and 20 items which are then measured using a Rasch model. Rasch model measurement through Winsteps software 4.3.3 is used to determine the validity and reliability of the instrument as a whole. Through its calibration, Rasch model can transform the ordinal data (Likert- rating scale) into an interval data called “logit” (Logarithm-odd units), which assesses the overall of the instrument for both item and person (Bond & Fox, 2015). Table 1. display the reliability score based on the overall fit statistics of Rasch model measurement. The quality of instrument statistics reported by a logit value for both person and item. Table 1. Instrument statistics Mean Reliability Separation Cronbach Alpha Person 0.26 0.33 0.70 0.43 Item 0.00 0.95 4.23 As shown in Table 1, statistically, the instruments show unreliable scores alongside with the Cronbach  which is lower than 0.60 (Cronbach’s  = .43). The score obtained indicating that the instrument possesses a poor reliability value (Sumintono & Widhiarso, 2014). The reliability values for the person and items show a great result for the item (0.95) but poor for the person (0.33). However, the reliability score for person and item indicates a great value as it possesses logit values of 0.70 and 4.23, respectively. The result suggests that the interaction between person and item is classified into a poor attribute (Linacre, 2012; Boone, Staver, and Yale, 2014). It is critical to note that poor attribution of instrument reliability was due to limited samples participating in the study and thus should be considered as limitation of the study. Interview with students was conducted to collect the qualitative data for the study. A total of 12 students participated in the interview. The interview was a semi-structured where students were asked to respond four questions, such as the advantages reading online, the factors contributing to reading articles online. The interview was employed in the native Bahasa Indonesia to allow them express their thought freely without any language constraints. The data obtained through semi-structured interview was used to support quantitative data. The Quantitative data collected from the questionnaire then will be analyzed into two different stages. First, the data are screened in Microsoft Excel to improve the quality of the data collected. Second, the raw data are then tabulated and converted into logit (logarithm odd unit) through a Rasch analysis using Winsteps software 4.3.4 to maintain an equal interval within a linear scale (Linacre, 2012). The purpose of using Rasch analysis is to ascertain the external consistency on inventories, and evaluate the distribution and the quality of responses. In addition to the quantitative data analysis, qualitative data analysis was performed using a thematic analysis. RESULTS AND DISCUSSION Students’ Perspective about Online Reading Course Item stratification The classification of items measured is 4.23 (rounded to 4), as informed in Table 1. The four strata of the questionnaires’ items listed two broad groups: more difficult items to be considered and easier items to be considered (see Table 2). The classification of each strata ranges from the most difficult item upon (item A4, LVI =1.16) to the easiest item to be agreed (item A10, LVI = -1.07). Precisely, the item stratification process uses percentile values of 25, 50, 75, and 100. Specifically, each item stratification is then classified by providing the logit value (LVI). The item logit scores were distributed very well and indicated that the items are capable of discriminating the individual response. Table 2. Item stratification Category Criteria Item/LVI More Difficult to be Considered Difficulty Strata I LVI > 0.49 A4 (LVI = 1.16) B5 (LVI = 0.82) A7 (LVI = 0.70) A1 (LVI = 0.61) A8 (LVI = 0.55) Difficulty Strata II 0.49 > LVI > 0.26 A3 (LVI = 0.49) A5 (LVI = 0.49) A2 (LVI = 0.34) A6 (LVI = 0.28) B2 (LVI = 0.28) Easier to be ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 10, Issue 2, June 2022 https://journal.uniku.ac.id/index.php/ERJEE 633 Category Criteria Item/LVI Considered Difficulty Strata V 0.26 > LVI > -0.74 B3 (LVI = 0.25) B9 (LVI = 0.02) B4 (LVI = -0.39) B1 (LVI = -0.66) B8 (LVI = -0.68) B10 (LVI = -0.68) Difficulty Strata VI LVI > -0.74 B6 (LVI = -0.75) B7 (LVI -0.80) A9 (LVI = -1.01) A10 (LVI =-1.07) As shown in Table 2, four items considered easiest to be agreed B6 (LVI = -0.75), B7 (LVI = - 0.80), A9 (LVI = -1.01), and A10 (LVI = -1.07). This implies that those four items are positively viewed refers to the effectiveness of an online reading course. In other words, few participants perceived that the online reading course was effective in promoting reading activity. They felt that ‘when doing the online test, the scroll bar helps me read the articles’ (item A9), ‘when doing the online test, the mouse helps me read the articles’ (item A10), ‘I think hyperlinks are helpful when I read on computer screens’ (item B6), and ‘I think the scroll bar is helpful when I read on computer screens’ (item B7) consider as the benefits of an online reading program. However, they also have a negative tendency toward five items considered difficult to agree on. Those items are A8 (LVI = 0.55), A1 (LVI = 0.61), A7 (LVI = 0.70), B5 (LVI = 0.82), and A4 (LVI = 1.16). This is an important notion that online reading course raised issues regarding ‘in the online test, the question asking when which, and who are more difficult than other questions’ (item A7), ‘in the online test, the vocabulary question is the most difficult question’ (item A8), ‘paper-based test is more difficult to read than online test’ (item A1), ‘in the printed text test, the questions asking when, which, and who are more difficult than other questions’ (item A4), and ‘to me, there is no difference between reading on computer screens and reading on paper’ (item B5). Students’ perceived benefits of attending online reading course Findings of the current study showed that the online reading course through a computer-mediated offered several benefits for the students, including the completeness of its features, i.e., the use of scroll bars (A9, LVI = -1.01), mouse (A10, LVI = - 1.07) and hyperlinks (B6, LVI = -0.75). Such features allowed the students to navigate the texts easily, particularly when the article had a sufficient number of pages. It is also interesting but not surprising that the online reading tool's supportive features helped the students explore their reading. It should be noticed that students felt more convenience reading on a computer screen during the reading course than on their smartphone reading application. Students’ perception of the instruction and online course Despite the benefits offered by the online reading course, which are able to facilitate learning activities, perceptions towards the utilizing online reading course were still under expectation. It can be seen in Table 3, which shows the overall perception. Table 3. Overall perception Perception N % Gender Ages Male Female 20 – 22 23 - 25 Positive 18 31 1 17 17 1 Negative 40 69 5 35 28 12 Table 3 shows that students had negative perceptions of the effectiveness of using online reading courses in learning. Of the total 58 students who voluntarily participated in this study, there were 40 students possessed negative perceptions (69%, LVI < 0.47), and 18 students took possession of positive perceptions (31%, LVI > 0.41) on the effectiveness of the online reading course. In terms of gender, the majority of students who have positive perceptions are females aged 20-22 years (94%), with a total of 17 (94%). Similarly, 35 female students (86%) figured out to hold negative perceptions with an age range of 20-22 years (70%). Based on these results, it can be concluded that the online reading course is still doubtful regarding its effectiveness, along with students' perceptions that tended to be negative while the course offered numerous benefits. Furthermore, findings revealed that students preferred conventional reading using paper to online, particularly when attending the reading test. Students found paper-based reading easier to work on the test compared to online reading tests (A1, LVI = 0.61). The questions contained in text-based questions such as when, which, and who are also considered far easier to finish (A4, LVI = 1.16). However, conveniences are also presented by online-based reading learning, questions such as when which, and who is easier to answer compared to other forms of questions (A7, LVI = 0.70). Also, vocabulary-themed questions are easier than other themed questions (A8, LVI = 0.55). Responding to these matters, students believe there is no difference between reading courses based on printed text and online digital text on the computer (B5, LVI = 0.82). Sri Kusuma Ningsih Integrating online reading activity in a critical reading course: a course evaluation 634 Factors contributing to students’ reading activity Students’ computer literacy In the semi-structured interview question, students’ technological competence in operating online reading tool feaures such as scroll bar, cursor, and hyperlink made them easier to search for information in digital texts, especially when there is a very long article. Some interviewees stated the following: “Yes, when reading the digital text I use the scroll bar, cursors and hyperlinks. It’s very helpful when doing reading in digital text because it makes me easier doing an online reading.” (Interviewee B) “Yes, I use scroll bar, cursor and hyperlink when reading online as it really helps me to make it easier to read online.” (Interviewee E) Students’ interest and motivation All participants’ motivation and interest are positive. It may indicate that they enjoyed reading online because they found many advantages. The following interviewees clarify this: “Firstly, the advantage of reading online, in my opinion, is easier to do anywhere and anytime. Secondly, reading online helps us more easily understand the contents of the text because if we have difficulty, we can immediately look for the things we are disposed to know. Then, it's easier to find a topic or articles that we consider reading” (Interviewee C) “So, when I carry a lot of stuff and have a lack of time to read a book, I am still able to read a material given by a lecturer. I can read digital text through a smartphone. Also, there is no need to bother visiting the library to read the material you want or recommended by lecturers because I can easily find and download it on Google or other search engines to read online.” (Interviewee D) “Because there’s no time to print it out. It is also waste of paper.” (Interviewee I) “Reading an online article is easy and can be done anywhere. Then, online articles are sometimes more interesting to read. If there were any difficult words to understand, we can directly look for the meaning online.” (Interviewee K) Challenges in reading online Although students were observed to be benefitted from attending online reading programs, many reported that they encountered one critical challenge during the online reading activity. Many students reported having health issues, particularly eye irritation caused by the screen light or make a notes on computer screens. “Spending too much time reading online may cause headache and eyestrain” (Interviewee B) “When doing an online reading, the technology devices we used automatically must be fully charged. Sometimes when I read online, I don't realize that my device is off due to a low battery. Then, the eyes become more easily exhausted and dizzy because staring at the screen for a long time period. And generally, constraints are found on our devices such as viruses, blanks, or suddenly freeze.” (Interviewee D) “I find the difficulty at the time when reading on a computer screen or cellphone, because easy to feel sore eyes and runny when staring too long at the computer screen or cellphone” (Interviewee H) “For some occasions, sometimes reading online will be difficult when you want to create notation on the article” (Interviewee I) CONCLUSION Based on the findings and discussion, it can be concluded that there was no difference between the Critical Reading course based on paper-based reading and online reading. They found that online reading could assist them in reaching the flexibility of reading and searching articles, particularly for those already familiar with electronic reading devices. They found it helpful to be able to read better on a computer screen. However, participants tend to negatively perceive the effectiveness of online reading tests. They found it easier to do paper-based reading tests than online-based reading tests. Moreover, the small screen size and lights on the screen appeared to issue students’ eye irritation. The research from this study has contributed to knowledge in the field of curriculum development that integrates online reading activity in a reading course. Since the participants in this study were from a single educational institution, the findings could not be generalized to other populations outside the institution. Future studies should involve a large number of participants from various educational institutions to generalize. ACKNOWLEDGEMENT I acknowledge the funding from The Research and Development Body, University of Muhammadiyah DR. HAMKA, Jakarta, Indonesia No. 345/F.03.07/2022 REFERENCES Aldaihani, H. A., Shuqair, K. M., Alotaibi, A. M., & Alrabah, S. (2015). Students’ perceptions of the effectiveness of the english program taught at the college of technological studies in Kuwait. English Language Teaching, 8(4), 80–89. https://doi.org/10.5539/elt.v8n4p80 Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). New York: Routledge. Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 10, Issue 2, June 2022 https://journal.uniku.ac.id/index.php/ERJEE 635 analysis in the human sciences. New York: Springer. Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Routledge. Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer. Chen, C. N., Chen, S. C., Chen, S. H. E., & Wey, S. C. (2013). The effects of extensive reading via e- books on tertiary level EFL students’ reading attitude, reading comprehension and vocabulary. Turkish Online Journal of Educational Technology. Ciampa, K. (2012). ICANREAD: The effects of an online reading program on grade 1 students’ engagement and comprehension strategy use. Journal of Research on Technology in Education, 45(1), 27–59. https://doi.org/10.1080/15391523.2012.107825 96 Gilbert, J. (2017). A study of ESL students’ perceptions of their digital reading. Reading Matrix: An International Online Journal, 17(2), 179–195. Hazaea, A. N., & Alzubi, A. A. (2016). The effectiveness of using mobile on EFL learners' reading practices in Najran university. English Language Teaching, 9, 8-21. Huang, H. C. (2013). E-reading and e-discussion: EFL learners’ perceptions of an e-book reading program. Computer Assisted Language Learning, 26(3), 258–281. https://doi.org/10.1080/09588221.2012.656313 Huang, H. M., Liaw, S. S., & Lai, C. M. (2016). Exploring learner acceptance of the use of virtual reality in medical education: a case study of desktopand projection-based display system. Interactive Learning Environments, 21(1), 3-19. Linacre, J. M. (2012). A user’s guide to Winsteps: Rasch model computer programs. Manalu, B. H. (2019). Students perception of digital texts reading: a case study at the English education department of Universitas Kristen Indonesia. JET (Journal of English Teaching), 5(3), 191. https://doi.org/10.33541/jet.v5i3.1312 Mizrachi, D. (2014). Online or print: Which do students prefer? In European Conference on Information Literacy (pp.733-742). Springer, Cham. Murphy, T. M., Chang, C. Y., & Dispenza, F. (2018). Qualitative clinical mental health program evaluation: models and implications for counseling practitioners and educators. Journal of Mental Health Counseling. https://doi.org/10.17744/mehc.40.1.01 Norris, J. M. (2016). Language program evaluation. Modern Language Journal. https://doi.org/10.1111/modl.12307 Pasaribu, T. A. (2020). Challenging EFL students to read: digital reader response tasks to foster learner autonomy. Teaching English with Technology, 20(2), 21–41. Perveen, A. (2018). Role of MOOCs in Pakistani English teachers’ professional development. Studies in Self-Access Learning Journal, 9, 33– 54. https://doi.org/10.37237/090104 Sumintono, B., & Widhiarso, W. (2014). Aplikasi model rasch untuk penelitian ilmu-ilmu sosial (B. Trim (2nd ed.). Trim Komunikata. Taki, S., & Soleimani, G. H. (2012). Online reading strategy use and gender differences: The case of Iranian EFL learners. Mediterranean Journal of Social Sciences, 3(2), 173-184. Tseng, M. (2010). Factors that influence online reading: an investigation into EFL students' perceptions. The Reading Marix, 10(1), 96-105. Wu, M., & Chen, S. (2011). Graduate students' usage of and attitudes towards e-books: Experiences from Taiwan. Program 45, 294-307. Yeh, H., Hung, H., and Chiang, Y. (2017). The use of online annotations in reading instruction and its impact on students' reading progress and processes. ReCALL, 29(1), 22-38. Sri Kusuma Ningsih Integrating online reading activity in a critical reading course: a course evaluation 636