Copyright©2017 P-ISSN: 1978-8118 E-ISSN: 2460-710X 79 Lingua Cultura, 11(2), November 2017, 79-83 DOI: 10.21512/lc.v11i2.1518 ASSESSING UNIVERSITY STUDENTS’ CRITICAL THINKING SKILL BY USING THE TOEFL ITP READING TEST Rentauli Mariah Silalahi Fakultas Teknik Informatika dan Elektro (FTIE), Prodi Sistem Informasi, Institut Teknologi Del Jl. Sisingamangaraja, Toba Samosir 22381, Indonesia rentaulisilalahi@gmail.com Received: 11th August 2016/ Revised: 18th November 2016/ Accepted: 31th March 2017 How to Cite: Silalahi, R. M. (2017). Assessing University Students’ Critical Thinking Skill by Using the TOEFL ITP Reading Test. Lingua Cultura, 11(2). 79-83. http://dx.doi.org/10.21512/lc.v11i2.1518 ABSTRACT This research investigated the students’ ability in applying the critical thinking skill while doing the TOEFL ITP reading test. The participants were 140 university students in one of the private universities (FI University/pseudonym) in Indonesia. To find out the answer, this research was firstly done by looking at the relationship of critical thinking and reading comprehension skill and investigating the level of students’ reading ability by contrasting their reading score to the Common European Framework of Reference for Languages (CEFR). The study finds that there is an inseparable connection between the students’ critical thinking skill and reading competence. And the majority of the participants is very weak in critical thinking and reading competence skills. Therefore they are placed at the bottom level of the CEFR. Keywords: critical thinking, reading competence, TOEFL, CEFR INTRODUCTION All teachers want their students to be critical in a positive way, yet generally, students in Indonesia are not very keen on improving their critical thinking ability. As this ability may be improved and developed, teachers should try to integrate it into their teaching experiences that eventually may result in critical students. When Indonesian’s young generations are critical enough, they will lead the country into prosperity. One good practice English teachers may do is by designing an English test that can assess students’ critical thinking. A good English test should be able to assess someone’s English competence and critical thinking at the same time. The English test should not at any form sounds like a pure logical test that requires a deep comprehension of mathematical formulation and calculation. The test should purely be based on English competence but test the critical thinking. However, critical thinking should not be misunderstood as a reasoning test. The purpose of the English critical thinking test is to enable students to be creative and critical when pursuing their study because the essence of being a student is to be always creative and critical, yet that is a rare situation in most of ASEAN countries’ schools, especially in Indonesia. The hardest part of a teacher’s work is to make the students active and responsive in class participation and creative in acquiring knowledge from other sources than solely depends on materials that are given by the teacher in the classroom. Therefore it is highly suggested that EFL (English as First Language) teachers become active in encouraging students to practice their critical thinking abilities by using different kinds of approaches such as practicing debates, problem-solving tasks, media analysis, (Rezaei, Derakhshan, & Bagherkazemi, 2011) and essay writing (Wang, 2016) in which by the end of the course may become an assessment tool. However, creating this kind of a test which is testing students’ critical thinking is not an easy task, but forces the teacher as the test maker to be able to be creative, critical, and logical in thinking. These skills acquired of a teacher may become a burden when not wisely comprehended and may stress the teacher out, but when the teacher realizes the importance of having critical thinking skill, he/she will immensely gain knowledge from it and be successful in his/ her career (Ghaemi & Taherian, 2011). There have been several kinds of English tests which are internationally tested and approved as a tool for assessing someone’s English proficiency. It is considered reliable to measure the potential of somebody to survive for a study for certain circumstances. Among them, they are the IELTS test and TOEFL test which are currently booming as the most favored tests by anyone who is having a purpose of studying overseas. These kinds of tests will be a promising business for the next many years to come. These two tests are different in forms yet with their own uniqueness can 80 LINGUA CULTURA, Vol.11 No.2, November 2017, 79-83 assess students for their critical thinking. There have been many kinds of research done related to assessing English test that may test the test taker’s critical thinking. Critical thinking itself has been defined by varied interpretation yet basically sounds the same. Khatib & Mehrgan have defined critical thinking as logical, analytical, reasonable, higher-order, reflective, and scientific thinking as well as reasoning skill (Khatib & Mehrgan, 2012). Meanwhile, it is defined critical thinking as the ability to analyze facts, generate and organize ideas, defend opinions, draw inferences, evaluate arguments, and solve problems (Hassani, Rahmany, & Babaei, 2013). On a different research, Fahim & Pezeshki have clearly stated that critical thinking involved logical reasoning, an ability to separate facts from opinions, examining things before accepting them, and asking oneself questions all the time (Fahim, Barjesteh, & Vaseghi, 2012). Among all definitions defined by many researchers, Paul’s definition is the one quite close to Bloom’s taxonomy in which he defined critical thinking as learning how to ask and answer questions of analysis, synthesis, and evaluation (Rezaei et al., 2011). Looking back at those definitions, it may be concluded that critical thinking is inseparable with creative and logical thinking. Considering the urgent need for students to be critical, Rezaei et al. (2011) have suggested teachers help students develop their students’ critical thinking by doing some active activities from the debate, problem-solving, self- assessment assignment, and peer assessment assignment. The ultimate objective of critical thinking is to help students make the correct judgment based on the careful weighing of available evidence. Meanwhile, Khatib & Mehrgan, 2012 have suggested that giving students short stories to read will improve their critical thinking. Another method is introduced by Fahim, Miri, & Najafi (2014), who find that by doing collaborative assessment; particularly between students, is more effective in fostering critical thinking skill. It can be concluded that almost all previous research accomplished is about the relation between reading and critical thinking or how reading activities may evolve students’ critical thinking (Hassani et al., 2013), (Khodadady & Ghanizadeh, 2011), (Fahim, Bagherkazemi, & Alemi, 2010), (Fahim et al., 2012) or more interestingly, someone’s critical thinking may help him/her to advance his/her writing skill (Assadi, Davatgar, & Jafari, 2013) or speaking skill (Ramezani, Larsari, & Kiasi, 2016). However, for this research, the researcher makes a scope to focus on the relationship between critical thinking and reading skill. The researcher chooses this focus based on a personal reason where she finds so many students in Institut Teknologi Del are very weak in reading skill both in understanding the content and interpreting the content critically. It happens to the most courses students learned at the college. However, the researcher focuses on students’ reading skill in English course taking TOEFL ITP reading test as an instrument. In the TOEFL ITP test, there are three type of questions commonly asked. Firstly is about ideas of the passages whether the main idea, stated or unstated detail, and implied one, secondly is about vocabulary, and thirdly is about cohesion for author’s tone or purpose and the location of specific information (Phillips, 2003; Phillips, 2004). All those types of questions require students to be able to apply three main skills which are closely related to Bloom’s taxonomy and the Watson-Glaser Critical Thinking Appraisal (Fahim et al., 2010). They are skills for comprehension or understanding, analysis, and evaluation. When students try to answer a TOEFL reading test item, they must understand the text that they read and analyze the information they gain from the text and if necessary make evaluation of the best possible answer which is sometimes very tricky. The Watson-Glaser Critical Thinking Appraisal is all about comprehension, analysis, and evaluation (Fahim et al., 2010). It comprises five subsections which measure five aspects of critical thinking, they are (1) drawing inferences based on factual statements, (2) recognition of assumptions in many assertive statements, (3) making deductions to determine if conclusions follow from information given in statements, (4) interpreting evidence to decide if conclusions are legitimate or not, (5) evaluating arguments as being strong or weak. To look at the level of the students’ reading comprehension proficiency closely, the researcher contrasts the students’ achieved reading score to the Common European Framework of Reference for Languages (CEFR) as it would enable the researcher to map the students based on their score. There are six levels of the CEFR as shown in Table 1 (Phillips, 2004). Table 1 The Six Levels of CEFR C2 Mastery Proficient user C1 Effective operational proficiency B2 Vantage Independent user B1 Threshold A2 Waystage Basic user A1 Breakthrough Therefore, the research questions are formulated as (1) Is there any relationship between students’ critical thinking skill and reading comprehension skill? (2) What are the levels of students’ reading comprehension proficiency and the level of students’ critical thinking skill when reflected the CEFR in FI University (pseudonym)? METHODS The participants of this study were 140 students aged between 21 to 22 years old from FI University. The students’ English proficiency as observed from the students’ rapport in their final semester in the sixth semester were varied from upper intermediate to lower intermediate because overall, the students’ average score in English course was 75,88 and the maximum score was 90,48 and the lowest score was 57,24. The students were all majoring Information Technology (IT) and were sitting in their third year at the university. The researcher used TOEFL ITP test as the main instrument for conducting this research. Apart from that, there were several supporting documents used such as the students’ rapport to know the students’ general English competence in the university and the students’ TOEFL ITP certificates to know the result of their TOEFL ITP test. However, because the TOEFL ITP test was designed by the official TOEFL ITP test provider (known as ETS) and was organized by the official TOEFL ITP test organizer in Indonesia called Indonesian International Education Foundation (IIEF), then there was not any exploration or 81Assessing University Students’ Critical .... (Rentauli Mariah Silalahi) access to the test material whatsoever. It was very obvious that the question tested on the day the students sat the TOEFL ITP test was not very far from the common types of questions asked in any TOEFL ITP test practices available from the TOEFL ITP books (Phillips, 2003; Phillips, 2004). This used a qualitative study and some of the analysis was done by literature analysis. The literature analysis was carried out to find out the relationship between students’ critical thinking and reading comprehension skill. The study was then continued by collecting the 140 student participants. The students were all mixed and no differences were made regarding their genders. The students were then given a TOEFL ITP test on the 28 of May 2016. The students’ participation was not voluntary but was a part of a program in the university in which all students in the third year must sit a TOEFL ITP test. The students had previously been given a TOEFL test preparation class for around one year before sitting the test. Having been given a preparation class, they were expected to already learn some strategies to use critical thinking to answer the test questions. After two weeks, the students’ TOEFL ITP test results were collected from the test organizer with some numerical explanations on the result sheet. The result was then observed and analyzed to answer the research questions. RESULTS AND DISCUSSIONS Being critical is exactly as already explained by the Watson-Glaser Critical Thinking Appraisal that required a higher order thinking and very much alike the criteria of a proficient reader (Brassell & Rasinski, 2008). Brassell & Rasinski have stated that being a proficient reader means being competent in reading comprehension skill. Table 2 will show how similar and interconnected the two components. Table 2 The Relationship between Critical Thinking and Reading Competence Characteristic of being a critical thinking reader Characteristic of being a competent reader Could draw inferences based on factual statements, Could do some series of activities while reading starting from asking questions about the text before, during, and after reading, Could recognize assumptions in a number of assertive statements, Could draw inferences from the text, Could make deductions: to determine if conclusions follow from information given in statements, Could monitor his/her comprehension, Could interpret evidence to decide if conclusions are legitimate or not, Could use ‘fix-up’ strategies when meaning breaks down, Could evaluate arguments as being strong or weak. Could figure out what is most important in whatever they read and synthesize new information to create new thinking. Therefore, to answer the first question of this research whether there is a relationship between students’ critical thinking skill and reading comprehension skill, then the answer is yes. There is an inseparable connection between the two. Therefore, it is worth noted that teachers should never try to separate the two but bring them together in their teaching practices when helping students to develop their reading skill. The reading test of the TOEFL ITP test is only one small part of the test. There are actually two other important skills tested that contribute to the overall score of the TOEFL ITP score; listening and structure. However, the focus of this study is the reading test of the TOEFL ITP test. What texts really appear on every reading test of the TOEFL ITP test may not be known for certain, yet it is still predictable because the reading test will usually consist of five reading passages. Each is followed by a number of reading comprehension and vocabulary questions. And the topics are varied but are often about informational subjects that might be studied in an American university which usually talks about history, literature, art, architecture, geology, geography, and astronomy (Phillips, 2003). To answer the questions on the reading part of the test, every test taker is required to be critical and be able to do multiple tasking jobs in their thoughts. Among the tasks, according to Cohen & Upton (2006), the students should be able to do at a very similar time the following tasks: (1) choosing the right reading skill strategies to use for reading the text; either skimming, scanning, or reading for details. (2) Looking ahead at the questions that follow a reading text. (3) Thinking quickly to determine the types of answers required for the questions of the text; either asking for a noun, verb, adjective, adverb, phrase, or a complete sentence. (4) Working out on the meaning of any unfamiliar word when stumbled with one. (5) Rereading the sentences quickly where the answer to a question might be found. (6) Choosing the right answer from four possible answer choices provided. The students’ participation is involved by sitting a TOEFL ITP test or also known as TOEFL PBT on May 28, 2016. After sitting the TOEFL ITP test, the result comes out with students’ scaled scores mean for the Listening test 46,40, Structure and Written Expression 43,19, and Reading Comprehension 47,82. The scores are released by the official test organizer in Indonesia; IIEF, two weeks after the students sat the test. Based on the result of the students’ TOEFL ITP test, it is found that the students’ TOEFL ITP lowest score is 357 and the highest score is 580. Specific for reading competence, the students’ lowest score is 32 and the highest score is 63. Meanwhile, for TOEFL ITP official scaled score, the reading score ranged from 31 to 67. It means that the students’ in FI University is varied in reading competence from very poor to approaching excellent. When reflected the CEFR, the students are categorized into four levels of competencies from A2 to C1 with the number of students’ distribution in Table 3. Table 3 The students’ Competence based on CEFR Level of Competency CEFR level of competency Meaning of level No. of students A2 Basic user 61 B1 Independent user 68 B2 10 C1 Proficient user 1 Total 140 82 LINGUA CULTURA, Vol.11 No.2, November 2017, 79-83 From Table 3, it is clearly shown that there is only one student who is proficient in reading skill which most probably the most critical student of all in reading comprehension. While, if referring to the definition of each level by CEFR (Trim, 2011), A2 means students can communicate in English within a limited range of contexts, B1 means students can communicate essential points and ideas in familiar contexts, B2 means students can use English effectively with some fluency in a range of contexts, and C1 means students are able to use English fluently and flexible in a wide range of context. Overall, a modest conclusion may be withdrawn that only 0,08% of the students can use their critical thinking skill in order to survive the reading comprehension test at the TOEFL ITP Test. In other words, the students in FI University have a very low level of reading comprehension proficiency. Table 4 shows the FI students’ achievement on the reading test of the TOEFL ITP test they sat on the 28 of May 2016. However, the data displayed on the table is a summary of the number of students achieving a certain score on the TOEFL ITP test and is already sorted from the lowest score achiever to the highest one. Table 4 The Students’ Achieved Score in Reading Test of the TOEFL ITP Test Student's score in Reading Test No. of students 32 1 34 3 36 2 38 1 39 7 41 5 42 8 43 3 44 5 45 9 46 9 47 8 48 14 49 5 50 8 51 14 52 13 53 6 54 5 55 3 56 5 57 1 58 3 59 1 63 1 Total 140 In fact, there is a significant relationship between the students’ critical thinking and reading comprehension skill. And the TOEFL ITP test for the reading section is always designed to meet the demand for the test takers’ critical thinking skills. Therefore, it could be concluded that the FI students; whether realized or not, must have been trying to use their critical thinking to answer the TOEFL ITP reading test, and their ability to use their critical thinking skill could be measured by their achievement on that test. Looking back to table 4 and keeping in mind that the reading test is scored from 31 to 67. It is obvious that most of the students have poor critical thinking skills because there are only 43% of the students who manage to achieve a score of 50 and above. Meanwhile, around 80 students or 57% of them perform badly in integrating their critical thinking skill towards the test as indicated by their low scores. CONCLUSIONS The most important strategy to apply while doing a reading test is to integrate the critical thinking and reading skills. Being critical will enable students to choose the right choice instead of wandering around four answer choices provided but unable to choose the right one because of very little comprehension of what being read. These two interconnected skills are still possible to be improved and developed if students are willing to spend more time to practice and get used to doing the test to minimize the tension of a test in a real test situation. However, it is found that the students at FI University still lack in critical thinking skill that consequently hindered them to succeed the reading test on the TOEFL ITP test. The effort to reflect the students’ current achievement by sitting a TOEFL ITP test to the CEFR should not remain in the findings, but it makes as a new starting point to guide the English teachers to help the students improve their quality in English proficiency to be able to survive in the global competition era. The data could be used by teachers to develop the English curriculum, lesson plans, and to design new materials, activities, and assessments. There are various activities which could be made by English teachers to develop students’ English reading skill. Among them are to motivate students to start reading a wide variety of materials and turn it into a hobby while always keeping in mind that they need to raise some questions while reading and work on unfamiliar vocabulary (Brassell & Rasinski, 2008). The most important thing is to be curious at all time while reading so that the interest will lead the students to be critical in a natural way. In the end, students will not realize that they have been critical in thinking while reading and by reading lots of and varied sources will result in knowledgeable and open-minded students. REFERENCES Assadi, N., Davatgar, H., & Jafari, P. (2013). The effect of critical thinking on enhancing writing among Iranian EFL learners. International Journal of Scientific & Engineering Research, 4(3), 1–7. Brassell, D., & Rasinski, T. (2008). Comprehension that works. California, USA: Shell Education. Cohen, A. D., & Upton, T. A. (2006). Strategies in Responding to the New TOEFL Reading Tasks. Princeton, USA: ETS. Fahim, M., Bagherkazemi, M., & Alemi, M. (2010). The Relationship between Test Takers " Critical Thinking Ability and their Performance on the Reading Section of TOEFL. Journal of Language Teaching and Research, 1(6), 830–837. http://doi.org/10.4304/jltr.1.6.830-837. Fahim, M., Barjesteh, H., & Vaseghi, R. (2012). Effects of Critical Thinking Strategy Training on Male/Female 83Assessing University Students’ Critical .... (Rentauli Mariah Silalahi) EFL Learners’ Reading Comprehension. English Language Teaching, 5(1), 140–145. http://doi. org/10.5539/elt.v5n1p140. Fahim, M., Miri, M., & Najafi, Y. (2014). Contributory Role of Collaborative Assessment in Improving Critical Thinking and Writing. International Journal of Applied Linguistics & English Literature ISSN, 3(1), 2200–3592. http://doi.org/10.7575/aiac. ijalel.v.3n.1p.1. Ghaemi, H., & Taherian, R. (2011). The Role of Critical Thinking in EFL Teachers’ Teaching Success The Role of Critical Thinking in EFL Teachers’ Teaching Success The Role of Critical Thinking in EFL Teachers’ Teaching Success. Mjal, 3(1), 1–11. Hassani, M. T., Rahmany, R., & Babaei, M. (2013). The Relationship between Iranian EFL Learners’ Critical Thinking and Reading Comprehension Performance in Journalistic Texts. Theory and Practice in Language Studies, 3(10), 1873–1878. http://doi. org/10.4304/tpls.3.10.1873-1878. Khatib, M., & Mehrgan, K. (2012). Achieving Critical Thinking Skills through Reading Short Stories 1. Advances in Digital Multimedia, 1(3), 140–145. Retrieved from www.worldsciencepublisher.org. Khodadady, E., & Ghanizadeh, A. (2011). The Impact of Concept Mapping on EFL Learners’ Critical Thinking Ability. English Language Teaching, 4(4), 49–60. Phillips, D. (2003). Longman introductory course for the TOEFL test. New York: Pearson Education, Inc. Phillips, D. (2004). Longman introductory course for the TOEFL test. New York: Pearson Education, Inc. Ramezani, R., Larsari, E. E., & Kiasi, M. A. (2016). The Relationship between Critical Thinking and EFL Learners’ Speaking Ability. English Language Teaching, 9(6), 189-198. http://doi.org/10.5539/elt. v9n6p189 Rezaei, S., Derakhshan, A., & Bagherkazemi, M. (2011). Critical Thinking in Language Education. Journal of Language Teaching and Research, 2(4), 769–777. Trim, J. (2011). Using the CEFR: Principles of Good Practice. Cambridge, UK: Cambridge ESOL. Wang, X. (2016). Reasoning Critical Thinking: Is It Born or Made? Theory and Practice in Language Studies, 6(6), 1323–1331. http://dx.doi.org/10.17507/ tpls.0606.25.