38 | IJET| Volume. 8, Issue 2. December 2019 Textbook Readability Assessment of K13 English Textbook for the Twelfth Grade Susilo Adi Perwira, addiniyoosha02@gmail.com, Language and Literature Education Study Program, Postgraduate Program of The State University of Surabaya. Surabaya, Indonesia. Dr. Aswandi, M. Pd., aswandi@unesa.ac.id, Language and Literature Education Study Program, Postgraduate Program of The State University of Surabaya. Surabaya, Indonesia. Drs. SlametSetiawan, M. A., Ph. D., slametsetiawan@unesa.ac.id, Language and Literature Education Study Program, Postgraduate Program of The State University of Surabaya. Surabaya, Indonesia Abstract. English textbook for the twelfth grade of SMA, which is published by the Ministry of Education and Culture, is a compulsory textbook that is based on national curriculum 2013 to support education in Indonesia. This study aims to investigate the readability level of the English textbook. It uses the descriptive quantitative method to examine the readability of this book. The data are in the form of texts derives from the transactional and interpersonal text, short functional text, reading text, instructional text inside the book. There are 180 data taken from those four texts. They are analyzed by using Miyazaki EFL readability formulae as a tool to measure the readability level, whether it is suitable or not for twelfth- grade students. According to the Miyazaki EFL readability test, the easiest text from the book is in 5th grade and the hardest one is in post- school/university grade. But the most frequent grade appear in this book is in 8th and 9th grade which is 77 data (texts). The average of this book score is 63.2 which means it is suitable for 8th and 9th grade. This book is too easy for the twelfth grade according to the Miyazaki EFL readability test. Keywords: Readability formula, K13 English textbook, transactional and interpersonal text, short functional text, reading text, instructional text 1.INTRODUCTION There is a close association between reading interest and reading materials. Iftanti (2012)studied the interest of reading of English college students in East Java and found out that most of the students who are good at English have a high reading interest. It can be seen that they read various texts in English regularly most of their time. Richards (2001)claimed that the textbook has several positive impacts on language learning. It also has the benefit of keeping the quality of the organization of the course and syllabus. English textbook for the twelfth grade of SMA which is published by the Ministry of Education and Culture is a compulsory textbook that is according to the national curriculum 2013 to make education in Indonesia stay in line, in certain aspects the existence of material for teaching. Based on the regulation, English textbooks are compulsory to be used in schools in the learning activity, including giving students tasks and guidance and becoming the reference for the exam questions. mailto:addiniyoosha02@gmail.com mailto:aswandi@unesa.ac.id mailto:slametsetiawan@unesa.ac.id 39 | IJET| Volume. 8, Issue 2. December 2019 Mikki (2000) identifies several characteristics that contribute to making a good textbook. A good textbook is made to be well structured to allow learners to follow and make sense of the content. It should also have graphics to help understanding by giving a pictorial representation of ideas, which facilitate the ease of remembering information. In the textbook, there should be references to other related sources. A good textbook also includes exercises, which learners can use to test their skills. Learners do not like ‘bulky textbooks’ (p. 21) and so a good textbook should be long enough to cover and clarify the important concepts but not too long that they can intimidate the learners. Textbooks should also trigger critical thinking for students. To select reading material for the students, three important criteria need to be considered. They are suitability, exploitability and readability (Nuttall, 1982). Suitability is one of the components that means the material of reading should match students’ needs to trigger the students’ interest and stimulates their motivation in learning a language. Exploitability means the way of text used in developing students’ reading competence. Meanwhile, readability stands for the complexity and difficulty of the text; somehow, students’ reading materials should not contain too difficult words and structures and match with students’ level. Readability is the way measuring the level of reading material difficulty. Therefore, proper text selection for students is needed. Readability is away in the teaching field which meant to measure text level of difficulty to find suitable reading materials that students read(Mesmer, 2008). Most EFL teachers do not understand the measuring instrument to determine the readability level of text. As a result, it can be recognized that there is a gap between the reading materials and students’ comprehension level. This gap becomes the factor of the reduction of the interest and motivation of students’ reading. This study tries the latest readability formulas that are used specifically in measuring EFL reading materials that are considered to have better compatibility than the classic ones. It is Miyazaki Readability Formula (developed by Greenfield in 2003. Miyazaki counts the sentence length, but in the other variable, Miyazaki Readability Formula focuses on the length of words that are believed to puzzles the EFL readers. Miyazaki readability formula has been used by several researchers in Indonesia to analyze the readability of textbooks. One of the studies was done by Sangia (2015).His research aimed to solve the problems related to the reading text material in EnglishTextbook for Eleventh Grader published by Ministry of Education and Culture based on theory of good reading text (Nuttall, 2005) He found that the overall reading texts by using Miyazaki EFL Readability, the readability level is fairly difficult, and it is good for eleventh grade. 2. METHOD This study belongs to the descriptive quantitative approach because of two reasons. First, it has data in numbers. The data are in the form of counted words, letters, and sentences. Second, it needs statistical analysis to analyze the data. The analysis will be in the form of calculation using the formula and then explaining the result in narration form. This research describes the readability level of English transactional and interpersonal text, short functional text, reading texts, and instructional text with the Miyazaki EFL readability formula. The objects of this study are transactional and interpersonal text, short functional text, reading texts, and instructional text, which are printed in English textbook entitled “Bahasa Inggris” for the twelfth grade of SMA, which is published by Ministry of Education and Culture in 2018. This book is designed to be used for the k13 curriculum which is the latest curriculum used in school. The object is chosen to get the data needed in this research. 40 | IJET| Volume. 8, Issue 2. December 2019 To get the answer to the research question, the data needed are letters, words, and sentences count from each transactional and interpersonal text, short functional text, reading texts, and instruction from each chapter in the book. It needs the letters, words and sentences count for the formula to know the readability level for the book. They are taken from the English textbook entitled “Bahasa Inggris,” which is published by the Ministry of Education and Culture in 2018. This book can be downloaded online from the official website of education bse.kemdikbud.go.id. This textbook is one of the textbooks which is provided online by the Ministry of Education and Culture. It can be downloaded by everyone for free. The data needed to answer research questions in this research is the words count. To gain that, the researchers searched word counter from the internet, which counts letters, words and sentences. There are lots of words counters on the internet, but most of them count to character, not letters. The only one which counts letters is countwordsworth.com. That makes the researcher chooses it to count the letters, words and sentences. To measure the readability level of the books, the researchers used the Miyazaki EFL readability formula. The data collection procedures are, first, researchers downloaded English Book entitled Bahasa Inggris can be downloaded online from BSE.kemdikbud.go.id. Second, the researchers read all the content of the books and selected only the transactional and interpersonal text, short functional text, reading texts, and instructional text from each chapter of the book. Third, the researcher makes the sentences into groups from the whole chapter into four types (transactional and interpersonal text, short functional text, reading texts, and instructional text) and coded accordingly. Fourth, the researcher inputs letters, words, and sentences into the formula. Fifth, the researchers matched the result with the table of Miyazaki data interpretation to know the grade of the text. Sixth, the researchers narrated the result, and the last, the researchers drew conclusion. To make the text easier to be referred, the researchers did text coding. There were two variables that are represented by letter and number. First is the text type (transactional and interpersonal text, short functional text, reading text, and instruction) that will be represented by capital letter, the second is text order which will be represented by number, for example, Text A1 means transactional and interpersonal text number 1, Text A2 means transactional and interpersonal text number 2, Text B1 means short functional text number 1, Text C1 means reading text number 1, Text D1 means instruction text number 1, etc. Since Miyazaki needs the count of letters, words, and sentences, the researcher needs to search a valid word counter. It can be found in site countwordsworth.com. A researcher copied and pasted the text from the book to the box, which is provided inside that site to count the words, letters, and sentences. It takes a second to get the result and all the information about letters, words and sentences from the passage will appear. Miyazaki EFL readability formula is used to measure the readability level of the passage. The formula can be seen below: EFL DIFFICULTY = 164.935 - (18.792 × ) – (1.916 × ) The researchers input the sum of the letter, words, and sentences of each passage to the formula manually. Then it can be seen the readability level of the passage. The result of Miyazaki EFL formulae is in the form of the number. The score range is between 0-100. The explanation can be seen in the table below. 41 | IJET| Volume. 8, Issue 2. December 2019 Table 1. Miyazaki Readability Score Interpretation Reading Ease Score Style Description Estimated Reading Grade 0 to 30 Very difficult University Graduate 31 to 50 Difficult Post-school/College 51 to 60 Fairly difficult 10 th to 12 th Grade 61 to 70 Standard 8 th and 9 th Grade 71 to 80 Fairly Easy 7 th Grade 81 to 90 Easy 6 th Grade 91 to 100 Very Easy 5 th Grade 3. RESULT Inside the K13 English Book for 12th grade, which is published by the Ministry of Education and Culture, the researcher found three texts which are included in the transactional and interpersonal text. All of them are in the form of dialogue. From the dialogue, the transaction and interpersonal communication between two or more speakers can be seen clearly. Table 2. Transactional and Interpersonal Text Composition No Text Code Number of Letters Number of Words Number of Sentences 1. A1 481 136 25 2. A2 515 132 22 3. A3 310 81 26 From table 2, it can be seen that the text A1 has the most number of letters, and the most number of words and text A3 has the most number of sentences. To measure the readability of transactional and interpersonal text, the researcher input the letters, words and sentences count into Miyazaki Readability tools. To minimize the mistake of the researchers in counting with formula, the researchers put the Miyazaki formula into MS Excel in column Score, so the researchers input the letter, words, and sentence count, and the formula will be counted by MS Excel. The researcher then matches the result to Miyazaki readability score interpretation to get the text suits to which grade. Table 3. Mizayaki Test Result No Text Code Readability Score Estimated Reading Grade 1. A1 88.05 6th Grade 2. A2 80.12 7th Grade 3. A3 83.32 6th Grade From table 3, it can be seen that the text A1 has the highest readability score, followed by text A3 and text A2. Inside the K13 English Book for 12th grade which is published by the 42 | IJET| Volume. 8, Issue 2. December 2019 Ministry of Education and Culture, the researcher found one text which is included in the short functional text. It is a vacancy that has the function of advertising that a company needs an employee with the conditions mentioned inside the vacancy. Text B1 can be found inside the book on pages 58-59. The component needed to be able to be input in the Miyazaki Readability tool are letters, words and sentence count. By using sites http://countwordsworth.com to count letters, words and sentences, the researcher found that there are 638 letters, 127 words and 15 sentences inside that text. Text B1 contains 638 letters, 127 words, and 15 sentences. The researcher puts them to each row inside Ms. Excel and finds 54.31 as the score. According to Miyazaki readability score interpretation, score 54.31 is in the range of 10-12th grade. Inside the K13 English Book for 12th grade, which is published by the Ministry of Education and Culture, the researcher found 15 texts which are included in reading text. The distribution of functional text can be seen in table 4 below: Table 4. Functional Text Composition No Text Code Number of Letters Number of Words Number of Sentences 1. C1 2083 451 33 2. C2 804 163 15 3. C3 894 197 16 4. C4 1043 197 14 5. C5 494 111 9 6. C6 504 100 6 7. C7 1088 219 13 8. C8 2616 535 24 9. C9 1190 238 14 10. C10 553 109 7 11. C11 501 113 12 12. C12 633 141 13 13. C13 2451 591 52 14. C14 2694 666 37 15. C15 2514 578 34 Miyazaki Readability of Reading Text 1. Text C1 has 52.00 as the score. According to Miyazaki readability score interpretation, score 52.00 is in the range of 10-12th grade. 2. Text C2 has 51.40 as the score. According to Miyazaki readability score interpretation, score 51.40 is in the range of 10-12th grade. 3. Text C3 has 56.10 as the score. According to Miyazaki readability score interpretation, score 56.10 is in the range of 10-12th grade. 4. Text C4 has 38.50 as the score. According to Miyazaki readability score interpretation, score 38.50 is in the range of post-school grade. 5. Text C5 has 57.67 as the score. According to Miyazaki readability score interpretation, score 57.67 is in the range of 10-12th grade. 6. Text C6 has 38.28 as the score. According to Miyazaki readability score interpretation, score 38.28 is in the range of post-school grade. http://countwordsworth.com/ 43 | IJET| Volume. 8, Issue 2. December 2019 7. Text C7 has 51.40 as the score. According to Miyazaki readability score interpretation, score 51.40 is in the range of 10-12th grade. 8. Text C8 has 30.33 as the score. According to Miyazaki readability score interpretation, score 30.33 is in the range of post-school grade. 9. Text C9 has 38.40 as the score. According to Miyazaki readability score interpretation, score 38.40 is in the range of post-school grade. 10. Text C10 has 39.76 as the score. According to Miyazaki readability score interpretation, score 39.76 is in the range of post-school grade. 11. Text C11 has 63.57 as the score. According to Miyazaki readability score interpretation, score 63.57 is in the range of 8-9th grade. 12. Text C12 has 59.78 as the score. According to Miyazaki readability score interpretation, score 59.78 is in the range of 10-12th grade. 13. Text C13 has 65.22 as the score. According to Miyazaki readability score interpretation, score 65.22 is in the range of 8-9th grade. 14. Text C14 has 54.43 as the score. According to Miyazaki readability score interpretation, score 54.43 is in the range of 10-12th grade. 15. Text C15has 50.62 as the score. According to Miyazaki readability score interpretation, score 50.62 is in the range of 10-12th grade. Instructional Text Inside the K13 English Book for 12th grade, which is published by the Ministry of Education and Culture, the researcher found 161 texts which are included in Instructional text. The elaboration in this section will not be presented with each data like the previous section because of the space issue. Instructional Text Component The detailed data can be seen in appendix 1. The components of instructional text which are needed to be counted by Miyazaki consist of letters, words and sentences. Miyazaki Readability of Instructional Text From 161 instructional texts, it can be seen that they are included in 5 out of 7 levels of Miyazaki estimated reading grade. They are 6th grade, 7th grade, 8th and 9th grade, 10th to 12th grade, and post-school/college. Table 5. Estimated Reading Grade Estimated Reading Grade Style Description Data count per level University Graduate Very difficult 0 Post-school/College Difficult 11 10th to 12th Grade Fairly difficult 27 8th and 9th Grade Standard 75 7th Grade Fairly Easy 39 6th Grade Easy 9 5th Grade Very Easy 0 Total data 161 According to the table above, there are no instructional text inside the book which is included in university graduate level and 5th-grade levels. It can be found 11 instructional 44 | IJET| Volume. 8, Issue 2. December 2019 texts that are included in post-school and college. From 10th to 12th grade, it can be found 27 instructional texts. Then it can be found 75 instructional texts which are included in 8th and 9th grade. For 7th grade, it can be found 39 instructional texts, and for the 5th grade, it can be found nine texts. Readability of Textbook The researcher will show the highest grade of readability and lowest grade of readability level and average score from four components (text types) that compose the book according to the readability score using tools Miyazaki readability tool from the texts. From those scores can be seen the range of readability level of this book and the average readability score of the book. From the data above, it can be seen that the lowest score of readability inside the book can be found in the reading text number 8 (C8) which has a score of 30.33. According to Miyazaki Readability score interpretation, the text is categorized to post-school/college grade. Meanwhile, the highest score of readability inside the book can be found in instruction text number 19 (D19) which has a score of 90.37. According to the Miyazaki Readability score interpretation, the text is categorized into 5th grade. The average score from all transactional and interpersonal text (3 texts) in this book is 83.8. According to Miyazaki Readability score interpretation, the text is categorized to 6th grade. The average score from all short functional text (1 text) in this book is 54.3. According to Miyazaki Readability score interpretation, the text is categorized to 10th-12th grade. The average score from all reading text (15 texts) in this book is 49.0. According to Miyazaki Readability score interpretation, the text is categorized to post-school grade. The average score from all instructional texts (30 texts) in this book is 69.3. According to Miyazaki Readability score interpretation, the text is categorized to 8th-9th grade. The overall score of this book can be taken from the average of those four components, and the score is 8th-9th grade. Discussion of Transactional and Interpersonal Text Readability There are only three texts in the book which are included in the transactional and interpersonal text. All of them are the form of spoken texts. One spoken text can be seen on page 4, and two spoken texts can be seen in page 5. After being input into Miyazaki readability test, it can be seen that text one (A1) and text three (A3) have scored more than 81. It means they belong to easy for style description and in 6th grade for estimated reading grade. Meanwhile, the second text (A2) has a score of 80.0. It means it belongs to fairly easy for style description and 7th grade for estimated reading grade. The highest score of the text which belongs in the transactional and interpersonal text in this book, is 88.0. It means it is easy for style description and belongs to 6th grade for the estimated reading grade. Meanwhile, the lowest score of this text is 80.0. It belongs to fairly easy for style description and 7th grade for estimated reading grade. The average of those three texts is 83.8 which means the transactional and interpersonal text in this book is too easy for twelfth grade. From the highest and lowest score and average stated above, according to Miyazaki readability test, it can be seen that the transactional and interpersonal text in this book still belongs in elementary grades in Indonesia. It is not suitable for twelfth grade. Discussion of Short Functional Text In this research, it is found only one text (B1) in the book, which belongs to short 45 | IJET| Volume. 8, Issue 2. December 2019 functional text. It is in the form of a job vacancy. This text can be seen on pages 58-59. This text is included in short functional text because it follows the criteria stated in short functional text theory. After being input into Miyazaki readability test, it can be seen that the text hasa score of 54.3. It means it belongs to fairly difficult for style description and in 10th to 12th grade for estimated reading grade. From the score stated above, according to Miyazaki readability test, it can be seen that the short functional text in this book belongs to 10th to 12th grade. It is suitable for twelfth- grade students which is senior high school grade in Indonesia. This research found only one short functional text in this book. It doesn’t matter about how many short functional texts. What matter is whether it fulfills the need of the learners as argued by Cunningsworth as quoted by Miftachul Jannah (2017). Now the question is how to determine the learner’s need. The answer is to look back into core competence and basic competence, which are issued by the ministry of education and culture. One of the basic competences is “membedakan fungsi social, struktur teks, dan unsure kebahasaan beberapa teks khususdalambentuksuratlamarankerja, dengan memberi dan meminta informasi terkait jatidiri, latar belakang pendidikan/pengalaman kerja sesuai dengan konteks penggunaannya.” (to differentiate social function, text structure, and linguistic component of specific text in the form of application letter, by giving and asking information related to character, education/working experience background based on usage) It is believed by understanding short functional text such as vacancy, the basic competence will be achieved. Discussion of Reading Text There are fifteen texts in the book which are included in reading text. All of them are in the form of passages. Those fifteen texts are spread into each chapter in this book. Each chapter can have more than one text, such as chapter 5 which has three texts. After being input into Miyazaki readability test, it can be seen that there are two texts (C13 with score 65.2 and C11 with score 63.5) that belong to the standard for style description and 8th to 9th grade for estimated reading grade. There are seven texts (C12 with 59.7, C5 with 57.6, C3 with 56.0, C14 with 54.4, C1 with 51.9, C2 with 51.4 and C15 with 51.0) which belong to fairly difficult for style description and 10th to 12th grade for estimated reading grade. There are six texts (C10 with 39.7, C7 with 39.2, C4 with 38.4, C9 with 38.4, C6 with 38.2, C8 with 31.0) which belong to difficult for style description and post-school grade for estimated reading grade. The highest score of the text, which belongs to reading text in this book is 65.2. It means it belongs to the standard for style description and 8th to 9th grade for estimated reading grade. Meanwhile, the lowest score of this text is 31.0. It is included in fairly difficult for style description and post-school grade for estimated reading grade. The average of those fifteen texts is 49.1, which means the reading texts in this book belong to into fairly difficult for style description and post-school grade for estimated reading grade. From the highest and lowest score and average stated above, according to Miyazaki readability test, it can be seen that the reading texts in this book belong to university grade in Indonesia. It is not suitable for twelfth grade. Discussion of Instructional Text There are one hundred sixty-one texts in the book which are included in the instructional text. All of them are in the form of instructions for reading passages, asking the student to do teamwork, doing an assignment, etc. Those one hundred sixty-one texts are spread into each chapter in this book. 46 | IJET| Volume. 8, Issue 2. December 2019 After being input into Miyazaki readability test, it can be seen that there are nine texts which belong to easy for style description and 6th grade for estimated reading grade. There are forty texts which belong to fairly easy for style description and 7th grade for estimated reading grade. There are seventy-five texts which belong to the standard for style description and 8th to 9th grade for estimated reading grade. There are twenty-six texts which belong to fairly difficult for style description and 10th to 12th grade for estimated reading grade. There are eleven texts which belong to difficult for style description and post-school grade for estimated reading grade. The highest score of the text, which belongs to instructional text in this book, is 90. It means it belongs to easy for style description and 6th grade for an estimated reading grade. Meanwhile, the lowest score of this text is 32.0. It is included in fairly difficult for style description and post-school grade for estimated reading grade. The average of those one hundred sixty-one texts is 65.4, which means the instructional texts in this book belong to the standard for style description and 8th to 9th grade for an estimated reading grade. From the highest and lowest score and average stated above, according to Miyazaki readability test, it can be seen that the instructional texts in this book belong to 8th to 9th grade or junior high school in Indonesia. It is not suitable for twelfth grade. Discussion of Readability of Textbook The total data researched in this book are 180 texts. They are divided into three transactional and interpersonal texts (A1-A3), one short functional text (B1), 15 reading texts (C1-C15), and 161 instructional texts D1-D161). The most frequent data is from the instructional text. The highest Miyazaki readability test score in this book appears from instructional text number 19 (D19) which has 90 points. The lowest Miyazaki readability test score in this book appears from reading text number 8 (C8) which has 31.0 points. Texts in this book have variation in points range according to Miyazaki readability test from sixth grade until post-school/ college. There are 17 texts from this book that belong to post-school grade, 34 texts belong to 10th to 12th grade, 77 texts belong to 8th and 9th grade, 41 texts belong to 7th grade and 11 texts belong to 6th grade. The most frequent appear data in this book is the text which belongs to 8th and 9th grade which is 77 texts. Graphic 1. Data count per level The average score from all four texts component researched in this book is 63.2, which belong to the standard for style description and 8th to 9th grade for estimated reading grade. It means that this book is suitable for 8th and 9th grade according to Miyazaki EFL readability 47 | IJET| Volume. 8, Issue 2. December 2019 test. 4. CONCLUSION The conclusion of this study was gained from the result of readability analysis from the book. Based on the findings that have been explained in chapter four, four text components are researched in fromEnglish textbook for the twelfth grade of SMA, which is published by the Ministry of Education and Culture. They are transactional and interpersonal text, short functional text, reading text, and instructional text. To check the readability level of each text, Miyazaki EFL readability test is used in this research. It is found variation in the amount of each text. It is found three transactional and interpersonal texts (A1-A3), one short functional text (B1), 15 reading texts (C1-C15), and 161 instructional texts D1-D161). The total amount of the data is 180 texts. The readability score from each text component also varies. According to Miyazaki EFL readability test, the easiest text from the book is in 5th grade and the hardest one is in post- school/university grade. But the most frequent grade appear in this book is in 8th and 9th grade which is 77 data (texts). The average of this book score is 63.2 which means it is suitable for 8th and 9th grade. This book is too easy for the twelfth grade according to Miyazaki EFL readability test. From the conclusion above, the researcher suggests the government improve the readability level of the textbook to achieve the standard of 12th grade according to Miyazaki EFL readability test since the book is composed for the 12th grade. But the suggestion from the researcher is not as simple as that. Researchers suggest further research about government have concerned or not to the readability aspect. If the government has not concerned about it, then the government should consider it because readability is one of the aspects needed to make the students are interested in reading. If the government has considered it, then there is a possibility that the government intentionally make the book easier for the 12th grade. That’s why it is needed to do further research. REFERENCES Iftanti, E. (2012). A Survey of the English Reading Habits of EFL Students In Indonesia. TEFLIN Journal, 23(2), 149–164. https://doi.org/10.15639/teflinjournal.v23i2/149- 164 Mesmer, H. A., E. (2008). Tools for matching readers to texts: research-based practices. New York: Guilford Press. Mikki, J. (2000). Textbook: Future of the Nation. Frankfurt: Peter Lang. Nuttall, C. (1982). Teaching reading skills in a foreign language. (Practical language teaching series). Retrieved from https://journals.sagepub.com/doi/10.1177/003368828301400214 Nuttall, C. (2005). Teaching Reading Skills in a Foreign Language (3rd edition). Oxford: Macmillan Education. Richards, J. C. (2001). Curriculum Development in Language Teaching (1st ed.). Retrieved from https://www.cambridge.org/core/books/curriculum-development-in-language- teaching/CF5B24C0ADF73DE76BC6638B892661AF Sangia, R. (2015). Assessing Reading Text in English Textbook for Eleventh Grader Published by Kementrian Pendidikan Dan Kebudayaan.