Doğan Kahtalı, B. & Cayhan, C. (2021). Evaluation of 7th grade Turkish skill questions in terms of PISA reading skills and levels of qualification. International Online Journal of Education and Teaching (IOJET), 8(4). 2317-2333. Received :15.05.2021 Revised version received :22.08.2021 Accepted :22.08.2021 EVALUATION OF 7TH GRADE TURKISH SKILL QUESTIONS IN TERMS OF PISA READING SKILLS AND LEVELS OF QUALIFICATION (Research article) Bahar Doğan Kahtalı 0000-0001-6184-2306 (corresponding author) İnönü University, Turkey bahar.dogan@inonu.edu.tr Cihan Cayhan 0000-0001-8910-1198 Ministry of Education, Turkey dilcihanc@gmail.com Biodatas: Bahar Doğan Kahtalı is an Assistant Professor at the Department of Turkish and Social Sciences Education in Faculty of Education at İnönü University. She works on listening education and Turkish teaching. Cihan Cayhan is a Turkish language teacher at the Ministry of Education, Turkey. Copyright © 2014 by International Online Journal of Education and Teaching (IOJET). ISSN: 2148-225X. Material published and so copyrighted may not be published elsewhere without written permission of IOJET. mailto:bahar.dogan@inonu.edu.tr mailto:dilcihanc@gmail.com http://orcid.org/xxxx International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2317-2333. 2318 EVALUATION OF 7TH GRADE TURKISH SKILL QUESTIONS IN TERMS OF PISA READING SKILLS AND LEVELS OF QUALIFICATION Bahar Doğan Kahtalı bahar.dogan@inonu.edu.tr Cihan Cayhan dilcihanc@gmail.com Abstract The aim of this study is to present the qualities of 7th grade Turkish skill-based questions due to levels of reading skills and proficiency that used to prepare PISA reading skills questions. With these questions, secondary school students are prepared for national and international exams. The case study pattern has been used in the study, which is one of the qualitative research methods. The data obtained through document review were analyzed by content analysis by three field experts in Turkish education using a question analysis table. As a result of the examination it is determined that Turkish skill questions; include examples of all text types in PISA reading skills; mostly the text types which have continuations are preferred amongst the text types in PISA reading skills; big part of the questions are in a comprehensible level in terms of cognitive processes that the reader interacts with the text; and Turkish skill questions have problems for ''1b, 1a, 2nd and 3rd'' levels in the proficiency levels of PISA reading skills. As the result of the study, it is emerged that there is a necessity to provide a balanced dispersion for PISA reading skills, its dimensions and leves. In addition, there is also another necessity for arrangements to meet high level qualifications and Turkish skills questions should be created by taking into account all of the cognitive processes used in PISA that the reader interacts with the text. Keywords: PISA, reading skills, Skill-based questions 1. Introduction Many countries participate in the Programme for International Student Assessment (PISA), carried out by the Organization for Economic Co-Operation and Development (OECD). The first PISA, in which Turkey participated, was realized in 2003 and a total of 41 countries, including non-OECD countries, participated in this practice. This number increased to 65 in 2009 PISA and 76 in 2015. PISA was last carried out in 2018, with the participation of more than half a million students from 79 countries, 37 of which are OECD members (MEB, 2019a). The number of countries that want to participate in PISA is increasing every semester. The desire of non-OECD countries to participate in the PISA shows that this practice is valued by the education policymakers of countries. After the assessment, OECD prepares a report on reading skills, science and mathematics literacy of each country and shares these reports with participating countries. Participating countries can access reports of successful countries along with their own reports. "The results obtained from PISA give an idea in terms of seeing the problems in the education system of these countries and finding solutions to them." (Temizyürek & İnce, 2019: 158). Based on the data obtained as a result of examining the education systems of the countries that have been mailto:bahar.dogan@inonu.edu.tr mailto:dilcihanc@gmail.com Doğan Kahtalı & Cayhan 2319 successful in PISA and the reports showing the results of PISA, many countries take precautions regarding the problematic aspects of their education systems, change or update the programs in the education system, improve the infrastructure of their schools, and increase the quality of teacher education. In this context, the Ministry of National Education (MEB) rearranged the Primary Turkish Education Program in 2005 with a constructivist approach (İnce, 2016). PISA is conducted under three main themes as reading skills, science, and mathematics literacy, and one of these three themes is determined as the main theme each semester. Reading skills among these three main themes has the characteristic of affecting the other themes as well. Because students with advanced reading skills also become successful in other areas (İnce, 2016; Benzer, 2019; Batur and Alevli, 2014). Reading skills scores of Turkey, which have participated in PISA since 2003, indicate a periodic increase, except for the 2015 PISA (Akkuş, 2014; MEB, 2019a). Turkey has reduced its score difference with the OECD countries in 2018 to 21 (MEB, 2019a). When the reading skills proficiency levels of the students participating in PISA are examined, it is seen that they mostly score at the second level and below at the rate of 56.7% in 2009, 52.5% in 2012, 72.6% in 2015 and 58.7% in 2018 (MEB, 2016; MEB, 2019a). "In the developing and changing world, education is shaped by the purpose of teaching information to individuals, as well making them use this information, transfer it to life and adapt it to new situations. This shaping can be seen in the changes of educational programs, teaching techniques and methods and measurement instruments during the assesment phase." (MEB, 2016: 1). In this context, in order to increase the quality of education and to reach the OECD average, MEB changes the curriculum, opens workshop classes in schools, reorganizes the purpose, content, and question types of exams, diversifies the criteria and assessment methods used in measuring success, and updates teacher training programs (Aşıcı, Baysal and Şahenk Erkan, 2012; Demiral and Menşan, 2017; MEB, 2019a cited in Tuzlukaya, 2019; MEB, 2018:35, MEB, 2017). According to İnce (2016), one of the reasons for the preparation of the 2006 Primary Education Turkish Course Curriculum is the failure in international exams such as PISA and TIMMS. According to Özmusul (2012), on the other hand, studies should be carried out to increase the quality of learning in areas such as program development, teacher training, and professional development of teachers in order to increase the performance of Turkey in PISA. One of the last studies conducted in order to reach the OECD average in the field of reading skills has been the publication of Turkish Skills-Based Tests for 5th, 6th, and 7th grades as of 2019-2020 Academic Year by the Ministry of National Education General Directorate of Measurement, Evaluation and Examination Services. In the 2023 Education Vision report, where such studies will be carried out, it is stated that "New generation digital measurement tools will be developed to support high- level skills so that students can achieve the desired results in international exams such as PISA." (MEB, 2018: 74). MEB uses central exam questions consisting of multiple-choice questions, acquisition comprehension tests, sample questions and activities in the textbooks as measurement materials for Turkish lessons. Turkish course teachers also carry out assessment and evaluation activities with true-false questions and matching questions, short-answer questions and written exams, as well as multiple choice tests. The measurement materials prepared by MEB and Turkish course teachers generally consist of questions to measure sub-cognitive skills (Çeçen and Kurnaz, 2015; Göçer, 2008; Üstüner and Şengül, 2004; Yıldız, 2015). It is considered that student success in the field of PISA reading skills and other lessons is possible with the development of qualified teaching and measurement tools that can evaluate this. International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2317-2333. 2320 The reading skills that are expressed with the concept of "reading literacy" in PISA are defined as the capacity to understand, use, evaluate, reflect on, and interact with texts in order to achieve goals, improve knowledge and potential, and be included in social life (OECD, 2019: 27). The questions created within the scope of PISA, which considers reading skills as a part of social life, are prepared according to various criteria. These criteria, which are stated as reading skills dimensions and proficiency levels (OECD, 2019), are explained in detail in PISA and the questions are structured within the framework of these criteria. In the reading skills section, text types considered when preparing PISA questions, cognitive processes, and explanations regarding the questions are included. Cognitive processes in PISA consist of fluent reading, access to information, comprehension, and evaluation and reflection. Text types are categorized as source, organization and interaction, text format, and text type (MEB, 2019a). Here, it is necessary to touch upon the concept confusion that stands out in the dimensions of reading skills. In the 2018 Turkey PISA National Preliminary Report, it can be said that there is a definition problem regarding the text types and text format terms described under the heading "dimensions of reading skills." Reading skill dimensions terms specified as "text format" and "text type" in 2009, 2015, and 2018 PISA original texts are referred to as "metin şekli" (text shape) and "metin türü" (text type) in Turkey 2009, 2015, and 2018 PISA national reports. In PISA, while the term "text format" is divided into sub-categories such as "continuous text" consisting of structures such as sentences and paragraphs, "non-continuous text" consisting of tables, lists, graphics, indexes, and catalogs, and "mixed text" consisting of a paragraph with a graph or image, the term "text type" consists of sub-categories of description, narration, exposition, argumentation, instruction, and transaction. With the concept called "text format" in PISA and translated to Turkish as "metin şekli" (text shape), the texts in question are defined through their structural features and their appearance. For this reason, it can be said that the concept of "metin şekli" (text shape) is appropriate since it covers the sub-categories. However, the word "format" also means "shape, size, form, style." Ince and Gözütok (2017) used the concept of "metin formatı" (text format) in their work. The concept "text type" in PISA and the framework of which was determined based on the questions of why the texts were written and how they were organized (OECD, 2019a) was transferred to Turkish as "metin türü" (text type). In PISA reports translated into Turkish (MEB, 2019a; MEB, 2016) and in studies conducted in this field (Batur, Ulutaş, and Beyret, 2019; İnce and Gözütok, 2017; Tuzlukaya, 2019), the concept of "text type" has been tried to met with "metin türü" (text type). However, the concepts of "description, narration, exposition, argumentation, instruction, and transaction" under the title of "text type" in PISA are included under the title of "expression forms" in Turkish education. For this reason, it is thought that considering the concepts of "description, narration, exposition, argumentation, instruction, and transaction" as text type in PISA reports translated to Turkish and studies conducted in Turkish in the field of PISA reading skills will cause confusion. Because the term text type in Turkish education consists of the main headings of "narrative texts, informative texts, and poetry." Bozkurt (2018) and Şimşek and Müldür (2020) stated based on the studies in the field that linguists and language teachers have different perspectives on text type and revealed that the terms text kind/style (genre), text type, type of discourse are used in the field of linguistics and the term expression style/form is used in the field of language education studies instead of text type. The researchers stated that this situation caused confusion in text type teaching and it would be more correct to use the term text kind/style instead of the term expression form in terms of text linguistics. In this study, it was deemed appropriate to use the term "text style" instead of the term specified as "text type" in the PISA 2018 Turkey National Report. Doğan Kahtalı & Cayhan 2321 PISA reading skills proficiency levels are evaluated starting from the lowest level 1c to "1b, 1a, 2, 3, 4, 5" and at the 6th level which is the highest level (MEB, 2019a). Through the questions created according to these criteria, students are brought together with the new text types and forms offered by the digital environment by going beyond the standard multiple- choice tests, and it is determined which cognitive process the student uses and at what level of competence. The Vision 2023 document clearly states that new generation digital measurement materials that support high-level skills will be developed instead of the questions of standard multiple-choice tests used by the Ministry of National Education in many exams. With this, it is aimed that students achieve the desired success in international exams such as PISA. It is necessary to reveal to what extent Turkish skill-based questions developed in this direction meet the reading skills dimensions and proficiency levels taken into account when preparing PISA reading skills questions. With this study, skill-based questions were evaluated within the framework of PISA criteria and the rate of questions meeting these criteria was tried to be revealed. Thus, to what extent the questions developed to achieve success in international exams serve the purpose was determined, moreover, a contribution was made to the literature, and feedback was provided to those who prepared questions based on Turkish skills. The purpose of this study is to reveal the characteristics of the 7th grade Turkish skill- based questions published by the Ministry of Education in the 2019-2020 academic year, based on the reading skills dimensions and proficiency levels used when preparing PISA reading skills questions. In this framework, an answer is sought to the question "to what extent the Turkish skills-based questions meet the reading skills dimensions and proficiency levels used in the preparation of PISA reading skills questions?" . To achieve this goal, the following were tried to be determined; 1. Distribution of the first three themes published by the Ministry of National Education under the title of 7th grade Turkish Skills Based Questions in the 2019-2020 academic year by PISA reading skills text types, 2. Distribution of the first three themes published under the title of 7th grade Turkish Skill-Based Questions according to PISA reading skills text forms, 3. Distribution of the first three themes published under the title of 7th grade Turkish Skill- Based Questions according to PISA reading skills and cognitive processes with which the reader interacts with the text, 4. The distribution of the first three themes published in the 7th grade Turkish Skills Based Questions according to PISA reading skills proficiency levels was tried to be determined. 2. Method 2.1. Research Pattern This research is a qualitative study and it was carried out as a case study, one of qualitative research designs. A case study focuses on the in-depth description and analysis of a limited structure (Merriam, 2013). Case study is a method in which qualitative and quantitative data can be used in harmony (Yin, 2003). Case study was used to determine in detail the compliance of 7th grade Turkish Skills Questions with PISA reading skills dimensions and proficiency levels criteria. In this context, each of the Turkish Skill Questions was examined according to the PISA reading skills criteria. International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2317-2333. 2322 2.2. Research Data Source In this study, three test documents, each consisting of 20 questions, from the 7th grade Turkish Skill-Based Tests published in the period when the study was carried out by the General Directorate of Measurement, Assessment and Examination Services affiliated to the Ministry of National Education since the 2019-2020 academic year was used. The Ministry of Education has created and published skill questions at the 5th, 6th, and 7th grade levels. Through these questions, students are prepared for the secondary education transition exam. The fact that the study material was chosen among the 7th grade skill questions was particularly effective in the fact that 7th grade students showed parallelism with the students in the 15 age group, which constitutes the student profile of PISA by age. 60 questions extracted from the 7th grade Turkish Skills Based Tests named “Theme 1, Theme 2 and Theme 3” which were published at the time of the research, constitute the study material with consisting of tests. There are a total of eight themes in the 7th Grade Turkish textbook. The themes in the 7th grade Turkish Skill Based Tests are based on the themes (Virtues, National Struggle and Atatürk and Emotions…) in Turkish textbooks. Table 1. Turkish Skill Questions Theme Name Grade Level Number of Questions 1. Theme 7th grade 20 2. Theme 7th grade 20 3. Theme 7th grade 20 2.3. Collection of the Data Study data were obtained through document review. “Document review covers the processes of finding resources, reading, taking notes, and evaluation for a specific purpose. Data is collected by scanning existing records and documents in document review.”(Karasar, 2015: 183). In the research, document review was used as Turkish skill questions were examined in detail by three experts who have doctorates in Turkish education. The 7th grade Turkish Skill Questions PISA published by the General Directorate of Assessment, Selection and Assessment Services affiliated to the Ministry of National Education were subjected to document analysis according to their reading skills dimensions and proficiency levels. In order to obtain the study data, a "Question Analysis Table" was prepared by the researcher. PISA 2018 reading skills dimensions and proficiency levels, previously announced in 2015, 2012, and 2009 PISA reports were examined to create a "Question Analysis Table." While creating the "Question Analysis Table," the data of the latest 2018 PISA was used. In this arrangement, the fact that the 2018 PISA application has up-to-date data and its main theme is a "reading area" has been effective. PISA reading skills sample questions were examined and the opinion of a doctoral lecturer in Turkish education and three experts who had a doctorate in Turkish education were consulted. As a result of the investigations, it was decided to put the "Question Analysis Table" under two main headings: 1. Reading skills dimensions a) Different Text Types (Text Type, Text Format) Doğan Kahtalı & Cayhan 2323 b) Cognitive Processes by which the Reader Interacts with the Text (accessing information, comprehension, evaluating and reflecting) 2. Reading Skills Proficiency Levels (1c, 1b, 1a, 2, 3, 4, 5, 6) 2.4. Analysis of the Data Content analysis was used in this study. Content analysis is the gathering of data related to each other and organizing them in an understandable way (Akbulut, 2018: 186). In this context, content analysis of a total of 60 questions from the 7th grade Turkish Skills Based Tests was carried out according to PISA reading skills criteria. Three field experts examined the 7th grade Turkish Skill-Based Questions and added the data they obtained to the Question Analysis Table. Considering the order of the Question Analysis Table, firstly the text type of each question, then the text form were determined and marked on the table by the field experts. Then, all questions were examined to determine the cognitive processes in which the reader interacted with the text, and the results were encoded in the relevant section. Finally, the reading skills proficiency levels of the questions were determined by the Turkish field experts and added to the table. Question Analysis Tables marked by three field experts were compared. The different evaluations that emerged as a result of the comparison were concluded with the help of PISA Turkish question samples, PISA reading skills dimensions and proficiency levels, and expert opinions. A common question analysis table was created and these data were processed in graphics under the headings of "text type, text format, cognitive processes, and competence level." The relevant content of each title in the charts was arranged in a way to include percentage and frequency information and was included in the findings section. Objectively addressing and presenting the subject under investigation and clearly revealing the process followed, increase the validity and reliability of qualitative studies (Baltacı, 2019). In order to ensure validity, 7th grade Turkish Skill questions were examined by three Turkish field experts, and a common decision was tried to be reached by comparing the results of the examination. In addition, PISA reading skills dimensions, proficiency levels and sample questions were used during the analysis. The process from creating the question analysis table to processing the data on the graphs has been explained. 3.Findings In this part of the study, the findings of the sub-problems of the research are presented in figures. 3.1. Findings Regarding Text Types of 7th Grade Turkish Skill-Based Questions Distribution of 7th grade Turkish Skill-Based Questions published by the Ministry of National Education in the 2019-2020 academic year by text types is given in Figure 1. International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2317-2333. 2324 Figure 1. Distribution of 2019-2020 academic year 7th grade Turkish Skill-Based Questions by text types When Figure 1 is examined, it is seen that 7th grade Turkish Skill-Based Questions are in six different text types, and the explanation text type is preferred in half of the questions, that is, in thirty questions. It is seen that there are nine questions in story text type with a rate of 15%, eight questions in discussion type with a rate of 13%, and 5 problems in description and interaction type with 9% and 8%. The directive is the least frequently used text type. 3.2. Findings Regarding Text Formats of 7th Grade Turkish Skill-Based Questions The distribution of the 7th grade Turkish Skill-Based Questions published by the Ministry of National Education in the 2019-2020 academic year is given in Figure 2. Figure 2. Distribution of the 7th grade Turkish Skill-Based Questions for the 2019-2020 academic year by text formats 9% 15% 50% 13% 5% 8% Percentage and Numeric Distribution of Questions According to Text Types Description- 5 Narrative- 9 Explanation- 30 Discussion- 8 Directive- 3 Interaction- 5 73% 15% 12% Percentage and Numeric Distribution of Questions According to Text Formats Continuous Text- 44 Non-continuous Text- 9 Mixed Text- 7 Doğan Kahtalı & Cayhan 2325 As seen in Figure 2, 44 of the 7th grade Turkish Skill Questions were formed as a "continuous text" with a rate of 73%. The most preferred text form is the continuous texts made up of structures such as sentences and paragraphs. It was found that nine questions in the form of “non-continuous text” consisting of tables, lists, graphs, indexes, and catalogs with a rate of 15% were asked, and seven questions were asked in the form of “mixed text” consisting of a graphic or a paragraph with an explanation at a rate of 12%. 3.3. Findings Regarding 7th Grade Turkish Skill-Based Questions Regarding Cognitive Processes In Which Readers Interact With The Text Distribution of 7th grade Turkish Skill-Based Questions published by the Ministry of National Education in the 2019-2020 academic year according to the cognitive processes in which the reader interacts with the text is given in Figure 3. Figure 3. Distribution of the 2019-2020 academic year 7th grade Turkish Skill-Based Questions according to the cognitive processes in which the reader interacts with the text When Figure 3 is examined, it has been found that 68% of the 7th grade Turkish Skill- Based Questions are in the "comprehension" dimension, one of the cognitive processes in which the reader interacts with the text. It is seen that eighteen questions in the dimension of "accessing information" among the cognitive processes where the reader interacts with the text with a rate of 30% and two questions in the dimension of "evaluation and reflection" are asked among the cognitive processes in which the reader interacts with the text with a rate of 2%. In the creation of 7th grade Turkish Skill-Based Questions, among the cognitive processes in which the reader interacts with the text, the "comprehension" dimension was preferred the most, and the "evaluation and reflection" dimension was the least preferred. 3.4. Findings Regarding the Reading Skills Proficiency Levels of 7th Grade Turkish Skill-Based Questions The distribution of the 7th grade Turkish Skill-Based Questions published by the Ministry of National Education in the 2019-2020 academic year is given in Figure 4, according to their reading skills proficiency levels. 30% 68% 2% Percentage and Numeric Distribution of Questions According to Cognitive Processes Accessing Information-18 Comprehension- 41 Evaluation and Reflection- 1 International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2317-2333. 2326 Figure 4. Distribution of the 2019-2020 academic year 7th grade Turkish Skill-Based Questions by reading skills proficiency levels When Figure 4 is examined, it is seen that the reading skills proficiency levels consist of a total of eight levels from level 1c to level 6. It has been found that 7th grade Turkish Skill- Based Questions concentrate only on 4 levels of reading skills as "1b, 1a, 2nd and 3rd" levels. In creating 7th Grade Turkish Skill-Based Questions, the second level was preferred with a ratio of maximum 37% and twenty-two questions from the reading skills proficiency levels. Students at this level can identify the main idea of medium-length texts where the necessary information is not given explicitly, reflect on a chapter, and understand the meaning. Based on the data in the graph, nineteen questions were identified at level 3 with 32%. Students at this level can make simple and advanced inferences from texts that are not too long in which the information in the texts is not explicitly given, and can determine the main thought of the text by combining different information in the text. At 28% in Level 1a, there are seventeen questions. Students at this level can find the main idea of the text and make simple links based on sentences and short paragraphs that they are familiar with and where the information is clearly presented. There are two questions at 1b level with a rate of 3%. It was found that there are no questions for the "1c, 4, 5 and 6th" levels among the 7th grade Turkish Skill-Based Questions. At the 4th level, students are faced with long paragraphs in more than one text. Students can search and find the information embedded in these texts, make various inferences, and evaluate the reliability of the information source. It includes 5th and 6th level high-level questions. 5th level is the level at which students can find hidden information in long texts, establish a relationship between information in texts in different sources and questions, form hypotheses for questions, and distinguish between reality and perception. At level 6, which is not included in skill-based questions, students can grasp the information stored in long and abstract texts. Students at this level use advanced assessment, synthesis and analysis skills. 4. Conclusion and Discussion "Turkish Skill-Based Tests" for 5th, 6th and 7th grades started to be published in the 2019- 2020 academic year by the Ministry of National Education. In this study, the extent to which 60 questions under the title of 7th grade Turkish Skills-Based Tests meet the PISA reading 0% 0% 32% 37% 28% 3% 0% Percentage and Numeric Distribution of Questions According to Reading Skills Proficiency Levels Level 6 Level 5 Level 4 Level 3- 19 Level 2- 22 Level 1a- 17 Level 1b- 2 Level 1c Doğan Kahtalı & Cayhan 2327 skills dimensions and proficiency levels were examined and the data obtained were discussed. When the 7th grade Turkish Skill Questions are analyzed in terms of the text types used while preparing the PISA reading skills questions, it is seen that all text types are used. In the 7th grade skill questions, 30 questions constituting 50% of the 60 questions examined were explanation type, 9 questions constituting 15% were narrative type, the eight questions constituting 13% were discussion type, the five questions constituting 9% -8% were description and interaction type, and three questions constituting 5% were interaction type. While the description text type with 30 questions was the most preferred type among the text types, the interaction text type was the least preferred text type with 3 (5%) questions. There have been many studies that examine TEOG, LGS, Turkish lesson questions and Turkish lesson exam samples, textbooks in terms of text types used in PISA, and their results do not coincide with this research. İnce (2016) in the study that examined the texts in Turkish 6th, 7th, 8th grade textbooks, stated that text diversity was not provided in the textbooks and that texts in the narrative type were generally used; Aşıcı, Baysal, and Şahenk Erkan (2012) found in their study that mostly personal texts were used, which they found by comparing the characteristics of the 2009 SBS Turkish questions with the PISA reading skills questions. Tuzlukaya (2019) revealed that questions in the "discussion, narrative, description, explanation, and poetry" text types were included in the 8th grade Center Exam Questions for the 2nd term of the 2016-2017 academic year, while no questions were included in the instruction text type. Tuzlukaya (2019) revealed in his study where he examined the 8th grade Turkish lesson central exam questions in terms of PISA reading skills proficiency in the 2016-2017 academic year Semester II that; the 8th grade Central Examination Questions included in the text types of "discussion, story, description, explanation and poetry", while there are no questions in the text type of instruction. The study of Tuzlukaya is largely in parallel with this study. The use of the questions, which exemplify all six text types used in PISA, in Turkish skill questions is a positive development for students. Because when students encounter texts they are familiar with, they will develop a more positive attitude towards the question than texts they have never encountered before. Temizyürek and İnce (2019) stated that including questions for different types and these text types at all levels in the reading skills course will increase the efficiency of the course. Although it is an important development to include all text types specified in PISA in Turkish skills questions, studies should be carried out to ensure a balanced distribution between text types. In this way, it can be ensured that students encounter sufficient numbers of each text type. When the 7th grade Turkish skill questions are examined in terms of PISA reading skills text forms, it is seen that most of the questions are formed as "continued text". In addition, non-continued and mixed texts are also included. 44 questions, which make up 73% of the 7th grade Turkish skill questions, were in the continued text form. It was determined that the nine questions that make up 15% are non-continued text form, while the seven questions that make up 12% of them are in mixed text form. This result of the research is not similar to the results of other studies (Coşkun, 2013; İnce, 2016; İnce and Gözütok, 2017; Tuzlukaya, 2019) comparing PISA text formats with central exam questions, acquisitions, and textbooks. İnce (2016) states that the fact that non-continuous texts are not included in the textbooks directly affects the academic achievements of students in the field of reading skills. With advances in science and technology, individuals now encounter non-continued texts such as graphics, tables, lists, diagrams and catalogs, as well as continued texts composed of sentences or paragraphs. In this study, there are questions about all text forms, although there is not enough Turkish skill tests. In this context, it can be said that the statement "All exams International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2317-2333. 2328 in our education system in the 2023 Education Vision Document will be reorganized in the context of its purpose, content, structure depending on question types and the benefit it will provide." (MEB, 2018) has been realized. When the 7th grade Turkish skill questions were evaluated in terms of the cognitive processes in which the reader interacted with the text used in PISA reading skills questions, it was concluded that most of the questions were in the comprehension dimension. 41 of the 60 questions are in the dimension of comprehension, 18 of them are in the dimension of accessing information and only 1 of them is in the dimension of evaluation and reflection. The following findings are in parallel with the results of this study: Ince and Gözütok's (2017) study of the texts in Turkish textbooks in terms of the reader's approach to the text in PISA shows that the approach of gathering and interpreting the information was adopted in most of the texts; Aşıcı, Baysal and Şahenk Erkan (2012) stated that SBS items are mostly in the dimension of gathering and interpreting information in terms of readers' approaches to the text; Tuzlukaya's (2019) 8th grade Turkish center exam questions are in the dimension of accessing, gathering and interpreting information, and Demiral and Menşan's (2017) 8th grade Turkish lesson questions were prepared for the sole purpose of measuring knowledge, and they measured students' basic level skills. Bozkurt, Uzun, and Lee (2015: 308) found in their study that question types in the dimension of accessing and remembering information are more numerous in Turkish textbooks. When the 7th grade Turkish skill questions are considered in terms of the proficiency levels used in the PISA reading skills questions, it is stated that the skill questions are concentrated between the "1b" level and the 2nd level, and it is seen that there are no questions regarding the levels "1c, 4, 5 and 6". This finding of the study is in parallel with other studies in the field. The fact that curriculum acquisitions and textbook questions, questions at the 2nd level and lower than the PISA reading skills proficiency levels were preferred in central exam questions was also determined in the studies conducted by Aşıcı, Baysal and Şahenk Erkan (2012), Batur and Ulutaş (2013), Batur, Ulutaş and Beyret (2019), Benzer (2019), Demiral and Menşan (2017), İnce (2016), Tuzlukaya (2019). According to Batur and Ulutaş (2013), the 6th, 7th and 8th grade reading skills acquisitions represent the lower level competencies specified in PISA and there are no acquisitions representing high- level competencies. Considering that the commission that prepared the skill-based questions based on the Turkish Education Program (MEB, 2019b) and the Turkish course acquisitions, it is normal that the questions do not represent high-level competencies. It can be said that the program and acquisition levels reflect negatively on the question levels. Because, as Batur and Ulutaş (2013) stated, there are no direct high-level skills in the Turkish course acquisitions. Batur and Alevli (2014) suggest a different solution to the issue. Batur and Alevli (2014) state that the critical reading achievements of the Elective Reading Skills course coincide with the PISA reading skills high-level competencies and that the missing aspects of the Turkish course can be completed at the point of high-level skills with the Elective Reading Skills course. The results achieved in the study can be listed as follows: • It was found that the 7th grade Turkish skills questions, which were created by the Ministry of National Education in the 2019-2020 academic year, were created using one of the types of description, story, explanation, discussion, instruction, and interaction in terms of text types. Among the text types, explanation and instruction were least frequently used. • It has been observed that most of the 7th grade Turkish skill questions published by the Ministry of National Education in the 2019-2020 academic year are in the form of Doğan Kahtalı & Cayhan 2329 "continuation text," and the text and mixed text forms that are not followed by a small number are also used. • It was found that the 7th grade Turkish skill questions published by the Ministry of National Education in the 2019-2020 academic year were created by focusing on the dimension of comprehension and accessing information in terms of cognitive processes. • It was found that the 7th grade Turkish skill questions published by the Ministry of National Education in the 2019-2020 academic year were concentrated on 4 levels as "1b, 1a, 2nd and 3rd" levels, and there were no questions for the "1c, 4, 5 and 6th" levels. Based on the data obtained as a result of evaluating the 7th grade Turkish skill questions prepared by the Ministry of National Education in the 2019-2020 academic year according to the dimensions and proficiency levels of PISA reading skills, the following suggestions can be made: • While creating Turkish skill questions, questions regarding the text form and all text types should be included in a balanced way, taking into account the features of each type and format. • Turkish skills questions should be created by taking into account all of the cognitive processes used in PISA that the reader interacts with the text. International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2317-2333. 2330 References Akbulut, Y. (2018). Veri çözümleme teknikleri. A. Şimşek (Ed), Sosyal bilimlerde araştırma yöntemleri (ss. 162-195). Eskişehir: Anadolu Üniversitesi. Akkuş, M. (2014). PISA, TIMSS ve PIRLS sonuçlarının değerlendirilmesi (Yayımlanmamış Yüksek Lisans Tezi). İstanbul Aydın Üniversitesi, Sosyal Bilimler Enstitüsü, İstanbul. Aşıcı, M., Baysal, N. & Erkan, S. (2012). A comparision of the reading comprehension questions in PISA 2009 and national level determination exam in Turkey. Journal of Research in Education and Teaching, 1(2), 210-217. Baltacı, A. (2018). Nitel araştırmalarda örnekleme yöntemleri ve örnek hacmi sorunsalı üzerine kavramsal bir inceleme. Bitlis Eren Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 7(1), 231–274. Baltacı, A. (2019). Nitel araştırma süreci: Nitel bir araştırma nasıl yapılır?. Ahi Evran Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 5(2), 368-388. Batur, Z. ve Ulutaş, M. (2013). Pisa ile Türkçe öğretim programındaki okuduğunu anlama kazanımlarının örtüşme düzeylerinin incelenmesi. The Journal of Academic Social Science Studies, 6(2), 1549-1563. Batur, Z., Ulutaş, M. ve Beyret, T. N. (2019). 2018 LGS Türkçe sorularının PISA okuma becerileri hedefleri açısından incelenmesi araştırma makalesi. Milli Eğitim Dergisi (Özel Sayı), 48(1), 595-615. Batur, Z., ve Alevli, O. (2014). Okuma becerileri dersinin PISA okuduğunu anlama yeterlilikleri açısından incelenmesi. Okuma Yazma Eğitimi Araştırmaları, 2(1), 22-30. Benzer, A. (2019). Türkçe ders kitaplarının PISA okuma yeterlik düzeyleri ile imtihanı. Okuma Yazma Eğitimi Araştırmaları, 7(2), 96-109. Bozkurt, B. Ü. (2018). Okuma ve yazma eğitiminde metin türlerini sınıflama sorunsalı ve çoklu/karma türlerin öğretimiyle ilgili öneriler. N. E. Uzun & B. Ü. Bozkurt (Haz.), Türkçenin eğitimi öğretiminde kuramsal ve uygulamalı çalışmalar-10 (ss. 75-99). İstanbul: Okan Üniversitesi. Bozkurt, B. Ü., Uzun, G. L. ve Lee, Y. (2015). Korece ve Türkçe ders kitaplarındaki metin sonu sorularının karşılaştırılması: PISA 2009 sonuçlarına dönük bir tartışma. International Journal of Language Academy, 3(4), 295-313. Coşkun, Y. D., (2013). Türkçe ders kitaplarının PISA sınavı okuma ölçütleri açısından incelenmesi. Mehmet Akif Ersoy Üniversitesi Eğitim Fakültesi Dergisi, 13(26), 22-43. Çeçen, M ve Kurnaz, H. (2015). Ortaokul Türkçe dersi öğrenci çalışma kitaplarındaki tema değerlendirme soruları üzerine bir araştırma. Karadeniz Sosyal Bilimler Dergisi, 7 (2). Retrieved from https://dergipark.org.tr/tr/pub/ksbd/issue/16219/169872 Demiral, H. ve Menşan, Ö. H. (2017). Sekizinci sınıf Türkçe dersinin PISA okuma becerilerine göre değerlendirilmesi. Ö. Demirel & S. Dinçer (Ed), Küreselleşen Dünyada Eğitim (ss. 567-580). Ankara: Pegem Yayınları. Göçer, A. (2008). Analysis of primary education Turkish course books in terms of measurement and evaluation. Atatürk University Journal of the Institute of Social Sciences, 11(1), 197-210. İnce, M. (2016). Türkçe 6, 7, 8. sınıf öğretim programının uluslararası öğrenci değerlendirme programı'nda (PISA) yoklanan "okuma becerileri" açısından analizi https://dergipark.org.tr/tr/pub/ksbd/issue/16219/169872 Doğan Kahtalı & Cayhan 2331 (Zonguldak örneği)(Yayımlanmamış Doktora Tezi). Ankara Üniversitesi, Eğitim Bilimleri Enstitüsü, Ankara. İnce, M. ve Gözütok, F.D. (2017). Türkçe 6, 7, 8. sınıf ders kitaplarının PISA okuma becerilerine göre incelenmesi. Turkish Studies 12(33), 213-225. Karasar, N. (2015). Bilimsel araştırma yöntemi (28.basım). Ankara: Nobel Yayın Dağıtım. MEB, (2010). 2009 Ulusal ön raporu. Millî Eğitim Bakanlığı, Eğitimi Araştırma ve Geliştirme Dairesi Başkanlığı, Ankara. Retrieved from Retrieved from http://pisa.meb.gov.tr/wp-content/uploads/2013/07/PISA-2009-Ulusal-On-Rapor.pdf MEB, (2016). PISA 2015 ulusal raporu. Millî Eğitim Bakanlığı, Ölçme, Değerlendirme ve Sınav Hizmetleri Genel Müdürlüğü, Ankara. Retrieved from http://odsgm.meb.gov.tr/test/analizler/docs/PISA/PISA2015_Ulusal_Rapor.pdf MEB, (2018). 2023 eğitim vizyonu. Millî Eğitim Bakanlığı. Retrieved from http://2023vizyonu.meb.gov.tr/doc/2023_EGITIM_VIZYONU.pdf MEB, (2019a). PISA 2018 Türkiye ön raporu. Millî Eğitim Bakanlığı, Eğitim Analiz ve Değerlendirme Raporları Serisi, Ankara. Retrieved from http://www.meb.gov.tr/meb_iys_dosyalar/2019_12/03105347_PISA_2018_Turkiye_O n_Raporu.pdf MEB. (2019b). İlköğretim Türkçe dersi öğretim programı (1-8. Sınıflar). Ankara: MEB. Retrieved from https://mufredat.meb.gov.tr/Dosyalar/20195716392253-02 Türkçe%20Öğretim%20Programı%202019.pdf Merriam, S. B. (2013). Nitel araştırma: desen ve uygulama için bir rehber ( Çeviren Ed. S. Turan). Ankara: Nobel Yayın Dağıtım. OECD (2019). PISA 2018 results (volume I): What Students Know and Can Do, PISA. OECD Publishing, Paris. Retrieved from https://doi.org/10.1787/5f07c754-en. Özmusul, M. (2012). Examination of the education system targets in vision 2023 study in terms of international indicators. Journal of Turkish Science Education, 9(3), 97-114. Şimşek N.D. ve Müldür, M. (2020). Pedagojik bakış açısıyla metin tipleri, türleri ve yapıları. Ankara: İksad Yayınevi. Temizyürek, F., & İnce, V. (2019). The content research of the outcomes of reading skills course and PISA reading skills level. International Journal of Language Academy, 7(3), 157-166. http://dx.doi.org/10.29228/ijla.25823 Tuzlukaya, S. (2019). 8th class Turkish lessons exam questions examined in regard to PISA reading sufficiency skills. The Journal of International Lingual, Social and Educational Sciences, 5(1), 92-100. https://doi.org/10.34137/jilses.505073 Üstüner, A. & Şengül, M. (2004). Çoktan seçmeli test tekniğinin Türkçe öğretimine olumsuz etkileri. Fırat Üniversitesi Sosyal Bilimler Dergisi, 14(2), 197-208. Yin, R. K. (2003). Case study research: Design and methods (3rd ed.). Thousand Oaks, CA: Sage. Yıldız, D. Ç. (2015). Türkçe dersi sınav sorularının yeniden yapılandırılan Bloom taksonomisine göre analizi. Gaziantep Üniversitesi Sosyal Bilimler Dergisi, 14(2), 479- 497. http://odsgm.meb.gov.tr/test/analizler/docs/PISA/PISA2015_Ulusal_Rapor.pdf http://2023vizyonu.meb.gov.tr/doc/2023_EGITIM_VIZYONU.pdf http://www.meb.gov.tr/meb_iys_dosyalar/2019_12/03105347_PISA_2018_Turkiye_On_Raporu.pdf http://www.meb.gov.tr/meb_iys_dosyalar/2019_12/03105347_PISA_2018_Turkiye_On_Raporu.pdf International Online Journal of Education and Teaching (IOJET) 2021, 8(4), 2317-2333. 2332 Appendices Appendix 1: Question Analysis Table Appendix 2: Sample Questions Sample Question 1. 7th grade Turkish Skill-Based Theme 1 Question 1 Doğan Kahtalı & Cayhan 2333 Sample Question 2. 7th grade Turkish Skill-Based Theme 1 Question 8