EEJ 9 (2) (2019) 164 - 171 English Education Journal http://journal.unnes.ac.id/sju/index.php/eej Developing Problem-Solving Based Assessment to Stimulate Critical Thinking and Creativity of Students’ Writing Skill Rifki Nurlaili Hidayat, Dwi Rukmini, Dwi Anggani Linggar Bharati Universitas Negeri Semarang, Indonesia Article Info ________________ Article History: Recived 19 October 2018 Accepted 15 February 2019 Published 20 June 2019 ________________ Keywords: Assessment, writing, problem-solving, critical thinking, creativity ____________________ Abstract ___________________________________________________________________ This study was focused on the issues related to the development of problem-solving based assessment model especially in term of writing assessment to stimulate the critical thinking and creativity of the students. Research and Development (R&D) was used as a research design in this study. The subjects of this study were the students of X MIPA 6 at SMA Sultan Agung 1 Semarang in the academic year of 2017/2018. Questionnaire, interview, observation checklist, and tests were used as the instruments in collecting the data. In addition, the researchers collaborated with the teacher in developing the problem-solving writing assessment module. The result of this study revealed that there was a significant improvement of the students’ writing skill, critical thinking skill, and creativity skill. It was proven by the mean score results of the pre-test and post-test. The students’ score improvement of the writing skill was 47.83 to 70.83, and critical thinking and creativity skill was 42.67 to 60.13. Moreover, the paired sample t-test results showed that there was significant different between the result of pre-test and post-test. In conclusion, the problem-solving based assessment is applicable to stimulate the critical thinking and creativity of the students’ writing skill. 2019 Universitas Negeri Semarang Correspondence Address: Kampus Pascasarjana Unnes, Jl. Kelud Utara III Semarang 5023, Indonesia E-mail: rifki.nh@students.unnes.ac.id p-ISSN 2087-0108 e-ISSN 2502-4566 Rifki Nurlaili Hidayat, Dwi Rukmini, Dwi Anggani Linggar Bharati/ EEJ 9 (2) 2019 164 - 171 166 INTRODUCTION Assessment is an important component in the implementation of the 2013 Curriculum. assessment and learning cannot be separated due to the assessment result is also used to enhance the learning. As Stiggins (2001) explained that assessment is the process of gathering information about student learning for improvement of learning. Thus, the assessment that is applied by the educators (teachers), educational units (schools), and government should be well implemented based on or referring to the 2013 Curriculum. The assessment in the 2013 Curriculum is closely related to critical thinking skills. According to the Fisher (2001), critical thinking is that mode of thinking about any subject, content or problem in which thinkers improve the quality of their thinking by skillfully taking over structures inherent in thinking and imposing intellectual standards on them. One way to develop critical thinking of the students is with applying problem-based learning and problem-based assessment. As has been stated by Shakirova (2007) that with the existence of critical thinking skills on students, it may enable students to deal effectively with social, scientific, and practical problems. It means students who are able to think critically are able to solve problems effectively. Therefore, merely having knowledge or information is not enough. In order for students to be able to solve problems and be able to make effective decisions in their personal lives, they must be able to think critically and creatively. The terms of assessment can be applied to various aspects of language skills, i.e. listening, speaking, reading, and writing. One of the four language skills that have a level of difficulty in learning that is writing skill. This skill involves high-level thinking skills because it requires students to come up with ideas and creativity in the form of works. Brown (2001) claimed that writing is a thinking process. Further, he states that writing can be planned and given with an unlimited number of revisions before its release. Thus, to enhance the learning, teachers should be able to integrate learning and assessment effectively in order to stimulate critical thinking and creativity of the students. Related to the research on the development of problem-solving writing assessment to stimulate the students’ critical thinking and creativity, several studies had become the background as a reference on this study. The first previous study was about assessment especially about authentic assessment that was conducted by Aliningsih and Sofwan (2015), Wati, Bharati and Hartono (2014), Marhaeni and Artini (2015), Nurgiyantoro and Suyata (2009), Wahyuni (2010) and Merta et al (2015). In their research, they generally stated that authentic assessment is an assessment requiring students to use the same competencies, or combinations of knowledge, skills and attitudes, that they need to apply in their criterion situation in professional life. The result of the study indicated that the teachers need assistance in providing various types of authentic assessment instruments that can be used in learning. The next previous study is about critical thinking and creativity itself. Hanurawan and Waterwoth (2007), Muhfahroyin (2009), Khoiri et al (2017), Areni and Syafri (2015), Zaida and Sofwan (2015), Kristanto and Susilo (2015), Sulaiman et al (2017), Barnet and Francis (2013) had conducted a study on critical thinking and creativity in different ways but with the same goal, that was to analyze and to enhance the critical thinking and creativity of the students. Generally, the results of their research stated that the ability to think critically and creatively was needed by the students to face the real- world problems today. Therefore, it was very important to be owned by every student. Their previous studies could be used as practical guidance for our study, because it was relevant with our present study especially in term of assessment, critical thinking and creativity related to the 2013 curriculum. However, our research was not the same as those above who only describe the authentic assessment, critical thinking and creativity in general as well as the problems faced by the Rifki Nurlaili Hidayat, Dwi Rukmini, Dwi Anggani Linggar Bharati/ EEJ 9 (2) 2019 164 - 171 167 teacher, but this research focused more on how to develop that assessment especially problem- solving based assessment model to stimulate the critical thinking and creativity on a certain skill in English language teaching. In conclusion, from the discussion above, it can be seen that the assessment model that should be implemented by the teacher in the teaching and learning process can influence the students’ learning outcomes. Therefore, the study on the development of assessment model was very important to be implemented. Thus, in this study, the researchers tried to conduct the research about the assessment model that used by the teacher and tried to develop it especially in the development of problem-solving based assessment model to stimulate the critical thinking and creativity of the students’ writing skill. METHODS In accordance with the problem and the purpose of the research, this research was designed in the form of Research and Development (R & D) based on Borg and Gall (1983) theory. The steps of research and development in this study was simplified into three main stages, and every stage have some main activities. The first stage was Define and Design Stage. This stage consists of two main activities; (a) research and collecting information, (b) planning to design writing assessment. The second stage was Development Stage. This stage also consists of two main activities; (a) developing of preliminary form of writing assessment, (b) tray out of developed product. And the third stage was Final Stage; producing final product in the form of a set of assessment (module) to develop problem- solving writing assessment to stimulate the students’ critical thinking and creativity. In addition, the subjects of this study were the English teacher and tenth grade students of SMA Sultan Agung 1 Semarang in the academic year 2017/2018. The school was chosen because the 2013 curriculum has been implemented in all classes, especially in tenth grade. Besides questionnaire, interview, observation checklist, and tests were used as the instruments in collecting the data in this study. RESULT AND DISCUSSIONS In this phase, the researchers present the findings and discussions of the developing problem-solving based assessment model to stimulate the critical thinking and creativity of the students’ writing skill. The sections in this part will follow the statement of the research objectives. Define and Design Stage This stage was a stage to identify the problems that exist in the learning process and assessment and become the basis for designing product in the form of a set of assessment to be made. There were two main activities that have been done by the researcher at this stage: (a) research and collecting information, (b) planning to design writing assessment. Research and Collecting Information The step that had been done for collecting information in this research were observation, interview and questionnaire. To get the information about the existing assessment used by the teacher in order to analyze the students’ need, the researchers conducted preliminary research by observing the documents relating to the writing assessment. Based on the analysis of documents, the researchers got the following findings: 1) the learning devices used by the teacher were complete enough, they were syllabus, lesson plan (RPP), textbooks and LKS. 2) the syllabus and lesson plan used by the teacher came from MGMP team that had been revised in accordance with the conditions of the school. 3) the students’ daily test scores were still under the KKM. 4) the assessment that was used by the teacher still did not reflect critical thinking or creative assessment. Based on this analysis, the researchers concluded that the documents used and those made by the teacher were quite suitable to be used for teaching, because if it viewed from the truth of the concept, it was included in the good criteria, it Rifki Nurlaili Hidayat, Dwi Rukmini, Dwi Anggani Linggar Bharati/ EEJ 9 (2) 2019 164 - 171 168 was in accordance with the curriculum that should be used, but unfortunately it was not in accordance with the problems that students must face in their daily lives, or in the other words it had not trained students to be critical thinking and creative. Besides using the observation, the researchers also collected the information through interview and questionnaire towards the English teacher and the students. Based on the result of questionnaire and interview indicated that the teacher knew about Problem-based Learning (PBL) and he had ever applied PBL for assessing writing, but it was not maximal. Because there were several obstacles such as lack of time in learning and lack of teacher’s knowledge about how to assess the students using this model of assessment. On the other hand, the lack of skills and knowledge of students in solving the problem was also one of the obstacles faced. This was due to the teacher only asked the students to write a text and assessed the students’ writing by looking at the text structure, and grammar. The students seemed to be passive since the activity in the teaching and learning process. The students just asked to write a text without any stimulation of being creative and critical learners. As a result, the researcher concluded that the teacher needed to develop problem-based writing assessment which could stimulate students’ critical thinking and creativity. Planning to Design Writing Assessment After collecting information about the students’ needs and problems, the next step the researchers made a preparation to design preliminary draft of the problem-solving writing assessment model, which was referred to the mapping of core competences and basic competences, syllabus, and lesson plan based on 2013 curriculum. At this planning stage, preparations were made relating to the preparation of making the problem-solving writing assessment model. The planning stages that was done in this research included several stages as follows: (1) Identification of aspects contained in Core Competencies and Basic Competencies and identification of material. (2) Preparation of syllabus, lesson plan (RPP) and questions for evaluation. (3) Preparation of validation sheet of expert judgment. Development Stage The next step was the development stage. There are two main activities that was done in this stage: Developing of Preliminary Form of Writing Assessment In this case, the researchers helped the English teacher to develop problem-solving writing assessment model with the purpose was to improve the students’ critical thinking and creativity. Before developing the preliminary product, the first thing we did was determine the core and basic competence that would be used in accordance with the syllabus of senior high school. In this study we would use the core and basic competence 3.8 and 4.8 which was related to the material of Narrative Text. After determining the core and basic competence, the next stage we made lesson plan that was in accordance with the demands of 2013 Curriculum. In this case, the lesson plan we made was focused on writing skill and problem-based learning. The implementation of the lesson plan was divided into 4 meetings. After making the lesson plan, the next stage the researchers assisted the teacher in making several questions related to the narrative text. The arrangement of questions was started from the simple questions to the critical questions which related to the problem-solving model, so that through these questions, it was expected that students can develop their creativity and critical thinking. In addition, to measure the critical thinking and creativity we made the rubric for measuring critical thinking and creativity. The rubrics were adapted from Assessing Outcomes and Improving Achievement: Tips and tools for Using Rubrics, edited by Rhodes, Copyright 2010 by the Association of American Colleges and Universities. Rifki Nurlaili Hidayat, Dwi Rukmini, Dwi Anggani Linggar Bharati/ EEJ 9 (2) 2019 164 - 171 169 After all the preliminary form of writing assessment was made, then it would be validated by the expert judgment. The expert judgment consisted of two experts, they were Prof. Dr. Suwandi, M.Pd (Senior Lecturer at UPGRIS) and Farida Fahmalatif, S.Pd., M.Pd (Senior English Teacher at SMAN 1 Jambu). The validation was carried out in two stages; the validation of lesson plan and the validation of assessment product. The following are the results of validation of two experts. Figure 1. The Validation Result of Lesson Plan Based on the figure above, it can be seen that the validation result of the appropriateness of learning objectives got the mean score 4.2; the appropriateness of contents got the mean score 3.5; the appropriateness of language use got the mean score 3.3; and the appropriateness of time allocation got the mean score 4. The acquisition of scores for each aspect came from the acquisition of the average score of each indicator. Figure 2. The Validation Result of Assessment Product Based on the figure 4.2 above, it can be seen that the validation result of the appropriateness of the contents got the mean score 4.2; the appropriateness of the presentation got the mean score 3; the appropriateness of the language got the mean score 3.4; and the appropriateness of the lay-out got the mean score 3.6. The acquisition of scores for each aspect came from the acquisition of the average score of each indicator. On the other hand, besides giving a score in every aspect they also gave some suggestion which their suggestions were very helpful for the researchers in perfecting the product. The first, Mrs. Farida Fahmalatif said that the main activities that were made in the lesson plan must be in accordance with the problem-based learning model, and students achievement indicators must be as detailed as possible. The second, Prof. Dr. Suwandi advised us to make some questions that lead to the problem-solving questions and the steps in problem-solving writing assessment module must in line with the lesson plan. Try-out of Developed Product (Field Testing) The try out was conducted in the class X MIPA 6 and the class consisted of 30 students. A try out or field tasting was conducted to test the effectiveness of product that had been developed. Product effectiveness testing was carried out by using an experimental design (Before-after / One Group Pre-test Post-test Design). The students were given an initial test (pre-test) to find out how far the initial ability of the students before being given learning material of writing using a problem-solving based assessment model. After being given the initial test (pre-test), the students were given the treatment. Further after being given the treatment, the students were given the final test (post-test) to determine the extent of the influence of the writing assessment based on the problem-solving model on the development of students’ critical thinking and creativity. The following table were the results of the try-out of the developed product which include the result of pre-test and post-test of writing skill, and the Rifki Nurlaili Hidayat, Dwi Rukmini, Dwi Anggani Linggar Bharati/ EEJ 9 (2) 2019 164 - 171 170 result of pre-test and post-test of critical thinking and creativity skill. Table 1. The Result of Pre-Test and Post-Test of Writing Skill Mean Pre-Test Mean Post Test 47.83 70.83 The table above shows that the mean score of pre-tests was 47.83 and the mean score of post-tests was 70.83. It means that the mean score from pre-test to post-test has increased. From the result above, it can be concluded that there was a significance difference between pre-test and post-test, because the students’ ability was different after got the treatment, so the developed problem-solving writing assessment model was effective, and it can be used as writing assessment in the process of teaching and learning process in the class. In addition to measure the level of students’ writing skill in general, here the researchers also measured the level of critical thinking and creativity of the students, because the main purpose of developing this product was to stimulate students' critical thinking skill and creativity. The following were the results of pre- test and post-test of critical thinking and creativity skill. Table 2. The Result of Pre-Test and Post-Test of Critical Thinking and Creativity Skill Mean Pre-Test Mean Post Test 42.67 60.13 Based on the results of the statistical test above it can be seen that there were significant differences in the ability of the students’ critical thinking and creativity before and after being given the treatment. The mean score of the students increased from 42.67 to 60.13. Therefore, it can be concluded that the developed problem-solving writing assessment model was effective to stimulate the students’ critical thinking and creativity. Producing Final Product After the product is tested and revised, the last stage in this research and development was producing final product. The product of this research and development was a set of assessment which it was finalized with some interface designs (module). Based on the findings presented above, it can be concluded that the findings of our study were in line with some of the findings of previous studies. One of the early findings presented that High Order Thinking (HOT) assessment instrument as assessment for learning was effective to train student’s High Order Thinking Skill (HOTS) and effective measured student's thinking skills in accordance with the level of each student's thinking (Kusuma, Rosidin, Abdurrahman, Suyatna, 2017 and Hairida, 2016). In addition, Widana (2017) also stated in his findings that HOTS assessment could increase the students’ motivation to learn and improve the students’ learning outcomes. Their findings were in line with our study since the results of our findings also produced an effective assessment product that can improve the students’ critical thinking and creativity. CONCLUSION AND SUGGESTION Based on the result and the discussion above, the researchers concluded that the teacher actually had ever applied problem- solving for assessing writing, but it was not maximal. Because there were some obstacles faced by the teacher. One of the biggest obstacles was lack of the teacher’s knowledge about how to assess the students using this model of assessment. Thus, here the researchers tried to help the teacher to develop problem-based writing assessment which is expected to stimulate students’ critical thinking and creativity. After the developed product was completely developed by us and validated by the expert judgment, then it was tested to determine its effectiveness. The results of the test showed Rifki Nurlaili Hidayat, Dwi Rukmini, Dwi Anggani Linggar Bharati/ EEJ 9 (2) 2019 164 - 171 171 that the problem-solving writing assessment had positive effect toward the students’ critical thinking and creativity. It can be proven from the students’ mean score in pre-test and post-test. The results of the statistical tests showed a significance value lower than the significance level α = 0.05, so it could be concluded that the developing problem-solving based assessment module was effective to stimulate the students’ critical thinking and creativity especially in writing skill. Furthermore, in order to give contribution to research development in the assessment field, the results of this study are expected to be used as a reference for developing similar research, especially the research and the development of assessment modules in learning English. The other researchers can develop assessment modules with different learning model characteristics and material. REFERENCES Aliningsih, F., & Sofwan, A. (2015). English Teachers’ Perceptions and Practices of Authentic Assessment. Language Circle: Journal of Language and Literature. X (1), 19-27. Areni, G. K., & Syafri, F. (2015). Critical Thinking in Teaching Writing Book Review. LANGUAGE CIRCLE Journal of Language and Literature. IX (2), 141- 150. Barnett, J. E., & Francis, A. L. (2012). Educational Psychology: An International Journal of Experimental Educational Psychology. Educational Psychology. 32 (2), 201-211. Brown, H. D. (2001). Teaching by principles: an interactive approach to language pedagogy. New York: Pearson Education. Fisher, A. (2001). Critical Thinking: An Introduction. UK: Cambridge University Press. Hairida. (2016). The Effectiveness Using Inquiry Based Natural Science Module with Authentic Assessment to Improve the Critical Thinking and Inquiry Skills of Junior High School Students. Jurnal Pendidikan IPA Indonesia. 5 (2), 209- 215. Hanurawan, F., & Waterworth, P. (2007). Teachers' Perception of Developing students' Critical Thinking thorugh Controversial Issues Discussion. Jurnal Ilmu Pendidikan. 14 (3), 185-194. Khoiri, Basuki, I. A., & Priyatni, E. T. (2017). Assessment of Exposition Text to Develop Critical Thinking Skills for Junior High School Students. International Conference on Language, Society and Culture in Asian Contexts, KnE Social Science, 347–357. Kristanto, Y. E., & Susilo, H. (2015). Pengaruh Model Pembelajaran Inkuiri Terbimbing terhadap Kemampuan Berpikir Kritis dan Hasil Belajar IPA Siswa Kelas VII SMP. Jurnal Pendidikan dan Pembelajaran. 22 (2) , 197-208. Kusuma, M. D., Rosidin, U., Abdurrahman, & Suyatna, A. (2017). The Development of Higher Order Thinking Skill (Hots) Instrument Assessment In Physics Study. IOSR Journal of Research & Method in Education. 7 (1), 26-32. Marhaeni, A., & Artini, L. P. (2015). Asesmen Autentik dan Pendidikan Bermakna: Implementasi Kurikulum 2013. Jurnal Pendidikan Indonesia. 4 (1), 499-511. Merta, I. M., Suarjana, I. M., & Mahadewi, L. P. (2015). Analisis Penilaian Autentik Menurut Pembelajaran Kurikulum 2013 pada Kelas IV SD No. 4 Banyuasri . e- Journal PGSD Universitas Pendidikan Ganesha. 3 (1), 1-10. Muhfahroyin. (2009). Memberdayakan Kemampuan Berpikir Kritis Siswa Melalui Pembelajaran Konstruktivistik. Jurnal Pendidikan dan Pembelajaran. 16 (1), 88-93. Nurgiyantoro, B., & Suyata, P. (2009). Pengembangan Model Asesmen Otentik dalam Pembelajaran Bahasa. Cakrawala Pendidikan. XXVIII (3) , 224-237. Shakirova. (2007). Technology for the shaping of college students’ and upper-grade students’ critical thinking. Russian Education & Society, 42–52. Rifki Nurlaili Hidayat, Dwi Rukmini, Dwi Anggani Linggar Bharati/ EEJ 9 (2) 2019 164 - 171 172 Stiggins, R. (2001). Student-Involved Classroom Assessment. New Jersey: Merrill Prentice Hall. Sulaiman, T. (2017). Implementation of Higher Order Thinking Skills in Teaching Of Science: A Case Study in Malaysia. International Research Journal of Education and Sciences. 1 (1), 1-3. Wahyuni, S. (2010). Pengembangan Model Asesmen Otentik dalam Pembelajaran Keterampilan Berbahasa Indonesia Lisan di Sekolah Menengah Atas (Sma). LITERA. 9 (1), 69-79. Wati, A., Bharati, D.A.L., Hartono, R. (2014). The scientific approach in teaching speaking for various texts. English Education Journal (EEJ), 4(2), 145-150. Widana, I. W. (2017). Higher Order Thinking Skills Assessment (HOTs). Journal of Indonesian Student Assessment and Evaluation (JISAE). 3 (1), 32-44. Zaida, N., & Sofwan, A. (2015). Incorporating Critical Thinking Skills Into An English Textbook for Junior High School Students of Semarang City. The Journal of Educational Development. 3 (2), 148- 157.