Indonesian Review of Physics, Volume 2, Number 1, 2019 9 An Analysis of Graduate Students’ Conceptual Understanding in Heat and Temperature (H&T) Using Three-Tier Diagnostic Test Moh. Irma Sukarelawan1, Jumadi2, Nurulhuda Abdul Rahman3, 1Program Magister Pendidikan Fisika, Universitas Ahmad Dahlan 2Department of Physics Education, Faculty of Mathematics and Sciences, Yogyakarta State University 3Department of Physics, Faculty of Science & Mathematics, Universiti Pendidikan Sultan Idris 1Jl. Pramuka, No.42, Pandeyan, Umbulharjo, Yogyakarta, Indonesia. 2Jl. Colombo No.1, Karang Malang, Caturtunggal, Depok, Sleman, Daerah Istimewa Yogyakarta 3Tanjung Malim, Perak, Malaysia. Email: moh.sukarelawan@mpfis.uad.ac.id Abstract – There have been many studies that investigate H&T misconceptions among students in Indonesia but the respondents are usually limited to senior high school students. This study aims to analyse the conceptual understanding in H&T concepts of Masters of Physics Education students with a focus on identifying their misconceptions. The study employs the survey research design and data were collected using a three-tier diagnostic test. Twelve Masters of Physics Education students who are enrolled in one of the private universities in Yogyakarta City took part in the study. They were selected using the convenient sampling technique. Data on conceptual understanding involve both quantitative and qualitative data. As such, the data was analysed based on the percentage of student response patterns. Results showed that the majority of students (62.5%) have some form of misconceptions about the concepts of H&T and only 22.9% hold the scientifically accepted conceptions. The most common misconception held by the students is that both H&T depend on the material of an object. Other descriptions of conceptual understanding are False Positive (4.2%), False Negative (4.2%), and Lack of Knowledge (6.2%). Interestingly, none of the students made a lucky guess in responding to the test items. We conclude that the pattern in conceptual understanding specifically that of misconceptions are ubiquitous regardless of the level of study. Years of learning have failed to address misconceptions among students even at the graduate level. This result has implications on how learning should be designed to actively resolve the issue of misconceptions among students. Keywords: Heat and Temperature, three-tier diagnostic test, conceptual understanding I. Introduction One important goal in science learning is so students understand concepts that are scientific and logical [1]. More than half a century prior, research relating to conceptual understanding of the topic of H & T has begun to be looked at. A number of studies have widely reported and listed the misconceptions found. The Indonesian government has long been aware of this. Therefore, explicitly through Permendikbud in 2018 in Core Competencies physics learning mandates that students be encouraged to have the capacity to understand and implement conceptual understanding [2]. Determination of conceptual understanding of heat and temperature is important to do [3]. Heat and Temperature are one of the most appropriate topics to instill scientific instincts [4]. Heat and Temperature are the basic subjects in science and the phenomenon we experience everyday but some aspects conflict with the interpretation of students [3]. As we know that students' understanding of H & Sukarelawan & Rahman An Analysis of Graduate Students’ Conceptual Understanding in Heat and Temperature… Indonesian Review of Physics, Volume 2, Number 1, 2019 10 T is basically cliched and scientific (Gunes, 2005 in [3]). Many students hold misconceptions in H & T. Various H & T misconceptions have been identified. Students and even prospective teachers often use the terms temperature and heat alternately and not in the context [5], [6]. Temperature is a form of energy and joule units and changes in the state of matter are caused by changes in temperature [3]. There are still many students in Indonesia who hold heat and temperature misconceptions. Students assume that changes in temperature and form can occur simultaneously [7]. The concept of expansion still often misconceptions [8]. Various instruments have been created and utilized by researchers to diagnose misconceptions. Like interviews [9], multiple choice [10], multiple- tier diagnostic such as two-tier tests [11] - [13] and three-tier tests [14] - [16]. Multiple choice tests are the best alternative for diagnosing large numbers of student misconceptions. Because it is more effective and efficient and has better validity and reliability than the interview method. Various studies of H & T misconceptions have been carried out, such as at the elementary school level [9], [17], Junior High School [18], Senior High School [19], [20] and Higher Education [21]. In Indonesia, research on H & T misconceptions is more dominant at the level of Senior High School. Like research conducted by [7], [8], [16], [22] - [28]. There have been no reports of misconceptions examined at the level of higher education in Indonesia. To date, diagnostic tests developed and used in Indonesia are limited to a three-tier model and implemented at the high school level. While the misconception of master level students is still very limited in information. Therefore, the purpose of this study is to provide an overview of the conceptual understanding possessed by physics education students at the master level. II. Literature review Misconception Intuitive understanding or views of students on natural phenomena can be constructed through the learning process carried out through formal education in schools and informally from everyday life experiences [29]. This intuitive view or understanding is generally still simple and tends to not be in accordance with the views of scientists today. There are various kinds of terminology used to describe differences in students' conceptual understanding with scientists. Common terminologies used are misconception [30], [31] and alternative conception [32], [33]. Apart from the differences, these two terminologies together emphasize the existence of differences in students' intuitive ideas/knowledge of current scientific concepts in explaining natural phenomenon [34], [35]. Because its use is common in almost all disciplines, in this study uses the terminology of misconception, to describe intuitive conceptions/knowledge that are not in accordance with scientific views [29]. Three-tier diagnostic test To examine a person's misconceptions on a particular topic, various diagnostic methods have been used such as short questionnaries, interviews, multiple-choice diagnostic tests, concept mind and conceptual surveys [32], [36]. In the last decade many practitioners and researchers developed measuring instruments in the form of multiple-choice diagnostic tests [19] and multiple-tier diagnostic tests [29]. For large sample sizes, multiple choice tests are the best alternative in diagnosing misconceptions. Another advantage of multiple choice tests is having high validity and reliability, more effective and efficient in the assessment. However, multiple choice tests cannot provide an overview of student thinking behind the response of the answers given [37]. Therefore, a number of researchers developed multiple-tier diagnostic test instruments. One type of multiple-tier diagnostic test is the three tier test. The three-tier multiple choice diagnostic test is a refinement of the two-tier version. As the name implies, the three-tier multiple choice test consists of three levels, namely the first tier is the usual multiple choice. The second tier is the reason/rational of the answer at the first tier. The third tier is a statement relating to the level of confidence of students in the answers given [38]. III. Method Qualitative descriptive research method has been used to get an overview of H&T misconceptions in Masters of Physics Education students. Sample This research has been conducted on Masters of Physics Education students (semester 1 – 3) one of the private universities in Yogyakarta. Students have been selected using convenience sampling techniques. The respondents involved were 12 students who were still active in the 2018-2019 academic year. Respondents consisted of 7 men (58%) and women 5 people (42%). The age of Sukarelawan & Rahman An Analysis of Graduate Students’ Conceptual Understanding in Heat and Temperature… Indonesian Review of Physics, Volume 2, Number 1, 2019 11 students is between 23 years and 30 years. The average age of respondents is 24.5 years with a standard deviation of 1.9. Instrument This study has used a three-tier H&T diagnostic test instrument that was developed by Eryilmaz [21]. Diagnostic test instruments consist of 4 questions that are focused on the concept of H&T variables dependent on the size and material made of objects. Table 1 shows the item description to diagnose H&T misconceptions. Table 1. Items Description of H & T Diagnostic Test Item Number Items Description Code 1 The concept of temperature dependence on the size of objects Q1 2 The concept of heat dependence on the size of objects Q2 3 The concept of temperature dependence on material made of Q3 4 The concept of heat dependence on material made of Q4 Data collection and analysis Before data collection is carried out, the diagnostic instrument has been formatted into the google form to facilitate the administration of responses from respondents. Respondents have been motivated to participate voluntarily and are convinced that all the responses they give will not directly affect their academic value. Each item is analyzed into six categories, namely Scientific Knowledge (SK), Lucky Guess (LG), False Positive (FP), False Negative (FN), Lack of Knowledge (LK) and Misconception (MISC) [39]. The categorization of student conceptual understandings for the three- tier diagnostic test is shown in Table 2. Table 2. Combination of Answer Patterns at the Three-Tier Diagnostic Test [39] 1st Tier 2nd Tier 3rd Tier Category correct correct Sure Scientific Knowledge Not Sure Lucky Guess incorrect Sure False Positives Not Sure Lack Of Knowledge incorrect correct Sure False Negatives Not Sure Lack Of Knowledge incorrect Sure Misconception Not Sure Lack Of Knowledge IV. Results and discussion Data collection has been done using google form as an electronic document. This is done to facilitate the administration of data from respondents. Scores for correct answers and misconceptions have been analyzed using MS-EXCEL. Table 3 shows a tabulation of student response data. Sukarelawan & Rahman An Analysis of Graduate Students’ Conceptual Understanding in Heat and Temperature… Indonesian Review of Physics, Volume 2, Number 1, 2019 12 Table 3. Tabulation of Student Response Data Respondents Q1 Q2 Q3 Q4 Category CBA CBA CCA CBA Q1 Q2 Q3 Q4 1 CBA CBA BAA BAA SK SK Misc Misc 2 AAA AAA BAA BAA Misc Misc Misc Misc 3 CBA CBA BAA BAA SK SK Misc Misc 4 CAC CCA BAA BAA FP FP Misc Misc 5 CBA CBA BAA BAA SK SK Misc Misc 6 AAA ABA BBA BAA Misc FN Misc Misc 7 CBA DCB BAB DCB SK LK LK LK 8 CBA AAA AAA BAA SK Misc Misc Misc 9 AAA AAA BAA BAA Misc Misc Misc Misc 10 CBA DBA BAA AAA SK FN Misc Misc 11 CBA CBA BAA BAA SK SK Misc Misc 12 BAA AAA BBA BAA Misc Misc Misc Misc Table 4 shows the distribution of student answers to the concepts of heat and temperature. Based on the results of the analysis, the overall answers of students were classified as Scientific Knowledge at 22.9%, 70.5% held misconceptions, namely 62.5% original misconceptions, 4.2% False Positives and 4.2% False Negatives. 6.2% of Lack of Knowledge students and 0% of Lucky Guess. Table 4. Distribution of Student Concepts in 3-Tier Concepts of Heat and Temperature Category Cod e Question Averag e (%) Q1 Q2 Q3 Q4 N % N % N % N % Scientific Knowledge SK 7 58. 3 4 33. 3 0 0 0 0 22.9 False Positive FP 1 8.3 1 8.3 0 0 0 0 4.2 False Negative FN 0 0 2 16. 7 0 0 0 0 4.2 Misconception MIS C 4 33. 3 4 33. 3 1 1 91. 7 1 1 91. 7 62.5 Lack of Knowledge LK 0 0 1 8.3 1 8.3 0 8.3 6.2 Lucky Guess LG 0 0 0 0 0 0 1 0 0.0 1 2 100 1 2 100 1 2 100 1 2 100 100 Heat and temperature depend on the size of the object. To diagnose the conceptual understanding of H & T for the size of objects, we used the case of two iron desk of different sizes, placed in the same space for a long time. As many as 58.3% of students believe that the temperature does not depend on the size of the object. However, only 33.3% understood that heat also does not depend on the size of objects. 8.3% of students believe that heat and temperature do not depend on the size of objects but they reason that the temperature and heat in this case cannot be discussed. Sixteen point seven percent of students know that heat does not depend on the size of objects but at the first tier they answer if the small desk has higher heat or the heat of the two desk cannot be compared. Sukarelawan & Rahman An Analysis of Graduate Students’ Conceptual Understanding in Heat and Temperature… Indonesian Review of Physics, Volume 2, Number 1, 2019 13 33.3% of students strongly believe that the heat and temperature of objects depend on their size. This shows that most students consider heat and temperature to be extensive quantities [20], [40]. These findings prove that the misconception is not only experienced by students [19] but the master level students still hold the same misconception [21]. The percentage of students who have Scientific Knowledge in Q1 is higher than Q2. This indicates that the conceptual understanding of heat as an intensive quantity is more difficult for students to understand than the concept of temperature [21]. Heat and temperature depend on material made of To diagnose the heat and temperature misconceptions of the constituent material, the case used is the same sized desk (the first and second desks each made of iron and wood) which are stored in the same space for a long time. Based on Table 4, no student who understands the concept of H& T does not depend on the material made of. The concept of H& T dependence on the material made of is spread into two types of errors, namely Misconception (91.7%) and Lack of Knowledge (8.3%). Students holding a misconception in Q3 were 91.7%, 91% believed that the temperature of an iron desk was higher than a wooden desk because they thought there were some substances that felt cooler than others. This is in line with the results of the study [20]. While 9% of students holding this misconception believe that iron table temperatures are higher than wooden tables on the grounds that the temperature depends on the type of material making up the object. In this case also, 8.3% of students were of the view that iron desk temperatures were higher than wooden desk because the temperature was dependent on material objects but they were not sure of the knowledge they had. Students holding a misconception in Q4 as many as 91.7%, 91% believe that iron desk are hotter than wood desk. These findings are not in line with findings from [5], [19] - [21]. While 9% of them believe the opposite, namely wood desk feel hotter than iron desk. The reason behind both of their answers is the same, namely heat depends on the material made of. If you look at the distribution pattern of student responses in Q3 and Q4, it can be concluded that the conceptual understanding of temperature and heat dependence on objects making up materials has the same level of difficulty. V. Conclusion Based on the results of the research that has been done, it can be concluded that the students of physics education master program semester 1-3 academic year 2018/2019 at one of the private universities in Yogyakarta indicated a misconception related to the concept of temperature and heat dependence on the size and type of constituent material object. The level of student conceptual understanding is categorized in Scientific Knowledge 22.9%, 4.2% False Positive, False Negative 4.2%, Misconception 62.5%, Lack of Knowledge 6.2% and Lucky Guess 0%. ot be separated from the word after it. Reference [1] N. S. N. Daud, M. M. A. Karim, S. W. N. W. Hassan, and N. A. Rahman, Misconception and Difficulties in Introductory Physics Among High School and University Students: An Overview in Mechanics, Educ. -Journal Sci. Math. Technol., vol. 2, no. 1, 2015, pp. 34– 47. [2] Kemdikbud. Peraturan Menteri Pendidikan dan Kebudayaan Republik Indonesia No. 37 Tahun 2018. Kemdikbud, 2018. [3] I. Aykutlu, S. Bezen, and C. Bayrak, An Assessment of High School Students’ Conceptual Structures of Heat And Temperature Through Concept Maps, in Turkish Physical Society 32nd International Physics Congress, 2017, vol. 1815, pp. 1–4. [4] A. A. di Sessa, A bird’s eye view of the ‘pieces’ vs. ‘coherence’ controversy (from the pieces side of the fence), in International Handbook of Research on Conceptual Change, S. Vosniadou, Ed. New York: Routledge, 2008, p. 13. [5] A. A. Alwan, Misconception of heat and temperature among physics students, in Procedia - Social and Behavioral Sciences, vol. 12, 2011, pp. 600–614. [6] M. Yeşilyurt, High School Students’ Views About Heat and Temperature Concepts, Int. J. Environ. Sci. Educ., vol. 1, no. 1, 2006, pp. 1–24. [7] Alfiani, Analisis Profil Miskonsepsi dan Konsistensi Konsepsi Siswa SMA pada Topik Suhu dan Kalor, in Prosiding Seminar Nasional Fisika (E-Journal) SNF2015, 2015, vol. IV, pp. 29–32. [8] S. N. W. Silung, S. Kusairi, and S. Zulaikah, Diagnosis Miskonsepsi Siswa SMA di Kota Malang pada Konsep Suhu dan Kalor Menggunakan Three Tier Test, J. Pendidik. Fis. dan Teknol., vol. II, no. 3, 2016, pp. 95– 105. [9] S. Paik, B. Cho, and Y. Go, Korean 4- to 11-Year-Old Student Conceptions of Heat and Temperature, J. Res. Sci. Teach., vol. 44, no. 2, 2007, pp. 284–302. [10] V. D. Kulkarni, H. R. Mahavidyalay, and P. S. Tambade, Assessing the Conceptual Understanding about Heat and Thermodynamics at Undergraduate Level, Eur. J Phys. Educ., vol. 4, no. 2, 2013, pp. 9–16. [11] T. C. Yang, S. Y. Chen, and M. C. Chen, An Investigation of a Two-Tier Test Strategy in a University Calculus Course: Causes versus Consequences, IEEE Trans. Learn. Technol., vol. 9, no. 2, 2016, pp. 146–156. [12] U. Kanli, Using a Two-tier Test to Analyse Students’ and Teachers’ Alternative Concepts in Astronomy, Sci. Educ. Int., vol. 26, no. 2, 2015, pp. 148–165. Sukarelawan & Rahman An Analysis of Graduate Students’ Conceptual Understanding in Heat and Temperature… Indonesian Review of Physics, Volume 2, Number 1, 2019 14 [13] P. Potvin, Y. Skelling-desmeules, and O. Sy, Exploring Secondary Students’ Conceptions about Fire Using a Two-Tier , True/False, Easy-to-Use Diagnostic Test, J. Educ. Sci. Environ. Heal., vol. 1, no. 2, 2015, pp. 63– 78. [14] A. K. Prodjosantoso, A. M. Hertina, and I. Irwanto, The Misconception Diagnosis on Ionic and Covalent Bonds Concepts with Three Tier Diagnostic Test, Int. J. Instr., vol. 12, no. 1, 2019, pp. 1477–1488. [15] Ş. Şen and A. Yilmaz, The Development of a Three-tier Chemical Bonding Concept Test, J. Turkish Sci. Educ., vol. 14, no. 1, 2017, pp. 110–126. [16] M. L. H. Abbas, Pengembangan Instrumen Three Tier Diagnostic Test Miskonsepsi Suhu dan Kalor, Ed- Humanistics, vol. 01, no. 02, 2016, pp. 83–92. [17] E. B. Kirikkaya and D. Güllü, Fifth Grade Students’ Misconceptions about Heat-Temperature and Evaporation – Boiling, Elem. Educ. Online, vol. 7, no. 1, 2008, pp. 15–27. [18] M. Shayer and H. Wylam, The Development of The Concepts of Heat and Temperature In 10-13 Year-Olds, J. Res. Sci. Teach., vol. 18, no. 5, 1981, pp. 419–434. [19] D. Gurcay and E. Gulbas, Development of Three-Tier Heat, Temperature and Internal Energy Diagnostic Test, Res. Sci. Technol. Educ., vol. 33, no. 2, 2015, pp. 197– 217. [20] H.-E. Chu, D. F. Treagust, S. Yeo, and M. Zadnik, Evaluation of Students’ Understanding of Thermal Concepts in Everyday Contexts, Int. J. Sci. Educ., vol. 34, no. 10, 2012, pp. 1509–1534. [21] A. Eryilmaz, Development and Application of Three- Tier Heat and Temperature Test: Sample of Bachelor and Graduate Students, Eurasian J. Educ. Res., no. 40, 2010, pp. 53–76. [22] R. Amalia, I. M. Sari, and P. Sinaga, Students’ Mental Model on Heat Convection Concept and Its Relation with Students Conception on Heat and Temperature, in IOP Conf. Series: Journal of Physics: Conf. Series, 2017, pp. 1–7. [23] L. Fitriah, Diagnosis Miskonsepsi Siswa pada Materi Kalor dengan Menggunakan Three-Tier Essay dan Open–Ended Test, Berkala Ilmu Pendidikan Fisika, vol. 5, no. 2, 2017, pp. 168–181. [24] F. Iradaty and D. Rosana, Pengembangan Instrumen Three-Tier Test Untuk Mengidentifikasi Miskonsepsi pada Materi Fisika Suhu dan Kalor pada Siswa SMA Kelas X, eJournal Univ. Negeri Yogyakarta, vol. 2, no. 2, 2013. [25] N. Maunah and Wasis, Pengembangan Two-Tier Multiple Choice Diagnostic Test Untuk Menganalisis Kesulitan Belajar Siswa Kelas X pada Materi Suhu dan Kalor, J. Inov. Pendidikan Fisika, vol. 03, no. 02, 2014, pp. 195–200. [26] H. N. P. A. Putri, Suliyanah, and L. Rohmawati, Identifikasi Kualitas Instrumen Soal Three-Tier Diagnostic Test Pada Materi Suhu, Kalor dan Perpindahannya, J. Inov. Pendidikan Fisika, vol. 05, no. 03, 2016, pp. 216–219. [27] E. Setyadi and A. Komalasari, Miskonsepsi Tentang Suhu dan Kalor pada Siswa Kelas 1 di SMA Muhammadiyah Purworejo, Jawa Tengah, Berkala Fisika Indonesia, vol. 4, no. 1 & 2, 2012, pp. 46–49. [28] S. N. Wahidah, S. Kusairi, and S. Zulaikah, Identifikasi Miskonsepsi Siswa SMA pada Materi Suhu dan Kalor serta Kemungkinan Penyebabnya, in Seminar Nasional Jurusan Fisika FMIPA UNESA 2015, 2015, pp. 180– 185. [29] D. Kaltakci-Gurel, A. Eryılmaz, and L. C. McDermott, A Review and Comparison of Diagnostic Instruments to Identify Students’ Misconceptions in Science, Eurasia J. Math. Sci. Technol. Educ., vol. 11, no. 5, 2015, pp. 989–1008. [30] H. N. Bani-Salameh, How persistent are the misconceptions about force and motion held by college students?, Phys. Educ., vol. 52, no. 1, 2017, p. 14003. [31] D. Kaltakci-Gurel, A. Eryilmaz, and L. C. McDermott, Development and application of a four-tier test to assess pre-service physics teachers’ misconceptions about geometrical optics, Res. Sci. Technol. Educ., vol. 35, no. 2, 2017, pp. 238–260. [32] U. Kanli, A Study on Identifying the Misconceptions of Pre-service and In-service Teachers about Basic Astronomy Concepts, Eurasia J. Math. Sci. Technol. Educ., vol. 10, no. 5, 2014, pp. 471–479. [33] A. S. L. Loh, R. Subramaniam, and K. C. D. Tan, Exploring students’ understanding of electrochemical cells using an enhanced two-tier diagnostic instrument, Res. Sci. Technol. Educ., vol. 32, no. 3, 2014, pp. 229– 250. [34] J. S. Gyoungho Lee, S. S. Jiyeon Park, Y. Kim, and L. Bao, Alternative conceptions, memory, & mental models in physics education, in AIP Conference Proceedings, 2005, pp. 165–168. [35] H. O. Arslan, C. Cigdemoglu, and C. Moseley, A Three- Tier Diagnostic Test to Assess Pre-Service Teachers’ Misconceptions about Global Warming, Greenhouse Effect, Ozone Layer Depletion, and Acid Rain, Int. J. Sci. Educ., vol. 34, no. 11, 2012, pp. 1667–1686. [36] E. Taslidere, Development and use of a three-tier diagnostic test to assess high school students’ misconceptions about the photoelectric effect, Res. Sci. Technol. Educ., vol. 34, no. 2, 2016, pp. 164–186. [37] P. B. Griffard and J. H. Wandersee, The Two-Tier Instrument on Photosynthesis: What Does it Diagnose?, Int. J. Sci. Educ., vol. 23, no. 10, 2001, pp. 1039–1052. [38] Suliyanah, H. N. P. A. Putri, and L. Rohmawati, Identification student’s misconception of heat and temperature using three-tier diagnostic test, in J. Phys.: Conf. Ser. 997 012035, 2018. [39] D. D. Milenković, T. N. Hrin, M. D. Segedinac, and S. Horvat, Development of a Three-Tier Test as a Valid Diagnostic Tool for Identification of Misconceptions Related to Carbohydrates, J. Chem. Educ., vol. 93, no. 9, 2016, pp. 1514–1520. [40] S. Yeo and M. Zadnik, Introductory thermal concept evaluation: assessing students’ understanding, Phys. Teach., vol. 39, 2001, pp. 496–504. Keywords: Heat and Temperature, three-tier diagnostic test, conceptual understanding ... Reference [40] S. Yeo and M. Zadnik, Introductory thermal concept evaluation: assessing students’ understanding, Phys. Teach., vol. 39, 2001, pp. 496–504.