845 Copyright © 2023 The Author IDEAS is licensed under CC-BY-SA 4.0 License Issued by English study program of IAIN Palopo IDEAS Journal of Language Teaching and Learning, Linguistics and Literature ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) Volume 11, Number 1, June 2023 pp. 845 - 853 Developing Digital-Based Speaking Skill Literacy Assessment Using Plomp Model to Support the National Literacy Movement Maitri Rahmadhani1 madani.edu19@gmail.com STKIP AL Makusm Langkat Received: 2023-04-11 Accepted: 2023-07-05 DOI: 10.24256/ideas. V 11i1.3058 Abstract This research aimed to develop a digital-based speaking literacy assessment model that credible, measurable and do able by using the PLOMP model in an effort to support the National Literacy Movement. Located in SMK Al Maksum Stabat, the subject of this research was the second graders of SMK Al Maksum Stabat by applying random sampling. The instrument used were questionnaires both used for the expert validator and for the teachers. The result showed that the digital-based speaking literacy assessment model has 90.88% of validation percentage which means it has a high level of validity but need to be revised. 85.5% respondents respond positively to the developed digital-based speaking literacy assessment model including its interest and its attractiveness. It indicated that the developed digital-based speaking literacy assessment model has fulfill the aspect of feasibility and effectiveness and has been tested successfully implemented at SMK Al Maksum Stabat. Keywords: Assessment; Digital; PLOMP; Speaking_Literacy Introduction Based on various international surveys on literacy, it is known that Indonesia has a low literacy culture. The fact is that Indonesia ranks 64th out of 65 countries in Program for International Students Assessment (PISA) regarding literacy culture. This is certainly be a spotlight for other countries in seeing Indonesia as well as being a whip for the Indonesian government to improve itself considering that Indonesia is a country rich in nature and culture, which if managed by reliable and quality human resources, God willing, it will become a great developed country. One of the government's contributions is to issue Permendikbud N0.23 in 2015. Based on the Permendikbud, we can see that it is important for schools to set aside time regularly for the habit of reading as part of developing character. In this case, it is implied that reading activities are believed to have a positive role on character. This reading activity is certainly not only intended for textbooks, but also other quality and useful reading http://u.lipi.go.id/1457703302 Maitri Rahmadhani1 Developing Digital-Based Speaking Skill Literacy Assessment Using Plomp Model to Support the National Literacy Movement 846 books or more familiarly what we call free reading. Because “literacy” is not just being free from illiteracy, but more than that. Thus, activating literacy for students at schools is expected to be able to develop their learning habits. In practice, this free reading activity is carried out at schools in the first fifteen minutes before starting the lesson. Of course, the activities that have been initiated by the government must also be intensified by the schools, both in big cities and in remote parts of the country. In fact, there are still many schools that have not implemented it. The lack of reading books and socialization to parents and even students are also the reasons. Not only that, the next obstacle is how teachers/education practitioners measure the literacy abilities of students so that they are able to see how far the literacy program is succeed. It should be noted that literacy skills are related to four basic language skills, namely: reading, writing, listening and speaking. These four basic abilities must be mastered in order to achieve the success of student literacy. To see the level of success in these four basic language skills, different assessments are needed based on each skill and must be measurable. In accordance to how importance of assessment to see the level of literacy success including the reading, writing, Listening and speaking skill, it is necessary to conduct research and a qualified model design of assessment. Which means that the assessmentt design model is measurable, credible, and do able. In 2021, the government has developed a reading literacy assessment which is be a part of the minimum competency assessment that then is used as a government policy in 2021. However, several other equally important literacy assessments have not yet been developed. Assessment is crucial in literacy. The used of this assessment is as a way to get information about the extent of the learning process achievement. In relation to the success of speaking literacy, assessment can be said as a measuring tool. Based on the Permendikbud No. 104 of 2014, the assessment instrument is a tool used to assess the achievement of student learning development, such as tests and attitude scales. If it is associated with speaking literacy, it can be concluded that the role of this assessment is very important, namely as a tool to measure the success/achievement of a speaking learning process. Therefore, the development of an assessment model for speaking literacy must also be accurate and good, which includes: measurable, credible, do able and refers to learning progression. Speaking is derived from the word “speak” which means to express opinions; to say; to converse. In other words, speaking is a way to express an opinion and tell what is going to be uttered. In broad, speaking has a larger scope. It is one of the skills or abilities in language that is used to provide information, provide comments, express and refute opinions, ask and answer questions about something, or even to create an atmosphere of mere familiarity. According to Shihabuddin (2009: 195) speaking is one aspect of productive language skills, namely in the form of skills in changing the form of thoughts or feelings into meaningful language sounds. According to him, there are six things that must be considered when assessing a person's speaking ability, namely: 1. Pronunciation and speech; 2. Grammar, linguistic structure in accordance with the variety of language used; 3. Vocabulary, choosing the right words according to the meaning of the information conveyed; 4. Fluency, ease, and speed of speech; 5. The content of the conversation, the topic of conversation, the ideas presented, the ideas put forward, and the flow of the conversation; IDEAS, Vol. 11, No. 1, June 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 847 and 6. Understanding, including the level of success of communication, and the communicativeness. Furthermore, Shihabuddin (2009:197) stated that Speaking skill is a productive skill which consists of several components, namely: 1. The use of spoken language that functions as a medium of conversation through vocabulary, language structure, pronunciation and intonation, and variety of languages; 2. Mastery of the content of the conversation that depends on what is the topic of conversation; 3. Mastery of speaking technique and appearance adapted to the situation and type of conversation, for example conversing, giving speeches, telling stories, and so on. The mastery of this technique very important in formal types of speaking, such as giving speeches, lecturing, and discussing. According to Plomp (2013) the research and development is a model of research that consists of 5 stages: (a) preliminary research, (b) design, (c) realization/Construction, (d) test, evaluation and revision, (e) implemetation. In preliminary research the needs analysis is carried out, that is by gathering all the important information needed and identifying the problem. In the design phase, the thing to do is to design problem solving raised in the initial investigative phase. In this phase, a work plan is carried out in the form of designing a digital-based speaking literacy assessment which will then be realized in the next phase. In the realization/construction phase, production activities are carried out in the form of developing a digital-based speaking literacy assessment. In the test, evaluation and revision phase validation of the practicality and effectiveness of digital-based speaking literacy assessment by the expert validators is carried out. The revisions are made if deficiencies are happened at the time of evaluation. In the implementation phase, the developed digital-based speaking literacy assessment model was implemented for the students and teachers of SMK AL Maksum Stabat. This is carried out to see all the obstacles and deficiencies when applying the developed digital-based speaking literacy assessment model to ease the researcher to revise it. Method This research was conducted at SMK Al Maksum Stabat, Langkat regency – North Sumatera Province. The subject of this research is the teachers and 2nd graders of SMK Al Maksum Stabat by using purpossive sampling. This research is considered as Research and Development (R&D) by using PLOMP Model. Maitri Rahmadhani1 Developing Digital-Based Speaking Skill Literacy Assessment Using Plomp Model to Support the National Literacy Movement 848 Fig.1 Road map The qualitative data in this research are the suggestions from the expert validators and the media as a as a consideration for revising the digital-based speaking literacy assessment model product. The quantitative data analyzed in this study was calculated using the percentage score obtained from the validation questionnaire sheet. P = ∑𝑋 ∑𝑋𝑖 x 100% Notes: P : Percentage ∑𝑋 : Score for each answer ∑𝑋 𝑖 : Total score for each answer The results of the data analysis are then interpreted in accordance with the assessment qualification criteria proposed by Riduwan (2011) which had previously been adapted. No Interval Criteria 1 80% < x ≤ 100 Very Valid 2 60% < x ≤ 80 Valid 3 40% < x ≤ 60 Fairly Valid 4 20% < x ≤ 40 Less Valid 5 x ≤ 20 Not Valid Results The developed digital-based speaking literacy assessment model was declared to have met the criteria for validity based on the results of the assessment of experts and education practitioners, with a percentage of validity of 90.88%. The developed digital-based speaking literacy assessment was stated to be practical IDEAS, Vol. 11, No. 1, June 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 849 and has met the practicality criteria. This can be seen from: a. Based on the testing result from the expert validators who stated that the developed digital-based speaking literacy assessment had a level of validity: Very Valid which means it can be used and implemented for both teachers and students, but it needed a minor revisions until it became the final draft of the digital-based speaking literacy assessment. b. Based on the observation given to both teachers and students, it was showed that the developed digital-based speaking literacy assessment can be used and implemented easily by both teachers and students seen from the positive increase in student’s activity during the teaching-learning process. Based on the trial-using of the developed digital-based speaking literacy assessment, it showed that the this digital-based speaking literacy assessment has met the criteria of effectiveness since the overall average percentage showed that 85.5% both teachers and students stated “Yes” on the aspects of fun and interest in using the developed digital-based speaking literacy assessment, found what was new and what was contained in the assessment, understanding the instructions and clear use of the assessment, and interest in the assessment design being developed. Discussion In need analysis phase, the distribution of the questionnaire was carried out to the second graders’ English teachers of SMK Al Maksum Stabat. This initial investigation was carried out to find out the problems that occurred in the process of assessing/testing speaking literacy for vocational students. Based on the results of the questionnaire which was distributed purposively to 6 respondents of the English teacher at SMK Al Maksum Stabat it was found that: a. a. 83.33% teachers stated “never” to the statement “Students understand the speaking literacy assessment scoring criteria”. 66.67% teachers stated “never” to the statement “Students know the speaking literacy assessment scoring transparantly”. 100% of teachers stated "never" to the statement "Students can conduct peer reviews of speaking literacy assessments". This means that the literacy assessment process that has been carried out so far is not transparent so that students do not know the speaking literacy assessment criteria and cannot conduct peer reviews. b. 50% of teachers stated “never”, 16.67% stated “rarely” and 33.33 stated “sometimes” in the statement “Teachers have standardized speaking literacy assessment”. 50% of teachers stated “never”, 33.33% stated “rarely” and 16.67% stated “sometimes” in the statement “Before the teacher conducts an assessment, the speaking literacy assessment is determined by the criteria”. 16.67% of teachers stated “sometimes”, 33.33% stated “often”, and 50% stated “always” on the statement “Teachers give assignments that are in accordance with the material being taught and the assessment”. 66.67% stated “never”, 33.33% stated “rarely” in the statement “Teachers establish practical and reliable assessment procedures”. 100% of teachers stated "always" in the statement "Teachers present the maximum results of the assessment". This means that so far the speaking literacy assessment has not yet determined the standardization of the assessment. Each teacher has a different perception of speaking literacy assessment c. 33.33% of teachers stated “never”, 33.33% stated “rarely”, 16.67 stated “often” and 16.67% stated “always” to the statement “Assessment has been able to describe the Maitri Rahmadhani1 Developing Digital-Based Speaking Skill Literacy Assessment Using Plomp Model to Support the National Literacy Movement 850 condition of student competence”. 83.33% of teachers stated “never” and 16.67% stated “rarely” to the statement “The assessment has been consistent and reliable”. 66.67% of students stated “never” and 33.33% stated “sometimes” to the statement “The assessment has been standardized and fair for all students”. 50% of teachers stated “never” and 33.33% said “rarely”, and 16.67% stated “always” o the statement “Assessment was in accordance with student needs”. 33.33% of teachers stated “never”, 33.33 “rarely”, 16.67 “sometimes” and 16.67% “often to the statement “Assessment has been able to describe student achievement truly”. Thus, it can be concluded that the speaking literacy assessment that has been used so far is not in accordance with Kepmendikbud Number 719/P/2020 regarding to the nature of learning assessment. d. 50% of teachers stated "never" and 50% "rarely" on the statement "The assessment has been related and in accordance with what has been taught". 66.67% of teachers stated “never” and 33.33% “rarely” on the statement “The assessment used was very efficient”. 50% of students stated “never”, 16.67% “rarely” and 33.33% stated “sometimes” in the statement “The assessment used is clear and measurable”. 33.33% of teachers stated “never” and 66.67% stated “rarely” in the statement “The assessment used was easy”. In this case, it can be concluded that the assessment used at this time is still not clearly measured and is still based on the perception of each teacher. Therefore, the result will be differ from one teacher to another in assessing the same student with the same testing aspect. This will be unfair for those students who have good literacy speaking skill but probably have some problems to certain teacher academically or personally, caused by the scoring of speaking literacy assessment which has not standard and objective. Based on the calculation data from the expert validators toward the developed digital- based speaking literacy assessment with several aspects assessed including content feasibility aspects, display feasibility aspects, language feasibility aspects, interface feasibility aspects and assessment design, the results obtained are 90.888% stated “very valid” as yet with minor revisions suggested by expert validators. The practicality of the developed digital-based speaking literacy assessment can be seen from how ease both teachers and students in using it. Formerly, it has been noticed that based on the result of the expert validators test on the developed digital-based speaking literacy assessment was 90.88% or stated “very valid” with minor revisions. Based on this data, the first element as a requirement for assessing the practicality of digital-based speaking literacy assessments has been fulfilled. In addition, there are other things that determine whether the developed digital-based speaking literacy assessment has met the criteria of practical value or not, namely by seeing a positive increase in teacher and student activities while using a digital-based speaking literacy assessment in testing the students speaking literacy during the teaching-learning activity. Based on the observation given to 20 respondents i.e., the second graders of SMK AL Maksum Stabat taken by 3 English teachers for 4 times meeting by using the developed digital-based speaking literacy assessment, it was found: The highest percentage of student activity lied in practicing speaking literacy by understanding the teacher's statements and questions and conducting simulations with confidence. In this case, the percentage reached 95% at the 4th meeting, having previously increased from the 1st (65%) to the 3rd meeting (85%) and peaked at the 4th meeting. Thus it can be concluded that the use of digital-based speaking literacy assessment with standardized assessment instruments so that it is clear for students and teachers about what must be achieved and the value. Students also confidently conduct simulations and can conduct peer-reviews due to clear assessment scoring on the assessments developed. IDEAS, Vol. 11, No. 1, June 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 851 After testing the developed digital-based speaking literacy assessment on both teachers and students, the data obtained as followed: The overall average percentage showed that 85.5% both teachers and students stated “Yes” on the aspects of fun and interest in using the developed digital-based speaking literacy assessment, found what was new and what was contained in the assessment, understanding the instructions and clear use of the assessment, and interest in the assessment design being developed. On the other hand, there were only 14.5% of teachers and students who said “no” to the questionable aspects of the digital-based speaking literacy assessment. This means that the developed digital-based speaking literacy assessment has met the feasibility aspect by being declared very feasible but needs minor revision according to the criteria needed by teachers and students. The purpose of this revision is to make the developed digital-based speaking literacy assessment can be better understood both in terms of use and in terms of its implementation by teachers and students. The results of the tests conducted on the second graders of SMK AL Maksum Stabat revealed that the developed digital-based speaking literacy assessment had succeeded in measuring students' ability of speaking literacy correctly. However, it should be noted that the focus of the tests given to the second graders of SMK Al Maksum Stabat in this study was to test the developed digital-based speaking literacy assessment and not to achieve an increase in the students' speaking skills. Therefore, the value displayed on the test is not to be assessed for the achievement of improvement, but only as proof that the developed literacy assessment has functioned optimally. Conclusion The developed digital-based speaking literacy assessment model was declared to have met the criteria for validity based on the results of the assessment of experts and education practitioners, with a percentage of validity of 90.88%. This developed digital-based speaking literacy assessment was also stated to be practical and has met the practicality criteria. Based on the trial-using of the developed digital-based speaking literacy assessment, it showed that this digital-based speaking literacy assessment has met the criteria of effectiveness. Therefore, it can be concluded that this digital-based speaking literacy assessment had succeeded the trial using and can soon be implemented by teachers and students in assessing the speaking literacy. Acknowledgement The researchers would like to thank the LPPM for helping the researchers in doing any administrations deals with this research and Kemdikbudristek for the opportunity and trust that has been given to the researchers in winning the Novice Lecturer Research Grant (PDP). We also would like to thank the expert validators for helping us in doing the trial- using of the developed assessment as well as doing the validity of the data. Special thanks due to all teachers, students, and headmaster of SMK Al Maksum Stabat to which the researchers conduct this research. References Arifin, Zinal. (2010). Evaluasi Pembelajaran. PT. Remaja Rosdakarya : Bandung Brown, H. D. & Abeywickrama, P. (2010). Language assessment: principles and classroom practice (2nd ed.). New York: Pearson Education Inc. Maitri Rahmadhani1 Developing Digital-Based Speaking Skill Literacy Assessment Using Plomp Model to Support the National Literacy Movement 852 Chastain, K. (1998). Developing second language skills (2nd Ed.). Chicago: Harcourt Brace Publishers. Hahn, at al. (2013). Assessing scientific literacy over the lifespan –A description of the NEPS science framework and the test development. Journal for Educational Research Online, Journal fu r Bildungsforschung Online. Volume 5 (2013), No. 2, 110–138. Lina, Y.R., et.al. (2018). Pengembangan Instrumen Asesmen Berbasis Literasi Sains tentang Materi Sistem Pencernaan Makanan, Zat Aditif, dan Zat Adiktif untuk SMP. Bioeducation Journal, 2(2), 145-155. Mahendrayana, Gede. 2014. Developing Project Assessment Instruments for Speaking Skll of the Eight Grade Students of Junior High School. Thesis English Education Study Program, Post Graduate Program, Ganesha University. Nazara, Situju. (2011). Students’ Perception on EFL Speaking Skill Development. Journal of English Language Teaching, 1 (1): 28-43. Nitko, Anthony J. (2001). Educational Assessment of Students. Third Edition. United States of America: Prentice-Hall, Inc. Nunan, David. (1991). Research Methods in Language Learning. Cambridge: Cambridge University Press. Nurgiyantoro, Burhan. (2012). Penilaian Pembelajaran Bahasa. PBFE UGM : Yogyakarta Nurgiyantoro, Burhan. (2008). Penilaian Autentik. Cakrawala Pendidikan, 27 (3): 250-261. Plomp, T and Nienkee, N. (2013). Educational Design Research. Enscede: University of Twenty. Purnomo, Edy. (2015). Dasar-Dasar dan Perancangan Evaluasi Pembelajaran, Bandar Lampung. Unila Pusat Penilaian Pendidikan Balitbang. (2011). Survei Internasional PISA. Jakarta. Kementrian Pendidikan dan Kebudayaan. Rahmawati,Yeni. & Ertin. (2014). Developing Assessment For Speaking. IJEE Journal, 1 (2):199-210. Rahmadhani, Maitri & Supriadi. (2021). The Development Of English Textbook By Using Addie-Model To Improve English Speaking Skill For Non-English Department Students. Anglo Saxon Journal, 12 (1): p. 65-79. Ratminingsih, N.M. (2014). Pengembangan Model Pembelajaran Bahasa Inggris Induktif Berbasis Lagu Kreasi. Jurnal Ilmu Pendidikan, 20 (1): p. 47-58. Riduwan. (2011). Skala Pengukuran Variabel-Variabel Penelitian. Bandung: Allfabeta. Robert Petrone. (2013). Linking Contemporary Research on Youth, Literacy, and Popular Culture With Literacy Teacher Education,Journal of Literacy Research 45(3) 240-266 Roth, Eric H, Aberson, Toni W. (2006). Copmpelling Conversations: Questions and Quotations On Timeless Topic. Charleston USA: Chimayo Press. Rusilowati, A., Lina, K., Nugroho, S. E., & Widiyatmoko, A. (2016a). Scientific Literacy Evaluation Instrument to Measure The Student‟s Scientific Literacy Skill on The Cycle Theme. International Journal of Environment al and Science Education, 11(12), 5718- 5727. Shihabuddin, H. (2009). Evaluasi Pengajaran bahasa Indonesia. Bandung: UPI. Sudjana, Nana. (2010). Penialain Hasil Proses Belajar Mengajar. PT. Remaja Rosdakarya: Bandung. Sari, E., & Agustina, M. (2022). Thematic Development in Students’ Argumentative Essay. IDEAS: Journal on English Language Teaching and Learning, Linguistics and Literature, 10(1), 166-174. doi:https://doi.org/10.24256/ideas.v10i1.2563 Syahari. (2010). Information Gap Task Dapat Meningkatkan Kemampuan Speaking Siswa. Bandung: UNISBA. Sidabutar, K. (2022). Grice's Types of Maxims in "Willoughbhys" Movie. IDEAS: Journal on https://doi.org/10.24256/ideas.v10i1.2563 IDEAS, Vol. 11, No. 1, June 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 853 English Language Teaching and Learning, Linguistics and Literature, 10(1), 326-337. doi:https://doi.org/10.24256/ideas.v10i1.2661 Templeton, Melody. (2010). Public Speaking and Presentation Demystified. New York: Mc Graw Hill Wahyuni, L.G.E. 2(013). Authenticity of Teachers’ Made Assessment and Its Contribution to Students’ English Achievement. Jurnal Pendidikan dan pengajaran Undiksha, 46 (2): p. 182-19 Wortham, Sue C. (2008). Assessment in Early Childhood Education. United States: Pearson Education. Yuniawan, T. 2014. Model Penilaian Kinerja Dalam Pembelajaran Membaca Berbasis Teks Narasi Bermuatan Pendidikan Karakter Cinta Budaya. Jurnal Pendidikan Karakter, 14 (01): p. 61-72 Yusuf, A, Muri. (2015). Asesmen dan Evaluasi Pendidikan. Jakarta. Prenada. https://doi.org/10.24256/ideas.v10i1.2661