Microsoft Word - 1-7_216.docx ISSN 2744-1741 Defense and Security Studies Original Research Vol. 4, January 2023, pp.1-7 https://doi.org/10.37868/dss.v4.id216 This work is licensed under a Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) that allows others to share and adapt the material for any purpose (even commercially), in any medium with an acknowledgement of the work's authorship and initial publication in this journal. 1 Analysis of SPBE and SWCSF measurement instruments using Flesch Reading Ease for state security Meylia Susiana Dewi Putri1*, Rudy AG Gultom2, Achmad Farid Wadjdi3 1,2,3 Republic of Indonesia Defense University, Faculty of Defense Technology, Indonesia *Corresponding author E-mail: meylia.putri@idu.ac.id Received Dec. 8, 2022 Revised Jan. 3, 2023 Accepted Jan. 7, 2023 Abstract The world has transitioned into a digital era where both individuals and governments require technology and the internet. The number of cybercrimes perpetrated online is impacted by the rising usage of computers and the internet. A measurement instrument that can stop cybercrime is necessary. The Six-Ware Cyber Security Framework (SWCSF) and the Electronic-Based Government System (SPBE) are two measurement tools that are expected to be able to stop cybercrime from happening in an agency or organization. But are all people able to use these two instruments? This research was conducted to answer this question by analyzing readability on the SPBE and SWCSF instruments using the Flesch Reading Ease method. The result show that the two instruments were extremely difficult for respondents of all grade levels to comprehend, with the exception of those at the university level or individuals who worked with computers, the internet, and other technologies. © The Author 2023. Published by ARDA. Keywords: SPBE; SWCSF; Measurement instrument; Flesch reading ease; Digitalization 1. Introduction The Industrial Revolution 4.0 has affected all disciplines, including the sphere of information and technology, and Indonesia, which is entering it, appears to have developed a strategy to prepare for it (science and technology). The Indonesian people have adopted this transition as a trend and way of life. The Industrial Revolution era's legacy includes society's growing reliance on information and communication technology. The digital era is one example of how technology and information have advanced. The digital revolution has arrived in Indonesia, where information can now be found online and through websites. This is seen in the rising and increasingly sophisticated demand for technological devices. The transformation of traditional government into electronic government (e-government) is a public policy regulated in Article 28F of the 1945 Constitution. The development of e-government has a vision, namely a review based on broadband networks, electronic document systems, information sharing, and government portal websites. The Indonesian government is currently developing an electronic-based government system [1]. The use of e-government in Indonesia aims to improve performance and productivity, improve budget efficiency, and increase transparency and accountability in the delivery of service activities to the community [2]. Therefore, the implementation of e-government in government must be carefully prepared and integrated between bureaucratic systems and policies so that they can be well received by service users, namely the DSS Vol. 4, January 2023, pp.1-7 2 community and internal government. In the context of e-government, user trust in using public services is important because it can make people feel comfortable using e-government [3]. The development of e-government that utilizes information and communication technology (ICT) must pay attention to the ICT and cybersecurity policies applied. In the current era, the development of ICT and digitalization is directly proportional to the increase in ICT crimes and cybercrimes. Based on a report from [4], Indonesia accounts for 40% of the ASEAN population with a gross domestic product proportion of around US$ 4,174.9. Indonesia is estimated to spend 1.6% of GDP in digital infrastructure, compared to Malaysia at 4.5% and Singapore at 6.6%. This shows that there is room to accelerate cybersecurity in Indonesia. A new challenge to the system is brought about by the growth of the government's computerized system. Akhmad Toha (2021), Deputy for BSSN Protection, stated at the Huawei Techday event that in order to realize an accountable, transparent, and democratic government system, the governance of an electronic-based government system (SPBE) must be supported by strong cyber security. Cyber attacks can cause disruption and damage to network systems that connect, integrate, synchronize, and control computerized and integrated government equipment and infrastructure [5]. In 2019, Indonesia was one of the countries with the highest recorded cases of malware attacks in the Asia Pacific region. The potential economic loss in Indonesia due to cybersecurity incidents could reach US$ 34.2 billion [6]. These problems can be prevented by measuring the level of cyber security and information and communication technology using measurement instruments. Two measurement tools that make use of information and communication technology (ICT) resources that can be employed in the central and regional domains are the SPBE and SWCSF. The authority for ICT development focuses on the balance of central and local authority, alignment of planning, and coordination of implementation. Currently, Indonesia is experiencing the COVID-19 pandemic, which has changed the system from manual to digital [1]. Changing systems requires building skills within the organization with the aim of helping them cope with and adapt to new circumstances. Therefore, in measuring the progress of implementation and readiness of SPBE and SWCSF, it is assumed that there will be different results. From this statement, a measurement framework that contains complexity and diversity is needed for all instruments. The application of SPBE aims to measure the implementation of national ICT policies within an agency. Research conducted by Gultom et al [5] is the current implementation of SWCSF is in defense institutions such as the Cyber Defense Center of the Ministry of Defense (Pushansiber Kemhan) and the Ministry of Defense Information Data Center (ind. Pusdatin Kemhan). The application of SWCSF aims to measure an agency's readiness for cybersecurity. Instrument portability is an important part of determining whether a model or measurement system is simple to implement, and it is also related to the effectiveness and efficiency of resource use. Portability is the ease with which the instrument is moved from one place to another [7]. Portability can be defined as the ease with which the instrument is used by all users. Trishadiatmoko [8] used statistical analysis to perform a portability analysis on the SPBE instrument. This study was conducted to analyze the portability of the SPBE and SWCSF measurement instruments. One of the methods to measure the portability of the instrument is by analyzing the readability of the questionnaire that will be distributed to the respondents. 2. Research method This research was conducted using qualitative methods with a descriptive analysis approach. This study analyzed the readability of the SPBE and SWCSF instrument questionnaires using the Flesch Reading Ease. Generally, this method is used to analyze the readability of a book, magazine or essay. Anita [9] used this method to analyze the readability of grade 12 English textbooks. Research data are statements or questions displayed on the questionnaire that will be distributed to respondents. The questionnaire that was distributed was a measurement instrument that was carried out digitally because it was distributed online via a link. This indicates that the analysis using this method is very important to do to find out which categories of respondents are suitable for filling out the SPBE and SWCSF questionnaires. DSS Vol. 4, January 2023, pp.1-7 3 The calculation results obtained using Flesch reading ease are then matched with the reference table in Table 2 to see the categories of the data results. Then, the results obtained using the Flesch-Kincaid grade level will be seen in the reference table (Table 1) to see the categories of the data results. Table 1. Flesch-Kincaid Grade Level Categories [10] Score Notes 90 – 100 Comprehensible to the average 11-year-old kid 60 – 70 13- to 15-year-old children can understand it with ease 0 – 30 University grads are more likely to understand Table 2. Flesch Reading Ease Categories [10] Raw Score Difficulty level < 30 Very difficult 30 – 50 Difficult 50 – 60 Fairly difficult 60 – 70 Standard 70 – 80 Fairly easy 80 – 90 Easy 90 – 100 Very easy The questionnaire that will be distributed to respondents contains statements and questions regarding indicators from the SPBE and SWCSF. These questions and statements constitute data that will be analyzed to calculate the readability score and grade level generated by the questionnaire. These questions or statements will be grouped according to the indicators contained in the SPBE and SWCSF. The flow chart of analysis data can be seen in Figure 1. Figure 1. Data Analysis Flow 3. Results and discussion The Electronic Based Government System (SPBE) and Six-Ware Cyber Security Framework (SWCSF) are two measurement instruments that utilize information and communication technology (ICT) used in central and regional domains. SPBE is used to measure the implementation of national ICT policies in agencies. ICT policy in an agency must be supported by the agency's readiness for cyber security because, in today's digital era, the use of technology must be supported by knowledge about cybercrime. The SWCSF instrument can be Make a list of statements for the SPBE and SWCSF questionnaires Count the number of words and sentences in the sample Perform reading level analysis using the Flesch Reading Ease Formula Perform grade level analysis using the Flesch-Kincaid Grade Level Formula The resulting data is described DSS Vol. 4, January 2023, pp.1-7 4 used to assess agency readiness for cyber security. The development of ICT continues to increase, so the level of cybersecurity needs to be increased. So, it can be said that the two measurement instruments can support each other in preventing cybercrime in today's digital world. Table 3. Assessment Indicators for Electronic-Based Government Systems [8] No. Indicators 1. Governance policy 2. Service policy 3. Institutional 4. Strategy and Planning 5. Information and communication technology 6. Administrative services 7. Public service The measurement instruments used in the SPBE and SWCSF are in the form of a questionnaire consisting of several indicators. The SPBE instrument has seven determining indicators in the assessment, which can be seen in Table 3, and the SWCSF instrument has six determining indicators, which can be seen in Table 4. Table 4. Assessment Indicators for Six-Ware Cyber Security Framework [11] Factors Defenition Indicators Brainware Human Factor Security Awareness Hardware Physical computers and peripherals No compromises Software Operating system and applications No pirated Appl. etc. Infrastructureware Physical and logical network infrastructure No network security breaches, etc Firmware Document and procedures Good Bussiness Pro-cesses Budgetware An estimate of income and expenciture for a set period Licences always updated, etc The portability of these two instruments must be taken into account. The portability of an instrument refers to its ease of use by individuals or institutions. In other words, every individual or institution is capable of and understands using the instrument. One of the requirements for a measuring instrument to be considered portable is its readability. SPBE and SWCSF are instruments in the form of a questionnaire, so what is meant is the readability of the respondents to the statements or questions given. If the statement is easy to read, then the respondent understands the statement. The reading ease measurement uses the Flesch Reading Ease and Flesch-Kincaid Grade Level methods. The formula used to calculate it is as follows [12]: 𝐹𝑅𝐸 = 206.835 − 1.015 𝑥 𝐴𝑆𝐿 − 84.6 𝑥 𝐴𝑆𝑊 (1) 𝐹𝐾𝐺𝐿 = 0.39 𝑥 𝐴𝑆𝐿 + 11.8 𝑥 𝐴𝑆𝑊 − 15.59 (2) Where : FRA: Flesch Reading Ease, FKGL: Flesch-Kincaid Grade Level, ASL: Average sentence length, ASW: Average number of syllables per word. Some examples of statements and questionnaire questions used are "apakah unit kerja anda memiliki kebijakan terkait perencanaan dan penganggaran (rengar) TIK?" The sentence consists of 12 words and 33 DSS Vol. 4, January 2023, pp.1-7 5 syllables. The sentence " menurut saya, Instansi perlu adanya peraturan, perjanjian kerjasama dalam menjaga keamanan data dan informasi". The sentence has 14 words and 40 syllables. The data is obtained by counting the number of sentences, words and syllables for each measurement indicator. The results obtained from these data can be seen in Table 5 for the SPBE instrument and Table 6 for the SWCSF instrument. Table 5. Result of SPBE Instrument Indicator Number of sentences Number of words Number of syllables ASL (X1) ASW (X2) FRA (Y) FKGL 1 7 78 214 11,14 2,75 -37,12 21,20 2 10 116 340 11,60 2,93 -53,23 23,57 3 2 19 45 9,50 2,37 -3,18 16,06 4 2 22 55 11,00 2,50 -15,83 18,20 5 3 28 74 9,33 2,64 -26,22 19,24 6 7 58 166 8,29 2,86 -43,71 21,41 7 4 43 105 10,75 2,44 -10,66 17,42 Average -27,13 19,59 The readability values for the SPBE and SWCSF instruments are -27,13 and -39,28, respectively. This score falls under the "extremely difficult" category according to Table 2. Some responders may find it quite challenging to understand the level of understanding required by the distributed questionnaire. Table 6. Result of SWCSF Instrument Indicator Number of words Number of syllables Number of sentences ASL (X1) ASW (X2) FRA (Y) FKGL 1 95 268 8 11,88 2,82 -43,88 22,33 2 58 148 5 11,60 2,55 -20,81 19,04 3 36 95 4 9,00 2,64 -25,55 19,06 4 73 196 5 14,60 2,68 -35,13 21,79 5 71 184 3 23,67 2,59 -36,43 24,22 6 60 173 4 15,00 2,88 -52,32 24,28 7 79 225 3 26,33 2,85 -60,84 28,29 Average -39,28 22,72 The results of the Flesch-Kincaid Grade Level computation, which used the results of the two instruments, show that this is true. The SPBE instrument's score was 19,59, and the SWCSF instrument's score was 22,72 as you can see at Figure 2. The two scores fall under the "best understood by university graduates" category, according to Table 1. The two surveys will be provided to respondents that have a high level of knowledge, and it can be said that they are best suited for those who have completed college. The FKGL value obtained by SWCSF is greater than that obtained by SPBE. Even though the two instruments are in the same category, this indicates that the SWCSF instrument is more portable than the SPBE. DSS Vol. 4, January 2023, pp.1-7 6 Figure 2. Grade Level of SPBE and SWCSF 4. Conclusions SPBE and SWCSF measurement instruments are two important instruments to be used in today's digital era. Measuring policies on the use of ICT and cybersecurity in an agency or organization is necessary for the prevention of cybercrime. The instruments displayed from SPBE and SWCSF are in the form of questionnaires, which will be distributed to respondents. Thus, the respondent's understanding of the questions or statements in the questionnaire is important. The respondent's understanding begins with the ease with which the respondent reads the questionnaire. According to the results, the respondent's grade level is university level, and both the SPBE and SWCSF instruments are highly challenging to understand. The SPBE and SWCSF instruments can be completed by respondents who work with computers, the internet, and other technologies in settings other than higher education. Declaration of competing interest The authors declare that they have no any known financial or non-financial competing interests in any material discussed in this paper. Funding information No funding was received from any financial organization to conduct this research. References [1] M. H. Muttaqin, T. D. Susanto, "The Effect of Website Components on User Trust in Increasing the Interest to Use Public Administration Service on E-Government Website," 2019 International Conference on Computer Science, Information Technology, and Electrical Engineering (ICOMITEE), pp. 30-36, 2019. [2] F. Novianto, “Electronic Government Development Strategies Using Frameworks COBIT 5”, Proceeding International Conference on Science and Engineering, Vol 3, pp. 263-271, 2020. [3] H. M. Al-Saghier, M. Ford, A.L. Nguyen, R. Hexel, “Conceptualising Citizen's Trust in e-Government: Application of Q Methodology”, Electronic Journal of e-Government, Vol. 1, pp. 204-230, 2009. [4] Australian Cyber Security Growth Network (AustCyber), “Cyber Security Opportunities in the ASEAN Region,” Australian Cyber Security Growth Network, Canberra, 2019. [5] R. A. Gultom, F. A. Lestari, C. Lahallo, R. N. Akbar, “Pengembangan Teknologi Pertahanan Berbasis Cyber dari Sixware Cyber Framework Untuk Meningkatkan Penyelenggaraan Sistem Pertahanan Negara di Kota Batam”, Int. J. Adv. Sci. Teknologi., vol. 29, pp. 3431-3436, 2020. 19.59 22.72 18 19 20 21 22 23 SPBE SWCSF G ra de L ev el Measurement Instrument DSS Vol. 4, January 2023, pp.1-7 7 [6] L. N. Amali, M. R. Katili, S. Suhada, L. Hadjaratie, “The measurement of maturity level of information technology service based on COBIT 5 framework”, Telkomnika (Telecommunication Computing Electronics and Control), 18(1), pp. 133–139, 2020. [7] J. A. McCall, P. K. Richards., G. F. Walters, “Factors in Software Quality Volumes I, II, and III”, US Rome Air Development Center Reports, US Department of Commerce, USA, 1977. [8] P. Trishadiatmoko, A. F. Wadjdi, “Public Policy Implementation During Pandemic Covid: Modifying the Measurement Framework of The Electronic-Based Governance System for Various Organization Levels of Indonesia Public Service Broadcasting (TVRI)”, RANEPA: AAPA, 2021. [9] A. Firdaus, “The Readability Level Of Reading Texts In English Textbook Entitled Bahasa Inggris For Senior High School Students Grade XII”, UNUSA, 2020. [10] W. DuBay, “Smart language: Readers, readability, and the grading of text”, Costa Mesa: Impact Information, 2006. [11] R. Gultom, Cyber Warfare : Sudah Siapkah Kita Menghadapinya?, UNHAN Press: Bogor, 2019. [12] N. Hikmah, T. O. Sanerita, J. T. Ignatius, “The Readability Level of Reading Texts in The English Textbook Entitled When English Rings a Bell For The Seventh Grade Junior High School”, KOMPETENSI: Jurnal Ilmiah Bahasa dan Seni, pp. 368-376, 2021.