In Pr es s *Corresponding Author P-ISSN: 1412-1212 E-ISSN: 2541-2388 131 The Winners, 23(2), September 2022, 131-141 DOI: 10.21512/tw.v23i2.7423 Hot-Fit Model to Measure the Effectiveness and Efficiency of Information System in Public Sector Elvira Esperanza Sala1*; Apol Pribadi Subriadi2 1,2Information System Department, Institut Teknologi Sepuluh Nopember Jl. Teknik Kimia, Keputih, Kec. Sukolilo, Surabaya 60111, Indonesia 1salaesperanza1@gmail.com; 2apolpribadi@gmail.com Received: 10th June 2021/ Revised: 19th October 2021/ Accepted: 22nd October 2021 How to Cite: Sala, E. E. & Subriadi, A. P. (2022). Hot-fit model to measure the effectiveness and efficiency of information system in public sector. The Winners, 23(2), 131-141. https://doi.org/10.21512/tw.v23i2.7423 Abstract - The research endeavored to determine the effectiveness and efficiency of the information systems’ implementation. The research used the modified Human Organization Technology- Fit (HOT-Fit) as the model to measure information systems’ effectiveness and efficiency. Each indicator of the HOT-Fit model is mapped into two categories, namely effectiveness and efficiency. Data collection was conducted through interviews with the users of population administration information system (SIAK), and observation of activities at the public office. The data was analyzed qualitatively by assessing the achievement of each indicator of effectiveness and efficiency found in the interview and observation. It is found that effectiveness indicators are level of use, attitude, perceived usefulness, communication, availability, security, completeness, accuracy, timeliness, empathy, and transparency. Meanwhile, indicators to measure efficiency are users, training, staff support, ease of use, ease of learning, response time, access to technical support, and quick responsiveness. The research also expands the scope of the HOT-Fit model in evaluating information systems. Practically, the research can be a reference for evaluating the effectiveness and efficiency of information system implementation in the public sector. It also can be used to improve the achievement of indicators that have not achieved maximum results to improve services to the community. Keywords: evaluation of information system, HOT- Fit model, effectiveness measurement, efficiency measurement I. INTRODUCTION The use of technology has covered all aspects of human life. Information systems are one of these technologies. In government, the term e-government is known for using information technology in government activities. Today every country needs to deal with ongoing changes in digital technology and focus on the right set of technology for the governments’ progress in delivering the public services (Mittal, 2020). There are various kinds of information systems used according to each government agency's needs. To implement information system, the government has spent an extensive amount of funds. It is hoped that by using the system, the data and services will be better for the community, provide complete and accurate data, and fast and satisfying services. The significant investment of procurement and implementation of government programs requires that an assessment be carried out after the performance. Evaluation is a way to evaluate the system to determine if the system is running as expected and useful for users. Evaluation has three main objectives, namely: 1) to assess the extent and accessibility of the functions of the system, 2) to assess the user experience to interact with the system, and 3) to identify specific problems with the system (Hendarti et al., 2018). Sistem Informasi Administrasi Kependudukan (SIAK), an information system that utilizes information and communication technology to facilitate the management of population administration information at every level of the government administration area. The system has been used in Ende district since 2006, but there has never been an evaluation of its implementation despite some problems in giving service, especially the number of queues and the length of waiting time to process population documents. Therefore, it is considered necessary to measure the effectiveness and efficiency of implementing information systems to ensure its implementation. Information system evaluation is a process to explore and find out, about the sustainability of an information system implementation activities, both from the user's perception and organization In Pr es s 132 The Winners, Vol. 23 No. 2 September 2022, 131-141 perspective, also in terms of information systems technology (Puspita, Supriyantoro, & Hasyim, 2020). The information system evaluation is known as the Human Organization Technology-Fit (HOT-Fit) model. Yusof, Paul, and Stergioulas (2006) suggest this model assign important components in information systems which are humans, organization, technology, as well their relationships’ suitability. Various studies have been carried out by implementing the HOT-Fit model. Following the initial idea of making this model evaluate information systems in the health sector, more research using the HOT-Fit model is carried out in the health sector. Kodarisman and Nugroho (2013) state that over time, studies with this model began to be carried out in another type of organization, such as the government. HOT-Fit model is advantageous in evaluating an information system because the aspects assessed are three main aspects in the evaluation process. However, to expand its implementation scope, the HOT-Fit model needs to be developed. One of the development directions is using the HOT- Fit model domains as the tools for measuring the effectiveness and efficiency of information systems. Measuring the effectiveness and efficiency of SIAK is essential because the result can show whether the implementation of SIAK in Ende has met the objectives of using SIAK initiated by the government. Hamilton and Chervany (1981) state that there are two general views regarding the definition of system effectiveness. Firstly, a goal-centered theory states that the system’s effectiveness is determined by comparing performance against goals. An example of a viewpoint centered on system effectiveness objectives is comparing real costs and benefits to budgeted expenses and benefits. Secondly, according to the system resource view, system effectiveness is determined by achievement at the mutual stage, for example, standards for "good" practice. Conceptualization of effectiveness is about the resource’s feasibility in specific task objectives. For example, communication and conflict between MIS and user personnel, user participation in system development, and user job satisfaction can demonstrate the point of a system in terms of human resources. In terms of technological resources, its quality or service level can show the effectiveness. As stated by Subriadi, Herdiyanti, and Ayundari (2015), product effectiveness is defined as the user’s achievement’s accuracy and completeness. Other literature describes it as the suitability of the output and the objectives set. On the other hand, Subriadi et al. (2015) suggest efficiency as the minimum use of resources to achieve optimum results. In quality management (QM), efficiency refers to doing things right. For example, whatever is done, is done most appropriately, given the availability of resources (high efficiency). On the other hand, effectiveness refers to doing the right thing, namely selecting and focusing on producing the output on demand. The word efficiency implies that one thing is better than others. In information systems, effectiveness and efficiency are often not defined explicitly but are implied in several criteria to fulfill information system objectives. Based on the above definition, information systems' effectiveness and efficiency can be said as information systems with various qualities to achieve information system goals that support organizational goals using appropriate resources. HOT-Fit is an evaluation model developed to ensure effective implementation and positively impact implementing the Health Information (Yusof et al., 2006). HOT-Fit is built on the existing information system evaluation model, namely the Information System (IS) Success Model (DeLone & McLean, 2003) and the IT-Organization Fit Model by Scott Morton. IS Success Model is considered not to summarize the organizational factors, so it is shown in the IT-Organization Fit Model, as well as IS Success Model provide specific evaluation dimensions and measures that are not appropriate for IT-Organization Fit. Therefore, after knowing the strengths and limitations of the two models, HOT-Fit is developed to complement each other in presenting a comprehensive evaluation framework. The HOT-Fit model and the relationship between dimensions as seen in Figure 1. The research aims to expand the scope of Figure 1 HOT-Fit Model In Pr es s 133Hot-Fit Model to Measure .... (Elvira Esperanza Sala; Apol Pribadi Subriadi) the HOT-Fit model into a model for measuring the effectiveness and efficiency of information systems. This measurement model is then used in SIAK to determine the level of effectiveness and efficiency of its implementation. II. METHODS There are six stages in conducting the research: 1) literature study stage, 2) modeling and indicators classification stage, 3) mapping stage, 4) data collection and analysis stage, 5) scoring stage, and 6) measurement and categorization stage. In the literature study stage, the HOT-Fit model and its domain are studied, including its background and implementation over the years. This stage allows to learn the meaning of the information system’s effectiveness and efficiency and what standards can be categorized into effectiveness and efficiency. The modeling and indicators classification stage is to determine the HOT-Fit domains and indicators mapped to the effectiveness and efficiency categories. The research conceptual model is designed based on the HOT-Fit Model (Yusof et al., 2006), which is developed referring to the HOT variable from Kodarisman’s research because of the suitability of the research object. The domains are the basis of research, namely: 1) human, 2) organization, 3) technology, 4) effectiveness, and 5) efficiency. The model is seen in Figure 2. Each of these domains has a subdomain. Subdomain and measurement indicators are: 1) system use, 2) human-user satisfaction, 3) organization structure, 4) technology-system quality, 5) technology- information quality, 6) technology-service quality, 7) effectiveness, 8) efficiency. The first sub-domain, system use, is also described as affecting efficiency. Wibiksana (2019) state that the advantage of using information systems is to increase efficiency. Buana and Wirawati (2018) suggest that companies’ use of accounting information systems helps workers perform tasks more efficiently. Four indicators used to represent the system use sub- domain, namely training, user, attitude, and levels of use. Secondly, human-user satisfaction as stated in Yusof et al. (2006), is the overall evaluation of user experience using information systems and the potential impact of information systems. User satisfaction is declared to affect effectiveness. Munap et al. (2018) state that user satisfaction has been widely used to measure the effectiveness of information system. User satisfaction is related to perceived usefulness, therefore, personal characteristics affect user satisfaction with information systems (Yusof et al., 2006). The third sub-domain, organization structure, is the organization's nature, hierarchy, and functional division (Yusof et al., 2006). Top management and management information systems are organizational structure elements. Therefore, organizational structure can be said to affect the effectiveness of information systems as well as efficiency. Cahyaningsih (2017) finds the efficiency indicators are described by sub-indicators related to organizational structure, including employee placement according to specialization, synchronization between fields, and bureaucratic mechanisms. The indicator structure for communication and staff support is based on the research object. Fourth, technology-system quality of information systems is defined as the desired characteristics of an information system with information quality according to the desired characteristics of information products. According to Yusof et al. (2006), measuring system quality means to measure information processing systems. System quality reflects the effectiveness of information systems. The statement is supported by Pradipta, Setiawan, and Lusanjaya (2017), which states that system quality is one factor that can potentially affect the effectiveness of an information system. Likewise, Handoko and Dharmadiaksa (2017) state that the effectiveness of information is effected significant positive by the quality of information system. Apart from being a factor affecting information Figure 2 The Proposed Model In Pr es s 134 The Winners, Vol. 23 No. 2 September 2022, 131-141 systems’ effectiveness, Buana and Wirawati (2018) suggest that the efficiency domain can measure information system quality. Based on previous research and conformity with the research object, the system quality indicators used in the research are ease of use, ease of learning, access to technical support, availability, response time, and security. The fifth sub-domain, technology-information quality, as suggested in Yusof et al. (2006), focuses on information systems’ information. Weber in Pradipta et al. (2017) regards information quality as one factor that can potentially affect an information system’s effectiveness. Information quality also determines efficiency, Wardhana and Astuti (2018) suggest that information systems are expected to provide accurate, fast, and precise information. In the end, this process can shorten time without reducing the quality of data and show efficient results in presenting information for all parties. Yusof et al. (2006) suggest, and Wirapraja (2020) finds that the information quality indicators used in this study are completeness, accuracy, and timeliness. Technology-service quality, as stated by Yusof et al. (2006), that all the support of the system or technology service provider is the focus of sub- domain service quality. In-Law Number 25 of 2009 concerning Public Services, there is a statement regarding increasing services' effectiveness, with cooperation between providers, that service quality is related to effectiveness. Vellayati and Dwihartanti (2018) state that service is directed at achieving work effectiveness and efficiency. The research will use quick responsiveness, empathy, and transparency indicators to assess service quality (Yusof et al., 2006). In brief, the HOT-Fit model elements proposed as indicators for measuring the effectiveness and efficiency of the population administration information system are summarized, as shown in Table 1. The next sub-domain is effectiveness, which has various criteria to define the effectiveness of information systems. Hamilton and Chervany (1981) suggest the effectiveness perspective in three levels of objectives, which is: (1) information provided by information systems and support obtained from the benefits of information systems to users; (2) use of information systems and their effect on user processes and performance in the organization; (3) the effect of information systems on organizational performance. Effectiveness is closely related to efforts to achieve goals, so the point of SIAK in question is following the objectives of SIAK in Government Regulation Number 37 of 2007, improving the service quality as well as the availability of data and information that is accurate, complete, up-to-date, and easily accessible. Effectiveness in the research is measured by giving a score to each indicator representing the category of effectiveness. The calculation results are percentages to categorize the current system's level of effectiveness. Lastly, Hamilton and Chervany (1981) suggest efficiency through: 1) definition of the need for information systems; 2) consumption of resources needed to produce information systems; 3) production capability or resource capacity; 4) the level of investment in resources. In general, the word efficiency is associated with the proper use of resources to achieve maximum results. The resources in question can be in the form of time, cost, people, and others. Another literature said that the efficiency criteria are contact handling, resource utilization, and cost-efficiency. In the research, information systems’ efficiency is related to time and resources. Measurement of efficiency is carried out by grouping indicators related to the appropriate use of time and resources. A score is given to each hand based on the informant's statement about the indicator's achievement. The results of the total score are categorized into efficiency levels. Mapping stage is the third in the research. After defining the domains, sub-domains, and elements of the research, the HOT-Fit domain mapping is carried out on effectiveness and efficiency. The effectiveness criteria refer to the objectives of the SIAK: 1) increasing the quality of service and 2) availability of being accurate, complete, up-to-date, and easily accessible. Meanwhile, the criteria for efficiency are time and resource. Based on the criteria, the grouping for the HOT-Fit model indicator mapping on effectiveness and efficiency is shown in Table 2. The mapping results show that information system implementation’s Table 1 Domains and Indicators Human Organization Technology System Use User Satisfaction Structure System Quality Information Quality Service Quality Who uses it Perceived usefulness Communication Ease of use Completeness Quick responsiveness Level of use Staff support Ease of learning Accuracy Empathy Training Availability Timeliness Transparency Attitude Response time Security Access to technical support In Pr es s 135Hot-Fit Model to Measure .... (Elvira Esperanza Sala; Apol Pribadi Subriadi) effectiveness is measured using indicators level of use, attitude, perceived usefulness, format, communication, availability, security, completeness, accuracy, timeliness, empathy, and transparency. While the indicators for measuring efficiency are users, ease of learning, training, staff support, ease of use, access to technical support, response time, dan quick responsiveness. Table 2 Mapping Result Domain-Sub Domain Indicators Effectiveness Efficiency Improved service quality Availability of accurate, complete, up-to-date, easily accessible data Time Resource Human - System Use Who uses it v Level of use v Training v Attitude (acceptance / resistance) v v Human - User Satisfaction Perceived usefulness v Organization - Structure Communication v Staf support v Technology - System Quality Ease of use v Ease of learning v Availability v Response time v Security v Access to technical support v v Technology - Information Quality Completeness v Accuracy v v Timeliness v Technology - Service Quality Quick Responsiveness v Empathy v Transparency v Table 3 Effectiveness Scoring Indicator Formula Data Requirements Score Range Level of use Scale 0 - 4 1. working period as SIAK officer 2. The researcher’s assessment of the mastery of SIAK officers, based on the results of interviews and observations. 0 all informants do not have the ability and experience in operating SIAK 1 there is 1 informant capable and experienced in operating SIAK 2 there are 2 informants capable and experienced in operating SIAK 3 there are 3 informants capable and experienced in operating SIAK 4 there are >3 informants capable and experienced in operating SIAK Attitude Scale 0 - 4 The researcher's assessment, based on the results of interviews and observations. 0 there is no informant who has a good attitude perception towards SIAK and is willing to use SIAK 1 there is 1 informant who has a bad perception of SIAK but wants to use SIAK 2 there is 1 informant who has a good perception of SIAK but is hesitant in using SIAK 3 there is 1 informant who has a good perception and attitude towards SIAK and wants to use SIAK 4 there are >1 informants have good perceptions and attitudes towards SIAK and want to use SIAK In Pr es s 136 The Winners, Vol. 23 No. 2 September 2022, 131-141 The fourth stage in the research is data collection and analysis. Since it is a qualitative research, data is collected by interviewing the officers of the Population and Civil Registration Office of Ende Regency and observing at the office. The research informants are determined by using purposive sampling, where the informants who match the research criteria, are officers from the Department of Population and Civil Registration of Ende Regency who were responsible as direct users of SIAK with minimum one year experience. Four informants are selected: 1) database administrator, 2) network administrator, 3) family Indicator Formula Data Requirements Score Range Perceived Usefulness Scale 0 – 4 The researcher's assessment, based on the results of interviews and observations. 0 There is no benefit felt by the informant 1 There is 1 benefit felt by the informant 2 There are 2 benefits felt by the informant 3 There are 3 benefits felt by the informant 4 There are > 3 benefits felt by the informant Communication Scale 0 – 4 The researcher's assessment, based on the results of interviews and observations. 0 Absolutely nothing 1 There is communication but it is not smooth 2 There is one-way communication 3 There is two-way communication but it is not smooth 4 Smooth communication is two-way Availability [(Average SIAK time available) / (total ideal time) x 100%] The researcher's assessment, based on the results of interviews and observations. 0 Percentage 0% 20% 1 Percentage 21% - 40% 2 Percentage 41% - 60% 3 Percentage 61% - 80% 4 Percentage 81% - 100%. Security [(Number of security categories on SIAK) / (total ideal categories) x 100%] Interview, observasion, and government rules 0 Percentage 0% 20% 1 Percentage 21% - 40% 2 Percentage 41% - 60% 3 Percentage 61% - 80% 4 Percentage 81% - 100%. Completeness (Number of informants who stated complete + researcher's assessment) / (total opinion givers) * 100%] The researcher's assessment, based on the results of interviews and observations. 0 Percentage 0% 20% 1 Percentage 21% - 40% 2 Percentage 41% - 60% 3 Percentage 61% - 80% 4 Percentage 81% - 100%. Accuracy (Number of informants who stated that they were accurate + researcher's assessment) / (total opinion providers) * 100%] The researcher's assessment, based on the results of interviews and observations. 0 Percentage 0% 20% 1 Percentage 21% - 40% 2 Percentage 41% - 60% 3 Percentage 61% - 80% 4 Percentage 81% - 100%. Timeliness (Number of informants who stated correct + researcher assessment) / (total opinion givers) * 100%] The researcher's assessment, based on the results of interviews and observations. 0 Percentage 0% 20% 1 Percentage 21% - 40% 2 Percentage 41% - 60% 3 Percentage 61% - 80% 4 Percentage 81% - 100%. Empathy Scale 0 – 4 - Empathy attitude in services assessed by researchers during interviews and observations. 0 no empathy 1 rarely shows empathy 2 occasionally shows empathy 3 quite often shows empathy 4 always shows empathy. Transparency Scale 0 – 4 2 out of 3 categories of transparency in government services are fulfilled in SIAK services The category of transparency of government services assessed by researchers, based on interviews and observations Table 3 Effectiveness Scoring (Continued) In Pr es s 137Hot-Fit Model to Measure .... (Elvira Esperanza Sala; Apol Pribadi Subriadi) card operator, and 4) birth certificate operator. The data are analyzed and interpreted the meaning of each informant's statement describing each indicator. The fifth is scoring stage, in which the results of the analysis that explain each indicator are given a score, from 1-4 according to their achievement, including statements with definite values. Formulas and scores for effectiveness are shown in Table 3, while efficiency is in Table 4. The final stage is measuring and categorization. The total score obtained from the scoring stage is calculated using the standard Key Performance Indicator (KPI) formula: total score earned x 100% maximum score The calculation produces the value of effectiveness and efficiency value. Based on these values, the effectiveness and efficiency levels are categorized, referring to Table 7. III. RESULTS AND DISCUSSIONS Based on interviews and observations at the Ende district population and civil registration office, each indicator's attainments are obtained: 1) level of use, 2) attitude, 3) perceived usefulness, 4) communication, 5) availability, 6) security, 7) completeness, 8) accuracy, 9) timeliness, 10) empathy, 11) transparency, 12) users, 13) training, 14) staff support, 15) ease of use, 16) ease of learning, 17) respond time, 18) access to Table 4 Efficiency Scoring Indicator Formula Data Requirements Score Range Who uses it [(Number of officers with appropriate qualifications for SIAK administrators) / (Total SIAK officers)] x 100% Information about SIAK officers' data through interviews and observations. 0 Percentage 0% 20% 1 Percentage 21% - 40% 2 Percentage 41% - 60% 3 Percentage 61% - 80% 4 Percentage 81% - 100% Training [(Number of informants who have attended training) / (Total informants)] x 100% The researcher's assessment, based on the results of interviews and observations. 0 Percentage 0% 20% 1 Percentage 21% - 40% 2 Percentage 41% - 60% 3 Percentage 61% - 80% 4 Percentage 81% - 100% Staff Support Scale 0 – 4 The researcher's assessment, based on the results of interviews and observations. 0 Absolutely nothing 1 There is little support 2 There is enough support 3 Frequently available support 4 Always available support Ease of use Scale 0-4 The researcher's assessment, based on the results of interviews and observations. 0 Very difficult to use 1 Rather easy to use 2 Quite easy to use 3 Easy to use 4 Very easy to use Ease of learning Scale 0-4 The researcher's assessment, based on the results of interviews and observations. 0 Very difficult to learn 1 Somewhat easy to learn 2 Fairly easy to learn 3 Easy to learn 4 Very easy to learn Respond time The average time the system responds to user commands The researcher's assessment, based on the results of interviews and observations. 0 > 5 minutes 1 1 min – 5 min 2 31 seconds - 60 seconds 3 11 seconds – 30 seconds 4 < 10 seconds Access to technical support Average time to get technical support The researcher's assessment, based on the results of interviews and observations. 0 > 1 year 1 > 1 month - 1 year 2 1 week - 1 month 3 1 day - 1 week 4 < 1 day Quick Responsiveness average time for people to input documents until they get a response The researcher's assessment, based on the results of interviews and observations. 0 > 1 year 1 > 1 month - 1 year 2 1 week - 1 month 3 1 day - 1 week 4 < 1 day In Pr es s 138 The Winners, Vol. 23 No. 2 September 2022, 131-141 technical support, and 19) quick responsiveness. Each system officer is competent and mastering the use of the system. The expert’s using the system remembers their minimum tenure of a year with work frequency every day, five days a week. Furthermore, the users are satisfied with the current system. Feelings of satisfaction present an attitude of accepting SIAK to continue operating. Regarding perceived usefulness, three benefits are felt directly by the informants, namely making it easier to manage population data, making work complete faster, and avoiding informants from making mistakes. In terms of communication, the activities shown by the system users indicate that there is good communication, so the process of managing population documents is smooth and quickly completed. It means the communication is frequent, continuous, and daily communication. The system is always available unless there is an internet network problem or hardware failure. Thus, based on observations, the system is available approximately 36 hours a week (5 working days). The network admin’s statement regarding the fulfillment of system security standards as required is supported by observations of no security problems, system, data, or system environment. However, regarding completeness, the data contained in the system is not entirely complete. There are still unfilled fields due to limited data support, and the system has not been updated. Moreover, there are still errors in inputting data and the lack of supporting data on the old version of the system, which is why the data in the system is less accurate. The timeliness promised to the community was fulfilled, following the document's time. Regarding empathy, the action taken by the officer shows the act of listening to complaints from the community who are experiencing problems. Although it is quite transparent in providing services, there is still a category of service transparency that has not been fulfilled, namely complaint service. Most system users have a background following their duties as system officers, but not all system officer positions are available at the research object. Hence, the level of achievement of this indicator is only 30%. Regarding training, from the interview, information is obtained that the SIAK officers had attended training to support their duties as system officers. This indicator's level of achievement is not optimal (30%) because the number of system officers in Ende district population and civil registration office is not by the rules due to a lack of human resources. Information from interviews and observations indicates that there is support from staff for system officers, by helping each other, working together to complete tasks. The found system is easy to use, with menus and navigation easy to understand and use. Besides, the system is easy to learn, especially for new users. Users can quickly complete tasks using the system. It appears that the system has a menu that is easy to understand, as well as straightforward navigation, and there is a warning system, making it easier for new users to learn. The system appears to have a fast response time immediately after a user's request. Once clicked, there is a response from the system. The period needed to get technical support, mostly when a technical problem occurs, is 1-7 days. Based on the interviews conducted, the two informants who serve as operators state that the time needed is between 1 and 2 weeks. Through observation, it is known that the time required to retrieve the queue number to enter files is a maximum of half an hour so that in a day, officers can receive more than 100 files for processing. Furthermore, the community is promised to take the documents at least four days after submitting the files. Therefore, the average quick responsiveness indicator is around 1-7 days. In the research, the HOT-Fit model is developed to measure the effectiveness and efficiency of SIAK. Therefore, after obtaining the informants’ assessment of each indicator of effectiveness and efficiency, the measurement process is carried out according to the effectiveness and efficiency measurement tools. Measurement of effectiveness uses effectiveness indicators, as presented in Table 5. Simultaneously, the measure of efficiency is provided in Table 6. The categorization of effectiveness and efficiency is shown in Table 5. Table 5 Effectiveness Measurement Indicator Finding Score Level of use The four SIAK officers in Ende District who were interviewed had years of experience operating SIAK. The entire menu, which is the users’ task, is controlled and can be handled by the SIAK officer. 4 Attitude Informants are satisfied with the current system and accept SIAK to operate. The development of SIAK, which is complete from day to day, also increases feelings of satisfaction with SIAK and positively influences the willingness of operators to work using SIAK. 4 Perceived Use- fulness Three benefits are felt directly by informants, making it easier to manage population data, make work complete faster, and avoid informants from being mistaken. 3 Communication Communication between SIAK officers is well established, two-way, and continuous every day 4 In Pr es s 139Hot-Fit Model to Measure .... (Elvira Esperanza Sala; Apol Pribadi Subriadi) Indicator Finding Score Availability The system is always available unless a technical problem occurs. At the time of observation, SIAK operates normally for 8 hours a day. Within a week (5 days), SIAK is available for 36 hours out of a total of 40 hours because there are days when SIAK experiences network connection problems. 4 Security Network security aspects have been met, while other security aspects follow SOPs. However, what security aspects are not explained. 4 Completeness Incomplete because there are developments in SIAK that require previously optional data input. The developments have an impact on the completeness of population data that has been stored in the database. The majority of opinion givers considered that the completeness indicator had not been fulfilled, so the percentage was 25%. 1 Accuracy Overall, it is not yet accurate due to inaccurate initial data and KK data without complete supporting data, resulting in errors in inputting or data inconsistencies with the population's condition. 3 out of 4 opinion respondents considered the SIAK data to be inaccurate so that the accuracy of the accuracy indicator was 25% 1 Timeliness According to the time frame for completing each document (1 day - 2 weeks), the timeliness promised to the citizen (1-2 weeks) can be fulfilled. 4 Empathy Informants felt empathy in providing services to residents who administer population documents, mostly if there were many queues for residents. This feeling of compassion is felt and expressed by providing tents and chairs to queue up and pay attention to the queues’ order. 3 Transparency 2 out of 3 categories of transparency in government services are fulfilled in SIAK services 3 Total Score 34 Based on Table 5, the total domain value on the effectiveness measurement tool is 44, so the calculation of the SIAK effectiveness score using the standard KPI formula is: The effectiveness of SIAK = 34 x 100% 44 = 77,27% SIAK’s effectiveness value of 77,27%, based on the percentage range of categories in Table 7, grouped into the very good category. This shows that the implementation of SIAK, from the SIAK management side, has met the objectives of population data collection, namely improving the quality of services and the availability of being accurate, complete, up-to- date, and easily accessible data. Table 6 Efficiency Measurement Indicator Finding Score Who uses it The compatibility between the educational background and the officer’s duties, the comparison is under Permendagri number 25 of 2011, the ratio is 3: 1. However, the SIAK, which ten managerial positions must operate, is only filled with three places, so that the percentage is 30%. 1 Training 3 out of 4 informants have attended training according to their duties as SIAK officers. 2 people have attended training more than once, the percentage is 75%. 3 Staff Support There is support from other staff, helping each other, especially if the informant has problems in his work 3 Ease of use The informants questioned stated that SIAK was easy to use 3 Ease of learning SIAK is easy to learn even though it takes some time to get used to. The condition can be categorized as SIAK as relatively easy to learn 2 Respond time The design responds to user requests immediately after clicking. 4 Access to technical support The time it takes to get access to technical support is 1 hour - 1 week, depending on the problems at hand. On average, informants stated that technical assistance could be obtained within one day 3 Table 5 Effectiveness Measurement (Continued) In Pr es s 140 The Winners, Vol. 23 No. 2 September 2022, 131-141 Indicator Finding Score Quick Responsiveness The time needed to complete one population document is a day - 2 weeks 3 Total Score 22 Table 7 Categorization (Herdiyanti, Hapsari, & Susanto, 2019) Percentage Range Category 81% - 100% Satisfactory 61% - 80% Very good 41% - 60% Good 21% - 40% Sufficient 0% - 20% Lacking Based on Table 6, the total domain value on the efficiency measurement device is 32, so the calculation of the SIAK efficiency score using the standard KPI formula is: SIAK's Efficiency = 22 x 100% 32 = 68,75% The SIAK efficiency score is 68,75%, in the range of categories classified in the very good category. It shows that from the SIAK management side, the implementation of SIAK in Ende District has an efficiency impact in terms of time and resources. Based on the stages and results of the research, the initial HOT-Fit model (Yusof et al., 2006) is expanded into a measurement model with each element of human, organization, and technology as an indicator. A significantly good results indicate services that have been effective and efficient. This is in line with previous research which finds that the effectiveness and efficiency of information systems in government improve the quality of government services (Durachman et al., 2020). IV. CONCLUSIONS The effectiveness and efficiency of SIAK can be a measure of the success rate of implementing SIAK in a region. Based on the research that has been done, several conclusions have been drawn. The HOT-Fit model’s domains and elements have been successfully measuring the effectiveness and efficiency of SIAK implementation in the Ende District. There are 11 indicators used in measuring the effectiveness of the implementation of information systems, namely level of use, attitude, perceived usefulness, communication, availability, security, completeness, accuracy, timeliness, empathy, and transparency. Meanwhile, eight indicators to measure efficiency are users, training, staff support, ease of use, ease of learning, response time, access to technical support, and quick responsiveness. According to the measurement results, the implementation of SIAK in Ende District reached good level for effectiveness (34 of 44), which means that the implementation of SIAK has realized the objectives of establishing this system well. Meanwhile, it is included in the very good category for efficiency (22 of 32). The result indicates that the implementation of SIAK has made the document processing process more efficient in terms of time and other resources. The research provides a model for measuring the effectiveness and efficiency of the successful implementation of information systems, especially those used in the government service sector. Furthermore, the research expands the scope of the HOT-Fit model in evaluating information systems. Practically, the research is one of the references for evaluating services using SIAK in Ende Regency. Improving services to the community can be achieved through improving indicators that have not yet achieved maximum results. The measurement model has only been tested on one research object in the government sector. In fact, with complete and comprehensive indicators both from the human side, as well as organization and technology, the model has the prospect of expanding its scope. Further research can implement this measurement model on research objects in other sectors. REFERENCES Buana, I. B. G. M. M. & Wirawati, N. G. P. (2018). Pengaruh kualitas sistem informasi, kualitas informasi, dan perceived usefulness pada kepuasan pengguna sistem informasi akuntansi. E-Jurnal Akuntansi, 22(1), 683-713. https://doi.org/10.24843/EJA.2018. v22.i01.p26 Cahyaningsih. (2017). Pengaruh struktur organisasi terhadap efektivitas organisasi pada dinas daerah. [Thesis, Universitas Lampung]. DeLone, W. H. & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30. https://doi.org/10.1080/07421 222.2003.11045748 Durachman, Y., Nuryasin, Harahap, D., Rodoni, A., Faisal Bakti, A. M., & Mansoer, M. (2020). Analysis of factors that affect the quality of e-government services: A case study in Ombudsman of the Republic of Indonesia. 2020 8th International Conference on Cyber and IT Service Management, CITSM 2020. https://doi.org/10.1109/CITSM50537.2020.9268796 Hamilton, S. & Chervany, N. L. (1981). Evaluating Table 6 Efficiency Measurement (Continued) In Pr es s 141Hot-Fit Model to Measure .... (Elvira Esperanza Sala; Apol Pribadi Subriadi) information system effectiveness - Part I: Comparing evaluation approaches. MIS Quarterly: Management Information Systems, 5(3), 55-69. https://doi. org/10.2307/249291 Handoko, R. & Dharmadiaksa, I. B. (2017). Faktor-Faktor yang memengaruhi pada efektifitas sistem informasi akuntansi hotel berbintang tiga di Denpasar. E-Jurnal Akuntansi, 21(2), 1101-1130. https://doi. org/10.24843/EJA.2017.v21.i02.p09 Hendarti, H., Christian, T. L., Permatasari, A., Maryani, & Nindito, H. (2018). Evaluation of web information system national land agency to improve user satisfaction. Proceedings of 2017 International Conference on Information Management and Technology, ICIMTech 2017, pp. 39-43. https://doi. org/10.1109/ICIMTech.2017.8273508 Herdiyanti, A., Hapsari, P. S., & Susanto, T. D. (2019). Modelling the smart governance performance to support smart city program in Indonesia. Procedia Computer Science, 161, 367-377. https://doi. org/10.1016/j.procs.2019.11.135 Kodarisman, R. & Nugroho, E. (2013). Evaluasi Penerapan Sistem Informasi Manajemen Kepegawaian (SIMPEG) di Pemerintah Kota Bogor. Jurnal Nasional Teknik Elektro dan Teknologi Informasi, 2(2), 24-32. Mittal, P. (2020). A multi-criterion decision analysis based on PCA for analyzing the digital technology skills in the effectiveness of government services. 2020 International Conference on Decision Aid Sciences and Application, DASA 2020, pp. 490-494. https:// doi.org/10.1109/DASA51403.2020.9317241 Munap, R., Ahmad, S. N. B., Hamid, S. A., & Beg, M. F. B. M. T. (2018). The influence of End User Computing System (EUCS) on User Satisfaction: The case of a logistic and courier service company. The International Journal of Social Sciences and Humanities Invention, 5(12), 5103-5110. https://doi. org/10.18535/ijsshi/v5i12.03 Pradipta, A., Setiawan, A., & Lusanjaya, G. R. (2017). Analisa faktor-faktor yang mempengaruhi efektivitas sistem informasi pada situs e-commerce terpilih dan hubungannya dengan kepuasan konsumen. Journal of Accounting and Business Studies, 1(2), 1-15. Puspita, S. C., Supriyantoro, & Hasyim. (2020). Analysis of hospital information system implementation using the Human-Organization-Technology (HOT) Fit Method: A case study hospital in Indonesia. European Journal of Business and Management Research, 5(6), 1-8. https://doi.org/10.24018/ ejbmr.2020.5.6.592 Subriadi, A. P., Herdiyanti, A., & Ayundari, S. R. (2015). Pengukuran efektivitas dan efisiensi e-goverment - Surabaya Single Windows. Seminar Nasional Sistem Informasi Indonesia, (November), 69-74. Surabaya: SESINDO. Vellayati, W. A. & Dwihartanti, M. (2018). Pengaruh kualitas pelayanan publik dan disiplin kerja pegawai terhadap kepuasan masyarakat di kantor balai desa Rowosari Kecamatan Ulujami Kabupaten Pemalang. Jurnal Pendidikan Administrasi Perkantoran, 7(2), 146-154. Wardhana, G. N. & Astuti, E. S. (2018). Pengaruh kualitas sistem, kualitas informasi dan kemudahan pengisian SIAM (Sistem Informasi Akademik Mahasiswa) terhadap kepuasan mahasiswa studi dilakukan pada Fakultas Ilmu Komputer Universitas Brawijaya tahun angkatan 2016-2017. Jurnal Administrasi Bisnis, 62(2), 158-165. Wibiksana, I. G. (2019). Implementasi kebijakan pelaksanaan e-government di Desa Sayang Kabupaten Sumedang. Jurnal Agregasi: Aksi Reformasi Government dalam Demokrasi, 7(1), 83-102. Wirapraja, A. (2020). Minat beli ulang pengguna social commerce instagram. [Thesis, Institut Teknologi Sepuluh Nopember]. Yusof, M. M., Paul, R. J., & Stergioulas, L. K. (2006). Towards a framework for Health Information System evaluation. Proceedings of the 39th Hawaii International Conference on System Sciences, vol. 5. http://dx.doi.org/10.1109/HICSS.2006.491