Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 9, No. 3, 1632-1639 2025 Publisher: Learning Gate DOI: 10.55214/25768484.v9i3.5646 © 2025 by the authors; licensee Learning Gate © 2025 by the authors; licensee Learning Gate History: Received: 13 January 2025; Revised: 17 March 2025; Accepted: 18 March 2025; Published: 21 March 2025 * Correspondence: andrajati@farmasi.ui.ac.id Development of instruments for implementation analysis and effectiveness evaluation of clinical pharmacy services as well as need identification of Telepharmacy services Rachmawati Felani Djuria1, Retnosari Andrajati2*, Nadia Farhanah Syafhan3, Bambang Wispriyono4 1,2,3Faculty of Pharmacy, University Indonesia, Depok, West Java, Indonesia; andrajati@farmasi.ui.ac.id (R.A.). 4Faculty of Public Health, University Indonesia, Depok, West Java, Indonesia. Abstract: Pharmaceutical services are activities designed to improve the quality of patients' lives. To measure their effectiveness, four indicators are often used, including input, process, results, and productivity. The purpose of this study is to develop instruments for implementation analysis and effectiveness evaluation of clinical pharmacy services, as well as to identify the needs of telepharmacy services. The study procedures were carried out using qualitative and quantitative methods. The stages of development included literature study, expert panel, validity and reliability testing, and instrument testing. The results showed that at the expert panel stage, using CVR-Lawshe analysis, all question items in instruments 1 and 2 were valid with a CVR value greater than 0.56. Based on the i-CVI values, which were greater than 0.80, all question items were relevant to the study objectives and had high validity. At the validity and reliability testing stage, the Alpha Cronbach values of instrument 2 on compliance and appropriateness of drug use were 0.709 and 0.866, respectively. Meanwhile, the Alpha Cronbach value of instrument 3 was 0.951. At the trial stage, respondents' understanding of the questions was assessed without any direction. All instruments developed in this study were valid, reliable, and could be effectively applied. Keywords: Clinical pharmacy, Instruments, Telepharmacy. 1. Introduction Pharmaceutical services are a series of activities carried out directly and responsibly for patients, which focus on pharmaceutical preparations. The primary aim of these services is to achieve definite results and improve the quality of life of patients [1, 2]. In addition, their implementation requires quality control on an ongoing basis through various assessments [1]. According to Sedarmayanti [3] there are 4 indicators for measuring effectiveness, including input, process, results (output), and productivity. Input indicator is in the form of human resources and infrastructure, while the process aspect comprises implementation (ways, methods, and techniques) of clinical pharmacy services. Several studies have shown that outcome indicator is based on quantity (number of services), quality (compliance and appropriate use of drugs), and time (average service time), while productivity is based on services development according to needs. One of the productivities of pharmaceutical services is the implementation of telepharmacy services by pharmacists through the use of telecommunications technology and information systems for patients [2, 4] who are located over long distances [5] and in rural areas [6, 7]. This method is an effective solution for implementing pharmaceutical services during the COVID-19 pandemic [8-10] and is needed in Indonesia. During the COVID-19 pandemic, patients received more drug information from social media, internet networks, and WhatsApp [11] with the least amount obtained from pharmacy staff. The 1633 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 9, No. 3: 1632-1639, 2025 DOI: 10.55214/25768484.v9i3.5646 © 2025 by the authors; licensee Learning Gate results were consistent with a study on patients’ compliance in using prescription antibiotics in Pangkalanbaru, Central Bangka Regency, Bangka Belitung Islands Province, which reported a low category. This was because drug information provided was incomplete and only limited to the usage rules [12]. Therefore, this study aims to develop instruments for analyzing the implementation and evaluating the effectiveness of clinical pharmacy services, and identifying community needs for telepharmacy services. By obtaining valid and reliable instruments, the quality of pharmaceutical services can be controlled to improve pharmaceutical services. The product is also expected to facilitate the implementation of telepharmacy services based on community needs. 2. Methods This was a qualitative and quantitative study that aimed to obtain draft question items in a study instrument that were valid and reliable. This consisted of 3 instruments, namely 1) an evaluation instrument for the implementation of clinical pharmacy pharmaceutical services in hospitals, 2) an instrument for the effectiveness of clinical pharmacy services in hospitals, and 3) an instrument for identifying community needs for telepharmacy services. The design for developing the study instrument was carried out in 4 stages, namely preparing instrument using literature study, expert panel, validity reliability testing, and instrument testing. The literature study was carried out in August 2023, and the preparation of the draft instrument (expert panel) was carried out in a hybrid manner (offline and online via the Zoom cloud meeting application) in September 2023. Furthermore, validity and reliability tests were carried out in September-October 2023 at RSUD Dr. Ir. H. Ibnu Saleh, MM, Namang, Central Bangka Regency. Instrument trial was carried out in September-October 2023 at Sjafrie Rachman Pudding Besar Regional Hospital, Bangka Regency. The hospitals selected were government hospitals that had implemented telemedicine. The number of expert panels that were involved was 5 people and the expert team involved in this expert judgment consisted of 7 pharmacists working in hospitals who were representatives from each district/city area. The study population consisted of 2 types, namely pharmaceutical services facilities and informants. Pharmaceutical services facilities were all hospitals in the Bangka Belitung Islands Province, and the informants were all residents of the Bangka Belitung Islands Province. The sample size of pharmaceutical services facilities for validity and reliability testing as well as instrument testing was one government hospital in each of the Bangka Belitung Islands Province which had inclusion criteria (Government Hospital Type D or D-Pratama and implements telemedicine services) and exclusion (had problems in terms of operational permits and located outside the territory of the Bangka Belitung Islands Province). The informants in this study consisted of key/main informants and supporting/additional informants who had the same characteristics as the sample of informants from the study locus. Key informants were the Head of the Pharmacy Installation and Pharmacist who provided clinical pharmacy services or saturated sampling (total sampling). The total key informants for validity and reliability testing were 3 pharmacists and 1 pharmacist for instrument testing. Additional informants in this study consisted of 7 people (namely other health workers who held the positions of Head Room Nurse, Head Room Midwife, Doctor, Head of the Pharmacy and Therapy Team, Head Nutritionist/Nutritionist, and Head of Medical Records as well as representatives of Hospital Management). Apart from pharmaceutical services variables, investigators also obtained data on accuracy and compliance with drug use from 30 community informants (patients and/or patient families), and the total number of additional informants was 37 people. The total number of informants was 40 people for validity and reliability testing, while for instrument testing there were 38 people. The inclusion criteria for informants were involved in clinical 1634 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 9, No. 3: 1632-1639, 2025 DOI: 10.55214/25768484.v9i3.5646 © 2025 by the authors; licensee Learning Gate pharmacy services both providing and receiving services, residing in the Bangka Belitung Islands Province, and willing to fill out informed consent. Exclusion criteria were moving assignments to another agency and experiencing serious illness at the time data collection took place. 3. Results 3.1. Literature Study The initial step of this report was to conduct a literature study on clinical pharmacy services in hospitals, telepharmacy clinical pharmacy services, patient compliance with treatment procedures, and appropriate use of drugs. All information obtained was then transcribed into written draft form. The results of the literature study were used as a basis for compiling the question items in instrument. The literature study produced 3 draft instruments that could be used, namely instrument for implementing clinical pharmacy services, instrument for evaluating the effectiveness of clinical pharmacy services and instrument for identifying community needs for telepharmacy services in the island region. 3.2. Expert Panel After preparing a draft study instrument through a literature study, a hybrid online and offline expert panel was conducted. When online, this was done synchronously (Focus Group Discussion/FGD method) and asynchronously (self-filling via spreadsheet link). While offline, direct interviews were conducted with a panel of experts. According to Taherdoost [13] the content validity assessment procedure required the presence of investigators and experts to facilitate validation. However, when this was not possible then it was necessary to carry out a comprehensive literature review. In addition, a validation survey of experts in the same study field using 3 scales (essential, useful but not essential, and not necessary), carried out CVR calculations using the Lawshe method and eliminated items that were not significant. The expert panel team consisted of 5 practicing pharmacists who were also implementers in assessing the implementation of pharmaceutical services in the Bangka Belitung Islands Province. The positions of the 5 expert panel were Head of the Pharmacy Section of the Bangka Belitung Islands Provincial Health Service, Head of the Health Resources Division of the Bangka District Health Service, Sub-Coordinator for Pharmacy, Medical Devices and PKRT, South Bangka Health, Population Control and Family Planning Service, Coordinator of the POM Pangkalpinang Hospital Service Assessment Team, and Chairman of the Hospital Pharmacy Association (HISFARSI) of Bangka Belitung Islands Province. During the expert panel activities, the panelists assessed the draft instrument that was prepared by assessing the relevance of the question items in instrument to the study objectives using Lawshe's CVR technique. According to Lawshe [14] each question item had 3 answer choices, namely essential, useful but not essential, and not necessary. The number of panelists who answered essentially was more than half, meaning the item was content-valid Ayre and Scally [15] and Gilbert and Prion [16]. Polit, et al. [17] also stated that 3 or more panelists who gave an "essential" opinion could be considered evidence of the content validity of a good question item. Question items that did not reach the minimum score could be deleted or removed from the question list. Based on the results of the expert panel discussion, all Instrument 1 question items were valid in content but there were changes and additions to several questions. Changing questions was done by revising the editorial writing of the questions. The revised editorial writing was contained in 6 questions, namely question 7 on the respondent's biodata and questions 1, 2, 3, 4, and 5 on the implementation of clinical pharmacy pharmaceutical services. The additions were made because there were things that the expert panel team considered important to ask about to perfect the data to be taken. Additional questions totaling 5 questions with details namely 3 questions on the implementation 1635 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 9, No. 3: 1632-1639, 2025 DOI: 10.55214/25768484.v9i3.5646 © 2025 by the authors; licensee Learning Gate of clinical pharmacy pharmaceutical services, 1 question on the documentation sheet, and 1 question on the observation sheet. In Instrument 2, all question items were content-valid, however, 2 questions were revised by the editors, namely question 7 on the biodata of pharmacist respondents and community respondents. Likewise, in Instrument 3, there was 1 question that was revised by the editor, namely question 7 in the respondent's biodata. The results of the discussion from this expert panel produced a draft of new questions for instrument which were then reassessed by experts through online synchronous expert judgment (Focus Group Discussion/FGD method via Zoom meeting) and online asynchronous (self-filling via spreadsheet link). The expert-judgment FGD discussion involved 7 hospital clinical pharmacy practitioner pharmacists who were representatives from each district/city area. Conceptually, all Pharmacists agreed on instruments, and for the validity of instrument to be unbiased, the investigators hoped that pharmacist could be advised to complete it independently via the spreadsheet link. Pharmacists were expected to provide input when there was anything that could be possible for improvement. The assessment of the relevance of the question items in instrument to the study objectives was the same as the previous expert panel activities using Lawshe's CVR technique. The difference was that in expert judgment activities, the content of instrument was determined until mutual agreement was reached, and the content validity ratio (CVR) was calculated. Based on the CVR analysis, all question items were valid in content and there were 2 additional questions in instrument 2 (documentation sheet and observation sheet with 1 question each). This was made to clarify the results because what the investigators wanted was the average time, and the expert judgment team thought it was important to add questions to the documentation sheet and observation sheet. According to Lawshe [14] the minimum CVR value for a panel of 12 people was 0.56. Based on CVR analysis, all question items had a Lawshe CVR value > 0.56 and were declared valid. The results in 1 also showed that in Instrument 3, 17 answer choice items were invalid because the CVR value was <0.56. According to Tilden, et al. [18] a good i-CVI value was at least 0.70. In contrast to Lynn [19] and Polit, et al. [17] stated that a study instrument was said to be relevant to the study objectives when it had an i-CVI value of at least 0.83. In line with Guilford and Fruchter [20] who also stated that study instruments had very high validity when the i-CVI value was between 0.800-1.00. Consequently, Davis [21] suggested a minimum CVI value of 0.80. The overall CVI score could be more efficient when compared to the CVR value of each question item. Based on Table 1, the i-CVI value for all instruments was > 0.80. Therefore, it could be concluded that the question items in all instruments were relevant to the study objectives and had very high validity. Table 1. Description of CVR and i-CVI Values for Research Instruments Instrument Number of Panelists Minimum CVR Value Total CVR Total Question Items i-CVI Value Number of Valid Questions Number of Invalid Questions 1 12 0.56 115.66 117 0.988 117 0 2 12 0.56 53.56 56 0.958 56 0 3 12 0.56 264.05 308 0.857 291 17 3.3. Validity and Reliability Test Instruments that were content-valid were then subjected to face validation. In face validity, a test was carried out on understanding the language used in instrument on 30 people/patients for the 1636 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 9, No. 3: 1632-1639, 2025 DOI: 10.55214/25768484.v9i3.5646 © 2025 by the authors; licensee Learning Gate quantitative study instrument and on key informants and supporting informants for the qualitative instrument. This consisted of 2 qualitative (instruments 1 and 2) and 1 quantitative instrument (instruments 3a and b). Validity and reliability test respondents consisted of pharmacists, health workers, hospital management, patients, and the public. Based on the qualitative study data collection process using instruments 1, 2, and 3b on respondents, it could be concluded that respondents could understand all questions well without any confusion in answering. Therefore, it could be said that instruments 1, 2, and 3b were facially valid. Data processing and analysis were carried out to test validity and reliability for instrument 2 data (for indicators of compliance and appropriateness of drug use) and 3a using the IBM SPSS Statistics 25 program, and the processing results could be seen in Table 2. Table 2. Description of Validity and Reliability Test Results for Instruments 2 and 3a with SPSS 25 Measurement Variables Number n r table Cronbach Alpha Value Compliance 32 0.349 0.709 Accuracy of drug use 32 0.349 0.866 The need for telepharmacy services 36 0.329 0.951 According to Puspasari and Puspita [22] the validity test aimed to find out whether there were statements that must be discarded or replaced because these were considered irrelevant. Hadisa, et al. [23] stated that the validity test was carried out using the Pearson correlation method. Product Moment Analysis was a simple technique for assessing the relationship between variables [24]. Instrument was declared valid when it correlated with the question items and the total score ≥ 0.3 [23, 25]. The validity test could also be seen by comparing the calculated r-value with the r-table value, and a statement was declared valid when it had a calculated r-value greater than the r-table value. Instrument was also said to be valid when the correlation value (Pearson correlation) showed a positive number and the probability value (2-tailed sig.) < α 0.05 [22]. Based on the results of the analysis in Instrument 2, there was 1 question about compliance that was invalid, and Instrument 3 had 155 invalid answer choices. This was because the question item had a calculated r value < 0.3 and/or had a calculated r value < r table and it was declared invalid. In instrument 3 there were 2 question topics for which all answers were invalid, namely regarding telepharmacy methods and the need for places to implement drug information services. In reliability testing, the most commonly used consistency measure was Cronbach's Alpha. There were no standard rules for determining Cronbach's Alpha value, but most investigators agreed that the minimum Cronbach's Alpha coefficient was 0.70 [26]. In line with that, Hadisa, et al. [23] also stated that reliability tests usually used internal consistency techniques with a Cronbach Alpha value ≥ 0.7 (α ≥ 0.7). The Alpha value was in the range 0-1, and zero showed that instrument was not reliable and conversely a value of 1 showed that it was perfectly reliable. There were 4 criteria for reliability, namely very good (Alpha value > 0.90), high (Alpha value 0.70-0.90), moderate (Alpha value 0.50-0.70), and low reliability (Alpha value < 0.50). Based on Table ii, the Cronbach's Alpha value for instrument 2 on the compliance question was 0.709, and on the question of appropriateness of medication use was 0.866. Therefore, it could be concluded that the reliability of Instrument 2 was high. The Cronbach Alpha value of instrument for the need for telepharmacy services in island communities was 0.951, and it could be concluded that the reliability of instrument 3 was very good. 1637 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 9, No. 3: 1632-1639, 2025 DOI: 10.55214/25768484.v9i3.5646 © 2025 by the authors; licensee Learning Gate 4. Instruments Testing The study instrument trial was carried out at the Sjafrie Rachman Pudding Besar District Hospital, Bangka Regency. The study instrument trial aimed to ensure that instrument could be used in other places on a small scale before it was applied on a large scale at the study locus. In the trial, 3 stages of the report were carried out following valid instruments, consisting of analysis of the implementation of clinical pharmacy pharmaceutical services, evaluation of the effectiveness of implementing clinical pharmacy services, and identifying community needed for telepharmacy services. 4.1. Analysis of the Implementation of Clinical Pharmacy Pharmaceutical Services This study stage used instrument 1, and the respondents were pharmacists and health workers related to pharmaceutical work (such as doctors, nurses, midwives, medical records, nutritionists, pharmacy and therapy committees, and representatives of hospital management). In addition, the total number of respondents at this stage was 8 people (1 pharmacist and 7 other health workers). Based on the results of data collection from in-depth interviews, FGDs, surveys, and direct observations in the field, it showed that all respondents were able to answer questions according to the investigator’s expectations, namely that all respondents understood all the questions without showing expressions of confusion, and it could be concluded that this Instrument 1 could be used in further studies, namely on a large scale. 4.2. Evaluation of the Effectiveness of Implementation of Clinical Pharmacy Pharmaceutical Services This study stage used instrument 2, and the respondents were 30 pharmacists and patients. For some patient respondents, specifically those who did not speak Indonesian, investigators used regional languages when asking questions to make data collection easier. Based on the results of data collection from in-depth interviews, surveys, and direct observations with respondents, showed that all respondents were able to answer questions without any confusion. When observing pharmaceutical services, investigators also did not find it difficult to fill in data into instrument. Therefore, it could be concluded that Instrument 2 could be used in further reports, namely on a large scale. 4.3. Identify Community Needs for Telepharmacy Services This report stage used instruments 3a and 3b, and in 3a, the respondents were the public as users of telepharmacy services and other health workers, and hospital management as providers of telepharmacy services. Instrument 3b which was the respondent was Pharmacist. Based on the results of data collection through filling out questionnaires by respondents independently, it showed that all respondents were able to answer questions without any confusion. Therefore, it could be concluded that instruments 3a and 3b could be used in further studies, namely on a large scale. 5. Conclusion The conclusion was all instruments were valid and reliable and could be used in the studies. Transparency: The authors confirm that the manuscript is an honest, accurate, and transparent account of the study; that no vital features of the study have been omitted; and that any discrepancies from the study as planned have been explained. This study followed all ethical practices during writing. 1638 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 9, No. 3: 1632-1639, 2025 DOI: 10.55214/25768484.v9i3.5646 © 2025 by the authors; licensee Learning Gate Acknowledgments: Authors would like to thank the Ministry of Health for providing financial assistance in collecting data for this research. Copyright: © 2025 by the authors. This open-access article is distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). References [1] Ministry of Health, Minister of health regulation No. 72 of 2016 concerning pharmaceutical service standards in hospitals. Jakarta: Indonesian Ministry of Health, 2016. [2] Indonesian Government, Government regulation of the republic of Indonesia number 28 of 2024 concerning implementing regulations of law Number 17 of 2023 concerning Health. Indonesia: Indonesian Government, 2024. [3] Sedarmayanti, Human resources and work productivity. Indonesia: Mandar Maju Publishers, 2009. [4] Ministry of Health, Minister of health decree No. HK.01.07/ MENKES/4829/2021 concerning guidelines for health services via telemedicine during the corona virus1disease 20191 (Covid-19) pandemic. Jakarta: Indonesian Ministry of Health, 2021. [5] A. Z. Win, "Telepharmacy: Time to pick up the line," Research in Social & Administrative Pharmacy: RSAP, vol. 13, no. 4, pp. 882-883, 2017. http://doi.org/10.1016/j.sapharm.2015.06.002 [6] A. Poudel and L. M. Nissen, "Telepharmacy: A pharmacist’s perspective on the clinical benefits and challenges," Integrated Pharmacy Research and Practice, vol. 5, pp. 75-82, 2016. https://doi.org/10.2147/iprp.s101685 [7] T. Le, M. Toscani, and J. Colaizzi, "Telepharmacy: A new paradigm for our profession," Journal of Pharmacy Practice, vol. 33, no. 2, pp. 176-182, 2020. [8] G. N. Iftinan, N. Wathoni, and K. Lestari, "Telepharmacy: A potential alternative approach for diabetic patients during the COVID-19 pandemic," Journal of Multidisciplinary Healthcare, pp. 2261-2273, 2021. [9] Z. I. Lubis, "Qualitative analysis of the use of telemedicine as a health service solution in Indonesia during the Covid- 19 pandemic," Physiotherapy Health Science, vol. 2, no. 2, pp. 76-82, 2021. https://doi.org/10.22219/physiohs.v2i2.15148 [10] W. U. Putri and I. A. Wicaksono, "Article review: Telepharmacy services during the Covid-19 pandemic," Farmaka, vol. 19, no. 3, pp. 93-102, 2021. [11] A. Gaviria-Mendoza et al., "Self-medication and the ‘infodemic’during mandatory preventive isolation due to the COVID-19 pandemic," Therapeutic Advances in Drug Safety, vol. 13, pp. 1-12, 2022. https://doi.org/10.1177/20420986211072376 [12] R. Djuria, "Compliance With the use of amoxcillin antibiotic during the Covid-19 pandemic by patient at pangkalanbaru central bangka health center," presented at the 2nd International Conference on Handling Non- Communicable Diseases Proceedings, 2-3, 2020. [13] H. Taherdoost, "The content validity assessment procedure: A method to facilitate validation," International Journal of Academic Research in Business and Social Sciences, vol. 6, no. 11, pp. 41-47, 2016. [14] C. Lawshe, "A quantitative approach to content validity," Personnel Psychology, vol. 28, no. 3, pp. 563-575, 1975. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x [15] C. Ayre and A. J. Scally, "Critical values for Lawshe’s content validity ratio: revisiting the original methods of calculation," Measurement and evaluation in counseling and development, vol. 47, no. 1, pp. 79-86, 2014. http://doi.org/10.1177/0748175613513808 [16] G. E. Gilbert and S. Prion, "Making sense of methods and measurement: Lawshe's content validity index," Clinical Simulation in Nursing, vol. 12, no. 12, pp. 530-531, 2016. https://doi.org/10.1016/j.ecns.2016.08.002 [17] D. F. Polit, C. T. Beck, and S. V. Owen, "Is the CVI an acceptable indicator of content validity? Appraisal and recommendations," Research in Nursing & Health, vol. 30, no. 4, pp. 459-467, 2007. https://doi.org/10.1002/nur.20199 [18] V. P. Tilden, C. A. Nelson, and B. A. May, "Use of qualitative methods to enhance content validity," Nursing Research, vol. 39, no. 3, pp. 172-175, 1990. https://doi.org/10.1097/00006199-199005000-00015 [19] M. R. Lynn, "Determination and quantification of content validity," Nursing Research, vol. 35, no. 6, pp. 382-386, 1986. https://doi.org/10.1097/00006199-198611000-00017 [20] J. P. Guilford and B. Fruchter, Fundamental statistics in psychology and education, 6th ed. McGraw-Hill: United States, 1978. [21] L. L. Davis, "Instrument review: Getting the most from a panel of experts," Applied Nursing Research, vol. 5, no. 4, pp. 194-197, 1992. https://creativecommons.org/licenses/by/4.0/ http://doi.org/10.1016/j.sapharm.2015.06.002 https://doi.org/10.2147/iprp.s101685 https://doi.org/10.22219/physiohs.v2i2.15148 https://doi.org/10.1177/20420986211072376 https://doi.org/10.1111/j.1744-6570.1975.tb01393.x http://doi.org/10.1177/0748175613513808 https://doi.org/10.1016/j.ecns.2016.08.002 https://doi.org/10.1002/nur.20199 https://doi.org/10.1097/00006199-199005000-00015 https://doi.org/10.1097/00006199-198611000-00017 1639 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 9, No. 3: 1632-1639, 2025 DOI: 10.55214/25768484.v9i3.5646 © 2025 by the authors; licensee Learning Gate [22] H. Puspasari and W. Puspita, "Validity and reliability test of research instruments on students' level of knowledge and attitudes towards selection of health supplements in facing Covid-19," Jurnal Kesehatan, vol. 13, no. 1, pp. 65-71, 2022. https://doi.org/10.26630/jk.v13i1.2814 [23] N. Hadisa, R. Susanti, and Robiyanto, "Validity and reliability test of the indonesian version of B-IPQ in chronic kidney failure patients at Soedarso Hospital Pontianak," Journal of Pharmaceutical Management and Services, vol. 7, no. 4, pp. 175-181, 2017. [24] A. Meivira, N. Dewi, and C. Puspitasari, "Validity and reliability test of the antibiotic use and storage questionnaire in Ampenan District," Archives Pharmacia, vol. 4, no. 1, pp. 10-16, 2022. [25] Sugiyono, Educational research methods: Quantitative, qualitative and R&D approaches. Bandung: CV. Alphabet, 2016. [26] J. Robinson, "Triandis theory of interpersonal behaviour in understanding software privace behaviour in the South African context," Masters Degree, University of the Witwatersrand, 2009. https://doi.org/10.26630/jk.v13i1.2814