142142 Dental Journal (Majalah Kedokteran Gigi) 2022 September; 55(3): 142–147 Original article INTRODUCTION Oral health problems have a high rate of occurrence in many countries worldwide and affect various aspects of an individual’s life, often causing pain and discomfort.1 According to the Data and Information Center of the Indonesia Ministry (2007 and 2013) and Baseline Health Research (2018), oral health problems in Indonesia continue to increase every year.2 However, most people with oral health problems have minimal knowledge, and are not concerned, about their situations.3 People who should understand and care about dental and oral problems are dental students. Dental students are future dentists responsible for maintaining the oral health and hygiene of the community. Consequently, they are expected to maintain their oral hygiene to be good examples for the community.4 Research on dental students’ oral hygiene levels conducted in several countries showed varying results. A study in Sudan showed a low score on dental hygiene behaviour, and another study in Saudi Arabia showed a high level in oral health attitudes, but this was not reflected in the oral hygiene and gingival statuses. Research conducted on Trisakti University dental students using the HU-DBI (Hiroshima University Dental Behavioural Inventory) questionnaire, which describes oral health perception and behaviour, showed good results. Thus, further studies using intraoral assessments are needed to confirm the findings of the prior studies.5–8 Oral hygiene assessments are usually carried out in dental practices with various oral hygiene indexes, but these assessments are not possible to perform during the COVID-19 pandemic.9 The COVID-19 pandemic began in early 2020 and caused a lockdown condition that impacted dental practices, mainly routine dental check-ups.10 Research conducted on the Brazilian population during the pandemic reported that oral hygiene behaviour decreased, and the anxiety level associated with visiting dental clinics increased because of the risk of virus transmission, meaning that oral hygiene could not be monitored appropriately, and eventually, this Oral hygiene assessment of dental students using the Oral Rating Index (ORI) Tirza Oktarina Setiabudi1, Fajar Hamonangan Nasution2 1Dental Student, Faculty of Dentistry, Trisakti University, Jakarta, Indonesia 2Department of Orthodontics, Faculty of Dentistry, Trisakti University, Jakarta, Indonesia ABSTRACT Background: Oral hygiene screening should be done on a regular basis, notably during the COVID-19 outbreak, during which lifestyle changes and government lockdown policies lower the oral hygiene level. The Oral Rating Index (ORI), established by Kawamura, is the suitable oral hygiene screening index during the COVID-19 pandemic. Dental students are supposed to be role models for the community by maintaining good dental hygiene. Purpose: To study the use of online oral hygiene screening using the ORI and to determine the oral hygiene of dentistry students at Trisakti University. Methods: This research is a cross-sectional descriptive observational study. A total of 100 preclinical dental students from Trisakti University took part in the study. Intraoral photos were taken and sent to the researcher via the internet. The ORI was used to evaluate the data. The reliability of the results was determined using a per cent agreement test and Cohen’s kappa coefficient. Results: The average oral hygiene score of dental students at Trisakti University was 0.58±0.88. The per cent agreement was 88%, and Cohen’s kappa coefficient was κ = 0.79, indicating excellent reliability. Conclusion: Based on their ORI score, it can be stated that dentistry students at Trisakti University have good oral hygiene and that online evaluation using the ORI is a useful tool for routine oral hygiene screening. Keywords: dental students; Kawamura; oral hygiene; Oral Rating Index (ORI) Correspondence: Fajar Hamonangan Nasution, Department of Orthodontics, Faculty of Dentistry, Trisakti University. Jl. Kyai Tapa, No. 260, Jakarta, 11440, Indonesia. Email: fajar@yahoo.co.jp Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 158/E/KPT/2021. Open access under CC-BY-SA license. Available at https://e-journal.unair.ac.id/MKG/index DOI: 10.20473/j.djmkg.v55.i3.p142–147 mailto:fajar@yahoo.co.jp https://e-journal.unair.ac.id/MKG/index https://doi.org/10.20473/j.djmkg.v55.i3.p142-147 143 Setiabudi and Nasution/Dent. J. (Majalah Kedokteran Gigi) 2022 September; 55(3): 142–147 led to dental and oral health problems.11,12 Particularly in dental students, the switch in activity to online learning may affect their oral health; however, there is no data yet about this. Dental health during the pandemic can be monitored by using an oral hygiene screening method that can be applied under lockdown conditions. Oral hygiene screening methods that are suitable during the pandemic are assessments that do not require face-to-face contact between dentists and patients and can be done online while still providing reliable results.13 An example of an oral hygiene assessment that can be done during the pandemic is the Oral Rating Index (ORI), which was developed by Kawamura and will be modified in this research by using remote examination through intraoral photos to adapt to the pandemic situation.14,15 The urgency of this research is based on the current conditions of the pandemic, during which oral hygiene cannot be monitored as usual, and there is no data yet regarding dental students’ oral hygiene. Therefore, the researchers are interested in conducting this study, which aims to obtain an overview of oral hygiene among dental students in the Faculty of Dentistry, Trisakti University – the most accessible population for testing the new screening method during the pandemic. This study will contribute to current knowledge as it reflects the community oral hygiene condition and reveals whether we will have good future dentists. Furthermore, the purpose of this research is to assess the reliability of online oral hygiene screening using the ORI to monitor oral health as an alternative reliable option during the pandemic. MATERIALS AND METHODS This research employed a descriptive observational method and a cross-sectional method. This research was conducted online from September to October 2021 and received ethical clearance from the Faculty of Dentistry Ethics Commission at Trisakti University on 23 July 2021, with letter number 471/S1/KEPK/FKG/7/2021. The population in this study comprised preclinical dental students from the Faculty of Dentistry, Trisakti University, Jakarta. The research sample was taken by consecutive sampling. The minimum sample size required was 97, which was obtained based on the Lemeshow formula, but 100 samples were taken to prevent data shortages.16 The inclusion criteria in this study included preclinical students of the Faculty of Dentistry, Trisakti University, who were willing to be research subjects and use the same brand of smartphone to take the intraoral photos according to the instructions given. The exclusion criteria for this study were those using fixed orthodontic appliances and students who had performed scaling after March 2020. Preclinical dental students were chosen because they had the same education and knowledge background, they were of a similar age range and they were the most accessible population on which to conduct the research during the pandemic. The variables in this study were the dental and oral hygiene of preclinical students who were assessed based on the ORI. The ORI is an index of dental and oral hygiene assessment, which is based on gingival condition and plaque and calculus accumulation by visual examination as mentioned and established by Kawamura. The area examined included the labial surfaces of the upper and lower anterior teeth and the lingual surfaces of the upper and lower right posterior teeth. The ORI rating uses an ordinal scale from +2 to –2. A score of +2 (very good) was assigned if the gingivae were healthy and no plaque and calculus were detected. A score of +1 (good) was given if there was a slight gingival inflammation but oral hygiene was generally good. A score of 0 (questionable) was given if the researcher found it difficult to determine a positive or negative score. A score of –1 (poor) was assigned when the gingival inflammation was visible and there was a lot of plaque and calculus. A score of –2 (very poor) was given if the gingival inflammation was severe and the oral hygiene was very bad.14 The data collection was carried out online through Google Forms, which included examples and instructions for taking intraoral photos. There were four intraoral photos that should have been taken: the labial surface of the anterior upper teeth and lower teeth and the lingual surface of the posterior upper and lower right teeth. The instructions for taking intraoral photos included using the same brand of smartphone with the specification of a 12-megapixel rear camera and activating the camera flash. The distance should have been adjusted accordingly so that the result would be focused and not blurry. The angle and the intraoral photo results should have followed the example from the researchers (Figure 1). The data were collected and assessed based on the parameters. By use of Microsoft Excel, the mean score of the data was measured and analysed based on the gender group and the academic year’s group. The differences among the gender groups and the academic year groups were analysed using the Mann–Whitney U test and the Kruskal–Wallis test, respectively, as well as Statistical Package for the Social Sciences (SPSS) Statistics 25 software. The intra-rater reliability test was carried out by one researcher who reassessed the data within two weeks of the first assessment by blinding the respondent's identity to prove that the assessment carried out had reliable results. Reliability analysis was carried out using Cohen’s kappa coefficients and SPSS Statistics 25 software. The results of the Cohen’s kappa assessment can be categorised as poor agreement (κ < 0.40), good agreement (0.40 < κ < 0.75) or excellent agreement (κ > 0.75). The data obtained in this study were then calculated using Microsoft Excel software to find the mean of oral hygiene and per cent agreement. The results of per cent agreement can be categorised as none (0%–4%), minimal (4%–15%), weak (15%–35%), moderate (35%–63%), strong (64%–81%) and almost perfect (82%–100%). Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 158/E/KPT/2021. Open access under CC-BY-SA license. Available at https://e-journal.unair.ac.id/MKG/index DOI: 10.20473/j.djmkg.v55.i3.p142–147 https://e-journal.unair.ac.id/MKG/index https://doi.org/10.20473/j.djmkg.v55.i3.p142-147 144Setiabudi and Nasution/Dent. J. (Majalah Kedokteran Gigi) 2022 September; 55(3): 142–147 Figure 1. Example for taking intraoral photos and the desired results. +2 (very good) Normal gingival condition and no detectable plaque or calculus +1 (good) Normal gingival condition; small amount of plaque and calculus accumulation detected; fairly good oral hygiene 0 (questionable) Localised gingival inflammation but small amount of plaque and calculus accumulation detected; questionable appearance –1 (poor) Gingival inflammation; notable amount of plaque and calculus accumulation detected; poor oral hygiene –2 (very poor) Gingival inflammation; large amount of plaque and calculus accumulation detected; very poor oral hygiene Figure 2. Examples of data collected and Oral Rating Index (ORI) assessment, in order from top to bottom: +2 to –2. Table 1. Respondents’ characteristics distribution (n = 100) Year Male Female Total First 6 21 27 Second 13 21 34 Third 3 15 18 Fourth 2 19 21 Table 2. Results distribution of Oral Rating Index (ORI) examination based on gender (n = 100) Gender ORI score (+2) (+1) (0) (–1) (–2) Male 2 7 10 2 3 Female 5 53 11 6 1 (n = 100, Mann–Whitney Test = 0.003 (p < 0.05)) Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 158/E/KPT/2021. Open access under CC-BY-SA license. Available at https://e-journal.unair.ac.id/MKG/index DOI: 10.20473/j.djmkg.v55.i3.p142–147 https://e-journal.unair.ac.id/MKG/index https://doi.org/10.20473/j.djmkg.v55.i3.p142-147 145 Setiabudi and Nasution/Dent. J. (Majalah Kedokteran Gigi) 2022 September; 55(3): 142–147 RESULTS Data were collected online via Google Forms from 100 preclinical dental students attending Trisakti University. Of the 100 respondents, 76 were female and 24 were male; they were distributed from first year to fourth year (Table 1). The collected intraoral photos were then compiled and assessed for each ORI score based on the gingival condition and the plaque and calculus that were visible in the intraoral photos. The acceptable intraoral photos were those that showed the examination parameters clearly. The researcher marked the inflammation signs of the gingivae and the accumulation of plaque and calculus (Figure 2). The majority of the students (60%) got a +1 score, which indicates good oral hygiene and good gingival healthcare levels, while only 4% of the total respondents registered the lowest score of –2 (very poor) (Table 2). The data were then analysed using Microsoft Excel, and the average ORI score was 0.58 ± 0.88, which could be categorised as good oral hygiene. The average ORI score can be reviewed based on gender; the oral hygiene in female dental students was 0.72 ± 0.76, while in the male students, it was lower, at 0.13 ± 1.12. There is a significant difference between the oral hygiene scores of males and females (p < 0.05). In addition to the gender distribution, the results of the ORI examinations can be assessed by year of study. The average ORI score of third-year students is the highest, at 0.72 ± 0.57, followed by first-year students at 0.55 ± 0.89, second-year students at 0.55 ± 1.49, and lastly, fourth-year students, whose average was 0.52 ± 0.87 (Table 3). There is no significant difference between oral hygiene scores among students in different academic years (p < 0.05). The intra-rater reliability test was then carried out by conducting a second examination using the same intraoral photos from respondents with a blinding method, which uses code numbers to replace respondents’ initials and randomises the examination order. The results of the first and second examinations were combined in a cross- tabulation, which was then processed with the per cent agreement using Microsoft Excel and Cohen’s kappa coefficient through SPSS Statistics (Table 4). In this research, the per cent agreement was 88%, which can be deemed almost perfect reliability. In this study, the Cohen’s kappa results obtained were κ = 0.79, with a significance value of 0.00 (0.00 < 0.05), which means the reliability was excellent. DISCUSSION Oral hygiene screening using the ORI on preclinical dental students was conducted online through intraoral photos. Instructions were given for standardising the photo results for the ORI examination as the respondents took photos of their oral cavity by themselves. Hitherto, there had been no available reference for the standardisation of intraoral photos for ORI assessment. However, the overall results of the intraoral photos obtained showed the parameters needed for the assessment, so they were considered adequate for ORI examination. Therefore, this method can be an alternative for conducting similar research remotely during the pandemic or after the pandemic ends, as long as the participants are able to take the photos according to the instructions. The average score in the oral hygiene of dental students at the Faculty of Dentistry at Trisakti University was considered good. The research results aligned with the dental and oral hygiene behaviour assessment using the Hiroshima University Dental Behavioural Inventory (HU-DBI) questionnaire, previously conducted in 2020 with similar respondents.6 The ORI and the HU-DBI are complementary tools; data from the two studies conducted at Trisakti University showed a direct, unidirectional relationship between the ORI and the HU-DBI. Specifically, the higher the HU-DBI score, the higher the ORI score. Table 4. Intra-rater agreement for Oral Rating Index examination First examination Second examination Total Very good (+2) Good (+1) Questionable (0) Poor (–1) Very poor (–2) +2 7 0 0 0 0 7 +1 1 57 2 0 0 60 0 0 4 13 4 0 21 –1 0 1 0 7 0 8 –2 0 0 0 0 4 4 Total 8 62 15 11 4 100 (n = 100, % agreement = 88%, κ = 0.79 (SD = 0.05)) Table 3. Results distribution of Oral Rating Index examination based on year of study (n = 100) Year ORI score (+2) (+1) (0) (–1) (–2) First 2 15 7 2 1 Second 4 18 8 1 3 Third 0 14 3 1 0 Fourth 1 14 3 4 0 (n = 100, Kruskal–Wallis Test = 0.928 (p < 0.05)) Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 158/E/KPT/2021. Open access under CC-BY-SA license. Available at https://e-journal.unair.ac.id/MKG/index DOI: 10.20473/j.djmkg.v55.i3.p142–147 https://e-journal.unair.ac.id/MKG/index https://doi.org/10.20473/j.djmkg.v55.i3.p142-147 146Setiabudi and Nasution/Dent. J. (Majalah Kedokteran Gigi) 2022 September; 55(3): 142–147 Good oral and dental hygiene can also be found in dental faculty students in Korea who were examined with the ORI index and in dental faculty students in India and Romania, where other intraoral clinical examinations, such as the gingival index, the plaque index and the Oral Hygiene Index were used.17–19 Dental students at the Faculty of Dentistry at Trisakti University have been learning subjects related to oral health and hygiene since the first year; hence, good oral hygiene may indicate that the students have implemented dental and oral hygiene knowledge, yet further analysis and assessment are required to confirm this conclusion. The ORI examination was given according to the condition of the gingivae and the plaque and calculus accumulation on a scale of +2 (very good) to –2 (very poor). The higher the ORI value, the better the level of dental and oral hygiene. The score of +2 (very good) was expected to appear the most in the respondents of this research as they are dental students. However, from the results data, only a few respondents (7%) who met the assessment criteria were very good, while the majority of respondents (60%) got a score of +1 (good). Healthy gingival conditions and the presence of minimal or even no calculus during the pandemic are signs that oral hygiene behaviour has been applied routinely and adequately considering that calculus is formed from mineralised plaque, which can only be prevented by daily plaque control, and if calculus has formed, it can be cleaned only by scaling at the dental clinic.20 Scores of –1 (poor) and –2 (very poor) were still found in a small proportion of respondents (12%). Although only slight, these results were worrying because, as prospective dentists who should provide awareness about the importance of maintaining oral and dental hygiene, these respondents had not yet taken care of their oral hygiene properly. Research conducted on students of the dental faculty in Denpasar also showed that there was still a small proportion of students from the faculty who had poor levels of oral and dental hygiene.21 A score of 0 (questionable) is a condition in which the researcher finds it difficult to determine a positive or negative value in the ORI assessment. In this study, a score of 0 was given to respondents who had asynchronous conditions, such as gingivae that looked healthy alongside quite a lot of plaque and calculus accumulation (or vice versa) seen on the photo. Although online assessment of the ORI using photographs is recommended, there are still limitations that cannot be avoided; in some cases, the researchers were hesitant to determine the extent of gingival inflammation and plaque and calculus accumulation. Therefore, it is necessary to conduct another study by face- to-face ORI examination with the respondent to confirm the results. If necessary, another dental and oral hygiene index examination can be added using an instrument to confirm and facilitate the determination of the level of oral hygiene. The results obtained in this research had a refractive factor because the research was carried out during a pandemic and a government-mandated lockdown policy under which conditions were not normal, so the existing results could not be used as baseline data. After the pandemic, oral hygiene examinations can be carried out again to obtain baseline data on the oral hygiene level of dental students and to be used as a basis of comparison between oral hygiene during a pandemic and oral hygiene in everyday situations. ORI assessment can be viewed from the gender perspective, where the average score in this study was higher among women than it was among men and where women got scores of 0.72 ± 0.76 while men got 0.13 ± 1.12. Consequently, it can be stated that oral hygiene among women is better than it is among men. Women have better oral and dental hygiene behaviour than men in terms of brushing teeth, using dental floss and visiting the dentist. Women tend to care more about their bodies and their appearance, and they form habits that support dental and oral hygiene even before getting lessons about dentistry.22 Reliability tests were carried out to prove that the assessments that have been conducted have consistent and reliable results. The reliability test of this research used the intra-rater test-retest method, in which one researcher performed two assessments on the same data so that agreement was obtained between the results of the first and second assessments. The ORI assessment reliability test results in this research were processed with per cent agreement and Cohen’s kappa coefficient. Excellent results were obtained (% agreement = 88%, κ = 0.79), meaning that the ORI examination index has a clear definition and is understood by researchers so that it gives the same results when repeated assessments are carried out.23,24 The limitations of this study are that dental and oral hygiene examinations were conducted remotely. Therefore, it was very dependent on the respondent because the researcher could not observe directly when the respondent took the intraoral photos. The accuracy of the intraoral photos was influenced by many factors, so it was not easy to standardise the results of these photos. Ideally, however, the intraoral photos collected in this research can still be assessed.25 The blinding that was carried out in this study was done by only one researcher, so there could still be an examination bias. Efforts that can be taken to standardise the photos include using the same smartphone brand, giving exact instructions to all respondents and providing examples of intraoral photos and how to take the photos. Above all the shortcomings and limitations, it can be stated that online assessment using the ORI is reliable and safe, and it can be used as an initial dental screening tool and routine oral hygiene examinations, providing an alternative reliable option during the pandemic. From this research, it can be concluded that the oral hygiene level of dental students from the Faculty of Dentistry at Trisakti University is classified as good based on ORI examination (0.58 ± 0.88), and online oral hygiene screening using the ORI can be implemented well. Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 158/E/KPT/2021. Open access under CC-BY-SA license. Available at https://e-journal.unair.ac.id/MKG/index DOI: 10.20473/j.djmkg.v55.i3.p142–147 https://e-journal.unair.ac.id/MKG/index https://doi.org/10.20473/j.djmkg.v55.i3.p142-147 147 Setiabudi and Nasution/Dent. J. (Majalah Kedokteran Gigi) 2022 September; 55(3): 142–147 REFERENCES 1. Baiju RM, Peter E, Varghese NO, Sivaram R. Oral health and quality of life: current concepts. J Clin Diagn Res. 2017; 11(6): ZE21–6. 2. Badan Penelitian dan Pengembangan Kesehatan. Riset kesehatan dasar (RISKESDAS). Jakarta: Kementerian Kesehatan Republik Indonesia; 2018. 3. Soesanto S, Octarina, Kusnoto J. Peningkatan kesadaran warga RT 014/RW 008 JatiPulo, Jakarta Barat mengenai kesehatan gigi pada masa pandemi Covid-19. Akal Abdimas dan Kearifan Lokal. 2021; 2(1): 22–9. 4. Mekhemar M, Ebeid K, Attia S, Dörfer C, Conrad J. Oral health attitudes among preclinical and clinical dental students: A pilot study and self-assessment in an Egyptian State-Funded University. Int J Environ Res Public Health. 2020; 18(1): 1–13. 5. Rahman B, Al Kawas S. The relationship between dental health behavior, oral hygiene and gingival status of dental students in the United Arab Emirates. Eur J Dent. 2013; 7(1): 22–7. 6. Saptari VS. Perilaku kesehatan gigi dan mulut mahasiswa Fakultas Kedokteran Gigi Universitas Trisakti : kajian dengan menggunakan Hiroshima University Dental Behavioral Inventory. Thesis: Jakarta, Universitas Trisakti; 2021. p. 1–35. 7. Kawamura M, Ikeda-Nakaoka Y, Sasahara H. An assessment of oral self-care level among Japanese dental hygiene students and general nursing students using the Hiroshima University--Dental Behavioural Inventory (HU-DBI): surveys in 1990/1999. Eur J Dent Educ. 2000; 4(2): 82–8. 8. Al-Shiekh L, Muhammed ME-D, Muhammed AE-R, El-Huda MA, Hashim NT. Evaluation of dental students’ oral hygiene attitude and behavior using HU-DBI in Sudan. Sci Postprint. 2015; 1(2): 2–6. 9. Marya C. A practical manual of public health dentistry. New Delhi: Jaypee Brothers Medical Publishers; 2012. p. 146–148, 156–160, 231. 10. Sinjari B, Rexhepi I, Santilli M, D Addazio G, Chiacchiaretta P, Di Carlo P, Caputi S. The impact of COVID-19 related lockdown on dental practice in Central Italy-outcomes of a survey. Int J Environ Res Public Health. 2020; 17(16): 1–14. 11. Pinzan-Vercelino C-R-M, Freitas K-M-S, Girão V-M-P, da Silva DO, Peloso R-M, Pinzan A. Does the use of face masks during the COVID-19 pandemic impact on oral hygiene habits, oral conditions, reasons to seek dental care and esthetic concerns? J Clin Exp Dent. 2021; 13(4): e369–75. 12. Peloso RM, Pini NIP, Neto DS, Mori AA, de Oliveira RCG, Valarelli FP, Freitas KMS. How does the quarantine resulting from COVID-19 impact dental appointments and patient anxiety levels? Braz Oral Res. 2020; 34: 1–11. 13. Estai M, Kanagasingam Y, Huang B, Checker H, Steele L, Kruger E, Tennant M. The efficacy of remote screening for dental caries by mid-level dental providers using a mobile teledentistry model. Community Dent Oral Epidemiol. 2016; 44(5): 435–41. 14. Kawamura M, Fukuda S, Inoue C, Sasahara H, Iwamoto Y. The validity and reproducibility of an oral rating index as a measurement of gingival health care and oral hygiene level in adults. J Clin Periodontol. 2000; 27(6): 411–6. 15. Camgoz M, Gurgan CA, Akkaya M. Turkish dental students’ and dentists’ ability to assess gingival health status with DAAGS software. J Dent Educ. 2011; 75(8): 1127–32. 16. Arifin WN. Introduction to sample size calculation. Educ Med J. 2013; 5(2): 89–96. 17. Kim KJ, Komabayashi T, Moon SE, Goo KM, Okada M, Kawamura M. Oral health attitudes/behavior and gingival self-care level of Korean dental hygiene students. J Oral Sci. 2001; 43(1): 49–53. 18. Lalani A, Dasar PL, Sandesh N, Mishra P, Kumar S, Balsaraf S. Assessment of relationship between oral health behavior, oral hygiene and gingival status of dental students. Indian J Dent Res. 2015; 26(6): 592–7. 19. Bobu LI, Saveanu CI, Ciceu O, Balcos C, Armencia A, Murariu A. Association between oral health perceptions and oral health status of dental students in Iasi, Romania. Rom J Med Dent Educ. 2020; 9(5): 84–90. 20. Perry DA, Beemsterboer PL, Essex G. Periodontology for the dental hygienist. 4th ed. Missouri: Elsevier; 2014. p. 54. 21. Astini NWRS, Susanti DNA, Handoko SA. Hubungan antara pengetahuan dan perilaku menjaga kesehatan gigi dengan oral hygiene pada mahasiswa Program Studi Pendidikan Dokter Gigi Fakultas Kedokteran Universitas Udayana. Bali Dent J. 2019; 3(2): 70–3. 22. Mamai-Homata E, Koletsi-Kounari H, Margaritis V. Gender differences in oral health status and behavior of Greek dental students: A meta-analysis of 1981, 2000, and 2010 data. J Int Soc Prev Community Dent. 2016; 6(1): 60–8. 23. Gwet KL. Handbook of inter-rater reliability. 4th ed. Gaithersburg: Advanced Analytics, LLC; 2014. p. 16,74-75,100. 24. McHugh ML. Interrater reliability: the kappa statistic. Biochem medica. 2012; 22(3): 276–82. 25. Ahmad I. Essentials of dental photography. Wiley; 2019. p. 5, 18. Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 158/E/KPT/2021. Open access under CC-BY-SA license. Available at https://e-journal.unair.ac.id/MKG/index DOI: 10.20473/j.djmkg.v55.i3.p142–147 https://e-journal.unair.ac.id/MKG/index https://doi.org/10.20473/j.djmkg.v55.i3.p142-147