Vol 51 No 1 Jan-Mrt 2018.indd 14 The RANKL expression and osteoclast in alveolar bone of rat diabetic model at different mechanical force application Nuzulul Hikmah, Amandia Dewi Permana Shita, and Hafiedz Maulana Department of Biomedics Faculty of Dentistry, Universitas Jember Jember - Indonesia ABSTRACT Background: Diabetes is a serious and important public health problem, especially in relation to dental treatment. Because of its complications in periodontal tissue, diabetes can be contraindicated in patients undergoing orthodontic treatment. The receptor activator of nuclear factor-κb ligand (RANKL) is an essential cytokine inducing osteoclastogenesis. Osteoblasts produce this cytokine which has been suggested to play an integral role in osteoclast activation during bone remodeling of orthodontic tooth movement. Purpose: The aim of this study was to determine the correlation between RANKL expression of osteoblast and the number of osteoclasts in the alveolar bone of diabetic rat models at different mechanical force application. Methods: This study used animal subjects, white rats (Rattus norvegicus) of the Wistar strain (n=24) divided into six groups. The mechanical force to which they were subjected ranged between 10, 20, and 30 gramforce (grf). The animal models with diabetes were injected with a stratified dose of Streptozotocin. An orthodontic appliance was inserted in both the maxillary incisors for seven days. The tissue was subjected to histological analysis of osteoclasts and immunohistochemistry analysis of RANKL expression on the pressure and tension side of the alveolar bone. Results: The results of this study showed that the increase in mechanical force produced a rise in RANKL expression and osteoclast number on the pressure and tension side of the alveolar bone of diabetic rat models. Conclusion: There was a correlation between the RANKL of osteoblast and osteoclast numbers in the alveolar bone of diabetic models with different mechanical force application. Keywords: RANKL expression; osteoclast; rat diabetic model; mechanical force Correspondence: Nuzulul Hikmah, Department of Biomedics, Faculty of Dentistry, Universitas Jember. Jl. Kalimantan no. 37 Jember 68121, Indonesia. E-mail: nuzulul.drg@gmail.com Research Report INTRODUCTION Diabetes is a chronic disease resulting from the inadequate production of insulin, a pancreatic hormone regulating blood glucose levels, or ineffective insulin use.1 Diabetes is a major global medical condition representing a complicating factor in dental treatment, especially the branch of orthodontics. This complicating factor results from elevated blood glucose levels that can induce vital organ failure and tissue damage in the heart, blood vessels, eyes, kidneys, nerves, and periodontal tissues.2,3 For this reason, orthodontics represent a contradictive treatment for diabetics. During orthodontic treatment, the mechanical force applied to the teeth affects periodontal tissue, included alveolar bone remodeling. There are two bone cell activities in alveolar bone remodeling, osteoclast activation leading to bone resorption and osteoblast activation resulting in bone formation.4 Both bone cell activities involve certain cytokines. The receptor activator of nuclear factor-κβ ligand (RANKL) is an essential cytokine produced by osteoblast and constitutes a tumor necrosis factor (TNF)- related ligand which plays a role in the osteoclastogenesis process, including osteoclast formation and activation during orthodontic tooth movement.5 Diabetes can influence orthodontic treatment. Certain studies have argued that diabetes affects the expression of cytokines which play a role in the osteoclastogenesis process. Previous studies using mice models indicated that the mechanical force of a 35-gram orthodontic appliance Dental Journal (Majalah Kedokteran Gigi) 2018 March; 51(1): 14–19 Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 32a/E/KPT/2017. Open access under CC-BY-SA license. Available at http://e-journal.unair.ac.id/index.php/MKG DOI: 10.20473/j.djmkg.v51.i1.p14–19 mailto:nuzulul.drg@gmail.com http://e-journal.unair.ac.id/index.php/MKG http://dx.doi.org/10.20473/j.djmkg.v51.i1.p14-19 15Hikmah, et al./Dent. J. (Majalah Kedokteran Gigi) 2018 March; 51(1): 14–19 increased the expressions of receptor activator of nuclear factor-κb (RANK), RANKL, matrix metalloproteinase-13 (MMP-13) and colony stimulating factor-1 (CSF-1) in the periodontal tissues of diabetic models.6 Furthermore, previous studies described how uncontrolled type 2 diabetes patients with chronic periodontitis expressed higher RANKL and osteoprotegerin (OPG) than both control group patients and healthy individuals.7,8 The aim of this study was to determine the correlation between RANKL expression of osteoblast and the number of osteoclast in alveolar bone of diabetic models under different mechanical force applications. From the application of varying mechanical forces, the optimal force that can be applied during the orthodontic treatment of diabetics could be estimated. MATERIALS AND METHODS This study was approved by the Ethics Committee of the Medical Faculty, Universitas Brawijaya, Malang, East Java, Indonesia. This study used animal subjects, namely; white Wistar strain rats (Rattus norvegicus). The criteria for the animal subjects comprised the following: healthy, 4-month old males, weighing 250-300 grams. The animal subjects (n=24) were divided into six groups (Table 1). The orthodontic appliance, which was inserted into both maxillary incisor teeth of the subjects and used to apply mechanical force was fabricated from stainless steel 0.012U wire (ClassOne Orthodontics, USA), configured into a special shape. The configuration was a coil, 2 mm in diameter, with two 10 mm-long arms connected by a round matrix band 2 mm in diameter (Meba, Germany) at the end of wire arms (Figure 1). The mechanical force applied to the animal models amounted to 10, 20, and 30 grf. The forces were measured using structural modeling design tools (the ANSYS software ver.14) and set by The Richmond Orthodontics Stress and Tension Gauge (Ormco, USA). The animal models were injected intraperitoneally with streptozotocin (Nacalaitesque Inc., Kyoto Japan, code 32238-91). The dose of streptozotocin administered was initially 40 mg/kgBW on the first day, with respective doses of 35, 30, 25 and 20 mg/kgBW being subsequently given up to the fifth day. After receiving an injection of streptozotocin, the diabetic animal subjects were incubated for 14 days during which period the blood glucose levels of samples taken from their tail vein were analyzed. The presence of diabetes was confirmed when the blood glucose levels of the subjects were above 300 mg/dl. Before insertion of the orthodontic appliance, the animal subjects were anesthetized by intraperitoneal administration of 10 mg/kgBW ketamine HCl (Anasject®, Danpac Pharma, Indonesia). The orthodontic appliance was inserted in both the maxillary incisor teeth, perpendicular to the teeth axis A c b a B Figure 1. The orthodontic appliance configured in the special shape (A) and inserted into both maxillary incisor teeth (red arrow) of animal models (B). a) arms; b) matrix bands; c) coil. Table 1. Types and periods of the treatment applied to the groups TimeTreatmentsGroups 14 days-Control groups I (CG I)Control groups 14 daysDiabetesControl groups II (CG II) 7 daysAppliance 30 grfControl groups III (CG III) 21 daysDiabetes + appliance 10 grfExperimental groups I (EG I)Experimental groups 21 daysDiabetes + appliance 20 grfExperimental groups II (EG II) 21 daysDiabetes + appliance 30 grfExperimental groups III (EG III) Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 32a/E/KPT/2017. Open access under CC-BY-SA license. Available at http://e-journal.unair.ac.id/index.php/MKG DOI: 10.20473/j.djmkg.v51.i1.p14–19 http://e-journal.unair.ac.id/index.php/MKG http://dx.doi.org/10.20473/j.djmkg.v51.i1.p14-19 16 Hikmah, et al./Dent. J. (Majalah Kedokteran Gigi) 2018 March; 51(1): 14–19 for seven days. The stabilization of orthodontic appliance was achieved by means of type IX Glass ionomer cement (Fuji IX, GC, Japan). The animal subjects were sacrificed by the administering of an overdose of an anesthetic agent. Thereafter, their incisor teeth in the maxillae were removed and fixed in 10% formalin for 24 hours to serve as tissue samples. After the fixation time had elapsed, the tissue samples were decalcified in 14% ethylenediaminetetraacetic acid (EDTA) for 30 days at room temperature. The tissue samples were subsequently embedded in paraffin. For the osteoclast analysis using a histological method, the tissue sample that had been embedded in paraffin was cut into slices of 5 mm thickness longitudinally and placed on object glass. Thereafter, the sliced samples on object glass were deparaffinized, rehydrated, and stained with hematoxylin-eosin for five minutes at room temperature. The samples were then dehydrated and mounted on a cover glass. The number of osteoclasts in the pressure and tension side of the alveolar bone area was counted using a light microscope at 400x magnification. The counting was completed on 4 slides from five selected areas. For RANKL expression using immunohistochemistry analysis, the tissue samples embedded in paraffin were sliced to 3 mm of thickness longitudinally and placed on poly L-lysine slides (Microscope, USA). After that, the slides were deparaffinized and incubated in 3% H2O2 dissolved in methanol for 15 minutes at room temperature. These procedures were undertaken to inhibit endogen peroxide activity. The slides were then washed with phosphate-buffered saline (PBS) and incubated with background sniper (Starr Trek Universal HRP Detection System, Biocare Medical, USA) for 30 minutes, before being stained with RANKL antibody (N-19) (sc-7628, Santa Cruz Biotechnology, USA; working dilution, 1:100) for 60 minutes at room temperature. After washing in PBS, the sections were incubated with secondary antibody, mouse antigoat IgG-B (sc-53799, Santa Cruz Biotechnology, USA; working dilution, 1:100) for 60 minutes at room temperature. RANKL was stained using an immunohistochemistry staining kit (Starr Trek Universal HRP Detection System, Biocare Medical, USA) following the manufacturer’s instructions. The sections were rinsed with PBS and the final color reactions performed using a 3.3’- diaminobenzidine (DAB) chromogen and buffer (working dilution, 1:200) for 15 minutes at room temperature. The sections were then counterstained with Mayer hematoxylin (working dilution 1:10) for 5 minutes. The RANKL expression was counted using the light microscope at 400x magnification in the pressure and tension side of the alveolar bone. The counting was completed on 4 slides from five selected areas. All of the data were statistically analyzed using the SPSS 20.0 software program (IBM-SPSS Inc., Chicago, USA). The statistical analyses was the Pearson correlation (p<0.05) and regression analysis. RESULTS Immunohistochemistry images of RANKL expression and histological images of osteoclast in the pressure and tension sides of diabetic rat subjects were shown in Figure 2. The average of osteoclast number and RANKL expression was shown in Table 2. From Table 2, it can be seen that the averages demonstrated a similar pattern, tending to be higher in the diabetes group with a higher mechanical force. According to correlation analysis, there was an association between RANKL expression of osteoblast and the osteoclast number in the pressure and tension sides of alveolar bone (p<0.05) (Table 3). Moreover, diabetic models subjected to for free of various mechanical forces Table 2. The number of osteoclasts and RANKL expressions in the pressure and tension sides of subjects’ alveolar bone The study group The number of osteoclasts (mean + standard deviation) RANKL expressions (mean + standard deviation) Tension sidePressure sideTension sidePressure side GG I 0.8 + 0.4 0.6 ± 0.01.0 ± 0.10.7 + 0.4 CG II 1.3 + 0.4 4.6 ± 0.54.9 ± 0.61.2 + 0.4 CG III 1.7 + 0.3 2.3 ± 0.31.9 ± 0.31.2 + 0.3 EG I 4.5 + 0.8 7.5 ± 0.95.6 ± 1.33.9 + 0.6 EG II 5.2 + 0.5 9.7 ± 1.07.3 ± 0.94.1 + 0.4 EG III 6.8 + 1.3 12.9 ± 1.512.6 ± 1.45.5 + 0.9 Table 3. The correlation and regression test between the RANKL expression of osteoblast with the number of osteoclast on the pressure and tension sides of the alveolar bone of a diabetic rat model The study group Pressure side Tension side r R2 Rp value r 2 p value 0.00*Experimental grups 0.835 0.698 0.00* 0.896 0.802 * = The Correlation was significant with p value<0.05; r = The Pearson’s correlation; R2 = The R square Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 32a/E/KPT/2017. Open access under CC-BY-SA license. Available at http://e-journal.unair.ac.id/index.php/MKG DOI: 10.20473/j.djmkg.v51.i1.p14–19 http://e-journal.unair.ac.id/index.php/MKG http://dx.doi.org/10.20473/j.djmkg.v51.i1.p14-19 17Hikmah, et al./Dent. J. (Majalah Kedokteran Gigi) 2018 March; 51(1): 14–19 demonstrated a similar blood glucose level of more than 300 mg/dl for the duration of the experiment. DISCUSSION The correlation between RANKL expression of osteoblast and the osteoclast number in the alveolar bone area of diabetic subjects under different mechanical force application can serve illustrate alterations to alveolar bone remodeling during the orthodontic treatment of diabetics. Osteoblast, osteoclast and RANKL are the major components playing a pivotal role in the periodontal tissue remodeling process, while osteoblast plays a role in the bone formation and osteoclast involved in bone resorption. The RANKL-RANK system regulates alveolar bone and periodontal tissue remodeling during orthodontic treatment.5 When mechanical force was applied to both incisors, there were pressure and tension sides in the periodontal tissue which involved the periodontal ligament and alveolar bone. The pressure side is an area of tooth movement direction located on the distal side of the incisors after mechanical force application. While the tension side is the opposite area to the pressure side located at the mesial side of the incisors after mechanical force application. This study showed that the increase in mechanical force influenced the enhancement of RANKL expression and osteoclast number in the pressure and tension side of the alveolar bone area of the diabetic subject. This study corresponded to a previous one in which diabetes altered alveolar bone turnover through an imbalance in osteoblast/ osteoclast activity and enhancement of pro-inflammatory mediator levels which induce increased bone resorption and tooth movement.6 In diabetes, osteoblast experiences apoptosis easily due to the activity of advanced glycation end products (AGEs) via the mitogen-activated protein (MAP) kinase and cytosolic apoptotic pathways.9 According to statistical analysis, there was a significant correlation between RANKL expression of osteoblast and osteoclast numbers on the pressure and tension side of the alveolar bone area in diabetic subjects. This meant that RANKL expressions of osteoblast and osteoclast were the major factors in increased tooth movement in diabetic subjects undergoing orthodontic treatments. Osteoclast activation and differentiation occurred when RANKL binds to RANK, a key preliminary step in downstream signaling is binding of tumor necrosis factor receptor-associated factors (TRAFs) to specific siter within the cytoplasmic domain of RANK, which is a transmembrane protein. However, not only did TRAFs play a role in the activation and differentiation of osteoclast. At least seven signaling C D AB PDL T T PDL AB AB AB T T PDL PDL A B Figure 2. Light microscopy images showing RANKL expression of osteoblast (black arrow) in the pressure (A) and tension (B) side and osteoclast (red arrow) in the pressure (C) and tension (D) side of the diabetic model with mechanical force application. AB, alveolar bone; PDL, periodontal ligament; T, teeth. A C B D Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 32a/E/KPT/2017. Open access under CC-BY-SA license. Available at http://e-journal.unair.ac.id/index.php/MKG DOI: 10.20473/j.djmkg.v51.i1.p14–19 http://e-journal.unair.ac.id/index.php/MKG http://dx.doi.org/10.20473/j.djmkg.v51.i1.p14-19 18 Hikmah, et al./Dent. J. (Majalah Kedokteran Gigi) 2018 March; 51(1): 14–19 pathways are activated by RANK-mediated protein kinase signaling, four of them directly mediate osteoclastogenesis (inhibitor of nuclear factor κB (NF-κB) kinase/NF-κB, c-Jun amino-terminal kinase/activator protein-1, c-myc, and calcineurin/nuclear factor of activated T cells [NFAT] c1), while three mediate osteoclast activation (src and MKK6/p38/MITF) and survival (src and extracellular signal-regulated kinase).10 Mechanical force also caused periodontal ligament cells to produce prostaglandin E2 (PGE2) and stimulate osteoclastogenesis via RANKL in osteoblast.11 Diabetes induces up-regulation of osteoclastogenic factor which stimulates osteoclast activation and differentiation. Inflammatory mediators such as TNFα and IL-1β incline in diabetes which can exert an influence on osteoblast to express RANKL protein, stimulate osteoclast differentiation and, finally, cause alveolar bone resorption in diabetes.12,13 Both diabetes and mechanical force can induce inflammation which activates T cells to produce inflammatory cytokine (TNFα, IL-1, IL-6) leading to enhancement of RANKL expression on osteoblast and bone marrow. T cells caused bone resorption directly via RANKL expression and indirectly via pro-inflammatory cytokine that mediated RANKL expression on non-T cells.14–16 Recent studies have also shown that blood glucose levels in the diabetes group and diabetes group with different orthodontic force were more than 300 mg/dl, causing hyperglycemia, during the experimental period. Hyperglycemia in diabetes drives the irreversible formation of advanced glycation end products (AGEs) that can have direct pro-inflammatory and pro-oxidant effects on cells. When AGEs bind their signaling receptor, advanced glycation end product (RAGE), cellular phenotype and function are critically impacted and enhance inflammation, oxidative stress, and tissue repair impairment.12,17 AGEs can lead to cellular stress by exerting pro-inflammatory/ oxidant effects directly, or through interaction with cell- surface receptors.12 Hyperglycemia also contributes to enhancement of reactive oxygen species (ROS) levels and a state of oxidative stress, both directly and indirectly through the AGE/RAGE axis, promoting quantitative and qualitative shifts in cytokine profiles.12 ROS stimulates pro-inflammatory cytokine production through activation of intracellular signaling pathways such as MAP kinase, NF-κB and the NALP3 inflammasome.18,19 ROS also have more wide-ranging effects, including those on bone formation and recently revealed pathways involving the interaction of ROS, Wnt signaling and activation of FoxO transcription factors in the regulation of osteoblast activity suggest another novel pathway which may link periodontitis and diabetes.20,21 Previous studies of the relationship between diabetes and periodontal diseases suggested that there were many factors related to the osteoclastogenesis process and reported that there was enhancement of RANKL level in diabetes. The AGE-RAGE axis might contribute to the osteoclastogenesis process. RAGE is involved in osteoclast and in reorganization, adhesion and activation, thereby contributing to reduced bone mass in diabetes. AGEs increased mRNA levels of RAGE and RANKL in osteoblasts, thus further suggesting the active participation of RAGE in osteoclastogenesis.22,23 RANKL/OPG ratio also is modulated by hyperglycemia in diabetes directly and indirectly, resulting in the stimulation of inflammation and tissue destruction.12 From this study, it can be concluded that mechanical force application in a diabetic rat model affects RANKL expression and osteoclast number in the pressure and tension side of alveolar bone area. The increased RANKL expression in osteoblast was caused by the application of increased mechanical force, the effect on the increase in the number of osteoclast in the pressure and tension side of the alveolar bone of diabetic rat models. Based on these results, the application of low mechanical force in orthodontic treatment under diabetic conditions is recommended. REFERENCES 1. World Health Organization. Global report on diabetes. Geneva: WHO Press; 2016. p. 11-9. 2. Leite RS, Marlow NM, Fernandes JK. Oral health and type 2 diabetes. Am J Med Sci. 2013; 345(4): 271–3. 3. Fowler MJ. Microvascular and macrovascular complications of diabetes. Clin Diabetes. 2008; 26(2): 77–82. 4. Hienz SA, Paliwal S, Ivanovski S. Mechanisms of bone resorption in periodontitis. J Immunol Res. 2015; 2015: 1–10. 5. Yamaguchi M. RANK/RANKL/OPG during orthodontic tooth movement. Orthod Craniofac Res. 2009; 12(2): 113–9. 6. Braga SMG, Taddei SR, Andrade I, Queiroz-Junior CM, Garlet GP, Repeke CE, Teixeira MM, da Silva TA. Effect of diabetes on orthodontic tooth movement in a mouse model. Eur J Oral Sci. 2011; 119: 7–14. 7. Santos VR, Lima JA, Gonçalves TED, Bastos MF, Figueiredo LC, Shibli JA, Duarte PM. Receptor activator of nuclear factor-kappa B ligand/osteoprotegerin ratio in sites of chronic periodontitis of subjects with poorly and well-controlled type 2 diabetes. J Periodontol. 2010; 81(10): 1455–65. 8. Ribeiro FV, de Mendonça AC, Santos VR, Bastos MF, Figueiredo LC, Duarte PM. Cytokines and bone-related factors in systemically healthy patients with chronic periodontitis and patients with type 2 diabetes and chronic periodontitis. J Periodontol. 2011; 82(8): 1187–96. 9. Alikhani M, Alikhani Z, Boyd C, MacLellan CM, Raptis M, Liu R, Pischon N, Trackman PC, Gerstenfeld L, Graves DT. Advanced glycation end products stimulate osteoblast apoptosis via the MAP kinase and cytosolic apoptotic pathways. Bone. 2007; 40(2): 345–53. 10. Boyce BF, Xing L. Biology of RANK, RANKL, and osteoprotegerin. Arthritis Res Ther. 2007; 9(Suppl 1): S1. 11. Mayahara K, Yamaguchi A, Takenouchi H, Kariya T, Taguchi H, Shimizu N. Osteoblasts stimulate osteoclastogenesis via RANKL expression more strongly than periodontal ligament cells do in response to PGE2. Arch Oral Biol. 2012; 57(10): 1377–84. 12. Taylor JJ, Preshaw PM, Lalla E. A review of the evidence for pathogenic mechanisms that may link periodontitis and diabetes. J Clin Periodontol. 2013; 40(Suppl 14): S113–34. 13. Cochran DL. Inflammation and bone loss in periodontal disease. J Periodontol. 2008; 79(8 Suppl): 1569–76. Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 32a/E/KPT/2017. Open access under CC-BY-SA license. Available at http://e-journal.unair.ac.id/index.php/MKG DOI: 10.20473/j.djmkg.v51.i1.p14–19 http://e-journal.unair.ac.id/index.php/MKG http://dx.doi.org/10.20473/j.djmkg.v51.i1.p14-19 19Hikmah, et al./Dent. J. (Majalah Kedokteran Gigi) 2018 March; 51(1): 14–19 14. Wu C-C, Sytwu H-K, Lu K-C, Lin Y-F. Role of T cells in type 2 diabetic nephropathy. Exp Diabetes Res. 2011; 2011: 1–9. 15. Yan Y, Liu F, Kou X, Liu D, Yang R, Wang X, Song Y, He D, Gan Y, Zhou Y. T cells are required for orthodontic tooth movement. J Dent Res. 2015; 94(10): 1463–70. 16. Kohli S, Kohli V. Role of R A NK L-R A NK /osteoproteger in molecular complex in bone remodeling and its immunopathologic implications. Indian J Endocrinol Metab. 2011; 15(3): 175–81. 17. Giacco F, Brownlee M. Oxidative stress and diabetic complications. Circ Res. 2010; 107(9): 1058–70. 18. Graves DT, Kayal RA. Diabetic complications and dysregulated innate immunity. Front Biosci. 2008; 13: 1227–39. 19. Martinon F. Signaling by ROS drives inflammasome activation. Eur J Immunol. 2010; 40(3): 616–9. 20. Almeida M, Han L, Ambrogini E, Weinstein RS, Manolagas SC. Glucocorticoids and tumor necrosis factor α increase oxidative stress and suppress Wnt protein signaling in osteoblasts. J Biol Chem. 2011; 286(52): 44326–35. 21. Galli C, Passeri G, Macaluso GM. FoxOs, Wnts and oxidative stress-induced bone loss: new players in the periodontitis arena? J Periodontal Res. 2011; 46(4): 397–406. 22. Yamagishi S. Role of advanced glycation end products (AGEs) in osteoporosis in diabetes. Curr Drug Targets. 2011; 12(14): 2096–102. 23. Yoshida T, Flegler A, Kozlov A, Stern PH. Direct inhibitory and indirect stimulatory effects of RAGE ligand S100 on sRANKL- induced osteoclastogenesis. J Cell Biochem. 2009; 107(5): 917–25. Dental Journal (Majalah Kedokteran Gigi) p-ISSN: 1978-3728; e-ISSN: 2442-9740. Accredited No. 32a/E/KPT/2017. Open access under CC-BY-SA license. Available at http://e-journal.unair.ac.id/index.php/MKG DOI: 10.20473/j.djmkg.v51.i1.p14–19 http://e-journal.unair.ac.id/index.php/MKG http://dx.doi.org/10.20473/j.djmkg.v51.i1.p14-19