Journal of Applied Engineering and Technological Science Vol 4(1) 2022 : 123-130 122 CLASSIFICATION OF MELINJO FRUIT LEVELS USING SKIN COLOR DETECTION WITH RGB AND HSV Dadang Iskandar1, Marjuki2* Sekolah Tinggi Ilmu Komputer Cipta Karya Informatika, East Jakarta, Indonesia1,2 aryatamamarjuki@gmail.com Received : 22 August 2022, Revised: 10 September 2022, Accepted : 10 September 2022 *Corresponding Author ABSTRACT This study aims to detect the ripeness of melinjo fruit using digital image method. Structured identification or division using image processing and computer vision requires the socialization of patterns based on training datasets. Melinjo (Gnetum gnemon L.) is a plant that can grow anywhere, such as yards, gardens, or on the sidelines of residential areas, as a result, produces melinjo into a plant that has relatively large potential to be developed. The process of image processing and pattern socialization is a highly developed research study. Starting based on the process of socializing an object, or a structured division of the object and about detecting the level of fruit maturity. The structured division process regarding ripeness into 3 classes, namely: raw, half-cooked and ripe where the process is carried out using Google Collaboratory which processes the RGB color space to HSV. In this study, the testing method for the system that will be used is a functional test where the test is carried out only by observing the execution results through test data and checking the functionality of the system being developed. The level of accuracy obtained from this study is 98.0% correct. Keywords : Digital Image, Classification Of Ripeness Of Melinjo Fruit, RGB, HSV 1. Introduction Melinjo ( Gnetum gnemon L. ) is a plant that can grow anywhere, such as yards, gardens, or on the sidelines of residential areas, as a result, produces melinjo into a plant that has relatively large potential to be developed. Young melinjo leaves and fruit can be processed into vegetables and old melinjo grains can be processed into standard ingredients for making chips(Siregar, 2014). Emping is a processed product of melinjo which is popular with the public, and has become a potential commodity for the small industrial sector. Emping melinjo is a processed product from melinjo whose manufacturing process is to use a method of flattening old melinjo fruit which was previously roasted (Widiantie et al., 2021). Before harvesting the melinjo fruit, it is very important to pay attention to the level of maturity of the fruit because it is a crucial factor in choosing the quality based on the melinjo fruit (Wardani et al., 2019). If the melinjo fruit is young, it can be prepared as processed vegetables and if the fruit that is harvested is not old enough, even though it is ripe, the quality is not good enough to be used as chips because the taste and aroma are not good and are easily destroyed. On the other hand, if the fruit is harvested too old, the taste and aroma of the fruit is strong, but has a short shelf life (Siregar, 2014). Therefore, the level of maturity of melinjo fruit to be harvested is closely related to the marketing reach and the purpose of using melinjo fruit. To follow up on these problems, this study tries to create a system where this system will be needed later and can be useful as a medium that can help in determining the maturity level of melinjo fruit. This system will detect based on the skin color of the melinjo fruit. Researchers will apply an RGB and HSV method in the melinjo grain maturity detection system to get good accuracy results(Phuangsaijai, et al., 2021; Herng, et al., 2021; Mohsin, et al., 2021; Din & Abdul Nasir, 2021; Kulakova, et al., 2022; Basak, et al., 2022). 2. Research Methods Quantitative research is the basis for selecting research methods, because basically the selection of quantitative methods focuses on the description of the results of a processed photo with the percentage accuracy of the method or process carried out. In this research, it is the Iskandar and Marjuki… Vol 4(1) 2022 : 123-130 123 accuracy in the level of maturity of the melinjo fruit by dividing the accurate amount by the total amount of data tested, while the stages are as follows: Fig 1. Steps of Quantitative Research a. Formulating a problem can be defined as the activity of a question sentence that is prepared based on the existence of the problem and the answer will be sought through data collection in a research process using the 5W + 1H formulation. b. Determine the theoretical basis where the process of seeking information from the theories of experts or previous research which will later become reference material. c. Data collection was carried out as research material to be tested. From 1,003 existing data sets, the data will be divided into two, namely 80% for training data and 20% for testing data. d. Analyzing data from various aspects to be tested using the method prepared as a test method. e. The conclusion of this study is the result of the testing process carried out and is usually written in writing that discusses before, during, and after testing. Research data In this study, the data needed by researchers as research material is melinjo fruit. The type of data used in this study is an image or a photo of melinjo fruit which later the image will be processed. The melinjo fruit, which was previously obtained from the market, are cleaned first to clean the remaining dirt that can interfere with the results of the photo which will be used as research material and then separate one by one from the stems. Test Design The design of the test to be carried out is to create a fruit ripeness classification system using two color spaces, namely the RGB and HSV color spaces, the results of which aim to increase the accuracy of fruit ripeness with the following stages: Making a Problem Formulation Determining the Theory Data collection Analyzing Data and Testing Conclusion Start End Iskandar and Marjuki… Vol 4(1) 2022 : 123-130 124 Fig 2. Test Design a. Designing on Google Collaboratory, this process can be mentioned as the first step in creating a system which contains Python language command code to run it. With the command code that has been prepared in advance, the design is made and tested one by one according to the stages. b. Changing the BGR color space to RGB along with its average value, in this study the image data or images obtained previously captured by the system have B, G, and R colors so that to improve color accuracy in the next process, the image must be converted to RGB. c. Changing the RGB color space to HSV, to reduce the effect of illumination on an image, the image color can be converted to another color space, one of which is the HSV color space. The first step that must be done is to change the RGB value by dividing each value by an RGB scale of 255 as follows: The next step is to convert RGB to HSV by using the following formula: d. After the HSV image is obtained, the next process is to change the HSV image to a black and white image through the masking process . Masking is the process of converting an image into a black and white image. The masking process aims to separate the foreground and background images by utilizing the thresholding process . The thresholding process uses a certain limit value to convert the pixel value in the original image into a binary image, where the foreground pixel value will be 1 and the background pixel value will be 0. e. In this study, the testing method for the system that will be used is a functional test where the test is carried out only by observing the execution results through test data and checking the functionality of the system being developed. Google Collaboratory Design BGR to RGB RGB to HSV Masking Result Recording Start End Iskandar and Marjuki… Vol 4(1) 2022 : 123-130 125 3. Results and Discussions Melinjo Fruit Dataset In this study, the dataset used was 1,003 images with the distribution of 333 images of raw melinjo fruit, 330 images of half-ripe melinjo fruit, and 340 images of ripe melinjo fruit. From 1,003 existing data sets, the data will be divided into two parts, namely 80% for training data and 20% for testing data. Fig 3. A Raw Image, B Half Ripe Image, C Ripe Image Based on Figure 3, it is found that the average RGB values are as follows: Table 1 - Average RGB values of melinjo images Furthermore, the results of the conversion of RGB to HSV images A, B, and C can be seen in the following table: Table 2 - HSV Score of Melinjo Image Melinjo Maturity Detection Test The stages of the melinjo fruit maturity detection test are carried out one by one and are divided into three processes, namely 67 images of the raw category, 66 images of the half -ripe category, and 68 images of the ripe category. a. Melinjo Fruit Detection Stages The first stages in Google Collabolatory are as follows: Image Name Color Average Image A Red 179 Green 192 Blue 193 Image B Red 177 Green 191 Blue 197 Image C Red 176 Green 186 Blue 193 Image Name Classification Characteristic Score Image A Raw H 0.112 S -58.7 V 188 Image B Half-cooked H 0.585 S -58.0 V 188 Image C Ripe H 0.985 S -57.7 V 185 Iskandar and Marjuki… Vol 4(1) 2022 : 123-130 126 Fig 4. Inserting the Melinjo A1 Image At the initial stage, the user can press the Choose Files button as shown in Figure 4 to select the image to be tested so that a search menu appears as shown in Figure 5 below: Fig 5. Select Image A1 Based on Figure 5, the user selects the melinjo image whose maturity level will be tested, namely "A1" then selects " Open " so that the image enters the Google Collabolatory as shown in Figure 6 below: Fig 6. Display BGR image A1 After appearing in Google Collabolatory as shown above, the user then processes the change from BGR to RGB so that it looks like Figure 7 below: Iskandar and Marjuki… Vol 4(1) 2022 : 123-130 127 Fig 7. BGR to RGB A1 image Furthermore, the RGB image is converted back into HSV form as shown in Figure 8 below: Fig 8. RGB to HSV image A1 The next process is to change the HSV image to a black and white image through the masking process. The masking process aims to separate the foreground and background images by utilizing the thresholding process as shown in Figure 9 below: Fig 9. Masking the A1 Citra Image After the masking process as shown in Figure 4.9, the next step is to detect the ripeness of the melinjo fruit by pressing the next button. Iskandar and Marjuki… Vol 4(1) 2022 : 123-130 128 Fig 10. Detection of Maturity Level of A1 Image Based on Figure 10 above, the process of detecting the level of maturity of the melinjo fruit was successfully detected with the results of "Raw Fruit" so that the results were correct. b. Melinjo Fruit Detection Results Based on the results of the melinjo maturity level detection test with the process carried out as many as 201 images consisting of 3 types of maturity levels, namely raw, half-cooked, and ripe, the results were as follows: Table 3 - Results of Testing Images A, B, and C From the table above, the results show that there are 197 images with true detection and 4 images with false detection. The next step is to calculate the level of accuracy based on the number of melinjo images tested, namely 201 with the formula as follows: No Image Name Maturity Level Detection Results Conclusion 1 A1.jpeg Raw Raw Correct 2 A2.jpeg Raw Raw Correct 3 A3.jpeg Raw Raw Correct ===================================================== 68 B1.jpeg Half-cooked Half-cooked Correct 69 B2.jpeg Half-cooked Half-cooked Correct 74 B7.jpeg Half-cooked Half-cooked Correct 75 B8.jpeg Half-cooked Half-cooked Correct ===================================================== 113 B46.jpeg Half-cooked Ripe Wrong ===================================================== 119 B52.jpeg Half-cooked Ripe Wrong ===================================================== 122 B55.jpeg Half-cooked Ripe Wrong ===================================================== 128 B61.jpeg Half-cooked Ripe Wrong ===================================================== 195 C62.jpeg Ripe Ripe Correct 199 C66.jpeg Ripe Ripe Correct 200 C67.jpeg Ripe Ripe Correct 201 C68.jpeg Ripe Ripe Correct Iskandar and Marjuki… Vol 4(1) 2022 : 123-130 129 Based on the results of the accuracy test, an accuracy value of 98.0% was obtained for the detection process of melinjo fruit maturity as much as 201 test data. 5. Conclusion Based on the results of tests carried out by detecting the ripeness of melinjo fruit, the following conclusions can be drawn: Detection of ripeness of melinjo fruit with RGB and HSV color space extraction method can be performed on images of ripe, half-ripe and unripe melinjo fruit. The results of the detection of the maturity level of the mangosteen using the RGB and HSV color space methods give an accuracy of 98.0% with 201 melinjo image data, it can be said that the accuracy level in this method is quite high. Based on the test results, the detection accuracy will certainly change with the increasing number of data being tested. For the advancement of a work system that excels in additional exploration, the authors can provide the following suggestions: To provide a better level of accuracy in detecting fruit ripeness, it is necessary to compare the RGB and HSV extraction methods with other extraction methods and for a better system development, it is necessary to add a test menu with the selection of melinjo fruit image folders simultaneously detection, to shorten the testing time if there are enough test images. References Ayllon, MA, Mendoza, JJ, & Tomas, MC (nd). Detection of ripeness of whole local fruit using Convolutional Neural Networks Through Image Processing Machine Translated by Google . 145–148. Basak, J. K., Madhavi, B. G. K., Paudel, B., Kim, N. E., & Kim, H. T. (2022). Prediction of Total Soluble Solids and pH of Strawberry Fruits Using RGB, HSV and HSL Colour Spaces and Machine Learning Models. Foods, 11(14), 2086. Cruz, J. C. D., Garcia, R. G., Dimaunahan, E. D., Labaclado, J. J., Reyes, G. A. B., Riomero, H. M. C. P., & Salamatin, P. M. (2019). Eczema, Hives and Psoriasis Detection with the Application of Local Binary Pattern, Color Histogram, SVM and RGB-HSV Color Space. In Proceedings of the 2019 9th International Conference on Biomedical Engineering and Technology (pp. 160-165). Din, A. F., & Abdul Nasir, A. S. (2021). Automated cells counting for leukaemia and malaria detection based on RGB and HSV colour spaces analysis. In Proceedings of the 11th National Technical Seminar on Unmanned System Technology 2019 (pp. 981-996). Springer, Singapore. Liantoni, F., & Annisa, FN (2018). Fuzzy K-Nearest Neighbor on Chili Maturity Classification Based on Hsv Image Features. JIPI (Scientific Journal of Informatics Research and Learning) , 3 (2), 101–108. https://doi.org/10.29100/jipi.v3i2.851 Herng, O. W., Nasir, A. S. A., Chin, O. B., & Tan, E. S. M. M. (2021, November). Harumanis Mango Leaves Image Segmentation on RGB and HSV Colour Spaces using Fast k-Means Clustering. In Journal of Physics: Conference Series (Vol. 2107, No. 1, p. 012068). IOP Publishing. Kulakova, A. D., Galkin, V., & Makarenko, A. V. (2022). Methoarial; analysis of methods of colour calibration of images using rgb ang hsv colour spaces in tasks of intelligent computer vision (for example, images ob-tained in industrial greenhouses. Upravlenie Bol'shimi Sistemami, 97, 87-107. Mohsin, R. H., Daway, H. G., & Rashid, H. G. (2021). Automatic Detection of Smoke in Videos Relying on Features Analysis Using RGB and HSV Colour Spaces. In Data Science and Security (pp. 182-190). Springer, Singapore. Mujahidin, S. (2015). Skin Color Classification based on the RGB Color Space. National Seminar on Information Technology Applications (SNATI) , 1(1), 17–19. http://journal.uii.ac.id/index.php/Snati/article/view/3530 Pambudi, SW, & Sustiono. (2015). Design and Build of Dragon Fruit Ripening Quality Selection Tool Using Image Processing Technique with HSV Image Segmentation Method. Journal of Science And Informatics , 1(2), 28–37. Iskandar and Marjuki… Vol 4(1) 2022 : 123-130 130 Phuangsaijai, N., Jakmunee, J., & Kittiwachana, S. (2021). Investigation into the predictive performance of colorimetric sensor strips using RGB, CMYK, HSV, and CIELAB coupled with various data preprocessing methods: A case study on an analysis of water quality parameters. Journal of Analytical Science and Technology, 12(1), 1-16. Sanusi, H., S., SH, & Susetianingtias, DT (2019). Making Leaf Image Classification Applications Using Rgb And Hsv Color Spaces. Scientific Journal of Computer Informatics , 24 (3), 180–190. https://doi.org/10.35760/ik.2019.v24i3.2323 Siregar, YDI (2014). Utilization of Red Melinjo Skin Extract (Gnetum gnemon) as a Natural Colorant for Lipstick Making. Journal of VALENCY Chemistry , 1(4), 98–108. https://doi.org/10.15408/jkv.v0i0.3607 Wardani, VR, Fatimah, S., Nadia, & Cahyani, IM (2019). Melinjo peel ethanol extract activity ( Indonesian Pharmaceutical Media , 14 (1), 1466–1470. Whidhiasih, RN, Wahanani, NA, & Supriyanto. (2013). Classification of Starfruit Based on RED- GREEN-BLUE Imagery. Journal of Computer Science Research, Systems Embedded & Logic , 1(1), 29–35. Wibowo, JS (2011). Detection and Classification of Image Based on Skin Color Using HSV. Journal of Information Technology DYNAMIC. 16 (2), 118–123. Widiantie, R., Setiawati, I., & Handayani. (2021). Utilization of melinjo and melinjo skin is an innovative product in improving the economy of the Sumbakeling village community . 58– 62. Yogi, M. (2016). Application of watermelon ripeness detection based on RGB value using thresholding method . JURIKOM: Journal of Computer Research , 3(6), 84–89.