Microsoft Word - ???? ????? ????? Al-Khwarizmi Engineering Journal Al-Khwarizmi Engineering Journal, Vol. 17, No. 2, June, (2021) P. P. 1-7 Effect of Environmental Factors on the Accuracy of a Quality Inspection System Based on Transfer Learning Ahmed Najah* Faiz F. Mustafa* Wisam S. Hacham** *,** Department of Automated Manufacturing/ Al-Khwarizmi College of Engineering/ University of Baghdad ***Department of Mechatronics/ Al-Khwarizmi College of Engineering/ University of Baghdad *Email: ahmednajah5049@gmail.com **Email: faizalrawy@yahoo.com ***Email: wisam@kecbu.uobaghdad.edu.iq (Received 5 October 2020; Revised 14 November 2020; Accepted 14 December 2020) https://doi.org/10.22153/kej.2021.12.004 Abstract In this research, a study is introduced on the effect of several environmental factors on the performance of an already constructed quality inspection system, which was designed using a transfer learning approach based on convolutional neural networks. The system comprised two sets of layers, transferred layers set from an already trained model (DenseNet121) and a custom classification layers set. It was designed to discriminate between damaged and undamaged helical gears according to the configuration of the gear regardless to its dimensions, and the model showed good performance discriminating between the two products at ideal conditions of high-resolution images. So, this study aimed at testing the system performance at poor settings of lighting, background, distance and camera resolution. Experimental results implied that the system was able to show high accuracies above 90% at very bad settings and around 99% at good settings, which assures that an inspection system with good performance can be built at low costs. Keywords: Convolutional Neural Networks, DenseNet121, Environmental Factors and Transfer Learning. 1. Introduction Automated quality inspection systems based on computer vision (CV) and deep learning (DL), can push the manufacturing facilities a step forward, with more accurate, reliable and cost-effective architectures. DL is considered as an advanced data analysis technique and this characteristic places it in a position to be one of the building blocks of smart manufacturing [1]. The importance of using DL with CV on the other hand, has arisen due to several qualities possessed by DL, in which it up- scaled traditional CV systems and made it more sophisticated and easier to implement and design. Machine vision with deep learning have been used for: defects detection in products [2], inspection of sugarcane varieties [3], process monitoring and quality control [4], welding inspection [5] optical laser welding inspection [6], laser welding defects detection [7] and gear faults diagnosis with convolutional neural networks (CNNs) [8]. As DL depends on deep neural networks to extract the necessary features from the data automatically, in contrast with traditional CV techniques where manual features extraction is mandatory [9]. Some of the previous researches that relied on traditional CV methods include: inspection of spur gears parameters with the algorithms of CV and image processing [10], measuring gears dimensions with machine vision algorithm [11], detecting defects in welds using CV and machine learning algorithms [12], measuring automobile parts dimensions with machine vision [13], O-rings inspection with CV algorithm [14], aerospace components inspection Ahmed Najah Al-Khwarizmi Engineering Journal, Vol. 17, No. 2, P.P. 1- 7 (2021)     2 using CV [15] and anomaly detection in additive manufacturing with CV [16]. Unlike conventional manual inspection systems, automated platforms have the advantage of being unaffected by psychological factors and fatigue, which normally occurs with human beings [17]. This characteristic improves the production efficiency and eliminates costs, as continuous operations are performed without being affected by those factors. Although, other factors that are not associated with humans emotions like environmental factors, can have major or minor impacts on the system performance depending on the strength of the designed architecture and the training data. So, providing the necessary tools to build a system can help with making it more tolerable to environmental factors. In this paper, a study is conducted regarding the effects of four environmental factors on the performance of a helical gears inspection system that is designed using transfer learning. A number of combined conditions has been used to test and evaluate the system. The remainder of this paper is organized in the following matter, in section two we briefly introduce transfer learning and the built inspection system, in sections three and four; the experimental data and the evaluation results are shown, respectively. Finally, we close with section five and six with the conclusions and future work. 2. Transfer Learning To improve learning within new tasks, knowledge can be transferred from a task that is already learned and related to the new one [18]. That is how transfer learning generally functions. DL and CNNs require a massive amount of information about the target task before being able to perform adequately [19]. But, sometimes there are not enough data resources to cover the necessary amount of training data, which results in inaccurate performance by the model, overfitting (high performance on the training data but low on the validation data) [20] and eventually failing to accomplish the target task. So, using a technique like transfer learning can help with eliminating those problems and reducing the amount of requested information. Giving a remarkable performance with small datasets and can be compared with custom CNNs [21][22]. A previously trained model can be thought of as two sets of tools, one that extracts features and one that classifies the objects according to those features [9]. When transferring layers from one network to another, a new classification layer must be added to classify the new target objects, taking into consideration that the transferred classification layer must be muted. 3. Inspection System Using transfer learning, a model was designed to detect and classify the helical gears according to their shape condition, DenseNet121 model was used as the transferred model as can be seen in Table 1. The model (Fig. 1) was deployed and evaluated on a set of random unseen images with random conditions. Evaluation results yielded high classification accuracy, which proved that using CNNs with CV was superior to classical methods, as it was easier to implement with automatic features extraction and transfer learning. Table 1, Inspection system layers. Stage Layers Transferred 1 (Conv.) Convolution Transferred 2 (Pooling) Max Pooling Transferred 3 (Dense Block 1) Convolution x 6 Convolution Transferred 4 (Transition 1) Convolution Average Pooling T a b l e 1 ( c o n t i n u e d ) Transferred 5 (Dense Block 2) Convolution x 12 Convolution Transferred 6 (Transition 2) Convolution Average Pooling Transferred 7 (Dense Block 3) Convolution x 24 Convolution Transferred 8 (Transition 3) Convolution Average Pooling Transferred 9 (Dense Block 4) Convolution x 16 Convolution Transferred 10 (Classification) Global Average Pooling Fully Connected To be trained (Classification) Global Average Pooling 256D Fully Connected 50 % Dropout 1D Sigmoid Ahmed Najah Al-Khwarizmi Engineering Journal, Vol. 17, No. 2, P.P. 1- 7 (2021)     3 Fig. 1. Inspection model architecture.   4. Methodology Classical analysis methods rely on descriptive type of analysis, which always include hypotheses assembled from the process information patterns, with a comparison between the model results and the genuine results for hypotheses verification [23]. However, models that rely on this method are unsafe, in which some variables might be left unmodeled due to misinterpretation of the issue by the engineers or lack of information [24]. Notwithstanding, predictive analysis, takes a phenomenon, unfold the underlying rules and construct a predictive model that takes all the variables into account and restricts errors between the predicted and the desired outcomes [23]. Since DL models rely on predictive models, CNNs granted an advantage over traditional CV methods as high accuracies for image classification can be guaranteed with automatic feature extraction, where classic CV applications depend on tedious manual features extraction. Nevertheless, every model requires some preprocessing steps before performing the target task. So, to conduct the research and study the effects of the previously mentioned factors, few earlier steps were taken to satisfy the prerequisites of the analysis process, which will be briefly discussed in the following subsections. 4.1 Data Preparation Prior to the evaluation process, imagery data of different gears with and without shape damages had to be collected to be classified into two classes, but due to lack of data resources, only two gears were used as the test samples, in which one was used as the healthy class and the other one as the unhealthy (damaged) class. The images for the damaged one were taken at several stages, in which it started with simple damages (surface damage like scratches were not targeted in this research) and then more impairments were added using an electric cutter, as shown in Fig. 2. Fig. 2. Damaged gear during preparation steps. Two phone cameras (24 and 8 mega-pixels) were used to capture the test images, moreover, a microcomputer called Raspberry Pi 3 was also used to collect images by employing an attached camera board with 5 mega-pixels resolution without using any additional lenses, as shown in Fig. 3. Fig. 3. Raspberry Pi 3 microcomputer (left) and the camera board (right). Ahmed Najah Al-Khwarizmi Engineering Journal, Vol. 17, No. 2, P.P. 1- 7 (2021)     4 Jupyter Notebook, an open-source development environment, provides the tools to develop deep neural networks and perform the necessary processing operations and data manipulation. So, to study the proposed factors, Python’s application programming interface (API) was used within the environment, to build the evaluation code, upload the network and initiate the prediction process. Prior to the test, a function was built to convert the inserted images by modifying the dimensions, to meet the DenseNet121 [25] standard image dimensions of (224 × 224) pixels that it was trained on in the ImageNet challenge [26]. Then by employing the model as the tool that can classify the images, the accuracy was predicted, and according to the cost function that is used in the model, which is binary cross-entropy function, the loss was also estimated. 4. 2. Experimental Work The used model was tested with random environmental factors, so, to make sure that the model is fully functional and can be used in real practical applications and industrial facilities, environmental factors must be taken into consideration. Factors that were studied within this research are: the intensity of light, the type of background, the resolution of the camera and the distance between the camera and the inspected object. Those factors are important to study to select the appropriate settings for the system. As can be seen in Fig. 4, a light-box with three 12Watts LEDs was used to take images with different settings of the factors to point out the model capabilities. Several images (Fig. 5) were captured at different conditions to evaluate the performance of the system. Fig. 4. Used light- box for data gathering.   Fig. 5. Some of the evaluation images. 5. Experimental Results The effect of the distance factor along with other factors on the system performance is shown in Table 2: Table 2, Illustrates the performance of the used system under different conditions. Applied Conditions Accuracy 24 Mega-pixels good lighting 15cm distance 99% 24 Mega-pixels poor lighting 15cm distance 96% 8 Mega-pixels good lighting 15cm distance 99% 8 Mega-pixels poor lighting 15cm distance 92% 5 Mega-pixels good lighting 15cm distance 99% 5 Mega-pixels poor lighting 15cm distance 85% 5 Mega-pixels good lighting 10cm distance 99% 5 Mega-pixels poor lighting 10cm distance 90% As it can be seen in Table 2, the model showed a good performance discriminating between damaged and undamaged gears. Despite the low- resolution cameras that were used like the 5 mega- pixels camera. Different image backgrounds were used during training, consequently; the model Ahmed Najah Al-Khwarizmi Engineering Journal, Vol. 17, No. 2, P.P. 1- 7 (2021)     5 classified the object at a high level of certainty despite the different backgrounds, indicating even lesser effect than the camera resolution to no effect at all on the system classification capability. Image lighting showed some noticeable effect on the system accuracy with low-resolution images, in contrast to the high resolution images, where lighting showed less effect on the classification accuracy. The distance between the object and the camera was the last studied factor, in which the position of the image capturing device with respect to the position of the inspected object, affects the model accuracy to identify and distinguish between the objects. In essence, the model classification accuracy is proportional to the camera resolution, lighting intensity and the distance, with zero effect from any used background. Compared to the lighting, the distance had more effect on the accuracy measures. Terms like good lighting and poor lighting refers to the intensity of the light source illumination and how much the object was obvious to the camera, as the better lighting provided the more certain the model would be. Good lighting (used lighting) was a minimum of 6.6Watt and poor lighting was below that minimum value. Briefly stated, the table shows the system performance at its best at each resolution and at the farthest point. 6. Conclusions In this paper, a transfer learning-based quality inspection system that was designed to inspect helical gears for faults, was evaluated under the effect of several environmental factors. Three cameras with differing resolutions were used to gather the evaluation data, which consisted of ten images at distances of 15cm and 10cm, different backgrounds and poor and good lighting intensities. The system showed good performance adapting to the different conditions, which conveyed the flexibility of the system and the power of CNNs. Also, this study implies the ability to build a quality inspection system at low costs as cheap setups can be sufficient to fulfil the system requirements. 7. Future Work This work can be further expanded by studying the effect of the previously mentioned conditions on other gear sizes, evaluating the system on lower resolutions to test its limits, involving more environmental factors and study the model detection speed when deployed onto low-end devices. Moreover, the same study can be performed on the model when implemented on those devices. 8. References [1] J. Wang, Y. Ma, L. Zhang, R. X. Gao and D. Wu, “Deep learning for smart manufacturing: Methods and applications,” Journal of Manufacturing Systems, vol. 48, pp. 144-156, 2018. [2] J. Wang, P. Fu and R. X. Gao, “Machine vision intelligence for product defect inspection based on deep learning and Hough transform,” Journal of Manufacturing Systems, vol. 51, pp. 52-60, 2019. [3] M. Alencastre-Miranda, R. R. Johnson and H. I. Krebs, “Convolutional Neural Networks and Transfer Learning for Quality Inspection of Different Sugarcane Varieties,” IEEE Transactions on Industrial Informatics, issue 1941-0050, pp. 1-1, 2020. [4] H. Würschinge, M. Mühlbauerl, M. Winter, M. Engelbrecht and N. Hanenkamp, “Implementation and potentials of a machine vision system in a series production using deep learning and low-cost hardware,” Procedia CIRP, vol. 90, pp. 611-616, 2020. [5] Y.J. Cruz, M. Rivas, R. Quiza, G. Beruvides and R. E. Haber, “Computer vision system for welding inspection of liquefied petroleum gas pressure vessels based on combined digital image processing and deep learning techniques,” Sensors, vol. 20, issue 16, pp. 4505, 2020. [6] Y. Yang, L. Pan, J. Ma, R. Yang, Y. Zhu, Y. Yang and L. Zhang, “A high-performance deep learning algorithm for the automated optical inspection of laser welding,” Appl. Sci., vol. 10, issue 3, pp. 933, 2020. [7] Y. Yang, R. Yang, L. Pan, J. Ma, Y. Zhu, T. Diao and L. Zhang, “A lightweight deep learning algorithm for inspection of laser welding defects on safety vent of power battery,” Computers in Industry, vol. 123, issue 0166-3615, pp. 103306, 2020. [8] P. Cao, S. Zhang and J. Tang, “Preprocessing- free gear fault diagnosis using small datasets with deep convolutional neural network-based transfer learning,” IEEE Access, vol. 6, pp. 26241-26253, May 2018 [9] J. Walsh, N. Mahony, S. Campbell, A. Carvalho, L. Krpalkova, G. Velasco- Ahmed Najah Al-Khwarizmi Engineering Journal, Vol. 17, No. 2, P.P. 1- 7 (2021)     6 Hernandez, et al. “Deep learning vs. traditional computer vision,” Computer Vision Conference (CVC), Nevada, 2019, pp. 128-144. [10] E. S. Gadelmawla, “Computer vision algorithms for measurement and inspection of spur gears,” Measurement, vol. 44, issue 9, pp.1669-1678, July 2011. [11] D.K. Moru and D. Borro, “A machine vision algorithm for quality control inspection of gears,” Int J Adv Manuf Technol, vol. 106, pp. 105–123, 2020. [12] J. D. Kothari, “Detecting welding defects in steel plates using machine learning and computer vision algorithms,” IJAREEIE, vol. 7, issue 2278-8875, 2018. [13] M. Y. Sallom, “Machine vision application in manufacturing: inspection of dimensions,” The Iraqi Journal for Mechanical and Material Engineering, vol. 16, issue 3, 2016. [14] G. Peng, Z. Zhang and W. Li, “Computer vision algorithm for measurement and inspection of O-rings,” Measurement, vol. 94, issue 0263-2241, pp. 828-836, 2016. [15] C. Beltr´an-Gonz´alez, M. Bustero and A. Del Bue, “External and internal quality inspection of aerospace components,” IEEE 7th International Workshop on Metrology for AeroSpace (MetroAeroSpace), Pisa, Italy, 2020, pp. 351-355. [16] L. Scime and J. Beuth, “Anomaly detection and classification in a laser powder bed additive manufacturing process using a trained computer vision algorithm,” Additive Manufacturing, vol. 19, issue 2214-8604, pp. 114-126, 2018. [17] E. Asoudegi and Z. Pan, “Computer vision for quality control in automated manufacturing systems,” Computers and Ind. Engng, vol. 21, issue 1-4, pp. 141-145, 1991. [18] P. Wang and R. X. Gao, “Transfer learning for enhanced machine fault diagnosis in manufacturing,” CIRP Annals - Manufacturing Technology, vol. 69, issue 1, pp. 413-416, 2020. [19] C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang and C. Liu, “A survey on deep transfer learning,” International Conference on Artificial Neural Networks – ICANN 2018, vol 11141. Springer, Cham. [20] S. Hou, B. Dong, H. Wang and G. Wu, “Inspection of surface defects on stay cables using a robot and transfer learning,” Automation in Construction, vol. 119, issue 0926-5805, 2020. [21] C.K. Shie, C.H. Chuang, C.N. Chou, M.H. Wu and E.Y. Chang, 2015, August. “Transfer representation learning for medical image analysis,” 37th Annual International Conference of the IEEE, Milan, 2015, pp. 711-714. [22] R. Zhang, H. Tao, L. Wu and Y. Guan, “Transfer learning with neural networks for bearing fault diagnosis in changing working conditions,” IEEE Access, vol. 5, pp.14347- 14357, June 2017. [23] G. Bonaccorso, Machine Learning Algorithms: Popular algorithms for data science and machine learning, 2nd ed. Birmingham: Packt Publishing, 2018. [24] N. Mahony, T. Murphy, K. Panduru, D. Riordan and J. Walsh. “Improving controller performance in a powder blending process using predictive control,” Irish Signals and Systems Conference (ISSC), Killarney, 2017, pp. 1-6. [25] G. Huang, Z. Liu, L. Van Der Maaten and K. Q. Weinberger, “Densely connected convolutional networks,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, 2017, pp. 2261-2269. [26] J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, and L. FeiFei, “Imagenet: A large-scale hierarchical image database,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Florida, 2009, pp. 248- 255.  )2021( 1- 7 صفحة ،2العدد ،17المجلد الهندسية الخوارزمي جلةم احمد نجاح 7 تأثير العوامل البيئية على دقة نظام فحص جودة مبني على التعلم بالنقل ***حجام خضير وسام سعدي** فائز فوزي مصطفى * احمد نجاح | كلية الهندسة الخوارزمي| جامعة بغداد قسم هندسة التصنيع المؤتمت *،** ** قسم هندسة الميكاترونكس| كلية الهندسة الخوارزمي| جامعة بغداد ahmednajah5049@gmail.com :البريد االلكتروني * faizalrawy@yahoo.com**البريد االلكتروني: wisam@kecbu.uobaghdad.edu.iq :البريد االلكتروني*** الخالصة تم إنشاؤه مسبقا، والذي تم تصميمه باستخدام نهج التعلم دراسة حول تأثير عدة عوامل بيئية على أداء نظام فحص الجودة الذي في هذا البحث، تم تقديم ) DenseNet121بالنقل القائم على الشبكات العصبية التالفيفية. يتألف النظام من مجموعتين من الطبقات، مجموعة طبقات منقولة من نموذج مدرب مسبقا ( يميز بين المنتجين في ظروف لفة، وأظهر النموذج أداًء جيداً لتالفة وغير التاومجموعة طبقات تصنيف خاصه. تم تصميمه للتمييز بين التروس الحلزونية ا مثالية لصور عالية الدقة. أظهرت النتائج التجريبية والخلفية والمسافة ودقة الكاميرا. من األضاءة السيئة الظروف في من هذه الدراسة هو اختبار أداء نظام الفحص ان الهدف الرئيسي ٪ في إعدادات جيدة، مما يضمن إمكانية إنشاء نظام فحص ٩٩٪ في إعدادات سيئة للغاية وحوالي ٩٠النظام كان قادًرا على إظهار دقة عالية أعلى من ن بأ بأداء رائع بتكاليف منخفضة.