FACTA UNIVERSITATIS Series: Mechanical Engineering Vol. 16, N o 3, 2018, pp. 297 - 305 https://doi.org/10.22190/FUME170620029S © 2018 by University of Niš, Serbia | Creative Commons License: CC BY-NC-ND Original scientific paper 3D DIGITIZATION OF FEATURELESS DENTAL MODELS USING CLOSE RANGE PHOTOGRAMMETRY AIDED BY NOISE BASED PATTERNS UDC 771:778 Ţeljko Santoši 1 , Igor Budak 1 , Mario Šokac 1 , Tatjana Puškar 2 , Đorđe Vukelić 1 , Branka Trifković 3 1 University of Novi Sad, Faculty of Technical Sciences, Serbia 2 University of Novi Sad, Faculty of Medicine, Serbia 3 University of Belgrade, School of Dental Medicine, Serbia Abstract. Development and improvement of 3D digitizing systems provide for the ability to digitize a growing number of materials and geometrical forms of greater complexity. This paper presents the application of 3D digitizing system using close range photogrammetry on the upper jaw cast in plaster in order to obtain its 3D model. Because of the low visual characteristics of gypsum, such as color and texture, many questions arise about the possibility of applying this particular method to this type of physical models. In order to overcome bad visual properties of gypsum, this paper analyzes the possibility of the photogrammetry method application supported by the projected light texture which is based on patterns in the form of noise-obtained mathematically modeled functions. In order to determine the selected image for light texture which gives the better results, an experiment was designed and carried out. Only two images were tested. One image is selected based on previous research and the other one was generated by the Matlab function for uniformly distributed random numbers. For validation and a comparative analysis of the results, an object of 3D digitization was generated with and without projected light texture. CAD inspection was applied for the analysis of the obtained 3D digitizing results. 3D model obtained by approved professional optical 3D scanner as a reference was used. The results in this paper confirm better accuracy of 3D models obtained with the use of light textures, but this approach requires additional hardware and setup adjustment for images acquisition. Key Words: 3D Digitization, Close Range Photogrammetry, Noise-based Patterns, CAD inspection Received June 20, 2017 / Accepted July 11, 2018 Corresponding author: Igor Budak University of Novi Sad, Faculty of Technical Sciences, Trg Dositeja Obradovića 6, 21000 Novi Sad, Serbia E-mail: budaki@uns.ac.rs 298 Ţ. SANTOŠI, I. BUDAK, M. ŠOKAC, T. PUŠKAR, Đ. VUKELIĆ, B. TRIFKOVIĆ 1. INTRODUCTION 3D technologies, new approaches and applications of innovative methods in the context of Reverse Engineering (RE) and 3D digitizing, lead away mechanical engineering from the conventional product design and toward its increasing search for another role and importance in other fields such as biomedical engineering, protection of cultural heritage, animation, criminal investigation, forensic science, etc. [1-6]. The dentists need the necessary technical resources and advancements that will facilitate their work with patients, and, therefore, provide their patients with a better treatment and healthcare service [7]. A high level of cooperation is especially required in the fields of dental prosthetics for dentures as well as in oral surgery for bone grafts [8-10]. The main task of all dental restorations is to meet both functional and aesthetic requirements. A conventional approach to making dental restoration usually requires a couple of visits to the dentist that can be stressful for the patient, and the dental restorations are produced with less precision when compared to the modern CAD/CAM manufacturing. Application of RE in dentistry greatly facilitates the modeling and designing process of dental restorations, as well as procedure planning and their successful implementation. In the field of RE several 3D digitizing methods are developed and/or adapted for acquisition of virtual 3D models in dental prosthetics, such as optical 3D digitizing methods based on structured light, laser triangulation or photogrammetric stereovision. In addition to the transmission methods (computed tomography - CT) that are suitable for obtaining a complete 3D model of the human anatomy, optical methods are also developed which, because of their simplicity and accessibility, provide good accuracy during 3D scanning of dental impressions. On the basis of optical 3D digitizing methods, different types of extraoral and intraoral digitizing scanners [10-12] are developed. This paper shows application of the advanced mechanical engineering techniques, from the RE and CAI (Computer-Aided Inspection) fields, to dentistry for the purposes of performing a sophisticated 3D geometrical analysis. Its aim is to test two different images used for projection of light texture and their influence on the 3D digitization results. Section 2 describes materials and methods used in this paper, where subsection 2.1 briefly describes the working principle of close range photogrammetry while subsection 2.2 shows the used images for projection of light texture. In section 3 experimental workflow and used equipment are described. Results and discussion are shown in the Section 4 and so are the conclusions in Section 5. 2. MATERIALS AND METHODS 2.1. Close range photogrammetry Close range photogrammetry has found its place in the group of non-contact optical 3D digitizing methods [4]. It is based on the Structure from Motion (SfM) approach [13, 14], that enables reconstruction of physical 3D models on the basis of 2D digital photos that are taken arbitrarily, so that specific points on the objects surface are visible on at least two common photos. This method is based on the principles of multi-view stereovision (Fig 1.), where the position of object’s characteristic points in the three- dimensional space and the positions of photos are determined simultaneously, with or without previous calibration (auto calibration) [4]. 3D Digitization of Featureless Dental Models Using Close Range Photogrammetry Aided by Noise... 299 Fig. 1 Multi-view stereovision approach, 3D net configuration of rays [4] The main precondition for the application of this method, in addition to the previously mentioned multi-view stereovision approach, is that the surface of the object has a unique and pronounced visual texture. The object’s photos are so captured that the object is covered from all sides. Since all photogrammetric measurements are dimensionless, it is necessary to use reference markers in order to properly scale generated 3D models. Regarding the errors, they can occur during the scaling of the 3D model. Also, errors can occur due to excessive distortions caused by imperfection of optical acquisition system, to an insufficient number of characteristic points, as well as an insufficient or excessive distance between two common images that form a stereopair. The consequence of these errors is incorrect calculation of the characteristic points' positions in the three-dimensional space. With an increasing number of the characteristic points on the object's surface, the captured images carry a lot more information which greatly reduces errors related to calculation of reconstructed photos as well as points to the positions in the three-dimensional space. Also, with a great number of detected points it can easily overcome the difference in the baselines distances between images, and in this way, minimize the error. This method provides flexible and fast image acquisition which enables easy coverage of the whole object. 2.2. Noise based patterns In order to perform 3D digitization of the object that has no characteristic visual texture, the techniques for improvement of visual characteristics of digitized surface must be applied. In practice, two techniques are applied in order to enhance the visual characteristics of the digitized objects surface. The first is based on physical application of paint or powder on the object, while the second includes projection of light textures [15, 16]. Physical application of paint or powder on the objects surface impairs the geometric accuracy to a small degree due to the fact that a thin layer of the filler material is applied to the object. Still, it is cheaper and it does not require the use of any additional hardware. On the other hand, for the projection of light texture, additional hardware in the form of a light projector is needed, but there is no influence on accuracy [4]. The type of light texture that is projected onto a 3D object depends on how well digitization will improve. Noise-based patterns are visual interpretations of mathematical models or functions, 300 Ţ. SANTOŠI, I. BUDAK, M. ŠOKAC, T. PUŠKAR, Đ. VUKELIĆ, B. TRIFKOVIĆ where each calculated value represents the intensity of a given pixel on the image, which is arranged in a m  n array. This paper will discuss the application of two different patterns, which are:  uniform random pattern [17], and,  wavelet pattern [18]. The uniform random pattern is selected because of its greatest histogram uniformity and random distribution of pixels, while the wavelet pattern has given the best results in recent research [15]. Fig. 2 shows the images of the used patterns and their histograms, on which intensity distribution can be seen. In Table 1 calculated statistics such as standard deviation, entropy, pixels minimum and maximum intensity values, pixel median, mode, range and number of colors are given. Fig. 2 Images of the a) used patterns: random pattern (left), wavelet pattern (right); b) their respective histograms Table 1 Pattern images' statistical parameters Pattern Standard deviation Entropy Min value Max value Median Mode Range Number of colors Random 73.9890 7.9623 0 255 127 / 255 255 Wavelet 44.8413 7.4508 3 245 126 81 242 238 In the recent research Koutsoudis et al. [15] conducted an analysis of the group of patterns based on mathematical functions and found that the wavelet pattern gives the best results in the case of a case of 3D digitization of a cultural heritage object. a) b) 3D Digitization of Featureless Dental Models Using Close Range Photogrammetry Aided by Noise... 301 3. 3D DIGITIZATION OF FEATURELESS DENTAL MODELS USING THE CLOSE RANGE PHOTOGRAMMETRY APPROACH As an object for 3D digitizing a working model of the upper jaw cast in plaster is chosen with its dimension 908030mm. Based on visual inspection it is assumed that the visual characteristics of this object are not suitable for applying photogrammetry based on the SfM approach because of its simple visual texture. Hence, it is considered as appropriate for testing of the generated patterns. The experiment was designed by using a video projector that projects a light texture pattern, generated on the basis of mathematical functions, onto the object. The whole setup is adopted according to the available equipment, but also due to the shape and size of the object. Images are captured and used to reconstruct and generate 3D models. On the basis of both projected light texture patterns an independent 3D model is created. CAD inspection is applied in order to determine which 3D model is better in terms of its geometrical accuracy. The reference 3D model of the plaster jaw was earlier obtained using an optical system for 3D scanning Atos II Triple Scan from GOM with scanning resolution from 0.02 to 0.79 mm. This type of system is considered amongst the most used optical measuring devices for 3D scanning, and it is based on a combination of structured light and stereovision. The workflow of the experiment is illustrated in Fig. 3. Wavelet pattern Random pattern Without pattern 3D models generation Nominal 3D model Acquisition CAD inspection Fig. 3 Experimental workflow 302 Ţ. SANTOŠI, I. BUDAK, M. ŠOKAC, T. PUŠKAR, Đ. VUKELIĆ, B. TRIFKOVIĆ The photo acquisition for the realization of this experiment is performed by means of a Canon 1200d DSLR camera equipped with 18-55 mm lens, set at the maximum value of 55 mm. Maximum focal length value provides a better field of view of object to camera distance ratio than minimum focal length. Photos are captured within distance of 50cm (± 5cm), the aperture size is set to F18, and those settings give approximately 5 cm depth of field. The video projector is set up at a distance of 75cm and at an angle of 50° approximately, compared to the plane of the table. In this setting the GSD (ground sample distance) is between 0.5 and 0.6 mm at a resolution of 1024x768 pixels of the video projector, while the GSD of the captured photos ranges between 0.06 and 0.07 mm. Experimental setup is shown on Fig. 4a. a) b) Fig. 4 a) The experimental setup; b) 3D digitization object Because of the specific shape of the plaster jaw (Fig. 4b) it is not possible to obtain a complete 3D model from a single set of photographs. Therefore, four groups with 8 photographs are made. During the acquisition of each group of 8 photos, projector and 3D digitization object are not moved. During that time the DSLR camera acquires stereopair from left and right sides of the projector on two levels. From each camera's position, three photos are captured: one with a random pattern, one with a wavelet pattern and the third one without projected light textured pattern. Once the first three groups are captured, each with 8 photos, the object is rotated by 90° in relation to the projector; thus the capturing process continues using the same procedure. Alignment of the corresponding groups is carried out using 12 equally distanced reference markers printed on an A4 paper sheet, arranged in a circle with diameter of 160 mm, and placed at the center where the object will be digitized. Based on the known distance between the markers, the actual size of the 3D model is determined. After the acquisition of photos, they are processed using Agisoft PhotoScan software based on SfM approach, and all three digitized 3D models of the object are obtained. 3D Digitization of Featureless Dental Models Using Close Range Photogrammetry Aided by Noise... 303 4. RESULTS AND DISCUSSION Using the CAD inspection it is possible to make a comparative analysis between the nominal geometry, which in this case is a 3D model obtained by the Atos II Triple Scan 3D scanner, and 3D models obtained by using photogrammetry. CAD inspection is carried out using GOM Inspect software and results are shown in Fig. 5. Once the models are aligned using the best fit method for alignment, the limit is set to search and display tolerances within ±1mm. Fig. 5 Result of CAD inspection: without pattern (upper left), random pattern (upper right) and wavelet pattern (down) The advantage of the CAD inspection, in relation to other inspection methods, is in its visual interpretation and easiness of perception of critical areas. Based on Fig. 5, and the results shown in Table 2, it can be seen that there is a very small difference in the results achieved using random and wavelet light projected patterns, while the results achieved without the use of any pattern showed the least accurate results. The advantage of the random and wavelet patterns is in unique textures which allow the photo editing software, which is based on the SfM approach, to easily detect characteristic points on the surface of a 3D object. Between the 3D model obtained using the textured light patterns there are small variations in accuracy, while compared to the 3D model generated without a textured light pattern, increasing accuracy is noticed. The reason for the slight variations in the 3D models obtained using the projected pattern can be seen from their histogram in Fig. 1, and the statistics in Table 1. A high quality pattern must have high randomness and disorder. Due to a smaller depth of the projector field 304 Ţ. SANTOŠI, I. BUDAK, M. ŠOKAC, T. PUŠKAR, Đ. VUKELIĆ, B. TRIFKOVIĆ with respect to the object size, when projecting the image, a slight blur occurs in the pattern that increases GSD of the projector. In the case of the pattern with a large number of intensities, as a result a reduced number of detected points is achieved, even though the subject is within the cameras' depth of field. Table 2 Results of CAD inspection 3D model Prealignment best fit standard deviation [mm] Max distance [mm] Min distance [mm] Mean distance [mm] Distance standard deviation [mm] Without pattern +0.096 +4.211 -9.875 -0.049 +0.184 Random pattern +0.081 +4.521 -6.201 -0.036 +0.115 Wavelet pattern +0.074 +4.454 -3.897 -0.038 +0.109 5. CONCLUSIONS Methodology based on the textured light pattern projection upon the object with reduced visual characteristics proved to be successful. Based on the assumptions and results related to the application of close range photogrammetry to the physical model of the plaster cast used in dental prosthetics, several conclusions come up:  gypsum as the material does not belong to the group of preferred materials for 3D scanning using the close range photogrammetry based on SfM due to its simple and monotonous visual texture,  due to their visual characteristics, objects made of plaster are necessary to undergo additional treatments in order to obtain a unique texture that is the key factor for obtaining high-quality 3D models using the close range photogrammetry,  3D digitizing of plaster models with the close range photogrammetry assisted with the textured light patterns, yields in better results compared to 3D digitizing without the textured light pattern. Further research will focus on developing a new pattern designs for application in the close range photogrammetry to the materials with different surfaces and visual qualities. Acknowledgements: This paper presents the results achieved within the Project no. 114-451-2723 / 2016-03 funded by the Province Secretariat for Higher Education and Scientific Research, and within the project TR - 35020, funded by the Ministry of Education, Science and Technological Development of the Republic of Serbia. REFERENCES 1. Stojkovic, M., Veselinovic, M., Vitkovic, N., Marinkovic, D., Trajanovic, M., Arsic, S., Mitkovic, M., 2018, Reverse modeling of human long bones using T-splines - case of tibia, Tehnicki Vjesnik, 25(6), pp. 1753-1760. 2. Sansoni, G., Trebeschi, M., Docchio, F., 2009, State-of-the-art and applications of 3D imaging sensors in industry, cultural heritage, medicine, and criminal investigation, Sensors, 9(1), pp. 568-601. 3. Apuzzo, N., Consulting, H., 2006, Overview of 3D surface digitization technologies in Europe, in: Corner B.D., Li P., Tocheri M. (Eds.), Three-Dimensional Image Capture and Applications VI, Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6056, San Jose (CA), USA. 3D Digitization of Featureless Dental Models Using Close Range Photogrammetry Aided by Noise... 305 4. Luhmann, T., 2010, Close range photogrammetry for industrial applications, ISPRS Journal of Photogrammetry and Remote Sensing, 65(6), pp. 558-569. 5. Santoši, Ţ., Šokac, M., Korolija-Crkvenjakov, D., Kosec, B., Soković, M., Budak I., 2015, Reconstruction of 3D models of cast sculptures using close-range photogrammetry, Metalurgija 54(4), pp. 695-698. 6. Arbace, L., Sonnino, E., Callieri, M., Dellepiane, M., Fabbri, M., Iaccarino Idelson, A., Scopigno, R., 2013, Innovative uses of 3D digital technologies to assist the restoration of a fragmented terracotta statue, Journal of Cultural Heritage, 14(4), pp. 332-345. 7. Belhadj, A., Boudjema, H. 2017, Recent Advances of Mechanical Engineering Applications in Medicine and Biology, Medical Technologies Journal, 1(3), pp. 62-75. 8. Taneva, E., Kusnoto, B., Evans, C., 2015, Chapter 9: 3D Scanning, Imaging, and Printing in Orthodontics, in Bourzgui, F. (ed.), Issues in Contemporary Orthodontics, IntechOpen. 9. Mirković, S., Budak, I., Puškar, T., Tadić, A., Šokac, M., Santoši, Ţ., Đurđević Mirković, T., 2015, Application Of Modern Computer-Aided Technologies In The Production of Individual Bone Graft. A Case Report, Vojnosanitetski pregled, 72(12), pp. 1049–1140. 10. Logozzo, S., Zanetti, E., Franceschini, G., Kilpelä, A., Mäkynen, A., 2014, Recent advances in dental optics – Part I: 3D intraoral scanners for restorative dentistry, Optics and Lasers in Engineering, 54, pp. 203–222. 11. Trifković, B., 2012, Analysis of metrological characteristics of the optical digitization devices in dental CAD/CAM technology, PhD Thesis, University of Belgrade, School of Dental Medicine, Serbia. 12. Budak, I., Trifkovic, B., Puskar, T., Vukelic, Dj., Vucaj-Cirilovic, V., Hodolic, J., Todorovic A., 2013, Comparative analysis of 3D digitization systems in the field of dental prosthetics, Tehnicki Vjesnik, 20(2), pp. 291-296. 13. Micheletti, N., Chandler, J., Lane, S., 2015, Section 2.2.2 Structure from Motion (SfM) Photogrammetry, in: Cook, S.J., Clarke, L.E., Nield, J.M. (Eds.), Geomorphological Techniques, (Online Edition), British Society for Geomorphology; London, UK. 14. Alsadik, B.S.A., 2014, Guided close range photogrammetry for 3D modelling of cultural heritage sites, PhD Thesis, University of Twente, Netherlands. 15. Koutsoudis, A., Ioannakis, G., Vidmar, B., Arnaoutoglou, F., Chamzas, C., 2015, Using noise function- based patterns to enhance photogrammetric 3D reconstruction performance of featurel ess surfaces, Journal of Cultural Heritage, 16(5), pp. 664-670. 16. Galantucci, L.M., Percoco, G., Dal Maso, U., 2008, Coded targets and hybrid grids for photogrammetric 3D digitisation of human faces, Virtual and Physical Prototyping, 3(3), pp. 167-176. 17. Moler, C., 2004, Chapter 9: Random Numbers, in: Numerical Computing with MATLAB, Society for Industrial and Applied Mathematics. 18. Cook, R., Derose, T., 2003, Wavelet Noise, ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH, 24(3), pp. 803-811.