________________________________________________________________________________ Geoinformatics CTU FCE 2011 89 PHOTO-REALISTIC 3D MODELLING OF SCULPTURES ON OPEN-AIR MUSEUMS Francesca DUCA1, Miriam CABRELLES2, Santiago NAVARRO2, Ana Elena SEGUI2 and José Luis LERMA2 1Università di Ferrara, Dipartimento Risorse Naturali e Culturali Corso Ercole I d'Este, 32, 44100 Ferrara, Italy francesca.duca@gmail.com 2Universitat Politècnica de València (UPV). Photogrammetry & Laser Scanning Research Group (GIFLE). Department of Cartographic Engineering, Geodesy and Photogrammetry. C° de Vera s/n, Building 7i, 46022 Valencia, Spain (micablo, ansegil, sannata, jllerma)@upvnet.upv.es Keywords: terrestrial laser scanning; cultural heritage; documentation; data acquisition; 3D modelling; visualization Abstract: Laser scanning is a high-end technology with possibilities far ahead the well-known civil engineering and industrial applications. The actual geomatic technologies and methodologies for cultural heritage documentation allow the generation of very realistic 3D results used for many scopes like archaeological documentation, digital conservation, 3D repositories, etc. The fast acquisition times of large number of point clouds in 3D opens up the world of capabilities to document and keep alive cultural heritage, moving forward the generation of virtual animated replicas of great value and smooth multimedia dissemination. This paper presents the use of a terrestrial laser sca nning (TLS) as a valuable tool for 3D documentation of large outdoor cultural heritage sculptures such as two of the existing ones inside the “Campus de Vera” of the UPV: “Defensas I” and “Mentoring”. The processing of the TLS data is discussed in detail in order to create photo-realistic digital models. Data acquisition is conducted with a time-of-flight scanner, characterized by its high accuracy, small beam, and ultra -fine scanning. Data processing is performed using Leica Geosystems Cyclone Software for the data registration and 3DReshaper Software for modelling and texturing. High-resolution images after calibration and orientation of an off-the-shelf digital camera are draped onto the models to achieve right appearance in colour and texture. A discussion on the differences found out when modelling sculptures with different deviation errors will be presented. Processing steps such as normal smoothing and vertices recalculation are found appropriate to achieve continuous meshes around the objects. 1. INTRODUCTION In the last decade, according to UNESCO Charter on the Preservation of the Digital Heritage 2003 [1], the resources of human knowledge or expression, whether cultural, educational, scientific and administrative, and other kinds of information are increasingly created digitally or converted into digital form from existing resources. Nowadays great efforts are dedicated to improve digital documentation technology in order to transmit knowledge to our future generations. In fact, world‟s digital heritage is at risk of either being lost or being damaged. Without any doubt, appropriate digital heritage recording techniques are required to measure the state and condition of objects, monuments and sites. There is a wide variety of techniques for three-dimensional measurement. The selection of the right technique should be based on the scale, the size and the complexity of the object [2,3]. Photogrammetry and laser scanning are widely used to provide large number of measurements and are usually suitable for simple and complex objects depending on the approach. Image-based photogrammetry, extracting 2D information from single imagery or 3D information from either stereoscopic plotting or automatic image matching techniques, provide geometric information and texture of the object's surface; range-based laser scanning, from a static position on the ground (terrestrial laser scanner) or from a moving platform such as an aircraft, unmanned aerial vehicle (UAV), or mobile mapping systems are becoming widely used, namely in combination with imagery, and considered as an efficient alternative to traditional survey techniques. Laser scanning is increasingly used to collect a large quantity of three-dimensional data in a short time, generating a point cloud with intensity values in a local coordinate system; additional information such as RGB values is usually provided by internal or external digital cameras. Laser scanning is generally used to record surface information to generate not only 3D models but also 2D sections, profiles and plans. It contributes to improve either the mailto:francesca.duca@gmail.com ________________________________________________________________________________ Geoinformatics CTU FCE 2011 90 geometric study of the monuments “as are”, the rigorous analysis of complex sites, the understanding of complex shapes, and, last but not least, the dissemination on multimedia platforms [4,5]. Recently many approaches suggest the integration of the different techniques to improve the resolution of the 3D model, the accuracy of the objects, the definition of geometries and/or the color enhancement [6,7,8,9]. The accurate recording in 3D of dimensions and shapes is essential in projects related to the restoration and documentation process. It allows users to explore the state of conservation of the structures, monitoring them over time. The complexity of multiple forms of archaeological sites and monuments and his cultural interest requires a high level of geometric detail and colour. Nowadays, the technology is used for many different applications like archaeological site surveys [7,8], digital conservation and creation of digital 3D repositories [10,11], Web geographic systems [12], etc. The integration of image-based and range-based solutions allow the generation of very realistic and accurate results in terms of geometry and texture, enabling to analyze shape and dimensions at high resolution. The 3D survey and modelling of complex objects at different resolutions is realistically possible preferably carrying out multi-resolution approaches, preferable for complex, detailed and large cultural heritage [13,14,15]. This paper presents the results achieved with a time-of-flight terrestrial laser scanner and an external off-the-shelf digital camera for the 3D documentation of two large sculptures placed in the open-air museum “Museo al Aire Libre” inside the “Campus de Vera” of the Universitat Politècnica de València : “Defensas I” - Arcadi Blasco Pastor 2003 - and “Mentoring” - Stephen J. Daly 2003 (Figure 1). The entire area is easily accessible, full of sculptures and trees, acting as a corridor among departments, institutes and facilities throughout the University [16]. “Defensas I” sculpture is like a tower with three sides, measuring approximately 3.5 m high and 2.5 m wide. The grayish surface is not smooth but rich in many intentional holes and incisions. At the top there are sets of sharp pyramids representing the tower‟s battlements. “Mentoring” sculpture is like a big metal doll, approximately 5 m high and 2 m wide, characterized by very complex objects on his upper part, like spheres, rings, spirals and cones. a b c d Figure 1: Sculptures “Defensas I” (a ,b) and “Mentoring” (c, d) 2. DATA ACQUISITION Leica HDS ScanStation2 time-of-flight scanner (Figure 2a), characterized by its high accuracy, small beam diameter and divergence and ultra-fine scanning resolution (up to 1 mm), was used for the survey. The detection range of this device is 300 m at ř0% and 134 m at 1Ř% of albedo, with a field of view of 360° (horizontal) and 270° (vertical) [17]. Artificial targets (Figure 2b-e) were placed in the scene around the sculptures at different heights in order to facilitate the alignment of the point clouds acquired from different positions around the sculptures. a b c d e Figure 2: : a) Leica HDS ScanStation2 time-of-flight; b-e) artificial targets used for the survey ________________________________________________________________________________ Geoinformatics CTU FCE 2011 91 2.1 “Defensas I” Arcadi Blasco Pastor 2003 For this sculpture the point cloud was acquired from two opposite scan positions with a distance to the sculpture of 7.5 m and 4 m, respectively. The resolution of each scanning was set to 5 mm on the object‟s surface. A mean absolute error of 3 mm was achieved within the registration process. A point cloud of about 1400000 points (Figure 3a) was used to create the 3D model. 2.2 “Mentoring” Stephen J. Daly 2003 For this sculpture the point cloud was acquired from four scan positions with a distance to the sculpture of 14.9 m, 41.7 m, 51.7 m and 50.3 m, respectively. The resolution of each scan world was 5 mm on the object‟s surface. A mean absolute error of 2 mm was achieved within the registration process. A point cloud of about 1083670 points (Figure 3b) was used to create the 3D model. 3. TERRESTRIAL LASER SCANNING WORKFLOW 3.1 Registration Leica Cyclone Register 6.0 software was used to align all the scans in a single and common coordinate system. It was performed following a target-to-target registration, by means of the artificial targets, and a cloud-to-cloud registration, manually picking corresponding points in the clouds. Point clouds were exported in the Leica .PTS interchange format. a b Figure 3: Point clouds: a) “Defensas I”; b); “Mentoring” 3.2 Modelling 3DReshaper 5.3 Software was chosen to carry out the 3D model creation: the most computational and time consuming step in the laser scanning workflow. The workflow included data cleaning, noise filtering, meshing, smoothing, joining parts and hole filling. Due to the morphological complexity of the sculptures, each sculpture was divided into different parts and modelled separately with different parameters (Figure 4). a b c d Figure 4: Sculptures decomposition: a,b) “Defensas I”; c, d) “Mentoring” Data cleaning/Noise filtering: the first step was removing noisy data, outliers, and all the unwanted parts from each point cloud. Two different functions were used: a) noise reduction; and b) explode with distance. For the former, a 0.1% of the total number of points was deleted; for the latter, all the isolated points separated more than 1.5 cm were deleted. ________________________________________________________________________________ Geoinformatics CTU FCE 2011 92 Meshing/triangulation: this step involved data triangulation to derive a 3D triangular mesh. A “good” mesh would be the one that keeps only the useful and valid points. Two criteria are used to achieve this goal. First, a quality criterion based on the noise of the measuring system. With this first criterion the idea is to keep only the most “right” points, eliminating the points that are above or below a theoretical surface. Second, a geometrical criterion: the system will keep points in the areas of high curvature based on a deviation error, which is the maximum distance between the theoretical surface and the triangulated irregular network. Figure 5 displays the way both criteria work on an idealized undulated surface [18]. The effects of the parameterization with the deviation error are presented in Section 4. Figure 5: Undulated surface after removing the noise and fitting the mesh to a maximum deviation error Smoothing: owing to the roughness of the surface, the shape of the mesh was noisy. Several smoothing values were selected for the different parts of the mesh. Following this way, the appearance of the eventual mesh improved. Joining the different parts: the different parts of the sculptures were joined to deliver single digital surface models. A summary of the different steps is shown in Figures 6 and 7. a b c d Figure 6: Modelling phases for “Defensas I”: a) Meshing; b) Smoothing; c) Joining parts; d) Holes filling a b c d Figure 7: Modelling phases for “Mentoring”: a) Meshing; b) Smoothing; c) Joining parts; d) Holes filling ________________________________________________________________________________ Geoinformatics CTU FCE 2011 93 3.3 Texturing 3DReshaper 5.3 Software was used to drape the texture to the 3D models, using images from an off-the-self digital camera, the Canon Powershot G11 at maximum resolution (10 MPixel). The position, the orientation and the inner parameters of the camera were determined during the orientation performance after selecting a minimum of three homologous points between the 3D surface and the corresponding images (Figure 8). As the digital camera was not calibrated beforehand, six to eight homologous points were measured on each picture in order to obtain the best orientation adjustment. a b c d Figure Ř: Texturing the images onto the models: a,b) “Defensas I”; c,d) “Mentoring” The final results achieved for the two sculptures can be visualized in Figure 9. The number of pictures draped onto the 3D model was three for “Defensas I”; more images were needed for “Mentoring” due to the complexity of the upper part (four for the main body and eight for the details). a b c d Figure 9: Final 3D models (a,c) and photorealistic 3D models (b,d): a,b) “Defensas I”; c,d) “Mentoring” 4. MESHING AND SMOOTHING ANALYSES One of the most critical steps for creating the 3D mesh is to choose the appropriate parameters regarding noise reduction and deviation error (Fig. 5). Noise reduction depends on the measurement noise of the laser scanner as well as on the sampling resolution. Unless there is oversampling or obvious noise in the data acquisition, the critical parameter for modelling is the deviation error. Figure 10 displays the point cloud and the image patch counterpart that was used to determine the best meshing parameters to model the sculptures. ________________________________________________________________________________ Geoinformatics CTU FCE 2011 94 a b Figure 10: a)Detail of the point cloud with intensity values; b) Corresponding image for “Defensas I” The different deviation error values selected for meshing as well as the reduction ratios achieved for the different solutions are presented in Table 1. The reduction ration ranges from 32.7% up to 95.5%; option 1 with 0 deviation error means that the deviation error is not taken into account, i.e. 100%. Therefore, whatever influence on the results will positively affect the output size of the mesh file, the more the higher the deviation error. The results of the different meshes after changing the deviation errors are displayed in Fig. 11. Table 1: Effects of the deviation error on the number of output triangles Option Deviation error (mm) Number of triangles Reduction ratio (%) 1 0 70807 0.00% 2 1 47704 32.7% 3 2 25047 54.6% 4 3 11994 83.1% 5 4 5992 91.5% 6 5 3206 95.5% a b c d e f Figure 11: Meshes of the same patch obtained with different deviation errors: a) 0 mm; b) 1 mm; c) 2 mm; d) 3 mm; e) 4 mm; f) 5 mm As can be realized in Figure 11, there is some noise in all the meshes, independently of the deviation errors. Moreover, the noise does not correspond with the true surface of the sculpture. On the one hand, there are tiny stripes that cannot be modeled with the selected sampling. On the other, there are some blobs of mortar irregularly spread on the surface‟s material. Therefore, smoothing was required to minimize the roughness (i.e. maximize the planar continuity in general) of the different surfaces (Figure 12). In particular, normal smoothing with controlled vertices recalculation was carried out to improve the appearance of the meshes. ________________________________________________________________________________ Geoinformatics CTU FCE 2011 95 a b c Figure 12: Meshes after normal smoothing with different deviation errors a) 0 mm; b) 1 mm c) 5 mm There were small differences in magnitude before and after normal smoothing but the regular appearances obtained after smoothing were visually significant (cf. Figures 11 and 12). Comparing metrically the unfiltered meshes with the corresponding filtered meshes, the differences were relatively small. Differences of ±1 mm yielded statistically equivalent values from 60.3% up to 90.3% for option 6 and option 1, respectively. In the contrary, maximum differences in the range of ±1-9 mm were computed with option 6 for 39.7% of the analysed area; for option 1, only ř.Ř% in the range of ±1-3 mm. Worth noticing is the small differences achieved with option 1 and 2 (Figure 13-2b) where Ř4.6% of the analysed area was up to ±1 mm. In fact, the difference images displayed in Figure 13 (a, b and c) show basically noise. Nevertheless, deviation errors equivalent to the error of the laser scanner yielded larger differences than expected for the patch area, minimizing the presence of the mortar among the pieces that shape the sculpture. a b c d Figure 13: Differences between meshes without and with filtering a) Option 1-1 smooth; b) Option 1-2 smooth; c) Option 6-6 smooth; and d) Option 6 smooth-2 smooth Whereas noise is self-evident for option 6 on Figure 13c, it is significant the average compensation of the noise after computing the differences between 6 smooth (Figure 13c) and 2 smooth (Figure 13d). The latter image displays the compensation of the noise coming from option 2 and option 6, yielding smaller differences in the range of ±1 mm for 71.3%, in other words, and improvement of 10% in average quality; only small number of peaks (2) reached up to 8 mm. After the research on the parameterization for meshing and smoothing, the recommended solution was option 2 after normal smoothing (Figure 12b). This option yields a balance among the number of triangles, the reduction ratio of the number of triangles (32.7%) and the eventual shape of the output model. This set of parameters was extrapolated from the test patch to the modelling of the whole sculptures, “Mentoring” and “Defensas I”. However, option 6 did not yield as bad results as expected beforehand and only around the break lines the output mesh was over-smoothed. The strong reduction ratio of the number of triangles up to 95.5% should not usually be rejected to yield fast and apparent photorealistic 3D models whether the output model is fully texturized with high resolution imagery. 5. CONCLUSIONS In this paper we have presented the results of a full 3D digitization of large contemporary art sculptures with a time -of- flight scanner. The output results demonstrate the chances of using terrestrial laser scanners to record geometrically the shape of cultural heritage assets such as large and complex sculptures in gardens, as well as the power of 3D modeling based on range-based solutions plus digital imagery for texturing. Nevertheless, the selection of the right parameterization for modeling is not a trivial step and should be carefully analyzed to yield acceptable metric ________________________________________________________________________________ Geoinformatics CTU FCE 2011 96 reconstruction. The improvement of the appearance on the surface of the objects based on imagery is thoroughly demonstrated herein with the two sculptures. Without texturing, a higher resolution 3D model should have been recorded to improve the quality of the digital reconstruction. However, owing to both the size and shape of the sculptures, other optical solutions such as structure light systems or triangulated laser scanners would have required both longer acquisition times and additional equipment to digitize the upper parts of the sculptures. The high quality photorealistic 3D models can be used for Web dissemination activities, to improve social awareness of cultural heritage value, for restoration and even for monitoring of the state of the objects over time. 5. ACKNOWLEDGEMENTS The authors gratefully acknowledge the support from the Spanish Ministry of Science and Innovation project HAR2010-18620. 6. REFERENCES [ 1 ] United Nations Educational, Scientific and Cultural Organization: Guidelines for the preservation of digital heritage, http://unesdoc.unesco.org/images/0013/001300/130071e.pdf, 2011-04-15. [2 ] Böhler,W. et al.: 3D scanning and photogrammetry for heritage recording: a comparison, 12th International Conference on Geoinformatics - Geospatial Information Research: Bridging the Pacific and Atlantic, Universidad de Gävle, Suecia, Junio 2004, 2ř1-298. [ 3 ] Patias, P.: Cultural Heritage Documentation, International Summer School “Digital Recording and 3D Modeling”, Aghios Nikolaos, Crete, Greece, 24-29 April 2006. [ 4 ] Lerma García, J.L., Van Genechten B. , Santana Quintero M.: 3D Risk Mapping. Theory and Practice on Terrestrial Laser Scanning. Training Material Based on Practical Applications. Universidad Politécnica de Valencia, Valencia 2008, 261 pp. [ 5 ] English Heritage: 3D laser scanning for heritage. Advice and guidance to users on laser scanning in archaeology and architecture http://www.helm.org.uk/upload/pdf/publishing-3d-laser-scanning-reprint.pdf , 2007. [ 6 ] Cabrelles N., Galcerá S., Navarro S., Lerma J.L., Akasheh T., Haddad N.: Integration of 3D laser scanning, photogrammetry, thermography to record architectural monuments, Proceedings of CIPA 2009, Kyoto, Japan, 11-15 Oct. 2009, pp. 6. [7 ] Al-kheder, S., Al-shawabkeh, Y., and Haala, N.: Developing a documentation system for desert palaces in Jordan using 3D laser scanning and digital photogrammetry, Journal of Archaeological Science, 2009, 36, 537-546. [ 8 ] Lerma J.L., Navarro S., Cabrelles N., Villaverde V.,: Terresrial Laser Scanning and close range photogrammetry for 3D archaeological documentation: the upper palaeolithic cave of Parpallò as a case of study, Journal of Archaeological Science, 37 (3), 2010, 499-507. [ 9 ] Remondino F., Rizzi A.: Reality-based 3D documentation of natural and cultural heritage sites-techniques, problems, and examples, Appl Geomat (2010) 2, 85-100 [ 10 ] Ikeuchi K., Miyazaki D.: Digitally archiving cultural objects. Springer, New York, 2008, 503 pp. ISBN: 978-0- 387-75806-0 [ 11 ] Ruther H.: Documenting Africa’s Cultural Heritage, Proceedings of VAST 2010, Archaeology and Cultural Heritage, Paris, September 2010. [ 12 ] Apollonio, F.I., Corsi, C., Gaiani, M., Baldissini, S.,: An integrated 3D geodatabase for Palladio’s work. Int. Journal of Architectural Computing, 2010, 2(8). [ 13] Ikeuchi K., Oishi T., Takamatsu J.: Digital bayon temple e-monumentalization of large-scale cultural-heritage objects, Proceeding of ASIAGRAPH 2007, Shanghaii, 2007, 1 (2), 99-106. [ 14 ] Levoy M, Pulli K, Curless B, Rusinkiewicz S, Koller D, Pereira L, Ginzton M, Anderson S, Davis J, Ginsberg J., Shade J., Fulk D.:The digital Michelangelo project: 3d scanning of large statues, Proceedings SIGGRAPH 2000, New Orleans, Louisiana, USA, July 2000, 131-144. [ 15 ] Remondino F., Girardi, S., Rizzi A., Gonzo, L.: 3D modeling of complex and detailed cultural heritage using multi-resolution data, Journal on Computing and Cultural Heritage 2(2009)1, 2:1-2:20 [16 ] Universidad Politécnica de Valencia: Campus en Tercera Dimensión: (primera fase), Universidad Politécnica de Valencia, Valencia 1993, 73 pp. [ 17 ] Leica Geosystems: Leica ScanStation 2 Product Specifications, http://hds.leica- geosystems.com/downloads123/hds/hds/ScanStation/brochures-datasheet/Leica_ScanStation%202_datasheet_us.pdf, 2011-05-19. [ 18 ] 3DReshaper 5.3 Tutorial, 2010, http://www.3dreshaper.com. http://unesdoc.unesco.org/images/0013/001300/130071e.pdf http://www.helm.org.uk/upload/pdf/publishing-3d-laser-scanning-reprint.pdf http://www.3dreshaper.com/