A Quick Method for the Texture Mapping of Meshes Acquired by Laser Scanner Francesco Gabellone1, Ivan Ferrari2, Francesco Giuri2 1Consiglio Nazionale delle Ricerche, Istituto per i Beni Archeologici e Monumentali (IBAM) Via Monteroni, 73100 Lecce, Italy f.gabellone@ibam.cnr.it 2Reaserch fellow at IBAM Via Monteroni, 73100 Lecce, Italy Abstract The methodology described in this article was developed in connection with two different projects and entails texture mapping by time-of-flight laser scanner. In order to verify its operational effectiveness and applicability to other contexts, sites with extremely different morphological characteristics were studied. The basic rationale of this simple method derives from the need to obtain differ- ent types of mapping – including RGB real colour images, infra-red images, false colour images from georadar scans, etc. – from the same scanned surface. To resolve this problem, we felt that the most appropriate step was to obtain a UVW mapping based on the high resolution real colour images and then use the same coordinates to rapidly map the false colour images as well. Thus we fitted a device to the camera to determine its trajectory (similar to a gun- sight); when scanned by the laser scanner in the same context as the monument, it makes it possible to know the exact coordinates of the viewpoint. Keywords: Laser scanning, texture mapping, projection camera Texture Mapping in Laser Scanner Applications 1. State of the art The progress of laser scanning technology is focused from several time on two different areas: I) the continuous and increasing demand, from the users, of more accurate and faster machines, II) a tendency to build the same in light alloy, easy to handle and more user-friendly. Into a few years it has gone from a standard of 50.000 pt/sec. to 1.000.000 pt/sec., with a performance increase of about twenty times higher than the past. In the same time, these machines have fully entered in the standard equipment of a small and medium companies, a process which has established, after a gestation period of about twenty years, their full usability even by users different from those one historically identified with universities and research centers. To this increasing diffusion of the “tool” laser scanner, encouraged by its relative ease of use, corresponds a transfer of advanced technologies, which have been merged from the research centers into big manufacturers, to improve not just the acquisition process of points in 3D space, but rather all post-processing data. The reference was to the process of data-fusion for alignment and registration of single shots, to the process to make the meshes and their optimization, to the inspection of models, and finally to the texturing. At Geoinformatics FCE CTU 9, 2012 17 Gabellone, F. et al.: A Quick Method for the Texture Mapping of Meshes . . . present exist a choice of several software solutions, able to manage successfully this critical stage of the acquisition and restitution process of historical and artistic artifacts. In this contest some research groups are included, now focused on the creation of tools that allow the total (or almost) automation of all post-processing processes of point clouds management, capturing data from the visible color of surfaces with Color-Per-Vertex (CPV) information, i.e. with point clouds that directly give spatial information and color information in RGB space, associated with each vertex. This approach have some problems that are not insignificant: the color information associated with the CPV points can’t be reworked or replaced with new color information. To point out it an example could be represented by laser scanning aimed at the documentation for the restoration, in which it is extremely useful to compare “ante and post rem” (before and after the restoration), or in case studies where it is useful to replace a color texture with images related to scientific analysis, such as the IR observation, or the false-color maps from GPR data. Contrarywise, the adoption of a classical texture mapping method allows the full flexibility in creating multi-resolution maps – very useful for the development of real-time and gaming applications – and an easy replacement of the same maps with scientific infor- mation about various themes, that may arise for the artifact studied. Leaving aside the most common mapping systems (cubic, spherical, planar, etc.), the most effective and advanced texture mapping technique available in software for the processing of point clouds, are sub- stantially referable to the point-to-point method [2]. Concretely, the operator has to identify the correspondences existing between geometry and texture to map, assigning each map some UVW coordinates that link a point on the surface with a pixel of the texture. This is all the more complex the more the surface is not well characterized in terms of three-dimensional topology, so on a perfectly flat (or smooth) surface will be very difficult to find corresponding points to the pixels of a texture. The research work of point-pixel correspondences is equally challenging in complex artifacts, which generally require a subdivision of the total geometry in small parts to map with different textures ( Fig. 01). Figure 1: Example of multiple patching for the application of many textures (M. delle Croci, Matera) Geoinformatics FCE CTU 9, 2012 18 Gabellone, F. et al.: A Quick Method for the Texture Mapping of Meshes . . . 2. Objectives The main purpose of this paper is to illustrate a simple texture mapping method that has been applied in two contests, very different in their morphology, in order to test the effective on site operability and the full usability also by users not highly specialized. The case studies related to two monuments with the same name, the rupestrian church of Santo Spirito in Monopoli and the church of Santo Spirito in Lecce. In both cases the mapping of the surfaces has been made using the images of the visible color, taken with a camera and with criteria set out below. The next step of this study will consist in the mapping of models with false-color images, coming from infrared observation (IR) and GPR data analysis, which will be referenced using the same UVW coordinates derived from the application of the simple method described in this paper. 3. Case studies The rupestrian church of Santo Spirito is located in Monopoli, a village few km south of Bari, Italy. The monument is part of a diagnostic project for it restoration sponsored by San Domenico’s Foundation, for many years active in the territory of the South of Italy for the preservation and the promotion of the rupestrian culture. This structure has a peculiarity in the history of architecture: every wall, vault, recess, obtained “per via di levare” (working subtracting material), is an unique and not repeatable piece. This value derives, for obvious reasons, from the very nature of the buildings, that imitate open-air (sub divo) architectures, but they do it in an opposite way, producing sometimes complex planimetric solutions, that could be called “organic”. Their development proceeds in a natural and sinuous way, the walls are often curved and not perpendicular, the ceilings and the vaults are on inclined planes, everything in the geometry of the surfaces is unpredictable. The morphological and architectural study of these hypogean spaces, just for the reasons mentioned above, requires a cognitive approach aimed at the acknowledgment of the constructive singularities, and since no architectural element can be traced back, a priori, to planar and regular forms, their survey must necessarily follow a procedure as rigorous and precise as possible, without simplifications. The survey of the rupestrian church of Santo Spirito in Monopoli has been carried out with the indirect active method, using Leica ScanStation 2 laser scanner. Considering the morphological characteristics of the studied monument and the large size of the area to be surveyed, a 3 mm point-to-point step was adopted for the survey, which also represents the most accurate value tested by the instrument. This parameter allow to obtain a high resolution mesh, suitable to the survey purposes, but at the same time it led to an huge complexity of the 3D model, which imposed complex post-processing solutions. The survey involved a total area of about 458 m2, which extends from the outer limits marked by the two roads that surround it, up to the apse on the opposite side of the present entrance. The internal area, measured at the floor, measures 136,12 m2 and it consists in two spaces, stylistically different, adjacent to each other, but forming a unitary space. The survey produced a total of about 40 million polygons, with a mesh resolution of 3 mm. Considering the difficulties of managing on desktop computer, the model has been divided into four high resolution parts, and later it has been decimated by a curve filtering parameter of 80◦, to bring it to a 5 Million polygons complexity. Of course, this filtering allowed to maintain a high level of details in areas with higher curvature (so more morphological complexity), and Geoinformatics FCE CTU 9, 2012 19 Gabellone, F. et al.: A Quick Method for the Texture Mapping of Meshes . . . Figure 2: The rupestrian church of Santo Spirito, Monopoli (Bari), general plan to eliminate redundant data in pseudo-planar areas (side walls). The second case study refers to the church of Santo Spirito in Lecce. The original structure of the church is dated 1392. It has been enlarged and reworked several times, first by the architect Gian Giacomo dell’Acaya, master of the hospital, and then rebuilt between 1691 and 1728 on the plans by Giuseppe Zimbalo, who died during the works, in 1710. The plant of the building, with a single nave, is marked by four arches and six niches, made of local stone and baroque stucco, which enclose the same number of altars. The morphology of the walls is extremely rich in details, as we would expect from a building of this period, compared to an overall structure very simple and regular. The surveys are functional to a diagnostic and research project, planned in the field of the AITECH project activities (Applied Innovation Technologies for Diagnosis and Conservation of Built Heritage), which is part of the “Rete dei Laboratori” (Laboratory Network) initiative, sponsored by the Ministry of Economic Development, the Ministry of University and Research and the Region of Puglia, in Italy. On this monument, the work team IBAM CNR of Lecce has set up some researches aimed at the mapping of structural cracks and at IR observation of masonry, activities for which a preliminary three-dimensional restitution of surfaces is required, necessary for the accurate knowledge of architectural features. Even for this monument the survey operations were performed using a laser scanner Leica ScanStation 2, where point clouds have been aligned, filtered and decimated with the same criteria used for the rupestrian church explained above. Geoinformatics FCE CTU 9, 2012 20 Gabellone, F. et al.: A Quick Method for the Texture Mapping of Meshes . . . Figure 3: The rupestrian church of Santo Spirito, Monopoli (Bari), axonometric view Mapping Using Camera Position Informations 4. Description of the method The correct mapping of the three-dimensional model acquired through laser scanning is usu- ally one of the most problematic issues in the process of accurate and verisimilar restitution of an artifact. Often, some important works by laser scanning are presented as well as mere geometric shapes, with simple shaded views without any applied texture. Certainly a simplified presentation of reality, in which the chromatic values of the surfaces are essential elements for a correct reading of the conservation status, of the characteristics of the constituent materials, of superficial cracks and other micro detail characteristics, impossible to reproduce with a time-of-flight laser scanner. For this reason, we tried to refine an old method of mapping, widely known, in the attempt to provide, even the beginner, the necessary tools to produce 3D textured models with good accuracy. In this kind of applications the use of quality images at high resolution will allow to get extremely realistic models, almost indistinguishable from the real one, to be used productively in the documentation for the knowledge of historical buildings, in restoration works, in faithful representations of the current state, in serious games, etc. So the basis of this method is the simple way of mapping according to a camera projection mode, Geoinformatics FCE CTU 9, 2012 21 Gabellone, F. et al.: A Quick Method for the Texture Mapping of Meshes . . . Figure 4: The rupestrian church of Santo Spirito, Monopoli (Bari), perspective section in some software called Camera Mapping, or Projection Mapping. This technique is generally known and applied in Computer Graphics especially for making interactive 3D models from two-dimensional images, very useful for the conversion of two-dimensional paintings, frescoes, engravings, in 3D explorable scenes. The usefulness of this method is even more appreciated in mapping operations starting from photos. It is indeed known that cylindrical, spherical or cubical projections, are applicable only in specific cases and can never be used to overlap the photos directly on a scanned object. The planar mapping, in particular, projects the texture on the object according to the plane normal, the plane orientationand so the projection direction is chosen according to the necessities and the kind of the object to texturize, but this type of projection is not coincident with a camera take. In a lot of case studies is wrongly used precisely this technique to map complex objects, by assigning small parts of the scanned object to specific photo-textures, with poor results and obvious signs of stretching in the areas with a different angle projection. This means, strictly speaking, that the planar mapping could be applied just on a planar object with an ortho-rectified texture, according to the rules of the orthogonal projection. Each photographic image is rather a perspective view, with a point of view, a target point of the perspective, a visual field and deformations dependent on the quality and nature of the lenses. Then in theory, knowing exactly these four parameters and mapping the photographic images according to the perspective rules, that is the same Camera Mapping method, you can get a mapping with an almost perfect texture-3D model overlap. The examination of the solutions proposed for the definition of each of these parameters is: shooting position (or point of view, XYZ position of picture center), target point of the perspective (look at), characteristics of the lenses (focal length and distortion). There are several criteria to establish with precision the camera position in a 3D scene. The Geoinformatics FCE CTU 9, 2012 22 Gabellone, F. et al.: A Quick Method for the Texture Mapping of Meshes . . . Figure 5: Church of Santo Spirito in Lecce, multiple texture applied on South wall and example of texture replacement using same UVW coordinates with color correct image first, widely experienced by the Information Technologies Lab of Lecce (ITLab), consists in the recognition of only the significant points of the scene with digital photogrammetric techniques, and then, after the orientation process, in the recovery of the camera positions of each shot after a patient work of restitution. However, this technique can produce also significant residual errors and lead to further uncertainty degrees in the research of the camera positions, moreover this technique, although it is classified as “low cost”, requires a long processing time, which should be added to that one needed for the post-processing of point clouds. Geoinformatics FCE CTU 9, 2012 23 Gabellone, F. et al.: A Quick Method for the Texture Mapping of Meshes . . . Figure 6: Screen shot of 3D model with and without texture applied. Figure 7: The rupestrian church of Santo Spirito, Monopoli (Bari), wireframe view with texture applied Geoinformatics FCE CTU 9, 2012 24 Gabellone, F. et al.: A Quick Method for the Texture Mapping of Meshes . . . The method proposed in this paper consists in shooting with laser scanner the camera po- sitions within the scene and in overlapping them with their respective virtual cameras in a 3D graphic software, respecting some important procedures. The first consideration concerns the difficulty of reproducing the exact point of view of the camera, which even though the double pass with the scanner set at the highest resolution (about 2 mm), can’t be identified with certainty. For this purpose a metal viewfinder long about 8 cm has been built, which mounted on the top of the photo camera provides a clear position reference that accurately identifies the point of view, and it has been also useful for a first orientation of the target point of the perspective. This first orientation is to be considered only rough, because a very little rotation fault of only few tenths of a degree on the basis of the virtual camera, produces significant phase shifts of the target point on the 3D surface. Therefore, the target point has been determined with certainty considering the center of the photographic image. Indeed, turning the camera around its point of view, now identified with certainty, is it possible to fit the center of the frame, marked by the intersection of the diagonals of the image, with the corresponding target point of the perspective, identified on the 3D model surface. This must of course be chosen on site, so it can be recognized during the mapping stages. Under low surface characterization, may be suggested the placing of a marker visible by the laser scan- ner. The last element, extremely important for the success of the method, is the elimination of the lens distortion. “Barrel” or “pincushion” distortions are always present, despite the use of aspherical professional lenses usually advertised with no distortions lens. These can be easily removed by the use of specific software, usually supplied by the manufacturers. The 3D model mapping with the Camera Mapping method, together with the identification of the point of view, the target point of the perspective and the elimination of distortions, creates in our opinion the best method of mapping complex surfaces. Of course this method can also be applied in an empirical way, manually moving the camera by making attempts up to find its exact position in space, but the results will be obtained only for approximation and they will not guarantee the best accuracy and the least time taken. 5. Conclusions The application of this method has proven an extremely easy to use and the full applicability on very complex and wide surfaces. In the two case studies proposed in this paper was possible to document with accurate precision both the morphological and the geometrical characteristics, both to a hyper-realistic representation of the current state of the surfaces. All that types of surface alteration that are outside from the maximum scanner resolution, have been documented by the mapping of images of about 20 M pixels, taken from a full frame digital reflex camera, Canon 5D Mark II. In the example in figure 5, concerning the south wall of the church of Santo Spirito in Lecce, three images were used, properly equalized and corrected, mapped on three separate parts of the model. The method gave excellent results both on planar parts and on round elements. Satisfactory results were obtained also in shots with great depth of field and short focal length (16 mm), showing that the elimination of the lens distortions makes fully usable also images with evident “barrel” deformations. A following application of this technique will allow using the coordinates of Camera Mapping, conveniently converted in UVW, to get textures from thematic maps, useful for providing false-color information about the conservation status of the structures. Geoinformatics FCE CTU 9, 2012 25 Gabellone, F. et al.: A Quick Method for the Texture Mapping of Meshes . . . References [1] Agathos, A., Fisher, R. B., Colour Texture Fusion of Multiple Range Images, Proc. 4th Int. Conf. on 3-D Digital Imaging and Modeling, Banff, to appear, 2003. [2] Callieri, M., Cignoni, P., Rocchini, C., and Scopigno, R., Weaver, an automatic texture builder, 3D Data Processing, Visualization and Transmission, Int. Conf., Padova, 2002, pp. 562-565. [3] Bernardini, F., Martin, I., and Rushmeier, H., High-quality Texture Reconstruction from Multiple Scans, IEEE Transactions on Visualization and Computer Graphics, 7(4), 2001, pp. 318-332 [4] Beraldin, J-A., Blais, F., Cournoyer, L., Picard, M., Gamache, D., Valzano, V., Bandiera, A., and Gorgoglione, M., Multi-resolution digital 3D imaging system applied to the record- ing of grotto sites: the case of the Grotta dei Cervi, in VAST, October 30 - November 4 2006. [5] Gabellone, F., Ancient contexts and Virtual Reality: From reconstructive study to the construction of knowledge models, Journal of Cultural Heritage, Journal number 9069, Elsevier B.V., 2009. [6] Gabellone, F., Virtual Cerrate: a DVR-based knowledge platform for an archaeological complex of the Byzantine age, in CAA 2008, Computer applications and quantitative methods in Archaeology, Budapest 2008. [7] Gabellone, F., Metodologie integrate per lo studio ricostruttivo e la conoscenza dello stato attuale dei Beni Culturali, In: Il dialogo dei Saperi, metodologie inte-grate per i Beni Cul- turali, a cura di F. D’Andria, D. Malfitana, N. Masini, G. Scardozzi, Edizioni Scientifiche, 2010. Geoinformatics FCE CTU 9, 2012 26