________________________________________________________________________________ Geoinformatics CTU FCE 2011 323 3DMADMAC|AUTOMATED: synergistic hardware and software solution for automated 3D digitization of cultural heritage objects Robert SITNIK1, Maciej KARASZEWSKI1, Wojciech ZAŁUSKI1, Eryk BUNSCH2 1Warsaw University of Technology, Faculty of Mechatronics Sw. Andrzeja Boboli 8, 02-525 Warsaw, Poland r.sitnik@mchtr.pw.edu.pl 2The Wilanow Palace Museum Stanislawa Kostki Potockiego 10/16, 02-958 Warsaw, Poland ebunsch@muzeum-wilanow.pl Keywords: 3D shape measurement, structured light, next best view, cultural heritage 3D digitization, automated data processing Abstract: In this article a fully automated 3D shape measurement system and data processing algorithms are presented. Main purpose of this system is to automatically (without any user intervention) and rapidly (at least ten times faster than manual measurement) digitize whole object’s surface with some limitations to its properties: maximum measurement volume is described as a cylinder with 2,8m height and 0,6m radius, maximum object's weight is 2 tons. Measurement head is automatically calibrated by the system for chosen working volume (from 120mm x 80mm x 60mm and ends up to 1,2m x 0,8m x 0,6m). Positioning of measurement head in relation to measured object is realized by computer-controlled manipulator. The system is equipped with two independent collision detection modules to prevent damaging measured object with moving sensor’s head. Measurement process is divided into three steps. First step is used for locating any part of object’s surface in assumed measurement volume. Second step is related to calculation of "next best view" position of measurement head on the base of existing 3D scans. Finally small holes in measured 3D surface are detected and measured. All 3D data processing (filtering, ICP based fitting and final views integration) is performed automatically. Final 3D model is created on the base of user specified parameters like accuracy of surface representation and/or density of surface sampling. In the last section of the paper, exemplary measurement result of two objects: biscuit (from the collection of Museum Palace at Wilanów) and Roman votive altar (Lower Moesia, II-III AD) are presented. 1. INTRODUCTION In recent years, many cultural institutions (museums, galleries etc) reached the conclusion that their presence in digital community is insufficient. The trend of making parts of collections available via Internet is rapidly growing, aided by excellent and still more and more affordable digitization hardware. Existing realizations are various, ranging from simple photos and two-dimensional scans, through three-dimensionally mapped photos (i.e. Google‟s Art Project [1] with contributors like National Gallery London, Palace of Versailles or Van Gogh Museum) to full three -dimensional presentations (Versailles Palace [2], the Khufu Pyramid [3], J. Iwaszkiewicz‟s Stawisko [4], Fryderyk Chopin‟s Piano [5]) with fully digitized objects presented as 3D models viewable from any direction. Multimedia presentations are not sole purpose of digitization – the resolution and accuracy of used measurement devices become so high it allows for creating relics‟ documentation and their eternal copies [6, 7]. In spite of all gains of 3D digitization of cultural heritage objects, it is still not very commonly used, what is a result of many problems associated with digitization process. 1.1 Difficulties in 3D digitization Development of hardware devices used in digitization (laser or structured light scanners) is generally much faster than progress of measurement strategies, at least partial automation and data processing software. This discrepancy often results in poor usability of even excellent scanning devices, leaving end-users without means to achieve professional final documentation. Typical digitization process contains of planning stage, when measurement strategies are created, scanning process, during which multiple directional measurements (each digitizing small part of an object) are taken, views integration (when directional measurements are integrated in full 3D model) and final processing (filtering, simplifying, triangle mesh creation etc.). By now, most of those stages are performed manually – measurement strategy and placement of scanner in subsequent measurements are done by skilled operator (obviously, those cannot be repeated ________________________________________________________________________________ Geoinformatics CTU FCE 2011 324 accurately during another measurements), views integration also very often requires human intervention. By lack of automation, each digitization process is unique (non-repeatable and subjective) which disqualifies this method as a mean for creating professional documentation [7]. Moreover, time and cost of manually - assisted digitization is very high. Automation of measurement process, connected with fully autonomous processing of obtained data is clearly an answer to above stated problems. This thesis was the idea behind development of 3DMADMAC|AUTOMATED, the three-dimensional digitization and data processing system for cultural heritage objects presented in this paper. 2. EXISTING DIGITIZATION SOLUTIONS To best of our knowledge, no system realizing all above stated assumptions for wide range of objects exist. Much work has been done on algorithms for views integration [8 – 12], however most of them require either initial fitting (by operator) or presence of unique objects (on object or in its vicinity), which often renders them unusable. As for automation of measurement process, rather simplified and not time-optimal solutions have been developed (i.e. scanning whole measurement volume by parts regardless if there is some part of an object or not) or ones based of a priori knowledge of object‟s shape (mainly for technical parts with CAD documentation [13]). Also, much work has been dedicated the problem of navigating measurement device in unknown space (with collision-free path calculated on-line during movement) – so called SLAM [14] problem [15]; however those solutions cannot be directly implemented in 3D shape digitization system because of different relations between size of measurement object and measurement device (in SLAM systems scanning head can be treated as point-like object travelling through vast 2.5 dimensional space, not able to change its attitude, in 3D digitizing systems scanning head is often of comparable dimensions with scanning object, cannot travel on the object and has to be positioned with different devices). 3. SYSTEM CONCEPT System realizing ideas presented in Introduction should allow for completely automated digitization of cultural heritage objects of given dimensions and weight with some constraints regarding surface parameters and shape complexity. The measurement process should not require any user intervention beside placement of object within measurement volume and entering few controlling parameters (like required resolution, final model format etc.). To obtain full digital representation of object‟s surface scanning head has to be positioned freely within measurement volume around digitized object; moreover, the transitions between subsequent head‟s positions have to be collision – free. Positions of head in next measurement have to be calculated automatically on the basis of hitherto obtained object‟s model. Positioning system should also report achieved head‟s position which could be used as a mean of initial fitting of directional measurement into whole model (thus eliminating the need of user-interaction during views integration or unique elements presence). Furthermore, processing software should be able to perform fine data fitting by means of ICP [18] or similar algorithm for vast datasets (with currently achievable resolutions, dataset size can easily be up to hundreds of gigabytes). Implementation of this concept, built in cooperation between Warsaw University of Technology and The Wilanow Palace Museum, uses commercial six-degree-of-freedom robot fixed to extendable column for scanning head‟s positioning, rotating table for object placement, structural light scanner returning measurement results as clouds of points (set of XYZ coordinates along with surface normal and color in sampled point). Measurement volume of scanning head can be set from 120mm x 80mm x 60mm up to 1200mm x 800mm x 600mm with accuracy better than 1/10 000 in respect to volume‟s largest dimension. Maximum dimensions of digitized object can be as large as cylinder of 600 mm radius and 2800mm height while its weigh cannot exceed 2000 kg. Photo of real measurement system‟s setup is at Figure 1. Developed software is composed of modules realizing: 1. Measurement strategy (next scanner position) calculation 2. Collision – free path planning with inverse kinematics 3. Positioning devices‟ control along with real-time collision-detection backup system 4. Directional measurements ordering 5. Initial data fitting (basing on achieved scanning head‟s position) 6. Fine data fitting (by ICP algorithm) 7. Final integration 8. Simplification algorithms, textured triangle mesh generation The application contains 64bit memory manager allowing for virtually unlimited processed data size (a custom implementation of page-file technique, using hard disks as an “extension” of operational computer memory). It is heavily optimized allowing multithreaded processing which utilizes modern CPU‟s capabilities. ________________________________________________________________________________ Geoinformatics CTU FCE 2011 325 Figure 1. Measurement system setup. 4. MEASUREMENT PROCESS WITH EXAMPLES Typical digitization process, beginning with preparation stage and ending with 3D model in format of triangle mesh with texture is presented in following subchapters. All elements of this process are illustrated by pictures from real digitization of cultural heritage objects, namely II-III AD Roman votive altar from Lower Moesia and 18th century ceramic figurine of Juno, made as imprint from the form created by Johann Friedrich Eberlein in year 1741. This statuette is a part of the collection of Museum Palace at Wilanów. 4.1 Preparation stage Measurement head, (in our implementation 3DMADMAC structural light scanner [19]) calibrated to required resolution and accuracy is fixed to robot‟s wrist, in a way ensuring stiff connection between those two devices. After this stage, the calibration of relations between coordinate system of robot and scanner has to be performed (clouds of points, obtained from the scanner have to be transformed into global coordinate system fixed to positioning devices). This process consists of taking few directional measurements (during which position of scanning head is known only in robot coordinates) of object of unique shape (complicated sculpture or artificial calibration body – Figure 2) to allow for automatic initial fitting of clouds. After views integration, the relation between coordinate systems of scanner and robot can be easily calculated. In similar way relations between rotating table‟s axis and robot fixture or axis of column‟s movement can be found. ________________________________________________________________________________ Geoinformatics CTU FCE 2011 326 Figure 2. Calibration unit used for identifying relations between coordinate systems. After calibration of coordinate systems, the object to be digitized is placed on the rotatable stage. It should be put in the middle of measurement volume. After fixing object to table, operator has to approximately measure dimensions of virtual cylinder enclosing digitized object ( Figure 3). Those parameters along with required resolution of final model and its format (cloud of points, triangle mesh) are put into controlling applications interface. Afterwards, the proper measurement process can be started. a) b) c) d) Figure 3. Virtual cylinder encompassing measured object: a) correct; b) radius too large; c) radius too small; d) object not centered. 4.2 Digitization Measurement process can be divided into three stages, namely locating an object in measurement volume (1), rough scanning (2) and filling of discontinuities (and areas with resolution lower than required)(3). Locating an object within measurement volume is tightly connected with proper values of parameters entered by operator at the end of preparation stage. To allow for collision-detection calculations, cylinder encompassing measured object has to be initially treated as unknown (i.e. it cannot be stated that moving scanner through it can be done without collision), and only already measured parts can be marked as empty or full, regarding if any part of an object is present there. Therefore, the first part of measurement process consists of locating any part of object‟s surface within the cylinder. This is done by “sculpting” the cylinder‟s volume with subsequent measurements ( Figure 4). After obtaining cloud of points representing a part of digitized object, the next stage (rough scanning) is started. During rough scanning, measurement planning algorithms calculate few positions of scanning head which give best possibility of obtaining large part of object‟s surface sample ( Figure 5). The algorithm used here locates areas near edges of already obtained model, and using surface normal vectors predicts probable shape of neighbourhood. Afterwards, it calculates position and orientation of scanner required to measure this predicted part of object. Those candidate positions are sent to module which analyzes if they are ________________________________________________________________________________ Geoinformatics CTU FCE 2011 327 possible to reach without collisions and if so, the scanner is transferred there. If it is impossible to place scanner in required point because of its collision with part of measurement space not yet known, the algorithm may order taking a scan there to complete space information. When the measurement head is positioned in required position, the scan is ordered. Obtained cloud of points, after some postprocessing (filtering, noise cancellation etc.), is transformed through coordinates of scanner in global coordinate system thus initially fitted into existing model. Afterwards, fine fitting is performed by automatic ICP algorithm (Figure 6). Figure 4. Locating measured object's surface. a) b) Figure 5. Next measurement directions proposed by rough scanning algorithm for the first measurement obtained from scanning of votive altar: a) all; b) selected. The process continues until view planning algorithm cannot distinguish any large border areas of digitized model (greater than 10% of measurement‟s head volume). Afterwards, the final stage is started. During third stage of digitization, areas of low point‟s resolution (lower than required) are identified. They are nested into groups of size similar to measurement head‟s working volume, and for each of those groups the position for scanning head required to measure it is calculated. After the process is completed, full digital model of an object is contained in computer memory. Of course, if analyzed object contains areas, which cannot be measured by used scanning head (for example occluded by another parts of object), cannot be digitized. 4.3 Final processing After measurement process is completed, the global relaxation algorithm is run, along with automated re -colouring of overexposed areas. In the next stage, data simplification is performed, according to parameter given at start by operator. If it is required to convert cloud of points into triangle mesh, the appropriate algorithm is run at the end of processing. Exemplary results are shown at Figure 7 and Figure 8. ________________________________________________________________________________ Geoinformatics CTU FCE 2011 328 a) b) c) d) Figure 6. Automatic processing of directional cloud of points from measurement: a) source cloud (up), noise filtering (down); b) cloud without transformations; c) inital fitting to global model; d) after ICP. a) b) c_ d) Figure 7. Votive altar: a) photo; b) cloud of points; c) triangle mesh; d) triangle mesh with texture. a) b) c) d) Figure 8. Biscuit: a) photo [20]; b) cloud of points; c) triangle mesh; d) triangle mesh with texture. 6. SUMMARY The 3D shape digitization system, 3DMADMAC|AUTOMATED, presented in this paper is the implementation of concept of a tool aimed at popularization of 3D scanning techniques in cultural heritage documentation and presentation. Its features address all problems present in commercially available digitizing devices, especially the need of skilled operator‟s supervision and assistance. It was developed as scalable and easily configurable universal measurement system capable of automatic digitization of various classes of objects, especially cultural heritage ones. Till now, it has been used to digitize more than 40 works of arts from collections of Museum Palace at Wilanów, Academy of Fine Arts at Warsaw and National Museum at Warsaw. ________________________________________________________________________________ Geoinformatics CTU FCE 2011 329 Parameters of exemplary objects, along with processing time and dataset sizes is presented in Table 1. Object Dimensions Material Directional scans (rough scanning) Directional scans (filling of discontinuities) Digitization time Point count Triangle count Votive altar from Lower Moesia 286mm x 168mm x 168mm Sandstone 32 8 19h 03m 20s 141231113 (resolution 0.05mm) 873510 (resolution 0.5 mm) Biscuit 335mm x 205mm x 260mm Biscuit 142 25 92h 05m 41s 720842124 (resolution 0.02mm) 1213158 (resolution 0.5 mm) Table 1. Parameters of measured objects 7. REFERENCES [1] Google Art Project, http://www.googleartproject.com/. [2] Grand Versailles Numérique, http://www.gvn.chateauversailles.fr. [3] The Khufu Pyramid, http://www.3dvia.com/3d_experiences/view_experience.php?experienceId=1. [4] J. Iwaszkiewicz‟s Stawisko, http://stawisko.pl/wirtualne/stawisko/index.html. [5] F. Chopin‟s Piano, http://www.culture.pl/chopin/index.html. [6] E. Bunsch, R. Sitnik, J. Michoński, Art documentation quality in function of 3D scanning resolution and precision, Proc. SPIE 7869, 2011, 78690D. [7] E. Bunsch, R. Sitnik, Documentation instead of visualization - applications of 3D scanning in works of art ana lysis, Proc. SPIE 7531, 2010, 75310I. [8] Dorai, C., Wang, G., Jain, A.K., Mercer, C.: Registration and integration of multiple object views for 3D model construction, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20 (1998) 1. [9] Zhou, H., Liu Y.: Accurate integration of multi-view range images using k-means clustering, Pattern Recognition, 41 (2008) 1, 152-175. [10] Kapoutsis, C.A., Vavoulidis, C.P., Pitas, I.: Morphological iterative closest point algorithm, IEEE Transactions on Image Processing, 8 (1999) 11, 1644 – 1646. [11] Zach, C., Pock, T., Bischof, H.: A Globally Optimal Algorithm for Robust TV-L1 Range Image Integration, IEEE 11th International Conference on Computer Vision, October 2007, 1 - 8. [12] Sappa, A. D., García, M.A.: Incremental Multiview Integration of Range Image, 15th IAPR International Conference on Pattern Recognition, Barcelona, September 2000, 546-549. [13] Ainsworth, I., Ristic, M., Brujic D.: CAD-Based Measurement Path Planning for Free-Form Shapes Using Contact Probes, International Journal of Advanced Manufacturing Technology, (2000) 16, 23–31. [14] Thrun, S.: Robotic mapping: A survey, Exploring Artificial Intelligence in the New Millenium San Mateo, CA, Morgan Kaufmann, 2002. [15] Gamini Dissanayake, M. W. M., Newman, P., Clark, S., Durrant-Whyte, H. F., Csorba, M.: A Solution to the Simultaneous Localization and Map Building (SLAM) Problem, Transactions on Robotics and Automation, 17 (2001) 3. [16] Menegatti, E., Pretto, A., Scarpa, A., Pagello, E.: Omnidirectional Vision Scan Matching for Robot Localization in Dynamic Environments, IEEE Transactions on Robotics, 22 (2006) 3. [17] Chang, H., J., Lee, , C. S. G., Lu, Y-H., Hu, Y.C.: P-SLAM: Simultaneous Localization and Mapping With Environmental-Structure Prediction, IEEE Transactions on Robotics, 23 (2007) 2. [18] Besl, P., McKay, N.: A method for Registration of 3-D Shapes, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(1992)2, 239 – 256. [19] Sitnik ,R., Kujawińska, M., Załuski, W.: 3DMADMAC system: optical 3D shape acquisition and processing path for VR applications, Proc. SPIE 5857, 2005, 106-117. [20] Szelegejd, B.: Wyrafinowany urok białej porcelany. Wilanowska kolekcja biskwitów, Warsaw 2006, 71-72. http://www.googleartproject.com/ http://www.gvn.chateauversailles.fr/ http://www.3dvia.com/3d_experiences/view_experience.php?experienceId=1 http://stawisko.pl/wirtualne/stawisko/index.html