________________________________________________________________________________ Geoinformatics CTU FCE 370 AUTOMATED 3D-OBJECTDOCUMENTATION ON THE BASE OF AN IMAGE SET Dipl.-Inf. (FH) Sebastian VETTER1, Dipl.-Ing. Gunnar SIEDLER1 1fokus GmbH Leipzig Lauchstädter Str. 20, 04229 Leipzig, Germany home@fokus-gmbh-leipzig.de Keywords: 3d-objectdocumentation, textured surface model, orthophotos, image matching, point cloud Abstract: Digital stereo-photogrammetry allows users an automatic evaluation of the spatial dimension and the surface texture of objects. The integration of image analysis techniques simplifies the automation of evaluation of large image sets and offers a high accuracy [1] . Due to the substantial similarities of stereoscopic image pairs, correlation techniques provide measurements of subpixel precision for corresponding image points. With the help of an automated point search algorithm in image sets identical points are used to associate pairs of images to stereo models and group them. The found identical points in all images are basis for calculation of the relative orientation of each stereo model as well as defining the relation of neighboured stereo models. By using proper filter strategies incorrect points are removed and the relative orientation of the stereo model can be made automatically. With the help of 3D-reference points or distances at the object or a defined distance of camera basis the stereo model is orientated absolute. An adapted expansion- and matching algorithm offers the possibility to scan the object surface automatically. The result is a three dimensional point cloud; the scan resolution depends on image quality. With the integration of the iterative closest point- algorithm (ICP) these partial point clouds are fitted to a total point cloud. In this way, 3D-reference points are not necessary. With the help of the implemented triangulation algorithm a digital surface models (DSM) can be created. The texturing can be made automatically by the usage of the images that were used for scanning the object surface. It is possible to texture the surface model directly or to generate orthophotos automatically. By using of calibrated digital SLR cameras with full frame sensor a high accuracy can be reached. A big advantage is the possibility to control the accuracy and quality of the 3d-objectdocumentation with the resolution of the images. The procedure described here is implemented in software Metigo 3D. 1. IMAGE RECORDING Two digital reflex cameras Canon EOS 5D Mark II were used for 3D-evaluation on the base of stereo models in an image set at the shown object (Castle of Katzenstein). These were previously calibrated on two focal lengths (24mm und 50mm); (Alternatively usable camera systems are Nikon D700 or Sony Alpha 950 as digital reflex camera with full frame sensor). Using a receiving rail (on a tripod), where both cameras have been attached, a set of stereo model was taken. Additional three dimensional reference points at the object have been measured by tacheometer. The accuracy of evaluation can be influenced by image quality and image scale. Another recording configuration for plastic objects can be the usage of a rotation plate and the recording of single images in suitable step size. Figure 1: Left: recording configuration at east wall and apse (castle of Katzenstein); right: used recording rail with 2 reflex cameras ________________________________________________________________________________ Geoinformatics CTU FCE 371 2. AUTOMATED MODEL ASSIGNMENT, AUTOMATED MODEL ORIENTATION After creating a project in the software the evaluation accuracy and resolution are defined, and the images are loaded into the project. The inner orientation is established for every image by linking the images to the corresponding camera. Control points are automatically detected by the evaluation software in the images (identical points at the object). On the base of these control points the arrangement of the images is analyzed, with the goal to detect suitable pairs of images or stereo models (parallel recording direction, a large region of overlap). The manual creation of stereo models is possible as an alternative. Figure 2: Left: six images with similar image part and automatically found control points; right: result of analyzing for one pair of images (shift vectors of control points) The calculation of relative orientation of both images of the stereo model is made on the base of the known control points. If there are not enough control points for model orientation, additional image coordinates could be detected automatically or measured by hand. By using proper filter strategies during calculation of relative orientation of the stereo model incorrect model coordinates are detected and removed. By additional measurement of reference points the absolute orientation of one stereo model into overall coordinate system is made. The absolute orientation can be made with the defined distance of camera basis alternatively. A relation graph, that describes the arrangement of the stereo models, is generated with the help of the existing control points. Therefore, it is sufficient to make absolute orientation of one stereo model (of one set of related stereo models) by hand. All the other stereo models of the set are automatically orientated absolute. In the absence or insufficient relation of the stereo models, additional reference points on the object surface can be measured and the absolute orientation can be calculated manually. Figure 3: Absolute orientated stereo model (green image coordinate: used control point; blue image coordinate: filtered control point; magenta image coordinate: control point was found in only one image; yellow image coordinate: not measured reference point) ________________________________________________________________________________ Geoinformatics CTU FCE 372 3. AUTOMATED GENERATION OF POINT CLOUD Due to the substantial similarities of stereoscopic image pairs, correlation techniques provide measurements of subpixel precision for corresponding image points. In addition to the single-point measurement object surfaces can be scanned with appropriate expansion algorithms [2]. With consideration of the evaluation accuracy for every stereo model the right step size (point distance) for matching is determined in dependence of the images scale. With batch processing all existing stereo models can be “scanned”. For every matched point the error is calculated by photogrammetric spatial intersection and the point cloud is coloured according to these errors. Incorrect points are filtered automatically by an adjustable treshold. Figure 4: Three partial point clouds in 3D window, coloring in dependence of point error 4. MERGE OF PARTIAL POINT CLOUDS TO A TOTAL POINT CLOUD With the help of the integrated Iterative Closest Point algorithm the partial point clouds are transformed by the identical control points and merged to a total point cloud. By usage of filter strategies the point cloud is thinned out to the evaluation accuracy in the overlapping areas, due to merge, these have a higher point density [3]. Figure 5: Merged point cloud; right: point colouring in dependence of origin (partial point cloud) ________________________________________________________________________________ Geoinformatics CTU FCE 373 5. AUTOMATED EVALUATION OF DIGITAL SURFACE MODEL With a triangulation algorithm [4] a digital surface model is generated by a point cloud. In a second step, after editing the surface model, the images are mapped on it. Thus a three-dimensional digital documentation is possible. 6. UNWINDING / DIGITAL ORTHO PROJECTION For the projection of images onto a plane or another unwinding geometry, user coordinate systems can be defined related to overall coordinate system or with the help of a partial set of points (balancing plane). Additional sectional profiles can be extracted and generalized from the existing point cloud or surface model. Figure 6: 3D-surface model with image texture Figure 7: Left: 3D point cloud and plane of a user coordinate system (for ortho projection); right: image plane of east wall (ortho projection to plane) ________________________________________________________________________________ Geoinformatics CTU FCE 374 The ortho projection onto the unwinding geometry is made in a user defined image scale and image resolution on the base of the orientated images, the sectional profiles and the surface model. Figure 8: Left: orientated image of calotte with projection of a barrel; right: unwound image plan of calotte (ortho projection onto barrel) Figure 9: Unwound image plan of apse (ortho projection onto barrel) 7. SUMMARY The here shown east wall with apse was covered with 13 stereo models and round about 100 reference points were measured by a tacheometer. The orientation and evaluation of these stereo models and the shown result were made with the software metigo 3D. The final image processing and colour matching were made with Adobe Photoshop CS5. For the described evaluation procedure for all stereo models (from loading the images to the final assembly of the image ________________________________________________________________________________ Geoinformatics CTU FCE 375 plans) nearly 8 hours were required. Additional nearly 8 hours calculation time was required during batch processing for image arrangement, generation of point clouds, triangulation and unwinding or ortho projection, and 4 hours were needed for image processing with Adobe Photoshop. The required time for evaluation generally depends on the amount of images, the resolution of images and the surface model and the existing computing power. The resulting orthophoto, processed in scale 1:10 and 400dpi image resolution, can be used as 2D mapping base for the documentation of the object. The digital surface model can be alternatively textured to use them for a 3D mapping. At the moment the 3D mapping is not used in restoring because of practical reasons like the amount of data, printing or data exchange with the final customer. Parts of the here described evaluation steps were developed in a cooperation project with the Society For the Promotion of Applied Computer Science and supported by: Federal Ministry of Economics and Technology on the basis of a decision by the German Bundestag. 8. REFERENCES [1] Henze, F.; Siedler, G.; Vetter, S.: Integration automatisierter Verfahren der digitalen Bildverarbeitung in einem Stereoauswertesystem, 26. Wissenschaftlich-Technische Jahrestagung der DGPF, Berlin, 11.– 13.09.2006, Band 15, S. 239 - 246 [2] Vetter, S.: Generierung digitaler Oberflächenmodelle (DOM) im Bereich der Architekturphotogrammetrie, diploma thesis (unpublished), HTWK Leipzig, Germany, 2005 [3] Heinrich, M.: Markante Punkte und 3D- Objektkanten in einem Oberflächenmodel, diploma thesis (unpublished), HTWK Leipzig, Germany, 2010 [4] Bernardini, F., Mittleman, J., Rushmeier, H., Silva, C., Taubin, G.: The Ball-Pivoting Algorithm for Surface Reconstruction. IEEE Transaction on Visualization and Computer Graphics, 5(4), Oct-Dec, 1999, pp. 349-359.