Acta Polytechnica DOI:10.14311/AP.2020.60.0288 Acta Polytechnica 60(4):288–302, 2020 © Czech Technical University in Prague, 2020 available online at https://ojs.cvut.cz/ojs/index.php/ap STRUCTURE FROM MOTION PROCESSING OF ANALOGUE IMAGES CAPTURED BY ROLLEI METRIC CAMERA, DIGITIZED WITH VARIOUS SCANNING RESOLUTION Adam Dleska,∗, Karel Vachb, Karel Pavelkaa a Czech Technical University in Prague, Faculty of Civil Engineering, Department of Geomatics, Thákurova 7/2077, 166 29 Prague 6, Czech Republic b EuroGV spol. s .r.o., Opletalova 1284/37, 110 00 Prague 1, Czech Republic ∗ corresponding author: adam.dlesk@fsv.cvut.cz Abstract. SfM processing of archived analogue images gives an opportunity to efficiently create new and valuable 2D and 3D results. The SfM processing of digitized analogue images brings some challenges. How to digitize negatives of photogrammetric images? What scanning resolution is the most beneficial for processing? How to preprocess the digitized images to be able to process them using the SfM method? What accuracy of results is possible to expect? This paper tries to deal with all these questions. For this paper, 7 negatives of former photogrammetric documentation of a vault were chosen. The negatives were captured by Rollei 3003 metric camera in 1999. Two pieces of software were chosen for the SfM processing. A commercial alternative Agisoft PhotoScan and free open-source alternative MicMac. The results of the SfM processing were compared to the results of an original photogrammetric method, which was used for former processing in 1999. Keywords: Structure from motion, Rollei metric cameras, scanning resolution, relative and absolute orientation, point cloud generation. 1. Introduction There are probably many archived analogue images, which were originally used for photogrammetric documentation of different kind of objects. The structure from motion method (SfM) is designed primarily for processing of images captured by digital cameras. Due to the algorithms of SfM, the time cost of the photogrammetric processing has been significantly reduced and it is possible to create two-dimensional (orthophoto) and three-dimensional (point cloud, textured mesh) results very efficiently. Processing of archival analogue images (e.g. images of cultural heritage objects) using current methods brings new possibilities to create 2D and 3D results with a potential historical value. Also, there is a potential to have an opportunity to examine the documented object in different time stages. The results of the processing could be used for further work and could be new and valuable data for information systems like GIS or BIM. The processing of analogue images using the SfM method brings some challenges. The information about images could be completely lost or prevailed only partially. In some cases, important information about the used camera and the used lens is missing. The material of the negatives of the analogue images could be deformed by time and the radiometric value of negatives could be deteriorated. There is a question, how to digitize the negatives and how to preprocess them correctly. Because the images were most likely captured for a different photogrammetric method than SfM, there could be too few images in the dataset captured in various angles with various overlap. All these factors must be considered during the processing. The main part of the paper is to suggest a method how to digitize, preprocess and how to process the digitized images using the SfM method. The results were compared to the results of an original method and were analysed and evaluated in terms of spatial accuracy. What accuracy is possible to achieve if the presented method is followed? For the SfM processing, Agisoft PhotoScan was chosen as a commercial alternative and MicMac as a free and open-source alternative. As a dataset of images for this paper, analogue photogrammetric images captured by Rollei 3003 metric camera were chosen. In 1999, a Czech company EuroGV s.r.o. has carried out a photogrammetric documentation of vaults in Saint Vitus Cathedral at Prague castle in Prague. The company provided negatives of the analogue images, coordinates of measured control points, the full documentation of the project and calibration report for the camera lens. Due to all the provided data, it was possible to fully analyse the results of the SfM processing of analogue photogrammetric images. A pilot project with processing of analogue images, captured by a Rollei metric camera, has been published [1]. Several projects of SfM processing of digitized analogue images are also presented. Processing of digitized glass plate negatives by UMK 288 https://doi.org/10.14311/AP.2020.60.0288 https://ojs.cvut.cz/ojs/index.php/ap vol. 60 no. 4/2020 Structure from motion processing of analogue images. . . cameras and TAN photo-theodolite was presented by [2]. In literature, Agisoft PhotoScan prevails as a photogrammetric software for processing analogue images. Agisoft PhotoScan was used also by [3] to create 3D model of historic buildings, by [4] to process analogue images captured by RolleiMetric camera and UMK camera of Omayyad mosque, by [5] to analyse historic changes of a glacier, by [6] to process archival images captured by Hasselblad H5D-60 of chosen archeological area, by [7] to process analogue non-metric images and by [8] to process analogue aerial images to create DEM and orthophoto. The open-source MicMac was used for processing analogue images by [9]. Mentioned papers do not analyse how various scanning resolutions affect the accuracy of the results. 1.1. Rollei Metric Before digital cameras, metric cameras were usually used for the photogrammetric documentation. Metric cameras are cameras with stable optics, ordinarily fix-focus lens with minimal distortion and with fiducial marks. Their parameters of interior orientation are calibrated in laboratory and are constant for a long time. The wide known metric cameras are, for example, UMK and TMK by Zeiss manufacturer and P31, P32 by Wild. [10] Figure 1. Camera Rollei 6006 metric. Rollei metric cameras are determined as a semi-metric or partial-metric cameras because the optical system is not completely stable. The parameters of interior orientation are not constant over the long time due to their interchangeable and focusable lenses. Rollei metric cameras are equipped with réseau plate. The réseau plate is placed in front of the image plane and the reference points are projected to the image during exposure (Fig. 2). The reference points are used for the image coordinate system definition. As a sensor of the cameras, ordinary photographic roll film was used. Rollei 6006 metric and its later model Rollei 6008 metric used an image format of Figure 2. Part of an image with projected reference points of réseau plate. 60 mm × 60mm [11, 12]. Rollei 3003 metric used 36 mm × 24mm format. Cameras were equipped with flash, had a wide variety of interchangeable lenses with good quality optics, were flexible, light and easy to operate. Because all the mentioned facts, Rollei metric cameras became very important in the field of architectural photogrammetry [13] and in many other fields of industry. There are many archived analogue images created in 90’s of the last century. 1.2. SfM processing The structure from motion is the process how to calculate relative orientation of the images, which are captured from different positions and different angles. Structure from motion processing is based on feature matching algorithm e.g. SIFT, SURF, BRISK, DAISY, HOG or GLOH and algorithm for model fitting RANSAC. In this paper, one commercial software and one open-source software (as a free alternative) was chosen for the SfM processing of archival analogue images. As a commercial software, Agisoft PhotoScan Professional in version 1.4.3 developed by Agisoft LLC was chosen. Agisoft PhotoScan has a graphic user interface and is easy-to-use. As a free and open-source alternative, MicMac in version v1.0.beta13 was chosen. MicMac is developed by the National Institute of Geographic and Forestry Information (IGN) and French National School for Geographic Sciences (ENSG). MicMac does not have a graphic user interface and is operated via command line by commands and their parameters. Workflow of the SfM processing and generation of a point cloud consist of these steps: (1.) Tie points identification (2.) Relative orientation 289 A. Dlesk, K. Vach, K. Pavelka Acta Polytechnica (3.) Absolute orientation (4.) Point cloud generation In Agisoft PhotoScan, the tie points identification and relative orientation is obtained by a single function Align Photos. In MicMac, the tie points identification is executed by Tapioca command and the relative orientation is computed by command Tapas. To scale the relative orientation, the model needs to be transformed to the control points. Computing the absolute orientation is obtained by function Update in Agisoft PhotoScan and by command GCPBascule in MicMac. By command Campari and function Optimize Camera Alignment the model is adjusted by bundle adjustment, and the final absolute orientation is computed. In the absolute oriented model, it is possible to measure single discrete survey points on the images. By this, it is possible to compute coordinates of the important points or to compute coordinates of check points. The check points were used for checking the accuracy of the relative and absolute orientation of the model. If the relative or absolute orientation is already known, it is possible to create point cloud of the object. In Agisoft PhotoScan, the point cloud is generated by Build Dense Cloud function and in MicMac by C3DC command. It is possible to export the point cloud into different formats and it can be used, for example, for creating hypsometry or other 2D or 3D models [14–17]. 2. Dataset For the documentation of vaults in the Saint Vitus Cathedral in Prague in 1999, EuroGV company used the intersection photogrammetry method. The results of the photogrammetric documentation were hypsographic plans of the vaults. For this paper, the vault of Zikmund Chapel was chosen. The company used Rollei 3003 metric camera with calibrated 28mm lens for capturing the images. As a storing medium, photographic roll film Konica Color VX400 Film with dimensions 36 mm × 24 mm convenient for capturing scenes under lower light conditions was used [18]. In total, 7 images of the vault were captured, see Fig. 3, with an approximate image scale M = 477. To achieve a sufficient accuracy in height with photogrammetric processing, it is crucial to have a proper intersection angle between the rays of observation. Because of the height of the vault, it was not possible to get a better intersection angle than 30°, see Fig. 4. There was a presumption that because of the too acute intersection angle, the accuracy in axis Z will be worse than accuracies in axis X and Y . Most of the vaults were textureless and it would have not been possible to identify discrete survey points on the images. To have the ability to identify discrete survey points on the images and to reference Figure 3. One of the images used for processing in this paper. them and compute their coordinates, it was necessary to project a grid on the vault, see Fig. 5. The spacing between points of the grid was approximately 0.2m on the vault. Although 7 images of the vault were captured, the grid was applied only on 4 images. In the cathedral, geodetic net was created. Control points on the vaults were measured by polar method from at least two different stations of the net and the resulted position of the control points was calculated as the mean. The geodetic net was connected to the Czech coordinate reference system and height reference system. The negatives of captured images were printed and digitized. Using manually selected control and tie points, relative orientation of the images was computed. Then, the bundle adjustment of the model was executed. From the adjusted model with the absolute orientation, it was possible to identify discrete survey points on the projected grid and compute their coordinates. The result of the photogrammetric documentation in 1999 was a hypsometric model created from the survey points measured in software Close-Range Digital Workstation (CDW). 3. Digitizing and preprocessing The negatives of Rollei metric images were digitized on photographic film scanner Nikon Super Coolscan 8000 ED. The scanner is equipped with holder, which makes the negative flat and has the declared highest optic resolution of 4000 dpi. The true resolution testing on USAF1951 target [19] showed that even if the flatbed scanners manufacturers declare a high optical resolution (sometimes even more than 4000 dpi) the effective resolution is much lower (barely more than 2000 dpi). On the contrary, tested scanners of Nikon Super Coolscan series reach an effective resolution of more than 3500 dpi when scanning on the 4000 dpi option. In the paper, four different scanning resolutions were chosen to compare the results to each scanning 290 vol. 60 no. 4/2020 Structure from motion processing of analogue images. . . Figure 4. Position of images related to the vault. Figure 5. Grid projected on the vault in detail. resolution and to determine which scanning resolution should have been chosen in this particular case. These different scanning resolutions were chosen after the discussion about the quality of image negatives and their resolving power. There was rough estimation by the authors of the paper, based on the resolving power and contrast of images, that advantageous scanning resolution should have been set around 2000 dpi. But since it was not possible to estimate it precisely, 4 different types of resolution were chosen: • 1000 dpi • 2000 dpi • 3000 dpi • 4000 dpi (highest possible) Scanning Pixel size Ground Sample resolution of image Distance 1000 dpi 0.0254 mm 12 mm/pix 2000 dpi 0.0127 mm 6 mm/pix 3000 dpi 0.0085 mm 4 mm/pix 4000 dpi 0.0064 mm 3 mm/pix Table 1. Parameters of scanned images. The testing of SfM processing in the following chapters shows if there was a benefit (in terms of accuracy) of scanning the negatives on the highest scanning resolution or if a lower scanning resolution would be enough. Ground sample distance (GSD) on vault has been calculated for each scanning resolution (Tab. 1). The scanning on the highest resolution was time consuming and many prints of dust on the digitized images were noticeable. Even though the images were scanned under the same color palette, some of the images have slightly different tone than others. This could also be caused by different aging of each negative. For the transformation of images with reference points of réseau plate, affine, bilinear or grid-wise transformation is suitable [20]. For simplicity, affine transformation was used in our case. The scanned images had to be transformed to have the same image coordinate system with the same origin for each image. Reference points of the réseau plate on the images were used for the transformation. Their positions were calibrated in laboratory by manufacturer and are stated in the calibration report. Average residuals of 291 A. Dlesk, K. Vach, K. Pavelka Acta Polytechnica Group of images Average residual [mm] 1000 dpi 0.0061 2000 dpi 0.0047 3000 dpi 0.0046 4000 dpi 0.0044 CDW 0.0040 Table 2. Average residuals on points used for affine transformation (Image scale M = 477). the affine transformation on reference points are stated in the table Tab. 2. For photogrammetric processing, the images were cropped by a rectangle. After the cropping, all the images of one group had the same image resolution. Digitized images are missing their EXIF file and information about the image important for the SfM software. The software needs the information about the focal length of the camera and physical size of the sensor of the camera. In the case of our dataset, the information about the camera was ascertainable from the calibration report of the camera. The information was inserted into the EXIF file of each image. To avoid potentially disturbing objects in images for photogrammetric processing (like reference points of réseau plate or other descriptive information on the images), the binary bitmaps as masks were created and implemented on each corresponding image before the start of the processing. 4. Processing in CDW The original processing of images in 1999 was carried out in software Close-Range Digital Workstation (CDW). For this paper, the original processing was repeated. While original processing was done with images created by digitizing of prints of negatives, nowadays, for purposes of this paper, digitized negatives of images were processed. For processing in CDW, images with 4000 dpi scanning resolution were used. The results of the processing in CDW for the highest scanning resolution images were considered as a reference results in this paper. All the results from the SfM processing were compared to them. For the processing in CDW, it was necessary to insert initial values of the interior orientation of the camera. The interior orientation parameters were read in the calibration report. Then, the reference points of réseau plate were semi-automatically digitized. Some of the reference points were not possible to detect, because they were in the dark or over exposed background and were not visible. The average accuracy of the transformation was 0.004 mm. Unfortunately, it was not possible to determine the type of the transformation used in the CDW software due to the missing documentation to the software. The control points were manually marked on each image. Then, the tie points were marked as well. The tie points were chosen for their conspicuousness in images and convenient distribution. The coordinates of control points were imported to the CDW and the rough position and rough rotation was computed using Single image function. After the Single image computation, the images were computed together, and an absolute orientation has been computed. The model was adjusted using bundle adjustment and the parameters of accuracy were checked and can be read in Chap. 5.1. For bundle adjustment, the weight between the accuracy of point marking and accuracy of coordinates of control points was set the same as it was in processing in Agisoft PhotoScan and MicMac. At the adjusted model, the coordinates of discrete survey points were measured on images. 313 survey points on the projected grid on the vault were measured. All the survey points lay on the vault and are used as check points for accuracy checking in this paper. 5. SfM processing — Relative and absolute orientation Agisoft PhotoScan as a commercial alternative and MicMac as a free and open-source alternative were chosen for the SfM processing of images. Four groups of images (ordered by the scanning resolution) were computed separately. It was important to know at least approximate focal length and size of the images. The processing without known focal length or with very different focal length could cause wrong relative orientation, or the computation of the relative orientation would not even be successful. Focal length was the same for all images of the dataset. The relative orientation of all 4 groups of images was computed in both software programs. A full resolution of images was used for the tie point detection. It means that, in the function Align Photos in Agisoft PhotoScan, the parameter accuracy as set to High and in MicMac, processing parameter −1 was set. All the images of all groups were successfully co-registered. According to the Fig. 6, the highest number of tie points of the relative orientation, for both software programs, was while processing of images with the scanning resolution 3000 dpi. Most of the tie points were at the position of the projected grid on the vault. The absolute orientation was calculated. For the transformation of the relative model, weights between the accuracy of marking of control points (in pixels) and accuracy of geodetically measured control points must be set . According to the reports from geodetic measurements, the accuracy of the control points coordinates was estimated as 0.02m. The accuracy of the marking of control points was set as 0.0254mm on a physical image, which corresponds to the 1 pixel in group 292 vol. 60 no. 4/2020 Structure from motion processing of analogue images. . . Agisoft PhotoScan MicMac CDW 1000 2000 3000 4000 1000 2000 3000 4000 4000 dpi dpi dpi dpi dpi dpi dpi dpi dpi RMSEx [m] 0.0078 0.0035 0.0030 0.0026 0.0134 0.0057 0.0073 0.0102 0.0029 RMSEy [m] 0.0066 0.0054 0.0048 0.0047 0.0171 0.0046 0.0060 0.0102 0.0038 RMSEz [m] 0.0091 0.0095 0.0099 0.0062 0.0237 0.0096 0.0150 0.0149 0.0075 RMSExy [m] 0.0102 0.0064 0.0056 0.0054 0.0217 0.0073 0.0094 0.0144 0.0048 RMSExyz [m] 0.0137 0.0114 0.0114 0.0082 0.0321 0.0121 0.0177 0.0207 0.0089 Table 3. RMSE values of control points after bundle adjustment. Figure 6. Number of tie points computed by SfM processing. of images scanned under 1000 dpi scanning resolution. The same weights were set in the processing in the CDW. The bundle adjustment was computed. By that step, the positions of optical centres and rotation angles of images were computed and remained unchanged. The accuracy of the absolute orientation is given by Root mean square error RMSE. According to the Tab. 3, in the case of Agisoft PhotoScan, there is a trend of decreasing RMSE with increasing scanning resolution. All the lowest RMSE are at 4000 dpi scanning resolution. MicMac gave the lowest RMSE at 3000 dpi. After the bundle adjustment, the residuals of image coordinates of control points and tie points were checked and compared to the residuals from the CDW processing. In Agisoft PhotoScan and MicMac, the residuals are given in pixels and then recalculated to millimeters, see Fig. 7. 5.1. Accuracy checking on check points After the absolute orientation and bundle adjustment, 100 survey points were measured on images in Agisoft PhotoScan and MicMac. Both software programs support to measure discrete survey points. The discrete point surveying could be used for points which are very difficult or impossible to identify in subsequent point cloud. The 100 survey points were placed on the projected grid on the vault. The grid Figure 7. Average residuals of tie points in SfM processing (Image scale M = 477). was projected only on 4 images, so the survey points were marked on 4 images and their coordinates were computed. It is presumed that the accuracy of the model is given by the highest possible accuracy of measurement in the given model. To check the accuracy, the measured survey points were used as check points and were compared to the reference coordinates computed in a CDW. The accuracy checking on measured check points is very important and should be carried out in every project to evaluate the processed absolute orientation. For each group of the scanning resolution, differences ∆X, ∆Y and ∆Z between the corresponding check points computed in CDW and in Agisoft PhotoScan and between the corresponding points computed in CDW and in MicMac were calculated. The differences were analysed for each group of the scanning resolution (1000 dpi, 2000 dpi, 3000 dpi, 4000 dpi). Statistical values of set of differences are presented, see Tab. 4 and Tab. 6 and RMSE of check points are presented in Tab. 5 and Tab. 7. The sets of differences ∆X, ∆Y and ∆Z have been tested if they have a normal distribution. The testing of normal distribution has been carried out by Shapiro-Wilk test of normality with a 95% confidence level (see Tab. 4 and Tab. 6) and by 293 A. Dlesk, K. Vach, K. Pavelka Acta Polytechnica Figure 8. The normal Q − Q plots of values of differences ∆X, ∆Y and ∆Z between check point coordinates determined by distinct point measurement in Agisoft PhotoScan and coordinates given by CDW processing. Agisoft PhotoScan Agisoft PhotoScan Agisoft PhotoScan Agisoft PhotoScan 1000 dpi 2000 dpi 3000 dpi 4000 dpi ∆X ∆Y ∆Z ∆X ∆Y ∆Z ∆X ∆Y ∆Z ∆X ∆Y ∆Z Mean [m] 0.0003 -0.0036 0.0011 -0.0044 1.11e − 6 -0.0061 -0.0043 0.0006 -0.0080 -0.0043 1.77e − 5 -0.0043 Median [m] 0.0002 -0.0035 0.0001 -0.0045 −3.3e − 5 -0.0054 -0.0042 0.0007 -0.0086 -0.0045 0.0001 -0.0047 Std. Deviation [m] 0.0029 0.0028 0.0121 0.0024 0.0025 0.0096 0.0023 0.0024 0.0077 0.0020 0.0021 0.0074 Range [m] 0.0148 0.0141 0.0596 0.0114 0.0121 0.0602 0.0116 0.0125 0.0379 0.0123 0.0101 0.0338 Shap. - Wilk 0.291 0.763 0.696 0.238 0.939 0.161 0.965 0.754 0.746 0.552 0.835 0.436 Table 4. Statistical values of differences ∆X, ∆Y and ∆Z between check points coordinates determined by distinct point measurement in Agisoft PhotoScan and coordinates given by CDW processing. Agisoft PhotoScan 1000 dpi 2000 dpi 3000 dpi 4000 RMSEx [m] 0.0029 0.0050 0.0049 0.0048 RMSEy [m] 0.0046 0.0024 0.0025 0.0021 RMSEz [m] 0.0121 0.0113 0.0111 0.0085 RMSExy [m] 0.0054 0.0056 0.0055 0.0052 RMSExyz [m] 0.0133 0.0126 0.0124 0.0100 Max.euq.dist. [m] 0.0323 0.0412 0.0294 0.0218 Table 5. RMSE values of check points measured by distinct point measurement in Agisoft PhotoScan compared to the coordinates given by CDW. 294 vol. 60 no. 4/2020 Structure from motion processing of analogue images. . . Figure 9. The normal Q − Q plots of values of differences ∆X, ∆Y and ∆Z between check point coordinates determined by distinct point measurement in MicMac and coordinates given by CDW processing. MicMac MicMac MicMac MicMac 1000 dpi 2000 dpi 3000 dpi 4000 dpi ∆X ∆Y ∆Z ∆X ∆Y ∆Z ∆X ∆Y ∆Z ∆X ∆Y ∆Z Mean [m] -0.0023 -0.0092 -0.0251 -0.0072 -0.0009 0.0146 -0.0015 -0.0014 0.0073 -0.0044 -0.0013 0.0043 Median [m] -0.0030 -0.0114 -0.0271 -0.0086 -0.0017 0.0137 -0.0023 -0.0021 0.0081 -0.0043 -0.0028 0.0030 Std. Deviation [m] 0.0066 0.0077 0.0157 0.0049 0.0043 0.0097 0.0071 0.0066 0.0081 0.0062 0.0077 0.0101 Range [m] 0.0289 0.0315 0.0666 0.0222 0.0208 0.0495 0.0307 0.0301 0.0403 0.0261 0.0320 0.0561 Shap. - Wilk 0.059 0.000 0.008 0.000 0.034 0.380 0.008 0.474 0.562 0.066 0.011 0.002 Table 6. Statistical values of differences ∆X, ∆Y and ∆Z between check point coordinates determined by distinct point measurement in MicMac and coordinates given by CDW processing. MicMac 1000 dpi 2000 dpi 3000 dpi 4000 dpi RMSEx [m] 0.0070 0.0088 0.0072 0.0076 RMSEy [m] 0.0120 0.0044 0.0067 0.0077 RMSEz [m] 0.0296 0.0174 0.0108 0.0109 RMSExy [m] 0.0139 0.0098 0.0098 0.0108 RMSExyz [m] 0.0327 0.0199 0.0146 0.0154 Max.euq.dist. [m] 0.0534 0.0439 0.0265 0.0407 Table 7. RMSE values of check points measured by distinct point measurement in MicMac compared to the coordinates given by CDW. 295 A. Dlesk, K. Vach, K. Pavelka Acta Polytechnica Figure 10. Distribution of check points on the vault. quantile-quantile plots (Q − Q plot), see Fig. 8 and Fig. 9. In the case of processing in Agisoft PhotoScan, all the datasets of differences had normal distribution. Shapiro-Wilk tests resulted with relatively confident p-values (lowest p-value = 0.161). Q-Q plots also show that the datasets have normal distribution even with some observed outliers. It has to be noted that the means of ∆X are not equal to means of ∆Y for all scanning resolutions. For scanning resolutions 2000 dpi, 3000 dpi and 4000 dpi, the Q − Q plots seem identical. The trend lines of ∆X and trend lines for ∆Y are more or less parallel. The trend lines of ∆Y are shifted approximately by −4mm in Y axis on the graphs. The Tab. 5 shows that RMSEx and RMSEy are not equal for all four scanning resolutions. However, std. deviations of ∆X and ∆Y are more or less equal, see Tab. 4. RMSEz is significantly higher as was expected by the presumption about too acute intersection angle in the Z axis. The planimetric accuracy represented by RMSExy is around 5mm for all scanning resolutions. RMSExyz is decreasing with a higher scanning resolution. The RMSExyz is 0.01m for the 4000 dpi scanning resolution. Maximal computed Euqlidian distance between the check points calculated in CDW and in Agisoft PhotoScan for the 4000 dpi scanning resolution is 0.021m. All these facts indicate that the relative and absolute orientation computed by the SfM method in Agisoft PhotoScan is relatively accurate and the method is applicable for processing digitized close-range analogue images captured by Rollei metric cameras. Also, the analysis showed that most of the statistical values e.g. mean or range show the best accuracy at the highest 4000 dpi scanning resolution. In the case of MicMac, calculated differences ∆X, ∆Y and ∆Z between 100 points computed from CDW and MicMac does not predominantly have normal distribution. P-values computed by Shapiro-Wilk test with 95% confidence level are, in most cases, lower than 0.05. The Q − Q plots confirmed the results of Shapiro-Wilk testing. As it could be seen in Fig. 9, the observations do not fit the trend lines and several outliers are apparent. RMSExyz of datasets decreases over higher scanning resolution except for the 4000 dpi scanning (approximately similar to the 3000 dpi scanning). Maximal computed Euclidean distance between a point calculated in CDW and in MicMac for the 3000 dpi scanning resolution is 0.027 m. In case of the 4000 dpi, the maximal Euclidean distance is 0.041m due to the two significant outliers, which could be seen in Q − Q plot for 4000 dpi (Fig. 9). The analysis of results of processing in MicMac shows that the accuracy is lower than the accuracy in Agisoft PhotoScan, but even in the case of processing in MicMac, it was possible to get a respectable accuracy of the relative and absolute orientation. 6. Interior orientation Part of the SfM processing is a definition of the interior orientation of images. The interior orientation was computed by self-calibration. Considered parameters of the interior orientation in this paper are: • Principal distance • Principal point • Parameters K1 , K2 and K3 of polynomial function describing radial distortion on images The Lens of Rollei metric cameras are interchangeable and focusable so there was a presumption that the principal distance and position of the principal point calculated by self-calibration could be different to the values stated in the calibration report. However, the values determined by the self-calibration should not differ between each other. Very similar results for all scanning resolutions were computed by self-calibration in Agisoft PhotoScan. However, values computed in MicMac are not similar to each other, see Tab. 8. The check of the computed principal distance and position of principal point showed reliability of self-calibration in Agisoft PhotoScan. In this paper, radial distortion is considered as ideally symmetrical. The comparison of radial distortions calculated from the self-calibration are stated in Fig. 11 and Fig. 12 where they are compared to the radial distortion stated in calibration report. In the case of Agisoft PhotoScan the radial distortion is more or less the same for all scanning resolutions and also fits to the radial distortion stated in the calibration report. Curves of the radial distortion of MicMac differ more. The largest difference at 15mm from the image centre is around 0.14mm on the image. Because the majority of the tie and check points were within 10mm, in Fig. 11 and 296 vol. 60 no. 4/2020 Structure from motion processing of analogue images. . . f xp yp [mm] [mm] [mm] Calibration report 28.76 -0.14 -0.14 PhotoScan 1000 dpi 28.80 -0.10 -0.02 PhotoScan 2000 dpi 28.85 -0.07 -0.02 PhotoScan 3000 dpi 28.81 -0.11 0.01 PhotoScan 4000 dpi 28.82 -0.06 -0.03 MicMac 1000 dpi 29.40 -0.18 0.26 MicMac 2000 dpi 28.51 0.25 -0.11 MicMac 3000 dpi 28.43 -0.16 0.12 MicMac 4000 dpi 29.51 -0.27 -0.51 Table 8. Values of principal distance and principal point determined by self-calibration of each processing (Image scale M = 477). Figure 11. Radial distortion determined by self-calibration in Agisoft PhotoScan (Image scale M = 477). Fig. 12 the curves of radial distortion are presented just until 15mm. In this case, it would not be relevant to present resulted radial distortion curves until the image corner. The results of the self-calibration in Agisoft PhotoScan do not differ much from each other and thus seem to be more reliable. 7. Point cloud generation The subsequent result of SfM processing is a point cloud. The point cloud is generated based on multi-view stereo matching or pair-wise stereo matching algorithms. The point cloud is obtained by Build Dense Cloud function in Agisoft PhotoScan and C3DC command in MicMac. The important parameter for the point cloud generation is the usable resolution on images (parameter Quality in Agisoft PhotoScan and ZoomF in MicMac). As a first attempt, the full resolution of images has been used for the point cloud generation. Agisoft PhotoScan generated the densest point cloud with the most points with full resolution parameter of images at the highest 4000 dpi scanning resolution. MicMac generated the point cloud with the highest number of points at Figure 12. Radial distortion determined by self-calibration in MicMac (Image scale M = 477). the 4000 dpi scanning resolution with a full resolution parameter as well. But in the case of MicMac, the points were not equally distributed at all. There were very large parts with missing points. Similar situation occurred in the case of 2000 dpi and 3000 dpi scanning resolution. Those point clouds would not be possible to use for further processing and modeling. To have a reasonable point cloud, which could possibly be used for further modelling and for analysis in this paper, it was necessary to set a lower used resolution as a parameter for the point cloud generation in Agisoft PhotoScan and in MicMac. A parameter, which each side of image resolution divides by 4, had to be used. All generated point clouds were compared to the 313 survey points computed in the CDW by Cloud-to-cloud method. For each point of 313 survey points computed in CDW, the nearest corresponding point in the generated point cloud was found. Then, the differences ∆X, ∆Y and ∆Z were computed between the corresponding points. The normal distribution of sets of differences were tested by Shapiro-Wilk test of normality with a 95% confidence level and by quantile-quantile plots (Q − Q plot). In the case of point clouds generated in Agisoft PhotoScan, p-values computed by Shapiro-Wilk test showed a sufficient confidence except for sets ∆X and ∆Y at the point cloud generated from the 1000 dpi scanning resolution images, see Tab. 9. The Q − Q plots show few outliers in each sets but most of the observations fit to the trend line of Q − Q plot. It has to be noted that trend lines of Q − Q plots of ∆X and ∆Y seem for all sets approximately equal, unlike the trend lines in Q − Q plots in relative and absolute orientation accuracy analysis (Chap. 5.1.). Also RMSEx and RMSEy (see Tab. 10) are almost equal, unlike Tab. 5. Std. deviations of ∆X and ∆Y are also similar to each other for all scanning resolution. RMSE, std. deviation and range show the increasing accuracy 297 A. Dlesk, K. Vach, K. Pavelka Acta Polytechnica Figure 13. The normal Q − Q plots of values of differences ∆X, ∆Y and ∆Z between point cloud generated in Agisoft PhotoScan and point coordinates given by CDW. Agisoft PhotoScan Agisoft PhotoScan Agisoft PhotoScan Agisoft PhotoScan 1000 dpi 2000 dpi 3000 dpi 4000 dpi ∆X ∆Y ∆Z ∆X ∆Y ∆Z ∆X ∆Y ∆Z ∆X ∆Y ∆Z Mean [m] 0.0012 0.0003 0.0025 -0.0004 0.0003 0.0003 -0.0001 -0.0003 -0.0007 6.6e − 5 -0.0002 -0.0021 Median [m] 0.0019 0.0006 0.0022 -0.0003 0.0008 −9.2e − 5 -0.0002 -0.0004 -0.0009 -0.0001 -0.0005 -0.0022 Std. Deviation [m] 0.0151 0.0138 0.0161 0.0073 0.0071 0.0079 0.0052 0.0052 0.0064 0.0044 0.0043 0.0059 Range [m] 0.0923 0.0679 0.1157 0.0416 0.0453 0.0482 0.0317 0.0341 0.0441 0.0260 0.0256 0.0321 Shap. - Wilk 0.051 0.010 0.527 0.122 0.480 0.179 0.824 0.864 0.735 0.387 0.228 0.407 Table 9. Statistical values of differences ∆X, ∆Y and ∆Z between point cloud generated in Agisoft PhotoScan and point coordinates given by CDW. Agisoft PhotoScan 1000 dpi 2000 dpi 3000 dpi 4000 dpi RMSEx [m] 0.0151 0.0073 0.0052 0.0044 RMSEy [m] 0.0138 0.0071 0.0052 0.0043 RMSEz [m] 0.0162 0.0079 0.0064 0.0062 RMSExy [m] 0.0205 0.0102 0.0073 0.0061 RMSExyz [m] 0.0261 0.0129 0.0097 0.0087 Max.euq.dist. [m] 0.0640 0.0280 0.0256 0.0191 Table 10. RMSE values of differences between point cloud generated in Agisoft PhotoScan and point coordinates given by CDW. 298 vol. 60 no. 4/2020 Structure from motion processing of analogue images. . . Figure 14. The normal Q − Q plots of values of differences ∆X, ∆Y and ∆Z between point cloud generated in MicMac and point coordinates given by CDW. MicMac MicMac MicMac MicMac 1000 dpi 2000 dpi 3000 dpi 4000 dpi ∆X ∆Y ∆Z ∆X ∆Y ∆Z ∆X ∆Y ∆Z ∆X ∆Y ∆Z Mean [m] -0.0012 0.0015 0.0114 -0.0001 -0.0014 -0.0191 -0.0005 -0.0007 -0.0114 -0.0007 -0.0012 -0.0050 Median [m] -0.0014 0.0021 0.0132 0.0013 0.0004 -0.0185 -0.0002 0.0005 -0.0118 0.0002 -0.0005 -0.0048 Std. Deviation [m] 0.0212 0.0216 0.0279 0.0175 0.0198 0.0191 0.0093 0.0093 0.0106 0.0064 0.0066 0.0083 Range [m] 0.1722 0.1801 0.1827 0.1012 0.1120 0.0999 0.0556 0.0609 0.0563 0.0429 0.0472 0.0539 Shap. - Wilk 0.003 0.007 0.036 0.023 0.022 0.566 0.428 0.000 0.194 0.000 0.000 0.582 Table 11. Statistical values of differences ∆X, ∆Y and ∆Z between point cloud generated in MicMac and point coordinates given by CDW. MicMac 1000 dpi 2000 dpi 3000 dpi 4000 dpi RMSEx [m] 0.0212 0.0175 0.0093 0.0064 RMSEy [m] 0.0216 0.0198 0.0093 0.0067 RMSEz [m] 0.0301 0.0270 0.0155 0.0097 RMSExy [m] 0.0303 0.0264 0.0132 0.0093 RMSExyz [m] 0.0427 0.0378 0.0204 0.0134 Max.euq.dist. [m] 0.1202 0.0845 0.0425 0.0359 Table 12. RMSE values of differences between point cloud generated in MicMac and point coordinates given by CDW. 299 A. Dlesk, K. Vach, K. Pavelka Acta Polytechnica Figure 15. Density of point cloud at different scanning resolution (parameter ZoomF = 4). with increasing scanning resolution, see Tab. 9 (the best results are at the 4000 dpi scanning resolution). Maximal Euclidean distance between corresponding points at the 4000 dpi scanning resolution is 0.019m. Differences ∆X, ∆Y and ∆Z between corresponding points of point clouds generated by MicMac and survey points in CDW are not, in most of the cases, normally distributed according to the Shapiro-Wilk test, see Tab. 11. The Q − Q plots show that most of the observations fit the trend lines but many outliers were observed. The analysis of point clouds generated by MicMac showed more present outliers than the point clouds generated by Agisoft PhotoScan. RMSEx and RMSEy are also more or less equal like in the case of the point clouds generated in Agisoft PhotoScan. Std. deviations of sets of ∆X and ∆Y are also very similar to each other. The maximal Euclidean distance is decreasing with increasing scanning resolution. The lowest maximal Euclidean distance 0.036m was observed in the point cloud generated in MicMac using images with the 4000 dpi scanning resolution. Few other statistical parameters show the best accuracy at 4000 dpi scanning resolution, see Tab. 11 and Tab. 12. Other results than point cloud could be textured triangulated model, orthophoto or DEM in raster form. For hypsometric model of vault, just the point cloud would be sufficient as the result. The generated point cloud can be seen at Fig. 4 or Fig. 15. 8. Conclusion The paper shows SfM processing of digitized analogue images captured by Rollei metric camera. The images were originally used for a different method of photogrammetry. The images were originally created for the purpose of photogrammetric documentation of vault of Zikmund’s chapel in Saint Vitus Cathedral. Nowadays, with the SfM processing, it is theoretically possible to get new and valuable 2D and 3D results. This paper shows the method how to digitize the analogue negatives, how to preprocess them and how to process them using the SfM method. As a software for SfM processing, Agisoft PhotoScan was chosen as a commercial alternative and MicMac as a free and open-source alternative. Then, the results were compared to the results in the CDW, by which the original processing was carried out.. Results from the CDW were considered as the reference. 300 vol. 60 no. 4/2020 Structure from motion processing of analogue images. . . Before the processing, it is necessary to deal with digitizing original negatives and with preprocessing, in order to obtain accurate results from the SfM processing. Seven negatives of images were chosen for the experiment. According to the resolving power, four different scanning resolutions were chosen - 1000 dpi, 2000 dpi, 3000 dpi and 4000 dpi. The digitized images were processed in Agisoft PhotoScan and in MicMac by the SfM method and relative and absolute orientation was computed. The relative and absolute orientation was checked on 100 check points computed in the CDW by the original method. The differences ∆X, ∆Y and ∆Z between corresponding check points were computed. In the case of Agisoft PhotoScan, the differences between check points in all axes had normal distribution, unlike differences computed in MicMac. According to the showed results, it is possible to conclude that Agisoft PhotoScan gave better results than MicMac in terms of accuracy. The resulted comparison showed lower RMSE, lower maximal Euclidian distance, lower range and lower std. deviation, etc. Despite that fact, MicMac was also able to compute relatively accurate relative and absolute orientation and the results could be sufficient for some purposes. Also, the analysis showed that with increasing scanning resolution, it is possible to get more accurate results. Parameters of interior orientation were determined by self-calibration by processing in Agisoft PhotoScan and MicMac. Resulted parameters in Agisoft PhotoScan were very similar for all scanning resolutions and it shows the reliability of processing. Results of interior orientation in MicMac differ to each other, some of them significantly. Curves of radial distortion computed in Agisoft PhotoScan fit to each other and to the radial distortion stated in calibration report. Curves of radial distortion computed in MicMac differ by approximately 0.14mm on image at 15mm distance from the image center. As one of the results of processing, point clouds were generated in both software programs for each scanning resolution. The point clouds were compared to the 313 survey points calculated in the CDW. For the comparison, cloud-to-cloud method was used and the differences ∆X, ∆Y and ∆Z between the points were analysed. The analysis of accuracy showed that Agisoft PhotoScan is a sufficient tool for point cloud generation based on digitized analogue images. Most of the differences in all three axis were normally distributed according to the Shapiro-Wilk tests and to the Q−Q plots. In the case of point clouds generated in MicMac, most of the differences were not normally distributed according to the Shapiro-Wilk tests. Most of the observed values fit to the trend lines of the Q − Q plot, which could be characteristic for the fact that the observations are normally distributed, but a lot of outliers, which could cause the results of Shapiro-Wilk tests, were presented. It could be stated that the chosen scanning resolution of images truly affects the accuracy of results. Almost all the results of analysis showed that with increasing scanning resolution we get better accuracy. The biggest differences were between the processing of images with a lower scanning resolution. It can be stated that scanning with the highest scanning resolution brought better results. However, one should consider that the higher resolution can be considerably more time consuming. So a compromise between the accuracy and the time should be made while dealing with a large set of images. The experiment in this paper showed that it is possible to use the SfM method for a processing of digitized analogue images captured by Rollei metric camera and originally used for a different method of photogrammetric processing. Agisoft PhotoScan gave more accurate and reliable results. Differences in coordinates between corresponding check points calculated by the SfM processing and by the CDW were almost in all cases normally distributed. In the case of MicMac, most of the differences were not normally distributed, possibly because of several outliers in the observations. Nevertheless, for some kind of purposes, even the results from MicMac could be sufficient. With respect to the fact that MicMac is free, MicMac could be an option for SfM processing of digitized analogue images captured by Rollei metric cameras as well. Acknowledgements This project was supported by grant of the Grant Agency of the Czech Technical University in Prague, grant No. SGS20/053/OHK1/1T/11. This project is a part of internal research of EuroGV s.r.o. company. References [1] A. Dlesk, P. Raeva, K. Vach. Possibilities of processing archival photogrammetric images captured by Rollei 6006 metric camera using current method. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2:319 – 323, 2018. doi:10.5194/isprs-archives-XLII-2-319-2018. [2] Z. Poloprutský, M. Fraštia, M. Marčiš. 3D digital reconstruction based on archived terrestrial photographs from metric cameras. Acta Polytechnica 59(4):384 – 398, 2019. doi:10.14311/AP.2019.59.0384. [3] Á. Rodríguez Miranda, J. M. Valle Melón. Recovering old stereoscopic negatives and producing digital 3D models of former appearances of historic buildings. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2/W3:601 – 608, 2017. doi:10.5194/isprs-archives-XLII-2-W3-601-2017. [4] K. Pavelka, J. Šedina, P. Raeva, M. Hůlková. Modern processing capabilities of analog data from documentation of the Great Omayyad mosque in Aleppo, Syria, damaged in civil war. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2/W5:561 – 565, 2017. doi:10.5194/isprs-archives-XLII-2-W5-561-2017. 301 http://dx.doi.org/10.5194/isprs-archives-XLII-2-319-2018 http://dx.doi.org/10.14311/AP.2019.59.0384 http://dx.doi.org/10.5194/isprs-archives-XLII-2-W3-601-2017 http://dx.doi.org/10.5194/isprs-archives-XLII-2-W5-561-2017 A. Dlesk, K. Vach, K. Pavelka Acta Polytechnica [5] L. J. Vargo, B. M. Anderson, H. J. Horgan, et al. Using structure from motion photogrammetry to measure past glacier changes from historic aerial photographs. Journal of Glaciology 63(242):1105 – 1118, 2017. doi:https://doi.org/10.1017/jog.2017.79. [6] M. V. Peppa, J. P. Mills, K. D. Fieber, et al. Archaeological feature detection from archive aerial photography with a sfm-mvs and image enhancement pipeline. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2:869 – 875, 2018. doi:10.5194/isprs-archives-XLII-2-869-2018. [7] F. Maiwald, T. Vietze, D. Schneider, et al. Photogrammetric analysis of historical image repositories for virtual reconstruction in the field of digital humanities. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2/W3:447 – 452, 2017. doi:10.5194/isprs-archives-XLII-2-W3-447-2017. [8] M. Cogliati, E. Tonelli, D. Battaglia, M. Scaioni. Extraction of DEMS and orthoimages from archive aerial imagery to support project planning in civil engineering. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences IV-5/W1:9 – 16, 2017. doi:10.5194/isprs-annals-IV-5-W1-9-2017. [9] S. Giordano, A. Le Bris, C. Mallet. Toward automatic georeferencing of archival aerial photogrammetric surveys. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences IV-2:105 – 112, 2018. doi:10.5194/isprs-annals-IV-2-105-2018. [10] T. Luhmann, S. Robson, S. Kyle, L. Harley. Close Range Photogrammetry: Principles, Methods and Applications. Whittles Publishing, 2006. [11] G. Suilmann. Rolleiflex 6008 metric - a new camera for industrial photogrammetry. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XXIX-Part B5:60 – 63, 1992. [12] G. Pomaska. Image acquisition for digital photogrammetry using off the shelf and metric cameras. Proceedings of the 18th International Symposium CIPA pp. 490 – 495, 2001. [13] A. Streilein, H. Beyer, T. Kersten. Digital photogrammetric techniques for architectural design. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XXIX-B5:825 – 831, 1992. [14] Agisoft. Agisoft Photoscan user manual, professional edition, version 1.4. http: //www.agisoft.com/pdf/photoscan-pro_1_4_en.pdf. Accessed: 15 July 2020. [15] Micmac, apero, pastis and other beverages in a nutshell! http://www.github.com/micmacIGN/ Documentation/blob/master/DocMicMac.pdf. Accessed: 15 July 2020. [16] E. Rupnik, M. Daakir, M. P. Deseilligny. Micmac–a free, open-source solution for photogrammetry. Open Geospatial Data, Software and Standards 2(1):1 – 9, 2017. doi:10.1186/s40965-017-0027-2. [17] J. M. Friedt. Photogrammetric 3D structure reconstruction using MicMac. http://jmfriedt.free.fr/lm_sfm_eng.pdf. Accessed: 15 July 2020. [18] Technical data sheet, konica color vx400 film. http: //www.photoweb.ru/exusr/pdf/konica/vx400.pdf. Accessed: 15 July 2020. [19] F. S. Patrick Wagner. Film scanner test-reports: complete overview of all the models. http://www.filmscanner.info/en/ FilmscannerTestberichte.html. Accessed: 15 July 2020. [20] R. Kotowski, B. Weber. A procedure for on-line correction of systematic errors. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XXV-A3:553 – 560, 1984. 302 http://dx.doi.org/https://doi.org/10.1017/jog.2017.79 http://dx.doi.org/10.5194/isprs-archives-XLII-2-869-2018 http://dx.doi.org/10.5194/isprs-archives-XLII-2-W3-447-2017 http://dx.doi.org/10.5194/isprs-annals-IV-5-W1-9-2017 http://dx.doi.org/10.5194/isprs-annals-IV-2-105-2018 http://www.agisoft.com/pdf/photoscan-pro_1_4_en.pdf http://www.agisoft.com/pdf/photoscan-pro_1_4_en.pdf http://www.github.com/micmacIGN/Documentation/blob/master/DocMicMac.pdf http://www.github.com/micmacIGN/Documentation/blob/master/DocMicMac.pdf http://dx.doi.org/10.1186/s40965-017-0027-2 http://jmfriedt.free.fr/lm_sfm_eng.pdf http://www.photoweb.ru/exusr/pdf/konica/vx400.pdf http://www.photoweb.ru/exusr/pdf/konica/vx400.pdf http://www.filmscanner.info/en/FilmscannerTestberichte.html http://www.filmscanner.info/en/FilmscannerTestberichte.html Acta Polytechnica 60(4):1–15, 2020 1 Introduction 1.1 Rollei Metric 1.2 SfM processing 2 Dataset 3 Digitizing and preprocessing 4 Processing in CDW 5 SfM processing — Relative and absolute orientation 5.1 Accuracy checking on check points 6 Interior orientation 7 Point cloud generation 8 Conclusion Acknowledgements References