Measuring Repeatability of the Focus-variable Lenses Jan Řezníček Laboratory of Photogrammetry, Department of Geomatics, Czech Technical University in Prague Thákurova 7, 166 29 Prague 6, Czech Republic reznicek33@centrum.cz Abstract In the field of photogrammetry, the optical system, usually represented by the glass lens, is used for metric purposes. Therefore, the aberration characteristics of such a lens, inducing deviations from projective imaging, has to be well known. How- ever, the most important property of the metric lens is the stability of its glass and mechanical elements, ensuring long-term reliability of the measured parameters. In case of a focus-variable lens, the repeatability of the lens setup is important as well. Lenses with a fixed focal length are usually considered as “fixed” though, in fact, most of them contain one or more movable glass elements, providing the focusing function. In cases where the lens is not equipped with fixing screws, the repeatability of the calibration parameters should be known. This paper derives simple mathematical formulas that can be used for measuring the repeatability of the focus-variable lenses, and gives a demonstrative example of such measuring. The given procedure has the advantage that only demanded parameters are esti- mated, hence, no unwanted correlations with the additional parameters exist. The test arrangement enables us to measure each demanded magnification of the optical system, which is important in close-range photogrammetry. Keywords: photogrammetry, camera, calibration, stability, repeatability, focus-variable lens 1. Introduction In the field of photogrammetry, the optical system, usually represented by the glass lens, is used for metric purposes. Therefore, the aberration characteristics of such a lens, inducing deviations from projective imaging, has to be well known. However, the most important property of the metric lens is the stability of its glass and mechanical elements, ensuring long-term reliability of the measured parameters. In the case of a focus-variable lens, the repeatability of the lens setup is important as well. Lenses with a fixed focal length are usually considered as “fixed” though, in fact, most of them contain one or more movable glass elements, providing the focusing function. In cases where the lens is not equipped with fixing screws, the repeatability of the calibration parameters should be known. 1.1. Related work Several papers have been published in recent years addressing the issue of the geometric stability of the camera calibration parameters. Shortis et al. (1997) describe the magnitude of instability of the calibration parameters of the Kodak DCS420 and 460 cameras when used Geoinformatics FCE CTU 13, 2014, doi:10.14311/gi.13.1 9 http://orcid.org/0000-0001-6279-1926 http://dx.doi.org/10.14311/gi.13.1 Řezníček. J: Measuring Repeatability of the Focus-variable Lenses as a metric system. Läbe and Förstner (2004) investigated the use of consumer grade cameras for photogrammetric applications in terms of their stability in time and usable functions, such as the zoom and auto focus. Shortis et al. (2006) compares the stability of the zoom vs. fixed focal lenses where the zoom lenses have been fixed with a piece of tape. Rieke-Zapp et al. (2009) evaluated the geometric stability and the accuracy potential of several fixed focal lenses, where some of them were fixed even in the focus mechanism. Sanz-Ablanedo et al. (2010) performed a comparison of the geometric stability of the interior orientation parameters (IOP) of six identical compact digital cameras. The measurement was performed in three model situations: during continuous use of the cameras, after the camera was powered off/on and after the full extension and retraction of the zoom-lens. All of the above mentioned works performed the measurement by computing the IOP, using the method of multi-pose network analytical calibration. The influence of different magnifications of each target on the calibration parameters is not covered and neither is the detailed correlation analysis between estimated parameters, which would give a more realistic view on the given results. 1.2. Motivation We have chosen to study the influence of the focusing mechanism play on the repeatability of the IOP of the camera, which is not covered in the previous works. In order to prevent the influence of the possible high correlations among IOP end EOP (exterior orientation parameters) we have decided to use a different, more rigorous, approach. It is also more suitable for analyzing additional systematic effects, as the method does not require an a priori given mathematical model describing camera’s internal geometry. The high correlations among parameters could cause an unreliable estimate. For example, the small change of the principal distance or the principal point can be reduced by a small change of the camera pose. 1.3. Interior orientation parameters Interior orientation parameters can be defined as “All characteristics that affect the geometry of the photograph” (Slama et al. 1980, p. 244). The most important IOP are given by: format (sensor) dimensions, principal distance, principal point position and lens distortion characteristics. Different types of cameras demand different characteristics. For example, the airborne cameras are usually assembled from much more components than the close- range cameras and therefore do requires much more parameters for the characterization of its interior geometry. Thus, depending on the camera type and demanding accuracy, additional characteristics could be needed: fiducials, axis scale, skew, reseau coordinates, point spread function (PSF), sensor unflatness characteristics, sensor noise characteristics, forward motion compensation characteristics, etc. In this paper, we focus only on the deviation in the location of the projection center (principal distance and the principal point position). 2. Procedure and test arrangement The measured lens is mounted to a digital camera body which has a fixed pose. The camera (we will use term camera for camera body with mounted lens) is directed toward the planar test field in such a way that the lens optical axis is perpendicular to the test field. The test field consists of several hundred (280) black circular dots (targets) printed on a white sheet Geoinformatics FCE CTU 13, 2014 10 Řezníček. J: Measuring Repeatability of the Focus-variable Lenses of paper in a regular interval. With such an arrangement, a pair of images is taken. Before the second shot, the lens is refocused either manually or by remote control. In the case of the remote control function, the contrast target has to be moving in front of the lens in order to drive the automatic focus function. Refocusing moves the optical elements inside the lens assembly, which cause the small deviations in repeatability of the focus setup. However, the position of the camera body should not be changed during image acquisition. Finally, the image coordinates of the target images are detected and referenced on both images. Figure 1: Test arrangement 3. Theoretical development In this section, we will describe the geometry of the situation and develop mathematical relations for it. The geometry of two marginal rays, corresponding to acquired pair of images, passing from the target (D) in object space to the sensor in the image space (E,F) is shown in Figure 2. Each ray is passing through the different projection center (A,B) of the same camera. The difference in the location of the projection center (denoted xB, yB, zB) simulates the error given by the refocusing of the lens. The goal is to express those differences in mathematic relations which could be enumerated by using images acquired by the procedure given above. Two coordinate systems are used here – local (denoted with superscript .loc) and global. The origin of the global coordinate system lies in the projection center A. The x-axis is vertical and directed towards the zenith, the z-axis lies in the optical axis of the camera and is directed towards the sensor, finally the y-axis is defined by the right-hand rule. The definition of the local coordinate system is same with one exception: The x and y component of the origin is shifted by values xR, yR, where R is a lower-right corner of the sensor (in the negative orientation). This definition allows us to read the image coordinates directly in the local coordinate system, because the pixels are organized in a matrix system (origin in the upper left corner) and because the orientation of the recorded image is positive (However, the Figure 2 shows the real negative orientation of the sensor). Notation aspects: The homogeneous (also called projective) system of coordinates will be used instead of Cartesian system of coordinates. Image point coordinates can be then defined in the same global coordinate system as the object point coordinates, which is unusual, but very Geoinformatics FCE CTU 13, 2014 11 Řezníček. J: Measuring Repeatability of the Focus-variable Lenses Figure 2: The geometry of two marginal rays practical. The z coordinate of every image point is therefore equal to the principal distance of cA. The projection of the object point D to the image points E and F can be written as Eloc = TA·D (1) F loc = TB·D (2) where D =   xD yD zD 1   , Eloc = zD ·   xlocE ylocE cA 1   , F loc = (zD − zB) ·   xlocF ylocF cA 1   (3) and TA = M2M1 =   cA 0 0 zD · xlocK 0 cA 0 zD · ylocK 0 0 cA 0 0 0 0 zD   . M1 represents the projection and M2 represents the transformation from global to local system M1 =   cA 0 0 0 0 cA 0 0 0 0 cA 0 0 0 0 zD   , M2 =   1 0 0 xlocK 0 1 0 ylocK 0 0 1 0 0 0 0 1   . Therefore xlocE = cA zD xD + xlocK , y loc E = cA zD yD + ylocK . Geoinformatics FCE CTU 13, 2014 12 Řezníček. J: Measuring Repeatability of the Focus-variable Lenses TB consists of a three matrices TB = N3N2N1 =   cB 0 0 xlocL (zD − zB) − cBxB 0 cB 0 ylocL (zD − zB) − cByB 0 0 cA −cAzB 0 0 0 zD − zB   where N1 represents the translation of the point D before projection, N2 represents the projection and N3 represents the transformation from global to local system N1 =   1 0 0 −xB 0 1 0 −yB 0 0 1 −zB 0 0 0 1  , N2 =   cB 0 0 0 0 cB 0 0 0 0 cA 0 0 0 0 zD − zB  , N3 =   1 0 0 xlocL 0 1 0 ylocL 0 0 1 0 0 0 0 1   . By combining equations (1) and (2), we get Eloc = TA · D = TA · T −1B · F loc = T · F loc (4) where T = TA · T −1B =   cA cB 0 0 zD ·x loc K (zD −zB ) − cA cB (xloc L (zD −zB )−cBxB ) (zD −zB ) 0 cA cB 0 zD ·y loc K (zD −zB ) − cA cB (yloc L (zD −zB )−cByB ) (zD −zB ) 0 0 1 cA zB(zD −zB ) 0 0 0 zD(zD −zB )   . (5) Therefore, according to (3), (4) and (5) zD ·   xlocE ylocE cA 1   = T · (zD − zB) ·   xlocF ylocF cA 1   which leads to   xlocE ylocE cA 1  =   cA cB (zD −zB ) zD 0 0 xlocK − cA cB (xloc L (zD −zB )−cBxB ) zD 0 cA cB (zD −zB ) zD 0 ylocK − cA cB (yloc L (zD −zB )−cByB ) zD 0 0 (zD −zB ) zD cA zB zD 0 0 0 1     xlocF ylocF cA 1   . For the third component cA we can write cA = cA (zD − zB) zD + cA zB zD which is after simplification zB zD = zB zD . Geoinformatics FCE CTU 13, 2014 13 Řezníček. J: Measuring Repeatability of the Focus-variable Lenses Therefore, only the first and second components constitute a system of independent equations which can be solved. Those two equations are given by xlocE = λ·x loc F + tx ylocE = λ·y loc F + ty (6) where λ = cA (zD − zB) zD (cA − zB) tx = xlocK − cA ( xlocK + xB ) (zD − zB) zD (cA − zB) + cAxB zD ty = ylocK − cA ( ylocK + yB ) (zD − zB) zD (cA − zB) + cAyB zD and cB = cA − zB xlocL = x loc K + xB ylocL = y loc K + yB. The system of equations given in (6) contain four measurements: xlocE , y loc E , x loc F , y loc F – repre- senting the image coordinates of the target corresponding to image A and B, three unknown parameters: xB, yB, zB – representing the deviation of the projection center, and four known constants: cA, xlocK , y loc K , zD. Because the test field consists of a several hundreds of targets, the system of non-linear equations is over-determined and needs to be solved by a proper estimator. One pair of images (A and B) gives one sample estimation of the projection center deviation. At least several tens of pairs have to be taken in order to get a reliable estimate and standard deviation of the projection center deviation x̄B = ∑n i=1 ∣∣xiB∣∣ n sxB = √∑n i=1 v 2 i n − 1 , where vi = ∣∣∣xiB∣∣∣ − x̄B and where n is the number of image pairs (same applies for yB and zB). 4. Demonstrative example The theoretical development, given in the previous section, is used in a following test. We have measured the repeatability of the focus mechanism of the Canon EF 40 mm F2.8 STM lens, mounted on a Canon 5D Mark II digital camera body (35 mm full-frame sensor). The whole procedure is described above. The test field consisted of 280 black circular dots (8 mm in diameter) and was placed in distance of 1230 mm from the camera sensor. Before each shot, the lens was refocused by using a remote control function. The focusing mode was Geoinformatics FCE CTU 13, 2014 14 Řezníček. J: Measuring Repeatability of the Focus-variable Lenses set to One-Shot AF mode. The target images were detected by using an ellipse fitting type of detector, using only the green channel in order to suppress the error given by chromatic aberration. The over-determined system of non-linear equations given in (6) has been solved by an estimator presented in Mikhail et al. (1976) and called “Adjustment with Conditions Only – General Case”. The program was written by the author of this paper in Matlab language. 4.1. Initial stability test The first set of images (101 = 100 pairs) have been taken with no refocusing or any other manipulation in order to confirm the stability of the test field and camera and to confirm the randomness of all processes (e.g. image pre-processing, target detector). The results are given in Table 1 and Figure 3. Table 1: Results based on n = 100 estimated values parameter nominal estimated standard [µm]value [mm] value [mm] deviation x̄B 40.0 39.999996 sxB 0.097 ȳB 12.0128 12.01280 syB 0.11 z̄B 18.0288 18.028820 szB 0.067 Conclusion: As can be seen from the Table 1, there is no significant change in the position of the projection center, which proves that no systematic error is present. The randomness is visible also in the Figure 3. 4.2. Focus repeatability test For the main focus repeatability test, 101 images have been taken in overall (this makes 100 pairs). The results of this test are given in Table 2 and Figure 4. Table 2: Results based on n = 100 estimated values parameter nominal estimated standard [µm]value [mm] value [mm] deviation x̄B 40.0 40.000 sxB 26.0 ȳB 12.0128 12.0128 syB 3.7 z̄B 18.0288 18.0288 szB 2.4 Conclusion: The results given in Table 2 show significant deviation of the projection center position induced by the refocusing operation. Figure 4, displaying the vector residuals (given by solving the system of equations) of two consequence images 94 and 95, shows no systematic pattern (same applies for all other image pairs also). This means that, for this particular case, no change of the lens distortion was measured (within the bounds of the measurement accuracy). Geoinformatics FCE CTU 13, 2014 15 Řezníček. J: Measuring Repeatability of the Focus-variable Lenses Figure 3: The vector differences between image coordinates of corresponding targets of two consequence images 94 and 95 (This pair was chosen randomly) Figure 4: The residual vectors between image coordinates of corresponding targets of two consequence images 94 and 95 (This pair was chosen randomly) Geoinformatics FCE CTU 13, 2014 16 Řezníček. J: Measuring Repeatability of the Focus-variable Lenses 5. Conclusion We have derived simple mathematical formulas, which can be used for measuring the repeata- bility of the focus-variable lenses. The procedure of such measuring has the advantage that only demanded parameters are estimated, hence, no unwanted correlations with the addi- tional parameters exist. The test arrangement enables the measuring under each demanded magnification of the optical system, which is important in close-range photogrammetry. The demonstrative example showed the error in repeatability, which can be modeled with a simple linear model. However, a more complicated, non-linear error progression can be modeled as well, without the need of a priori known model. Because the measured lens was not calibrated at the time of the test, we used nominal values of the principal distance and principal point position instead of the calibrated ones. The differences in the resulting values are so small, that, for the purpose of this test, it can be neglected. The results (standard deviation of the parameters) given by measuring repeatability have to be considered in planning photogram- metric project accuracy, as the corresponding estimates from a single camera calibration are not realistic (in long-term sense). Acknowledgement This work was supported by the Czech Ministry of Culture, under grant NAKI, no. DF13P01- OVV002 (New Modern Non-invasive Methods of Cultural Heritage Objects Exploration). References [1] Läbe T, Förstner W. Geometric stability of low-cost digital consumer cameras. In: Pro- ceedings of the XX ISPRS Congress, Commission I: 2004 Jul 12–23; Istanbul, Turkey. IAPRS 2004;35(Pt B1):528–34. [2] Mikhail EM, Ackermann F. Observations and Least Squares. New York: IEP–A Dun- Donnelley; 1976 [3] Rieke-Zapp D, Tecklenburg W, Peipe J, Hastedt H, Haig C. Evaluation of the geometric stability and the accuracy potential of digital cameras – Comparing mechanical stabili- sation versus parameterisation. ISPRS Journal of Photogrammetry and Remote Sensing 2009;64:248–58. [4] Sanz-Ablanedo E, Rodríguez-Pérez JR, Armesto J, Taboada MFA. Geometric Stability and Lens Decentering in Compact Digital Cameras. Sensors 2010;10:1553–72. [5] Shortis MR, Beyer HA. Calibration stability of the Kodak DCS420 and 460 cameras. In: Proceedings of the SPIE 3174, Videometrics V, 94 (July 7, 1997). doi:10.3390/s100301553 [6] Shortis MR, Bellman CJ, Robson S, Johnston GJ, Johnson GW. Stability of Zoom and Fixed Lenses Used With Digital SLR Cameras. In: Proceedings of the ISPRS Commission V Symposium “Image Engineering and Vision Metrology”: 2006 Sep 25–27; Dresden, Germany. IAPRS 2006;36(Pt 5):285–90. [7] Slama ChC, Theurer Ch, Henriksen SW. Manual of Photogrammetry. Virginia, USA: American Society of Photogrammetry, Falls Church; 1980 Geoinformatics FCE CTU 13, 2014 17 http://dx.doi.org/10.3390/s100301553 Geoinformatics FCE CTU 13, 2014 18