Microsoft Word - ETASR_V12_N5_pp9357-9363 Engineering, Technology & Applied Science Research Vol. 12, No. 5, 2022, 9357-9363 9357 www.etasr.com El-Ashmawy: Simulated Photogrammetric Data for Testing the Performance of Photogrammetric … Simulated Photogrammetric Data for Testing the Performance of Photogrammetric Instruments and Systems Khalid L. A. El-Ashmawy Department of Civil Engineering College of Engineering and Islamic Architecture University of Umm Al-Qura Makkah, Saudi Arabia khalid85_2002@yahoo.com Received: 26 July 2022 | Revised: 12 August 2022 | Accepted: 18 August 2022 Abstract-The generation of mathematical photograms (photos) as data is universally accepted as the basis for photogrammetric studies. New development in the field of computer technology has had a major influence on photogrammetry. This paper aims to describe the development of a computer system called SimuPhotos designed to produce simulated data in different forms for testing photogrammetric methodologies, software, and instruments. SimuPhotos is capable of generating photogrammetric blocks with different specifications. It generates the simulated data for the whole block, strips, or selected photos. The simulated free error data include the camera interior orientation parameters, ground coordinates of points, and for each photograph, camera exterior orientation parameters, and the photo coordinates of imaged points. To increase the capabilities of the developed system, it has a special error generator to get normally distributed error(s) with arbitrary mean(s) and standard deviation(s). The obtained errors are then, if required, applied to the error-free photo and ground coordinates of points. The developed system interfaces CAD technology to generate simulated photogrammetric data in DXF and raster formats which are suitable for testing the photogrammetric instruments and softcopy systems. SimuPhotos has been used for testing the analytical photogrammetric system PhotoMap. The results showed that PhotoMap is free of system error and is functional. Also, the results showed that SimuPhotos can effectively provide a convenient, economical, and accurate testing tool for photogrammetric systems. The developed system is quite versatile and affordable. Keywords-simulated photogrammetric data; numerical data; graphical data; SimuPhoto; testing photogrammetric systems; softcopy photogrammetry I. INTRODUCTION Photogrammetry has experienced significant changes caused by advances in optics, electronics, imaging techniques and computer technology. It is applied in the form of mechanical systems such as stereo plotters, combinations of mechanical and computerized systems such as analytical plotters, and fully computerized systems like softcopy systems. Testing the results of the developed software or the performance (calibration) of the instrument’s mechanism requires error free and well-defined input and output data in a compatible form. Generally, data enter the available photogrammetric system in the form of paper-based photographs, diapositives, or numbers or pixels. The data of actual photographs are characteristically incomplete because the object coordinates of all image points are unknown. Furthermore, the actual data are full of errors that disturb the results of computations and analysis, and it is difficult to estimate how the data, the geometry of the problem, the software, and the instrument’s mechanism might affect the results. The main sources of actual data errors are the variation of the irregular photo errors from one photo to the next, the correlation of the irregular errors and the difficulty of determining their degree, incomplete elimination of systematic errors, investigation of undetected errors, poor image quality and control data, poor image identification, and inadequate redundancy of control data [1]. The specifications of data required for photogrammetric research studies include high quality stereo photographs from cameras of precisely known interior orientation, considerably more than the minimum required ground control data, so that a large number of check points may be selected for the study of error vectors. Ground control data must be of considerably higher precision than the errors expected and ground control points must be suitably located, precisely described, and unmistakably identified on the photographs [2]. Actual photographs taken with the best photogrammetric cameras on the best quality film may not meet the above specifications. Further, available methods of ground survey cannot produce control data of the highest desired precision (1/10 micron in photo coordinates and equivalent in ground control for studying the effects of computational rounding off, and photogrammetric errors). On the other hand, entering the actual photo in pixel form causes a series of problems to image matching [3-6]. These problems Corresponding author: Khalid L. A. El-Ashmawy Engineering, Technology & Applied Science Research Vol. 12, No. 5, 2022, 9357-9363 9358 www.etasr.com El-Ashmawy: Simulated Photogrammetric Data for Testing the Performance of Photogrammetric … include foreshortened effects such as changing the value of the sampled gray level due to the elements smaller then pixel size, lab processing noise, e.g. scratches or spots on the photographic material due to uneven processing or aging chemicals, digital camera radiometric calibration differences like integration time, gray level range definition and exposure setting, digital camera noise during image digitization and textural problems such as existence of distinguishable structures, repetitive texture, hanging surfaces, ambiguous levels, and thin objects. Fortunately, the problems of non-availability of error free photogrammetric data and using actual photographs for testing the photogrammetric systems and instruments can be alleviated by a combination of the simulated data generation through software and merging them with the capabilities of the Computer Aided Drafting (CAD) technology for preparation of the data in a compatible form for the photogrammetric instruments and systems. This paper deals with: • The development, description and evaluation of a system called SimuPhotos for automatic generation of simulated photogrammetric data in numerical and graphical forms. SimuPhotos is an acronym that specifies the generation of simulated photographs for testing the available photogrammetric instruments, methodologies, and systems. • The use of the capabilities of SimuPhotos for testing the results of a photogrammetric system called PhotoMap [2]. II. CONCEPT OF GENERATING THE SIMULATED PHOTOGRAMMETRIC DATA A block of simulated photographs for aerial platforms is planned to be obtained in much the same way as in actual photography. The present work is limited to near vertical photography. A set of mathematical photogrammetric data should consist of the following values for strips or photogrammetric blocks: • Interior orientation of camera selected for photography. • Ground coordinates of each exposure station. • Angular orientation of each photograph. • Ground coordinates of some ground points. • Photo coordinates of all ground points available within the format of each photograph. To provide a means for performing computational operations, for generating the mathematical photogrammetric data, the following steps are taken: • Adopting standard specifications for longitudinal and lateral overlaps, camera angular orientations (ω, φ, κ), camera displacements ( , y z b b ), etc. • Assignment of the ground coordinates ( , , o o o X Y Z ) to each camera station in the block. • Assignment of height values and planimetric positions to the ground points. • Computation of photo coordinates of image points on each photograph using the collinearity condition [7]. III. SPECIFICATIONS FOR GENERATING THE SIMULATED DATA The specifications for generating the simulated photogrammetric data include built-in and floating specifications. Built-in specifications outline the general layout of the ground, the location of points whose values are fixed during the computational session, and the proposed photography. Floating specifications refer to camera specifications and the terrain type. Their values can be changed by the researcher, depending on the study aims and the available data. A. Built-in Specifications The following specifications were taken into account during the development of the software. They outline the general layout of the simulated block: • Unit block: The maximum size of the block has been adopted to be 5 strips of 21 photographs each, to a total of 105 photographs. This block size represents the average size that is generally used in photogrammetric practice. A strip of 21 photographs may be considered as of medium length. The ground area covered by a block is variable depending on the camera and format used. • Scale: The hypothetical ground system is chosen such that photos of near-vertical specification are produced at an approximate scale of 1:1. This decision conforms to the International Society of Photogrammetry's relevant resolution on the desirability of reporting all photogrammetric errors at the negative scale [8]. Fig. 1. Points transferred per photograph (single photograph numbering system). Engineering, Technology & Applied Science Research Vol. 12, No. 5, 2022, 9357-9363 9359 www.etasr.com El-Ashmawy: Simulated Photogrammetric Data for Testing the Performance of Photogrammetric … Fig. 2. Points transferred per photograph (block numbering system). • Precision of image coordinates: The photo coordinates of the image points are computed mathematically to a precision of ±0.1 micron. A precision of about ±0.1 micron may be considered high enough to reduce into insignificance the effect of image errors on photogrammetric data processing [8]. • Numbering code for strip and photos: For identifying each photograph, an alpha-numerical code system is used. Strips and photographs are identified by serial numbers 1 to 5 and 1 to 21 respectively. • Number of points transformed per photo: On each photograph, as shown in Figure 1, 30 points are selected for transformation from ground system into the photo coordinate system and 6 points (31-36) for transformation from the photo coordinate system to the ground system. Of the 30 points, 9 (points 1-9) are primary, 18 (points 10-27) auxiliary, 2 (points 28 and 29) are transferred principal points from preceding and following photographs, and one is the nadir point (point 30). Points 31-36 are the image (fiducial) points and their ground coordinates help in defining the extent of the photograph's ground coverage. For photogrammetric studies, primary points may be used as orientation points, while auxiliary points may be used as auxiliary orientation and scale transfer points [2]. • Numbering codes for points: Points are identified by 2 numbering systems, single photograph and block numbering. In single photograph numbering system, as shown in Figure 1, the points are directly identified by serial numbers (1 to 36). This numbering system is suitable for all single photograph studies, for example, studying the effects of dimensional distortions [9]. In the block numbering system, the alpha-numerical code system is used for identifying each point of the block points. Each of the three longitudinal sections of a strip has (n+2) and 2(n+2) primary and auxiliary points respectively, n being the number of photographs per strip. For a block with m laterally overlapping strips, there are (n+2) (2m+1) and 2(n+2) (2m+1) primary and auxiliary block points respectively. Thus, the block has 23*11=253 and 2*23*11=506 primary and auxiliary points, and each strip 69 and 138 primary and auxiliary points, which are common with the adjoining strip. To differentiate the primary points from the auxiliary points, CP and AP numbering codes are added to their numbers respectively as shown in Figure 2. • The block numbering system is suitable for block studies such as determination of strip deformations, studying the effects of systematic and random errors on the results of block adjustment, comparison of the results of the different procedures of block adjustment, e.g. sequential and bundle, and helping in the development of new methodologies for block adjustment and investigation of the effect of changing one or all block specifications on the results. • Ground coordinates of block points: The planned X and Y coordinates of the perspective center of the first photo of the first strip in the block are taken as the coordinates of block origin ( , o oG G X Y ). • Applied mathematical model: As mentioned above, perfect perspective projection (collinearity condition [7]) from ground to photograph and from photograph to ground, and absence of all error sources have been assumed for generating the mathematical photogrammetric data. B. Floating Specifications Floating specifications can be changed, depending on the study and the available data and are entered to the software operating system by editing the data file. • Camera focal length and format: These values depend on the camera used. • X and Y ground coordinates of block origin: X and Y ground coordinates of block points and camera stations are referred to the coordinates of block origin , . o oG G X Y • Height variation: Height variation of ground points is taken as a percentage of the flying height. This factor defines the terrain type. • Longitudinal and lateral overlaps: These are entered as percentages of the width of the photograph (longitudinal%, lateral%), and are necessary for strips and block formations, and photogrammetric studies. • Flight specifications: These include the values of , , , y z b b ω ϕ and κ for each of 105 photographs of the block. Engineering, Technology & Applied Science Research Vol. 12, No. 5, 2022, 9357-9363 9360 www.etasr.com El-Ashmawy: Simulated Photogrammetric Data for Testing the Performance of Photogrammetric … IV. THE SIMUPHOTOS SYSTEM Software suites designed to support CAD have become popular. The most well-known software package in this regard is AutoCAD, from Autodesk, Inc. [10]. Of equal importance has been the development of software designed to utilize efficient user interfaces, e.g. window-driven and menus [11]. These advances in technology have enabled the development of the SimuPhotos system. The applied methodologies in SimuPhotos development provide automatic generation of the simulated data in numerical and graphical forms. The numerical form enables the testing of analytical processing methodologies and systems. The graphical form in drawing exchange format (DXF) and numerical forms are used for testing photogrammetric instruments, e.g. stereoplotters and analytical plotters. Furthermore, the graphical form is suitable for testing the digital systems by scanning the plotted photos using CCD cameras or scanners. Also, the graphical form in DXF format can be directly converted to pixel or raster format using the capabilities of software such as CorelDraw [12] and AutoCAD [10]. One of the most important aspects of automatic generation of graphical simulated data is the prevention of manual graphical presentation, e.g. using hand or software drawing of the numerical simulated data which is time consuming and may have additional mistakes. The structure of generating the simulated data process is shown in Figure 3. Four main modules (generating simulated photos MathP module, generating AutoCAD files DXFGEN module, control interface, and AutoCAD software) can be seen. Fig. 3. SimuPhotos system architecture and its intermediate communication to the hardcopy devices. Central to the system is the MathP module that reads the floating block specifications from the input data file (3 data files are available to the system) and uses these specifications along with built-in block specifications for generating the free error simulated photos. The module has a special error generator to get normally distributed error(s) with arbitrary mean(s) and standard deviation(s). The obtained errors are then, if it is required, applied to the error free photo and the ground coordinates of points of the generated blocks. MathP module stores the generated data in ASCII format for further processing. MathP module is a window-driven program consisting of two master windows. The main master window consists of 5 menus, depending upon the user's choice. These menus are basically for generating the simulated data for the whole block, selected strips and selected photos, helping the user and exiting to the system main window. The second master window consists of 2 menus for choosing the numbering system, single photo or block numbering system. Also, the user interface helps the user to move across the generated data and send the required data to the available hardcopy interfaces. The DXFGEN module reads the generated data and prepares the suitable AutoCAD drawing files in DXF format. A menu pad is provided, allowing the user to work without knowledge of AutoCAD command sequences. Each key on this pad invokes a different mapping function. Map scale, legends, title blocks, border, and coordinate gridding can be entered. The DXFGEN module is supported by default values for automatic map generation without user choices. The main functions of using AutoCAD are (1) enabling the user to add his modification(s) to the resulting map, (2) sending the generated maps to the hardcopy interfaces, and (3) Engineering, Technology & Applied Science Research Vol. 12, No. 5, 2022, 9357-9363 9361 www.etasr.com El-Ashmawy: Simulated Photogrammetric Data for Testing the Performance of Photogrammetric … converting the DXF file into a raster format file, such as BMP, JPG, PNG, and TIFF, with the required resolution. The SimuPhotos system generates the necessary data for both analytical and digital processing systems. Basically, these data are classified as numerical and graphical simulated data. • Numerical simulated data: They include photos per each strip, ground coordinates of the origin point, camera format, height variation, and camera interior orientation parameters and Photo-Oriented Elements (which include photo and strip numbers, camera exterior orientation parameters, point numbers with respect to the chosen numbering system, and photo and ground coordinates of points). • Graphical simulated data: They consist of the generated maps for location of the camera stations, location of points and for each photo, showing the image locations and their numbers as indicated in Figures 1 and 2. V. TESTING PHOTOMAP The PhotoMap system [2] is an analytical data processing and presentation photogrammetric system. PhotoMap is under continuous development and new capabilities are developed and added to widen its applications. PhotoMap contains 7 modules namely BUNDLE [2], Self-Calibration [9], DLT [13], Coplanarity [9], Distance-Control [14], No-Control [15], INTERSECTION [2] and GENERATE [2]. SimuPhotos was used for generating the necessary data for testing all computational stages of the developed modules of PhotoMap. The BUNDLE, Self-Calibration, DLT, Coplanarity, Distance- Control modules were developed to generate control points, compute the parameters of camera exterior orientation, and store the output in the system data bank. The INTERSECTION module utilizes the computed/known parameters of camera exterior orientation for computing the object coordinates of desired number of points for specific application. These are stored in the system data bank. The GENERATE module reads the stored data in the system data bank and generates the desired graphical output in digital format compatible with the available CAD and DTM (Digital Terrain Model) packages for plotting and statistical error analysis. The most important function of GENERATE module is its ability to generate the maps in DXF file format which is suitable for many commercially available GIS (Geographic Information System) packages. Testing a photogrammetric system is a complex task. It includes the availability of appropriate data for testing, decisions regarding the number and type of tests to be accomplished, photogrammetric tasks for each test to be achieved, and many other considerations. After completing the development stages, PhotoMap was subjected to a series of tests. These tests provided an opportunity to verify that the developed system satisfies the required general performance, specifically the flexibility and feasibility of photogrammetric data processing. Testing the system error and functionality of BUNDLE and INTERSECTION modules could be offered right here. System errors consist of (a) error due to the rounding off of values throughout intermediate computations which may be minimized by using double precision computations as far as possible and (b) errors because of truncation of higher order terms while forming the linearized observation equations from the nonlinear condition equations. For instance, in bundle adjustment the errors in triangulation process may be generated because of the use of simplified linearized correction equations derived from the non-linear collinearity condition equations by truncating a few terms [2, 7]. A. Planning for Testing PhotoMap The simulated photogrammetric block utilizing the depicted specifications in Table I was used for testing the system error of PhotoMap. TABLE I. SPECIFICATIONS FOR THE USED SIMULATED PHOTOGRAMMETRIC BLOCK Specification Value a. Photograph scale 1:1 b. Camera format 230.0*230.0 mm c. Camera focal length 150.00 mm d. Longitudinal and Lateral overlaps 65% and 30% respectively e. Total number of points available per model 18 f. Terrain configuration Hilly type with height variation of 25% of flying height. B. Testing the Results of BUNDLE Module BUNDLE module carries the photogrammetric operations using the following methodologies [2]: • Exterior orientation determination, • Bundle block adjustment. Each of these methodologies has been independently tested for the system error as described below. 1) Testing the System Error for Camera Exterior Orientation Determination The available 18 points in the model were considered control points and this control distribution pattern was used to test the results of camera exterior orientation determination. Error free photo and ground coordinates of the 18 control points and their a priori standard deviations were processed simultaneously in the BUNDLE module. The known quantities were photo and ground coordinates of the control points and camera interior orientation parameters. The unknowns to be estimated were the 6 exterior orientation parameters of each photograph. The residual errors between the actual and the computed values of camera exterior orientation parameters were found out, along with standard errors [16] as tabulated in Table II. TABLE II. PHOTOMAP SYSTEM ERROR FOR CAMERA EXTERIOR ORIENTATION DETERMINATION Standard Errors Camera ground coordinates (μm) Orientation angle (rad) X σ Y σ Z σ ϕ σ ω σ κ σ 0.001 0.001 0.001 0.000 0.000 0.000 * Values at 1:1 photograph scale Engineering, Technology & Applied Science Research Vol. 12, No. 5, 2022, 9357-9363 9362 www.etasr.com El-Ashmawy: Simulated Photogrammetric Data for Testing the Performance of Photogrammetric … These standard errors are small and negligible, and it can be concluded that BUNDLE module is free from system error for camera exterior orientation determination. 2) Testing the System Error for BUNDLE Block Adjustment For testing the results of bundle block adjustment, suitable location and number of control points were chosen [2]. The block size was 5 strips, each with 21 photographs. In this case, the block contained 253 and 506 control and check points respectively. Error free photo coordinates of control and check points, ground coordinates of the control points and their standard deviations were processed simultaneously with the BUNDLE module. The output was the adjusted camera exterior orientation parameters and the ground coordinates of the check points. Table III lists the standard errors for camera exterior orientation parameters and ground coordinates of the check points. TABLE III. PHOTOMAP SYSTEM ERROR FOR BUNDLE BLOCK ADJUSTMENT a. Camera exterior orientation parameters Standard error Camera Ground Coordinates (μm) Orientation Angle (rad) X σ Y σ Z σ ϕ σ ω σ κ σ 0.002 0.002 0.003 0.000 0.000 0.000 b. Ground coordinates of check points Standard error (μm)* X σ Y σ Z σ 0.001 0.001 0.002 * Values at 1:1 photograph scale These results lead to following conclusions: • The bundle block adjustment has system error for the Z coordinate larger than for planimetric (X, Y) coordinates which was expected [9, 14, 15] and • The maximum system error is 0.002μm for the ground coordinates of check points, which is negligible. From the above, it is obvious that the BUNDLE module is free from system error and that it is functional. C. Testing the System Error of the INTERSECTION Module The ground coordinates of the 18 points of each model for which image coordinates were known, were computed with the INTERSECTION module. The error free values of camera exterior orientation parameters for each photograph as generated by SimuPhotos were used. The residual errors between the actual and the computed ground coordinates of these intersected points were obtained for computing the shown standard errors in Table IV. TABLE IV. PHOTOMAP ERROR FOR TESTING SPACE INTERSECTION METHODOLOGY Standard error of ground coordinates of check points (μm)* X σ Y σ Z σ 0.002 0.002 0.004 * Values at 1:1 photograph scale The results indicate negligible standard errors and hence the INTERSECTION module of PhotoMap is also free from system error in performing space intersection operation and is functional. VI. CONCLUSIONS This paper discusses a new application of CAD technology for automatic preparation of the necessary data for calibrating photogrammetric instruments. Using the colored facilities of the hardcopy devices, inkjet or laser printer or plotter, the gray level of graphical simulated data can be changed. SimuPhotos provides a real time solution for testing the photogrammetric systems and eliminates the need of manufactured plates which are costly and not always available, and preventing manual graphical presentation such as using hand or software drawing of the numerical simulated data which is time consuming and may pose additional mistakes. The numerical and graphical simulated data are useful for academic studies such as error behavior investigations, determination of the suitable number of control points and their location, testing the results of the suggested methodologies of data processing, etc. The PhotoMap testing results clearly showed that SimuPhotos system can effectively provide a convenient, economic, and accurate testing tool for photogrammetric systems. Moreover, the results showed that PhotoMap is free of system errors and is functional. SimuPhotos provides a window-driven system that is both portable and suitable for use by non-technical users after a short training period. The developed system is quite versatile and provides an affordable tool to the researchers in universities and academic canters. REFERENCES [1] F. J. Doyle, "Fictitious data generator for analytical aerotriangulation," Photogrammetria, vol. 21, no. 5, pp. 179–194, Oct. 1966, https://doi.org/ 10.1016/0031-8663(66)90011-1. [2] K. L. A. El-Ashmawy, "A Cost-Effective Photogrammetric System for Engineering Applications," Ph.D. dissertation, University of Roorkee, Roorkee, India, 1998. [3] S. Rani, Y. Chabrra, and K. Malik, "An Improved Denoising Algorithm for Removing Noise in Color Images," Engineering, Technology & Applied Science Research, vol. 12, no. 3, pp. 8738–8744, Jun. 2022, https://doi.org/10.48084/etasr.4952. [4] S. Banerjee, A. Bandyopadhyay, A. Mukherjee, A. Das, and R. Bag, "Random Valued Impulse Noise Removal Using Region Based Detection Approach," Engineering, Technology & Applied Science Research, vol. 7, no. 6, pp. 2288–2292, Dec. 2017, https://doi.org/ 10.48084/etasr.1609. [5] R. Vadhi, V. S. Kilari, and S. S. Kumar, "An Image Fusion Technique Based on Hadamard Transform and HVS," Engineering, Technology & Applied Science Research, vol. 6, no. 4, pp. 1075–1079, Aug. 2016, https://doi.org/10.48084/etasr.707. [6] Yu. B. Blokhinov, D. A. Gribov, and A. S. Chernyavskiy, "Image matching problem for certain cases of perspective photography," Journal of Computer and Systems Sciences International, vol. 47, no. 6, pp. 959– 973, Dec. 2008, https://doi.org/10.1134/S1064230708060105. [7] S. K.Ghosh, Fundamentals of Computational Photogrammetry, 1st ed. New Delhi, India: Concept Publishing Company Pvt. Ltd., 2005. [8] Manual of Photogrammetry. Bethesda, MD, USA: ASPRS, 2004. [9] K. L. A. El-Ashmawy, "Coplanarity Condition for Photogrammetric Simultaneous and Self Calibration Block Adjustments," International Engineering, Technology & Applied Science Research Vol. 12, No. 5, 2022, 9357-9363 9363 www.etasr.com El-Ashmawy: Simulated Photogrammetric Data for Testing the Performance of Photogrammetric … Journal of Advances in Scientific Research and Engineering, vol. 7, no. 2, May 2021, https://doi.org/10.31695/IJASRE.2021.33970. [10] I. Autodesk, Autocad Reference Manual. Sausalito, CA, USA: Autodesk, Inc., 1986. [11] K. Gregory, Using Visual C++ 6, Special edition. Indianapolis, IN, USA: Que Pub, 1998. [12] CorelDRAW® 2020 User Guide. Corel Corporation, 2020. [13] K. L. A. El-Ashmawy, "Using direct linear transformation (DLT) method for aerial photogrammetry applications," Geodesy and Cartography, vol. 44, no. 3, pp. 71–79, Oct. 2018, https://doi.org/ 10.3846/gac.2018.1629. [14] K. L. A. El-Ashmawy, "Block Adjustment Using Control Distances Constraint," International Journal of Innovation Scientific Research and Review, vol. 3, no. 3, pp. 948–953, Mar. 2021. [15] K. L. A. El-Ashmawy, "Photogrammetric block adjustment without control points," Geodesy and Cartography, vol. 44, no. 1, pp. 6–13, Apr. 2018, https://doi.org/10.3846/gac.2018.880. [16] C. D. Ghilani, Adjustment Computations: Spatial Data Analysis, 6th ed. Wiley, 2017.