Faez M. Hassan, Hussein Abdelwahab Mossa. Image Mosaicking Using Low-Distance...| 44 Image Mosaicking Using Low-Distance High-Resolution Images Captured by an Unmanned Aerial Vehicle Faez M. Hassan, Hussein Abdelwahab Mossa Physics Department, College of Education, Mustansiriyah University, Baghdad, Iraq A B S T R A C T S A R T I C L E I N F O Regional surveys will have a high demand for coverage. To adequately cover a large area while retaining high resolution, mosaics of the area from a variety of scenes can be created. This paper describes a mosaicking procedure that consists of a series of processing steps used to combine multiple aerial images. These images were taken from CropCam unmanned aerial platform flight missions over the desired area to quickly map a large geographical region. The results of periodic processing can be compared and analyzed to monitor a large area for future research or during an emergency situation in the covered area. Digital imagery captured from the air has proven to be a valuable resource for studying land cover and land use. For this study, airborne digital camera images were chosen because they provide data with a higher spatial resolution for trying to map a small research area. On board the UAV autopilot, images were captured from an elevation of 320 meters using a standard digital camera. When compared to other airborne studies, this technique was less expensive and more cost effective. According to this study, onboard a UAV autopilot, a digital camera serves as a sensor, which can be helpful in planning and developing a limited coverage area after mosaicking. Article History: Received 18 Nov 2021 Revised 20 Nov 2021 Accepted 25 Nov 2021 Available online 26 Dec 2021 __________________ Keywords: Image Mosaic, Crop Cam UAV, Aerial photography International Journal of Informatics Information System and Computer Engineering Journal homepage: http://ejournal.upi.edu/index.php/ijost/ International Journal of Informatics Information System and Computer Engineering 2(2) (2021) 44-52 45 | International Journal of Informatics Information System and Computer Engineering 2(2) (2021) 44-52 1. INTRODUCTION Aerial photography serves as a common foundation for large-scale mapping. It has been widely used for creating and updating maps, as well as for keeping GIS databases up to date (Neteler, M., & Mitasova, H. 2004; Xu, Y et al., 2016). In remote sensing and geoformation sciences, the term "mosaicking" is frequently used when two or more contiguous images are stitched together to create a single image file (Aber, et al., 2010). Stitching functions are now available in the majority of digital camera software, and there is a wide range of both free and commercial panorama software available for fully automating the merging of combining multiple photographs into larger composites. These tools have the potential to produce visually appealing results, but they are not geometrically correct, which means they can still detect the image tiles with an angular skew and straight edges (Aber, et al., 2010). One of the most significant image data processing techniques in UAV systems is mosaic in real time, which allows the UAV images that have been georeferenced to be combined with geographic information for quick reaction to time-sensitive events (Zhou, et al., 2006; Kim, et al., 2017). Aerial photographs can be used to study changes in the earth's features as time passes. Those images are especially useful in analyses of land cover because they compare older data sets with new data sets, which can be available for a wide range of studies (Ren, et al., 2017). Information on current land use allows agencies and researchers to identify patterns in land cover and, as a result, make more informed decisions about analyses of development suitability, proposed land uses, and long-term planning (Gómez-Candón, et al., 2014). The data can show how development has changed over time, which can be applied as a guide for future research on land cover (Ahmad, A., 2011; Hassan, et al., 2010; Gomarascs, M. A. 2009). The digital images captured are available in a short time and are accompanied by latitude, longitude, and altitude coordinates (Zhao., et al., 2019). By manipulating the visualization of digital images, the user can keep track of what is going on at the ground, observe the most recent developments, and prevent problems from spiraling out of control. The unmanned aerial vehicle (UAV) can be hand-launched and can autonomously fly from takeoff to landing (CropCam, 2008; Felderhof, et al., 2008). Both flights were made to capture visible imagery with a resolution of ground level of 9 cm over the selected area in each flight, and all of the images obtained were in jpeg format. The visible images that resulted demonstrated a clear distinction between urban and green land surfaces (Avola, et al., 2018). Flights of UAV’s have been completed successfully at all of the research sites selected for this study. The number of photos taken during each flight plan was adequate for covering the research area. A mosaic image is a fabricated composition created from a series of images obtained by comprehending the geometric relationships between images (Fuyi, et al., 2012; Lim, et al., 2009; Hassan, et al., 2011). The entire survey area should be covered by aerial images with a sufficient amount of overlap between them. Typically, the degree of overlap in route direction should be between 60% and 65% with no less than 53%, while in the lateral direction, it should be between 30% and 40% with no less than 15% (Wang, et al., 2007). Each fly file was set up in this study with a 60% Faez M. Hassan, Hussein Abdelwahab Mossa. Image Mosaicking Using Low-Distance...| 46 overlap along the flight end lap runs and a 30% overlap between the side lap runs. The images that were taken in this study had a lot of overlap between them, both inside and outside of the runs. This meant that the images could be stitched together into a good mosaic for further analysis. 2. METHOD Penang island, which is located in northern Malaysia between latitudes 5o 12' N and 5o 30' N and longitudes 100o 09' E and 100o 26' E, was chosen as a study area. The CropCam UAV system was used to collect images for this study, as shown in figure 1. The aircraft was outfitted with navigation and autopilot systems that allowed it to follow predetermined waypoints and thus acquire the target area. Furthermore, to collect digital remote sensing images of the study area, a Pentax Optio A40 digital camera was employed in the form of a low-cost imaging sensor system affixed to the body of the UAV. Pentax's first digital camera with a resolution of 12.0 effective megapixels, the Optio A40 is capable of producing images with extreme precision and high resolution. Fig. 1. CropCam UAV system used in this study. Flights were conducted for each study site to capture visible imagery in order to cover the entire study site, with high ground resolution. In this study, the imagery was captured by the UAV at a low altitude (320 meters) above ground level, allowing imagery to be obtained even when there was cloud cover, giving it a competitive advantage over manned aircraft and satellite imagery. The software packages AutoPano Giga 2.2 Pro and PTGui 8.3.10 were used to stitch or mosaic a number of images captured by the CropCam UAV platform. To cover a larger area, each flight's raw images were mosaicked together. To achieve a good mosaic, a couple of control points shared by photos taken in succession were manually added. Because it was pre-programmed into the fly files, the overlap between adjunct photos was ideal. This has enabled the 47 | International Journal of Informatics Information System and Computer Engineering 2(2) (2021) 44-52 stitching of images into seamless final mosaics, as well as the improvement of the georeferencing process (Mengxiao, et al., 2018). Figure 2 depicts the mosaicking procedure used in this study. The seemliness between individual mosaic pieces can be placed manually or automatically to be as inconspicuous as possible, and radiometric matching techniques can be used to account for color and brightness differences (Tian, et al., 2020). All of the image mosaics in this study were created with the software packages AutoPano and PTGui. The original images (raw images) from each flight were mosaicked with lens distortion correction and color equalization. As shown in figure 2, a near-neighbor interpolator and smart blend bending algorithms were used to render images in order to make the mosaics shown in figure 2. Fig. 2. Modular workflow for image mosaicking process. 3. RESULTS AND DISCUSSION Figures 3 and 4 show examples of unprocessed digital images collected during CropCam UAV flight missions over the chosen study sites. Figures 5 and 6 depict image mosaics created from raw images collected after each flight over the study sites. The RMSE of image mosaicking is displayed in table 1. Faez M. Hassan, Hussein Abdelwahab Mossa. Image Mosaicking Using Low-Distance...| 48 Fig. 3. Samples of CropCam raw images (First Flight on June 20th, 2011) Fig. 4. Samples of CropCam raw images (Second Flight on December 12th, 2011) 202011). 49 | International Journal of Informatics Information System and Computer Engineering 2(2) (2021) 44-52 Fig. 5. Uncontrolled image mosaic of the selected area in Penang Island created with AutoPano Pro and PTGui software from 65 images taken by CropCam UAV on June 20th, 2011 (First Flight) Fig. 6. Uncontrolled image mosaic of the selected area in Penang Island created with AutoPano Pro and PTGui software from 86 images taken by CropCam UAV on December 12th, 2011 (Second Flight). Faez M. Hassan, Hussein Abdelwahab Mossa. Image Mosaicking Using Low-Distance...| 50 Table 1. Mosaicking process results Flight Mission Number of Stitched Images Panorama FOV RMSE (cm) Quality Status Panorama 100 dpi(m) First Flight 65 68.03º×51.31º 2.6 V. Good 3.57×2.69 Second Flight 86 79.71º×40.10º 2.1 V. Good 3.76×1.59 An examination of the generated image mosaics reveals that they are preferable. Clearly, the characteristics (roads, buildings, etc.) in those images are joined perfectly with the minimum distortion. Furthermore, due to the efficient method (image blending) used to create a mosaic of high quality, Those image mosaics are in good enough shape to be used in further image analysis. In UAV-acquired images, radiometric variations of overlapping views are common. As a result, each image region retains its own color, brightness, and contrast during the image blending process. Therefore, these overlapping regions blend into one another with no discernible pattern. There is sufficient evidence in this study to show that the CropCam flight missions were successful to obtain the desired images with high resolution, and the mosaicking results are visually pleasing. Image mosaics frequently reveal differences in exposure across or between photographs. Finally, color matching between the stitched images is required to hide the seams. Clearly, the mosaicking results show that the software used in this study can be used to solve the problem of uneven brightness in mosaics. 4. CONCLUSION This research paper describes a simple but effective procedure for UAV high-resolution mosaicking images obtained from images captured by UAV flying at a low distance. The proposed method's performance was demonstrated in a case study on Penang Island, Malaysia. The method proposed outperformed the highly developed commercial software on UAV images to achieve better mosaicking results. Our proposed method generates mosaicked images with reduced spectral distortion and increased spatial accuracy. Moreover, the process of mosaicking is much faster than that of other software packages. The RMSE value in the experimental results is quite high. The method presented in this paper saves 40% of the time. Furthermore, the mosaicked images generated by our proposed technique are strikingly similar to the original UAV images. In this paper, we propose a small-scale UAV-based system for creating low-altitude image mosaics that are incremental and geo- referenced in real time. ACKNOWLEDGMENTS We acknowledge Mustansiriyah University, Baghdad, Iraq and Universitas Komputer Indonesia, Indonesia. 51 | International Journal of Informatics Information System and Computer Engineering 2(2) (2021) 44-52 REFERENCES Aber, J. S., Marzolff, I., & Ries, J. (2010). Small-format aerial photography: Principles, techniques and geoscience applications. Elsevier. Ahmad, A. (2011). Digital mapping using low altitude UAV. Pertanika Journal of Science and Technology, 19(S), 51-58. Avola, D., Cinque, L., Foresti, G. L., Martinel, N., Pannone, D., & Piciarelli, C. (2018). A UAV video dataset for mosaicking and change detection from low-altitude flights. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50(6), 2139-2149.. CropCam, (2008). CropCam User’s Guide-Application Version, Canada. Felderhof, L., Gillieson, D., Zadro, P., & Van Boven, A. (2008). Linking UAV (unmanned aerial vehicle) technology with precision agriculture. Fuyi, T., Chun, B. B., Jafri, M. Z. M., San, L. H., Abdullah, K., & Tahrin, N. M. (2012, November). Land cover/use mapping using multi-band imageries captured by Cropcam Unmanned Aerial Vehicle Autopilot (UAV) over Penang Island, Malaysia. In Unmanned/Unattended Sensors and Sensor Networks IX (Vol. 8540, pp. 147-152). SPIE. Gomarascs, M. A. (2009). Basics of Geomatics. London and New York, Springer. Gómez-Candón, D., De Castro, A. I., & López-Granados, F. (2014). Assessing the accuracy of mosaics from unmanned aerial vehicle (UAV) imagery for precision agriculture purposes in wheat. Precision Agriculture, 15(1), 44-56. Hassan, F. M., Lim, H. S., & Jafri, M. M. (2011). CropCam UAV for land use/land cover mapping over Penang Island, Malaysia. Pertanika Journal of Science & Technology, 19(S), 69-76. Hassan, F. M., Lim, H. S., MatJafri, M. Z., & Othman, N. (2010, July). An Assessment of Low‐Cost Cropcam UAV Images for Land Cover/Use over Penang Island, Malaysia. In AIP Conference Proceedings (Vol. 1250, No. 1, pp. 23-26). American Institute of Physics. Kim, J. I., Kim, T., Shin, D., & Kim, S. (2017). Fast and robust geometric correction for mosaicking UAV images with narrow overlaps. International Journal of Remote Sensing, 38(8-10), 2557-2576. Lim, H. S., Jafri, M. Z. M., Abdullah, K., Hassan, F., & Saleh, N. M. (2009). Feasibility of Using multi-band Imageries Captured by Cropcam Unmanned Aerial Vehicle Autopilot for Land Cover Mapping. Journal of Materials Science and Engineering, 3(12), 26-31. Faez M. Hassan, Hussein Abdelwahab Mossa. Image Mosaicking Using Low-Distance...| 52 Mengxiao Song, Zheng Ji, Shan Huang & Jing Fu. (2018). Mosaicking UAV orthoimages using bounded Voronoi diagrams and watersheds. International Journal of Remote Sensing, 39(15-16), 4960-4979. Neteler, M., & Mitasova, H. (2013). Open source GIS: a GRASS GIS approach (Vol. 689). Springer Science & Business Media. Ren, X., Sun, M., Zhang, X., & Liu, L. (2017). A simplified method for UAV multispectral images mosaicking. Remote Sensing, 9(9), 962. Tian, Y., Sun, A., Luo, N., & Gao, Y. (2020). Aerial image mosaicking based on the 6- DoF imaging model. International Journal of Remote Sensing, 41(1), 74-89. Wang, P. and Xu, Y., (2007). Photogrammetry, Wuhan: Wuhan University Press, pp.16- 17 . Xu, Y., Ou, J., He, H., Zhang, X., & Mills, J. (2016). Mosaicking of unmanned aerial vehicle imagery in the absence of camera poses. Remote Sensing, 8(3), 204. Zhao, J., Zhang, X., Gao, C., Qiu, X., Tian, Y., Zhu, Y., & Cao, W. (2019). Rapid mosaicking of unmanned aerial vehicle (UAV) images for crop growth monitoring using the SIFT algorithm. Remote Sensing, 11(10), 1226. Zhou, G., Wu, J., Wright, S., & Gao, J. (2006). High-resolution UAV video data processing for forest fire surveillance. Old Dominion Univ., Norfolk, VA, Tech. Rep. National Sci. Foundation.