PHYSICAL, CHEMICAL AND BIOLOGICAL ASPECTS OF HUMAN IMPACTS ON URBAN SOILS OF SZEGED (SE HUNGARY) Journal of Env. Geogr. Vol. I. No. 3-4. pp. 21-26 SMALL FORMAT AERIAL PHOTOGRAPHY – REMOTE SENSING DATA ACQUISITION FOR ENVIRONMENTAL ANALYSIS Tobak, Z. 1 – Szatmári, J. 1 – van Leeuwen, B. 1 1 Department of Physical Geography and Geoinformatics, University of Szeged, Hungary Abstract Since February 2008, an advanced system has been developed to acquire digital images in the visible to near infrared wavelengths. Using this system, it is possible to acquire data for a large variety of applications. The core of the system consists of a Duncantech MS3100 CIR (Color-InfraRed) multi- spectral camera. The main advantages of the system are its affordability and flexibility; within an hour the system can be deployed against very competi- tive costs. In several steps, using ArcGIS, Python and Avenue scripts, the raw data is semi-automatically processed into geo-referenced mosaics. This paper presents the parts of the system, the image processing workflow and several potential applications of the images. Keywords: small format aerial photography, data acquisition system, image processing, Python INTRODUCTION Acquiring aerial photographs and their digital analysis are traditionally long and expensive processes (Warner W. S. et al. 1996). As digital cameras, computers and GPS re- ceivers became available at lower price ranges, the amount of time and cost needed for the acquisition process and the analysis is gradually reduced, which highly in- creases the operativity of the system (Licskó B. – Ditzendy A. 2003, Bakó G. 2010). Earlier the colour infra- red small format digital cameras were mainly used for surface measurements (Warner W.S. et al. 1996). The near infrared spectrum is mainly used in vegetation monitoring but it also promotes the identification of the areas covered with water (Rakonczai J. et al. 2003, Tucker D. et al. 2005). THE ADVANTAGE OF USING SMALL FORMAT AERIAL PHOTOGRAPHY The development of a small format aerial photography system that is able to take colour infrared (CIR) aerial photographs was started at the spring of 2008 by the Department of Physical Geography and Geoinfomatics at the University of Szeged. Out of the numerous ad- vantages of the system, its cost efficiency and its opera- tivity can be highlighted. Beyond the price of the digital camera, that was a single investment of the department, only the costs of the flights and the wages of the human resources participating in the processing have to be paid. The fact that the system is easy to operate becomes par- ticularly important in projects that investigate quickly changing phenomena (Bakó G. 2010). Inland excess water, as a temporal water surface – depending on the weather conditions – can evolve rather quickly, but its extension can diminish relatively fast as well. To discover a mapping methodology for excess water and to be able to model its development, it is inevi- table to know the actual extent of the area covered with water (Licskó B. – Ditzendy A. 2003). The country-wide aerial photography campaigns carried out every 5 years provide photographs with inappropriate time resolution. By using our small format aerial photography system, photographs can be taken at any time at any frequency, which provides basic data for further analyses. Not only the time resolution, but the spatial and spectral resolution of the photographs adjusts better to the needs of the research. The size of the smallest object (pixel) on the surface that can be mapped falls within the sub-meter interval and it can be altered depending on the altitude of the flight. This harmonizes well with the size range of the examined excess water coverage. In com- parison: while the satellite images provide only 4, 10 or 30-meter resolution data, the country-wide aerial photog- raphy data provides 1-meter resolution. Our camera is able to record in 3 spectral bands of the electromagnetic spectrum: in the visible green (G), the red (R) and the near infrared (NIR) bands. Out of these, the near infrared band is of particular importance, as in this band the water surfaces nearly completely absorb the incoming radiation and thus they appear as dark, black territories, which are easy to detect both visually and also by using image-processing techniques. By using the red (R) and near infrared (NIR) spectra together, the vegetation can be differentiated. On the other hand, the traditional aerial photographs – provided by the country-wide aerial photography, for example – cover only the spectrum of visible light and thus they provide much less spectral information regarding the excess water areas. INTRODUCING THE RECORDING SYSTEM A Duncantech MS 100 CIR (colour infrared) digital multispectral camera forms the basis of the acquisition system. Additional components are an attached data storage computer – equipped with a framegrabber card – 22 Tobak Z. – Szatmári J. – van Leeuwen B. JOEG I/3-4 and GPS receivers (Mobile Mapper CE and Garmin GPSMAP 296) that help the navigation and also record the flight path (Fig. 1). The Duncantech MS3100 digital multispectral cam- era contains 3 single CCD sensors (Fig. 2). The sensors detect the photons of the red (R), green (G) and near infrared (NearIR) spectra separately, depending on the prism (which splits the light) in front of the sensors (Ta- ble 1). The 3 sensors can be programmed separately, which means that not only their gain but also the applied integrating time can be adjusted to the needs of the users. This way it is possible to strengthen the IR spectrum – which is used to take photographs of vegetation and water surfaces – taking advantage of their special NIR reflectance. Fig. 2 Sensitivity of the green, red and near-infrared sensors The sensors are built up of 1392x1040 pixels, the physical size of each is 4.65x4.65 micron. The radio- metric resolution of the detectors is 10 bit therefore they are able to differentiate maximally between 2 10 , i.e. 1024 intensity values. This way relatively low reflectance differences between recorded object can be identified. The ground resolution of the data always depends on the optics and flight height. Using the high-speed Tokina AT-X 17 AF Pro objective at a flight altitude of 2000 m the spatial resolution is 62 cm. Table 1 Recording spectrum of the CIR camera CIR configuration (nm) Band Range Center Width Blue - - - Green 530 – 570 550 40 Red 640 – 680 660 40 NIR 768 – 832 800 65 A mini computer with a National Instruments IMAQ 1428 type framegrabber PCI card was used to store the photographs. The transportation of the data from the camera to the framegrabber is done through Camera Link connection, in three channels, each of which transports 10 bit data to the framegrabber. The further settings of the data recording – the frequency of exposure, the name and the place of the saved photos, the integration time and the sensitivity of the sensors – happens at the time of acquisition with the DT Control software supplied with the camera. During the acquisition, navigation was carried out by a Garmin GPSMAP 296 aviation GPS device follow- ing the planned flight. The actual flight path was record- ed with a Thales Mobil Mapper CE type GPS receiver. While the Digiterra Explorer mobile GIS software run- ning on the Windows CE based device helps the naviga- Fig. 1 Hardware elements of the recording system (left) and the Cessna 127 airplane with the CIR camera fixed into its luggage space (right) JOEG I/3-4 Small format aerial photography – remote sensing data acquisition for environmental analysis 23 tion, the GPS Status application of the Thales records the GPS data in NMEA format. Subsequent processing of the GPS data provides one meter accuracy flight track. To carry out the flight a Cesna 172 type airplane was used, which is the property of the partner company. In the four-seater airplane the person sitting next to the pilot directs the navigation and operates the camera. While the camera is in a construction fixed to the side of the plane, the data recording and energy supplier com- ponents – the batteries and the inverter – are in the lug- gage space of the plane. The hardware elements of the system presented above cost nearly 13,000 Euro. The cost of operation is about 250 Euro for an hour long flight, which of course changes depending on the distance of the destination and the shape of the acquisition area. As our point of depar- ture is the Szeged Airport, our primary destination is the southern part of the Great Hungarian Plain. Nevertheless the flexibility of our system allows its installation into any airplane that has got a door on its luggage space. FLIGHT PLAN While preparing the flight plan, it has to be decided in which way to cover the area to be recorded. Not only the planned flight paths have to be recorded, but also the necessary time intervals between the exposures. To do this not only the speed of the flight and the size of the area covered by a single photo – in case of a fixed objective, the latter depends only on the altitude of the flight – has to be known, but also the distance between the photos (b) and the distance between the rows of photos (d) have to be defined. To facilitate later processing, the photos have to overlap each other by a minimum of 50% in the flight direction by 20-30% on adjacent flightpaths. Taking these requirements into consideration, on a flight altitude of 2000 m, at the speed of 150 km/h the frequency of expo- sure is 1 photo/ 4 seconds and the distance of the neigh- bouring paths – in case the camera is fixed perpendicular to the flight direction – is 600 meters (Fig. 3). Fig. 3 A part of the recorded area with the planned and the actual flight paths, some sample images and characteristic parameters Our inland excess water project had 3 sample areas (Fig. 4), the total extension of which was 69.7 km 2 . Out of these sample areas area I (Tápairét) and area II (Bati- da) was recorded two times during the excess water period of spring 2010: on 24th March 2010 and on 9th June 2010. The recording was carried out based on the same flight path on both occasions. The 10 lines of flight, the nearly 100 CIR images captured at each line, resulted in 1804 (895+909) images by the end of the second day. Acquisitions have already been carried out for pre- vious projects of the department and for external partners as well. In the course of these projects approximately an area of 1000 km 2 was recorded (Tobak Z. et al. 2008, Kitka G. et al. 2010). For urban ecology research high spatial resolution – 50 cm pixel size – colour infrared (CIR) data has been acquired for the total area of Szeged (van Leeuwen B. et al. 2009). 24 Tobak Z. – Szatmári J. – van Leeuwen B. JOEG I/3-4 Fig. 4 Study areas covered with excess water and their sur- roundings with flightlines and the footprints of the images IMAGE PROCESSING During the preparation of the aerial photography, the flight and the processing of the raw data, different types of software had to be used. All through the work these software were customized to our needs and therefore the processing of the numerous photos could be made more automatic. During the preparation of the flight plan, the flight altitude was determined on the basis of the size and the shape of the area under survey and the desired resolu- tion. Having known the image sizes calculated from the altitude of the flight, the flight paths were determined. It is practical to orientate the lines in north-south or east- west directions, although it is possible to rotate the im- ages in any directions at any angles by the processing software. The planned lines were loaded in Digiterra Explorer in shape file format based on the Hungarian national projection (EOV). The image exposure can be controlled and its pa- rameters can be set in real time during the flight. The data logger and the control software of the camera allow the modification of Gain and the Integration Time. This makes the adaptation to the different light conditions possible. New series of images were created for each line of flight for which separate log files, containing the pa- rameters of the camera and the exact time of exposure, were generated by the program. The images were rec- orded in three-band TIFF files with 8-bit colour depth per band. In the first step of processing the x, y and z coordi- nates and the time data of the flight were extracted from the NMEA file recorded every second by the GPS. Then based on these records a dBASE table was generated. The time field of the table is joined (Table Join) with the log file of the camera thus the records of the table are joined with the images. With this operation real EOV coordinates (x, y) were assigned to the central point of each image. In the next step these coordinates – and the spatial resolution – were used for the generation of the so-called world files. World files are simple ASCII text formats, with which geographical coordinates can be assigned to JPEG and TIFF files. In this way, using the coordinates record- ed by the GPS, geo-referenced images can be made from our images quickly and automatically. Only in case of image rotation angles of 0 or 180 degrees is the applica- tion of the world files an effective method, therefore the flight directions were chosen in a way to make this pos- sible. The generation of the .tfw world files connected to the TIFFs requiring a rotation angle of 0 or 180 degrees was carried out with Avenue script in ArcView 3.2 soft- ware. In case of a different rotation angle the ArcGIS Rotate tool was also used in a Python script besides the world files. By means of the world files, a rough geomet- ric correction was attained, the accuracy of which was better than 150-200 m, depending on the circumstances of the flight. In the next step the aim was to create an orthophoto mosaic of the whole studied area by the as- sembling and geo-correction of the single images. Dur- ing the preparation of this process the TIFF with world file was converted to IMG format by another Python script. In the ERDAS Imagine image-processing soft- ware tie points between the images of the block (image series) were automatically generated, and after their filtering the block was transformed to EOV. A single, coherent image file was made from the individual imag- es by mosaicing. Fig. 5 summarizes the image processing workflow and demonstrates the data and operations used, the applied GIS and image processing technologies, and different types of software. APPLICATION OF PROCESSED IMAGES The geometrically corrected CIR images can be used in a wide range of studies. The common characteristic of these studies is that they require high spatial and tem- poral resolution and visible as well as near infrared spec- tral information. All of the above-mentioned requirements are of ma- jor importance during the exact delineation of the areas covered with inland excess water. In the classification procedure based on the artificial neural network (ANN) under development in our department these images are the most important input data besides a high resolution JOEG I/3-4 Small format aerial photography – remote sensing data acquisition for environmental analysis 25 digital elevation model. The images supplied by the three bands are used as separate input layers both in the training and the simulation phases. Besides the above-mentioned, ANN-based, method visual information can also be interpreted by the conven- tional image analyzing methods. During the mapping of sources of pollution harmful to the environment – for example, illegal landfills, water reservoirs etc. – high spatial resolution and operativity are key issues (Mucsi L. et al. 2004, Szatmári J. et al. 2008). A GIS database can be created from the different type of contaminating objects identified on the basis of the images (Warner W. S. 1994, Tobak Z. et al. 2008, Kitka G. et al. 2010). The red and near infrared bands of the sensor are well applicable for the monitoring of vegetation. These ranges of the electromagnetic spectrum are used for most of the vegetation indices. The estimation of the amount of biomass and chlorophyll, the separation of different forest types and the assessment of vegetation health are research topics for which good quality data can be pro- vided (Ladányi Zs. et al. 2011). The monitoring of vegetation can be carried out in the urban environment as well. The remote sensing anal- ysis of the complex and heterogeneous urban surfaces, with adequate spatial accuracy, can only be carried out if the geometric resolution is high. However, the spectral information content of the colour infrared images is narrow compared to the hyperspectral data. By their combination and by the development of multi-level clas- sification methods, the advantages of both can be ex- ploited. FUTURE OPPORTUNITIES, CHALLENGES Our system – in its present state – is not suitable to completely replace the traditional methods of aerial photography. It does not contain inertial or other spatial reference system therefore there is no possibility to elim- inate errors arising from the irregular movements of the airplane. Positional errors depending on the weather conditions can partially be corrected by automatic switching point measurement. Conversion of the energy values measured by the sensor (DN) to surface reflectance values would need further developments. Field reference samples should be used for this during acquisition. The differences between the histograms of the images can be corrected by histo- gram matching. Both hardware and software developments are needed in the future. For greater spatial coverage and thus more cost-effective recording, the replacement of the small-format camera (1392x1040 pixels) to a medi- um-format one (7220x5410 pixels) is among our plans. The expansion of the GPS measurements with an inertial measurement unit (IMU) is planned for faster and more accurate geometric correction. The self-made programs carrying out automatic processing also need further de- velopment by which the time elapsed between the image Fig. 5 Flow diagram of image processing from the raw images to the image mosaic 26 Tobak Z. – Szatmári J. – van Leeuwen B. JOEG I/3-4 exposure and the production of processed – and even analyzed – images can be reduced. During the developments the targets set at the crea- tion of the system are kept in mind. Although the value of the entire system increases due to financial invest- ments, the costs of operation and processing are kept on a cost-effective level. In addition, operativity, which is the main advantage of the system, remains and is further improved. Acknowledgement The research was supported by the project “Development of an BEL- VÍZ-INFO system” (Economic Operative Program: GOP - 1.1.1 - 08 / 1 -2008 – 0025). The deployment of the system was supported by the Hungarian Scientific Research Fund (OTKA T048903). References Bakó G. 2010. Igen nagyfelbontású légifelvétel-mozaikok készítése kis- és középformátumú digitális fényképezőgé- pekkel. Geodézia és Kartográfia 62/6: 21-29 Kitka G. - Szatmári J. - Mucsi L. - Tobak Z. - van Leeuwen B. 2010. Digitális légifelvételek alkalmazása a környezetvédel- mi ellenőrzési folyamatokban. „Térinformatikai Konferencia és Szakkiállítás. Az elmélet és gyakorlat találkozása.” Debre- cen. 430-431 Ladányi Zs. – Rakonczai J. – van Leeuwen B. 2011. Evaluation of precipitation-vegetation interaction on a climate-sensitive landscape using vegetation indices. Journal of Applied Remote Sensing, 5, 053519, doi:10.1117/ 1.3576115 Licskó B. - Ditzendy A. 2003. Az 1999-2000. évi belvizek légi felmérésének tapasztalatai. In: Szlávik L. (ed.) Vízügyi Köz- lemények Különszám IV. kötet: Elemző és módszertani ta- nulmányok az 1998-2001. évi ár- és belvizekről. Budapest: VITUKI. 305-315 van Leeuwen B. – Szatmári J. – Tobak Z. – Német Cs. – Hauberger G. 2009. Opportunities for the generation of high resolution digital elevation models based on small format aerial photography. HUNDEM 2009 konferencia, Miskolc Mucsi L. - Kiss R. - Szatmári J. - Bódis K. - Kántor Z. - Dabis G. - Dzsupin M. 2004. Felszín alatti vezetékek környezet- szennyező hatásainak felmérése távérzékeléses technológiá- val. Geodézia és Kartográfia 56/4: 3-8 Rakonczai J. - Csató Sz. - Mucsi L. - Kovács F. - Szatmári J. 2003. Az 1999. és 2000. évi alföldi belvízelöntések kiértéke- lésének gyakorlati tapasztalatai. In: Szlávik L. (ed.) Vízügyi Közlemények Különszám IV. kötet: Elemző és módszertani tanulmányok az 1998-2001. évi ár- és belvizekről. Budapest: VITUKI. 317-336 Szatmári J. - Tobak Z. - van Leeuwen B. - Olasz A. - Dolleschall J. 2008. An effective and low-cost method to detect environmental contaminations: the promise of CIR small format aerial photography (SFAP). XII. Geomatematikai Szimpózium, Mórahalom Tobak Z. - Kitka G. - Szatmári J. - van Leeuwen B. - Mucsi L. 2008. Kisgépes, kisformátumú CIR légifelvételek készítése, feldolgozása és alkalmazása környezeti vizsgálatokban. IV. Magyar Földrajzi Konferencia, Debrecen Tucker D. - Chavez P. - Bogle R. 2005. Rapid Response Low Cost Airborne Digital Imaging System. In: Proc. of Land Remote Sensing Science Fair, Reston, Virginia Warner W. S. 1994. Evaluating a low-cost, non-metric aerial mapping system for waste site investigators. Photogrammetric Engineering and Remote Sensing 60: 983- 988 Warner W. S. - Graham R. - Read R. E. 1996. Small Format Aerial Photography. Caithness: Whittles Publishing. 370 http://www.geo.u-szeged.hu/PUB/pdf_ml/60-1.pdf