_______________________________________________________________________________________ Geoinformatics CTU FCE 2011 185 CULTURAL HERITAGE RECORDING UTILISING LOW-COST CLOSE- RANGE PHOTOGRAMMETRY Melanie KIRCHHÖFER1, Jim CHANDLER1, Rene WACKROW1 1Loughborough University, School of Civil and Building Engineering Loughborough, LE11 3TU, United Kingdom m.k.kirchhoefer@lboro.ac.uk, j.h.chandler@lboro.ac.uk, r.wackrow@lboro.ac.uk Keywords: close-range photogrammetry, heritage recording, orientation sensor, low-cost, smartphones Abstract: Cultural heritage is under a constant threat of damage or even destruction and comprehensive and accurate recording is necessary to attenuate the risk of losing heritage or serve as basis for reconstruction. Cost effective and easy to use methods are required to record cultural heritage, particularly during a world recession, and close-range photogrammetry has proven potential in this area. Off-the-shelf digital cameras can be used to rapidly acquire data at low cost, allowing non-experts to become involved. Exterior orientation of the camera during exposure ideally needs to be established for every image, traditionally requiring known coordinated target points. Establishing these points is time consuming and costly and using targets can be often undesirable on sensitive sites. MEMS-based sensors can assist in overcoming this problem by providing small-size and low-cost means to directly determine exterior orientation for close-range photogrammetry. This paper describes development of an image-based recording system, comprising an off-the-shelf digital SLR camera, a MEMS-based 3D orientation sensor and a GPS antenna. All system components were assembled in a compact and rigid frame that allows calibration of rotational and positional offsets between the components. The project involves collaboration between English Heritage and Loughborough University and the intention is to assess the system’s achievable accuracy and practicability in a heritage recording environment. Tests were conducted at Loughborough University and a case study at St. Catherine’s Oratory on the Isle of Wight, UK. These demonstrate that the data recorded by the system can indeed meet the accuracy requirements for heritage recording at medium accuracy (1-4cm), with either a single or even no control points. As the recording system has been configured with a focus on low-cost and easy-to-use components, it is believed to be suitable for heritage recording by non-specialists. This offers the opportunity for lay people to become more involved in their local heritage, an important aspiration identified by English Heritage. Recently, mobile phones (smartphones) with integrated camera and MEMS-based orientation and positioning sensors ha ve become available. When orientation and position during camera exposure is extracted, these phones establish off-the-shelf systems that can facilitate image-based recording with direct exterior orientation determination. Due to their small size and low-cost they have potential to further enhance the involvement of lay-people in heritage recording. The accuracy currently achievable will be presented also. 1. INTRODUCTION Cultural heritage plays a vital role in education about the past, in creating cultural or individual identity, and even in providing economical support for local communities [1,2,3]. Despite these widely acknowledged benefits, cultural heritage is at a constant risk by neglect and decay, deliberate destruction and damage due to social and economic progress, disasters, and armed conflict [3,4,5]. From this risk, an increased need to record spatially can be recognised. Comprehensive and accurate documentation can attenuate the risk of losing heritage and in the worst case serve as a basis for reconstruction [5]. The suitability of properly calibrated consumer-grade cameras for many heritage recording tasks has been demonstrated in [6,7,8]. Recognising the desirability to record within a three-dimensional (3D) national reference system, establishing known coordinated target points for exterior orientation estimation remains time consuming and costly and requires surveying expertise. Direct exterior orientation estimation for close-range applications could overcome this problem by avoiding expensive target point surveys and enabling non-experts to record cultural heritage within an appropriate national reference system. In that way the cost is reduced even further by the possibility to employ volunteers [9]. Direct exterior orientation estimation in close-range photogrammetry can be achieved using orientation sensors based on Micro Electro Mechanical Systems (MEMS) technology that have emerged on the market in recent years. Although their accuracy is lower than that of their large-size counterparts, results of utilising them for mobile mapping projects and photogrammetry look promising [10,11]. Direct positioning can be _______________________________________________________________________________________ Geoinformatics CTU FCE 2011 186 achieved using Global Positioning System (GPS) devices. Although positioning with current low-cost, handheld GPS devices does not meet the requirements for some applications of close-range photogrammetry, there is potential for improvements in the future [12]. One example is the announcement of GENEQ Inc. to release a small-size, high accuracy GPS receiver (SXBlue III) that is available for much lower cost than conventional survey-grade GPS receivers [13]. This paper presents the development and testing of a low-cost recording system for cultural heritage recording that utilises a low-cost orientation sensor and GPS for direct exterior orientation determination. Furthermore, the potential of utilising smartphones with integrated camera, orientation and position sensors for low-cost cultural heritage recording is investigated. First the recording system and its components are presented and the data collection and analysis process is explained. This is followed by a description of a recording system performance test at Loughborough University and of a case study on the Isle of Wight, UK. The results of these tests are presented in Section 4. In Section 5 the methodology of the smartphone test is described and the results of this test are presented. After discussing the results of the recording system and smartphone tests, this paper finishes in a conclusion. 2. METHODOLOGY 2.1 Recording System The recording system presented here comprises a calibrated consumer-grade digital camera (Nikon D80) for image acquisition, a small-size 3D orientation sensor (PNI TCM5) for orientation measurement, a survey-grade differential GPS (DGPS) (Leica System 500) for 3D positioning, and a laptop for operating the orientation sensor (Figure 20a). Figure 20: Full recording system (a) and mounting frame (b). Camera, orientation sensor, and DGPS antenna were attached to a purposely built mounting frame that constrains the components in their orientation and position (Figure 20b). This enables a reliable calibration of the rotational and positional offsets between components. When the recording system was assembled in early 2010, no low-cost, small- size DGPS receivers were available on the market to provide centimetre accuracy required in this project. Therefore, it was decided to use a survey-grade DGPS receiver, enabling positioning with centimetre accuracy. Although this is certainly not a low-cost component, it facilitates the testing of the principles of direct exterior orientation determination for close-range photogrammetry. The TCM5 orientation sensor is capable of measuring heading, pitch and roll using magnetometers and accelerometers. The expected accuracy of the measured angles is 0.3° in heading and 0.2° in pitch and roll [14]. 2.2 Offset calibration In order to achieve accurate exterior orientation parameters of the camera, the rotational offset between camera and orientation sensor and the positional offset between camera and DGPS antenna need to be calibrated. Exterior orientation parameters for a set of images acquired using the recording system were derived indirectly in a Leica Photogrammetric Suite (LPS) bundle adjustment. These parameters were used as truth data and compared to orientation sensor and DGPS measurements acquired at the time of exposure. For this purpose a routine was coded in MathWorks‟ _______________________________________________________________________________________ Geoinformatics CTU FCE 2011 187 Matrix Laboratory (MatLab) that used truth and measured data to estimate offset calibration values and their precision. Calibration values are defined by the arithmetic mean of the offsets calculated for each image and precision is indicated by the standard deviation. The calibration values were applied to the directly measured orientation and position values in order to derive direct exterior orientation parameters for each image. The MatLab routine also included an algorithm to convert the true omega, phi, and kappa values into equivalent heading, pitch and roll values, in order to enable comparison between indirectly derived (omega, phi, kappa) and directly measured (heading, pitch, roll) orientation angles. Another algorithm was needed to convert the corrected heading, pitch, and roll values into omega, phi, and kappa that were suitable for utilisation in a bundle adjustment. A detailed description of the offset calibration process will be presented in a future publication. 2.3 Data collection and analysis For testing the performance of the recording system, data was recorded from a varying number of camera stations adjacent to a test object which included coordinated points. A camera station here is defined as the position and orientation of the mounting frame at the time of image acquisition. For each acquired image, orientation and position at the time of exposure was measured by the orientation sensor and the DGPS receiver, respectively. Imagery, orientation and position data of all camera stations acquired on a particular date establish a data set. Calibration values were derived from the collected data and applied to the measurements of the same data set. Because the camera had been detached from the mounting frame between collection of differing data sets, no independently derived offset calibration values that were considered suitable to correct orientation and position measurements were available. Assuming that the best suitable calibration values are derived from the same data set, the results of accuracy assessment indicate the theoretically highest accuracy achievable. The corrected orientation sensor and DGPS measurements were used to provide initial exterior orientation parameters, constrained by the estimated calibration precision, in a bundle adjustment software known as GAP [15]. For each data set the GAP bundle adjustment was run twice. For the first run no control points were used, relying on the exterior orientation parameters derived from orientation sensor and DGPS only. The coordinated points of the test object were used as check points and their coordinates were estimated in the bundle adjustment. In the second run one coordinated point was used as control point with corresponding image point coordinates in only one image. In this bundle adjustment coordinates for the remaining check points were estimated. For both runs the estimated coordinates were compared to the known coordinates of the points, so allowing the calculation of the Root Mean Square Error (RMSE) for easting, northing, and height to quantify absolute accuracy. Relative accuracy was assessed also. 3D distances between all possible pairs of coordinated points were calculated from the check point coordinates estimated in the bundle adjustment. These distances were compared to corresponding distances calculated from the original check point coordinates. The RMSE of the distance differences indicates the 3D relative accuracy. 3. TESTING 3.1 Initial test The recording system was initially tested at Loughborough University. A metal piece of art located on Loughborough University campus was chosen as test object ( Figure 21a). The test object is a vertical structure with a small diameter on the ground and is accessible from all sides. It was considered representative for the type of heritage object that was also found at the case study site (Section 3.2). On the southern side of the test object 17 points with known Ordnance Survey National Grid (OSGB36) coordinates were established. In the lower part that could be reached without auxiliary means (approximately up to 2m) survey targets were used to mark the points. In the upper part of the test object natural points defined by distinctive features, such as corners and intersections of steelwork, were selected. Imagery, orientation and position data was collected at 11 camera stations arranged in an arc around the southern side of the test object with an approximate camera-to-object distance of 6m. At this distance some images were acquired with the mounting frame tilted up to 33°, in order to cover the entire height of the test object (approximately 6m). The data collected was processed and analysed using the methods described in Section 2.3 and the results can be found in Section 4. 3.2 Case study The aim of the case study was to test the performance of the recording system at a real heritage site. St. Catherine‟s Oratory ( Figure 21b) on the Isle of Wight, UK, was chosen as case study test site. St. Catherine‟s Oratory is an approximately 11m high, octagonal tower built in 1328. It is located in the south of the Isle of Wight on one of the highest parts of the Island [16]. On the eastern side of the tower 22 points with known OSGB36 coordinates were established. Analogous to the test object at Loughborough University, targeted points were used in the lower part and natural points were used in the upper part of the tower. Two data sets were collected during the case study. The first data set (DS1) consists of data _______________________________________________________________________________________ Geoinformatics CTU FCE 2011 188 collected from 12 camera stations arranged in an arc around the eastern side of the tower with an approximate camera- to-object distance of 10m. The second data set (DS2) consists of data collected from 12 camera stations arranged in an arc around the eastern side of the tower with an approximate camera-to-object distance of 6m. Due to the camera-to- object distance and the height of the tower, the mounting frame was tilted up to 21° in DS1 and 2Ř° in DS2 in order to cover the entire height of the tower. Each data set was processed and analysed separately using the methods described in Section 2.3. The results of the analysis can be found in Section 4. Figure 21: Test object at Loughborough University (a) and case study site St. Catherine‟s Oratory, Isle of Wight, UK (b). 4. RESULTS 4.1 Absolute accuracy Absolute accuracy quantifies the recording systems capability to provide data for measurements that are accurate in relation to a national coordinate reference system. It is indicated by the RMSE of the differences between object coordinates of check points estimated in a GAP bundle adjustment and their original coordinates. Figure 22 depicts the absolute accuracy achieved in the initial recording system test and in the case study using zero or just one single control point (CP). _______________________________________________________________________________________ Geoinformatics CTU FCE 2011 189 Figure 22: Absolute accuracy achieved in recording system test. The best accuracy is achieved in the initial test with values not exceeding 7.0mm. There is no significant difference between using zero or a single control point. The RMSE achieved in the case study using no control points is significantly higher than the RMSE of the initial test, with values up to 41.2mm in DS1 and 43.7mm in DS2. The accuracy in DS1 and DS2 is enhanced by using a single control point in the GAP bundle adjustment. However, the RMSE in DS2 (26.0mm) is significantly higher than the RMSE in DS1 (5.9mm). The accuracy variations between the three data sets indicate that their direct exterior orientation parameters used in the GAP bundle adjustment are of different accuracy. 4.2 Relative accuracy The relative or inner accuracy quantifies the recording system capability to provide data for measurements that are accurate in relation to each other. This was assessed by comparing 3D distances between check point coordinates estimated in a GAP bundle adjustment with equivalent distances derived from the original coordinates. The RMSE of the distance differences indicates the relative accuracy. Figure 23 depicts the relative accuracy achieved in the initial recording system test and in the case study using zero or a single control point. Figure 23: Relative accuracy achieved in recording system tests. The best relative accuracy is achieved for the initial test, with 2.5 mm when zero control points were used. Similar to the absolute accuracy, the relative accuracy achieved in the case study is worse than the relative accuracy achieved in the initial test. The relative accuracy achieved is also significantly different between the case study data sets, DS1 and 0 10 20 30 40 50 0CP 1CP 0CP 1CP 0CP 1CP R M S E ( m m ) Easting Northing Height 0 2 4 6 8 10 12 14 16 18 20 0CP 1CP 0CP 1CP 0CP 1CP Initial test DS1 DS2 R M S E ( m m ) _______________________________________________________________________________________ Geoinformatics CTU FCE 2011 190 DS2. When zero control points are used, the RMSE increased from DS1 (9.7mm) to DS2 (17.7mm) by 8mm. Similar to the results of the absolute accuracy assessment, this indicates accuracy differences between the exterior orientation parameters derived from the three data sets. The utilisation of one single control point seems to have no significant effect on the achievable relative accuracy. 5. Smartphone Test Smartphones with integrated camera and MEMS-based orientation and positioning sensors have potential to facilitate image-based recording with direct exterior orientation determination. When orientation and position during exposure can be extracted these phones establish off-the-shelf systems that are in principle similar to the recording system presented in this paper. The usability of smartphones for image-based heritage recording was tested using the “htc desire” smartphone. This smartphone integrates a 5 mega pixel camera, a GPS antenna, a digital compass and accelerometers [17]. In March 2011 the camera of the smartphone was calibrated and the smartphone used to acquire imagery at a test field established on an outside wall at Loughborough University using 22 coordinated points. Orientation and position at the time of exposure were extracted using a smartphone application (“Imageotag”) that prints the data derived from GPS, compass, and accelerometers on a copy of the original image. Imagery, orientation and position data was processed and analysed using the methods described in Section 2.3. This resulted in indicators for absolute (Figure 24a) and relative (Figure 24b) accuracy achievable when zero or one single control point is used. The results of the smartphone test are presented using the unit meters (m) instead of the unit millimetres (mm) used for the recording system test results. Figure 24: Absolute (a) and relative (b) accuracy achieved using a smartphone. Figure 24a demonstrates that the smartphone can achieve an absolute accuracy of 1.15m without using control points in the bundle adjustment. When a single control point is used in the bundle adjustment a significant increase in accuracy is only achieved for Easting where the RMSE drops from 1.04m to 0.68m. Using a single control points also improves the relative accuracy (Figure 24b). The RMSE of the relative accuracy changes from 0.85m achieved when no control point was used to 0.66m when a single control point was used in the GAP bundle adjustment. 6. DISCUSSION 6.1 Performance of the original recording system The results of the absolute accuracy assessment demonstrated that an accuracy level of 44mm can be achieved without control points when suitable exterior orientation parameters are available. With the utilisation of a single control point the absolute accuracy level can be improved to 26mm. As expected, the relative accuracy is better than the absolute accuracy, achieving 18mm without using any control points. The accuracy assessment also revealed significant differences in absolute and relative accuracy between the three data sets. This could be caused by variations in the accuracy of the direct exterior orientation parameters used in the GAP bundle adjustment. Because the calibration values and exterior orientation parameters were derived from the same data set, the standard deviations of the calibration values are also indicators of the accuracy of the directly measured values from where the exterior orientation parameters were derived. Investigating this issue, it was revealed that the standard deviations of the positional offset calibration values varied significantly between the three data sets (Table 1). 0 0,2 0,4 0,6 0,8 1 1,2 1,4 Easting Northing Height R M S E ( m ) a 0CP 1CP 0 0,2 0,4 0,6 0,8 1 0CP 1CP R M S E ( m ) b _______________________________________________________________________________________ Geoinformatics CTU FCE 2011 191 Easting (mm) Northing (mm) Height (mm) Initial test 7.86 9.21 9.35 DS1 13.40 14.65 15.64 DS2 24.62 37.57 16.74 Table 1: Standard deviations of positional offset calibration values. The standard deviations increase from the initial test data set to DS1 and also from DS1 to DS2, demonstrating the decrease in accuracy of the directly measured positions from the initial test to DS2. Because the case study standard deviations exceed the expected accuracy of DGPS, which is 10mm in plan and 30mm in height [18], the decrease in positioning accuracy is either caused by instability of the recording system components fixture to the mounting frame or by a decrease in DGPS accuracy. A decrease in DGPS accuracy during data collection at St. Catherine‟s Oratory could have been caused by tilting the mounting frame for some images, which also tilts the DGPS antenna. However, in the initial test, data was collected under similar conditions. Further investigations will be conducted in order to identify the reason for the decrease in positioning accuracy. The results of the absolute and relative accuracy assessment were achieved by correcting direct orientation and position measurements using offset calibration values derived from the same data set. Therefore, the calibration values are not independently derived and the results indicate only the theoretical accuracy achievable when well suited calibration values are available. After analysis of the data sets presented here, further test data sets were collected that enabled accuracy assessment using independently derived calibration values. Preliminary results suggest that the level of accuracy achieved in the tests presented here can also be achieved with independently derived calibration values, when stable offset calibration is maintained. These results will be presented in a future publication. 6.2 Performance of a system based upon a smartphone As expected, the accuracy achieved using the “htc desire” smartphone is substantially worse than the accuracy achieved using the developed recording system. The smartphone achieved 1.15m absolute and 0.68m relative accuracy without using control points. This significant difference to the results achieved with the recording system is caused by the smartphone sensor accuracy. The accuracy of the smartphone orientation and position sensors is expected to be lower than the accuracy of the recording system DGPS and orientation sensor. No information could be found about the compass and accelerometer accuracy, but the standard deviations derived during offset calibration can be used as indicators for orientation accuracy. Here standard deviations for heading, pitch, and roll between 2° and 3° were achieved. The accuracy of the integrated GPS can be expected to be no better than the theoretical positioning accuracy of code-based GPS, which is 6-10m [18]. This is higher than the displacement that would result from a rotational error of 3° in the exterior orientation rotation parameters at a camera-to-object distance of 10m. Therefore, at close-range, the positioning accuracy of the smartphone is probably the limiting factor on the currently achievable absolute accuracy. However, the absolute accuracy achieved in this smartphone test is better than the expected GPS positioning accuracy. This can be explained by the offset calibration partly compensating the positional error. Similar to the processing and analysis of the recording system data, calibration values and exterior orientation parameters were derived from the same data set. In order to test how well independently derived calibration values can compensate positioning errors, further data collection and analysis will be carried out. 7. CONCLUSION The results presented in this paper demonstrate that an absolute accuracy of 44mm can be achieved with an image- based recording system combined with direct exterior orientation determination. When a single control point is available for data processing the accuracy can be improved to 26mm. The recording system also achieves relative accuracy levels of 20mm and below. Preliminary results derived from further tests have indicated that this accuracy level can also be achieved when independently derived offset calibration values are used. The recording system is therefore believed to be suitable for many cultural heritage recording tasks. When the survey-grade DGPS receiver is replaced by a low-cost device for positioning with centimetre accuracy, the recording system will be a proper low-cost system that is suitable for heritage recording by non-specialists. The results of the smartphone test (1.2m absolute and 0.8m relative accuracy) demonstrate that even with well suited calibration values the currently achievable accuracy of the GPS positioning does not meet requirements for most cultural heritage recording tasks. However, the usability of smartphones for image-based recording was demonstrated and with in future potentially more accurate integrated orientation and position sensors, smartphones could be used for low-cost heritage recording by non-specialists. _______________________________________________________________________________________ Geoinformatics CTU FCE 2011 192 8. ACKNOWLEDGEMENT The authors wish to acknowledge the investment in a TCM5 orientation sensor by English Heritage, which made this project possible. Thanks are due to the National Trust for granting permission to conduct the case study at St. Catherine‟s Oratory. In addition the authors wish to thank the Remote Sensing and Photogrammetry Society (RSPSoc) for partly funding the first authors attendance at the CIPA 2011 conference in Prague. 9. REFERENCES [1] Uzzell, D.L.: Introduction: The Natural and Built Environment. In: Heritage Interpretation, Vol. 1, The Natural and Built Environment, London, Pinter, 1-14, 1989. [2] Herbert, D.T.: Preface. In: Heritage, Tourism and Society, London, Pinter, xi-xii, 1995. [3] Power of Place Office, English Heritage: Power of Place: The Future of the Historic Environment, London, Power of Place Office, 50 pages, 2000. [4] UNESCO: Convention concerning the protection of the world cultural and natural heritage, http://whc.unesco.org/archive/convention-en.pdf, 2009-05-05. [5] Palumbo, G., Ogleby, C.L.: Heritage at risk and CIPA today: a report on the status of heritage documentation, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2004, 35(2004)B5, 239-842. [6] Bosch, R.et al.: Non-metric camera calibration and documentation of historical buildings. Proceedings of CIPA 2005, Torino, September 2005, 142-147. [7] Chandler, J.H., Fryer, J.G.: Recording Aboriginal rock art using cheap digital cameras and digital photogrammetry, Proceedings of CIPA 2005, Torino, September 2005, 193-198. [8] Wackrow, R et al. Geometric consistency and stability of consumer -grade digital cameras for accurate spatial measurement. The Photogrammetric Record 2007, 22(2007)118, 121-134. [9] Bryan, P., Chandler, J.H.: Cost-effective rock-art recording within a non-specialist environment, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2008, 37(2008)B5, 259-264. [10] Niu, X. et al.: Directly georeferencing terrestrial imagery using MEMS-based INS/GNSS integrated systems, Proceedings of XXIII FIG Congress 2006, Munich, October 2006, 16 pages [11] Guarnieri, A. et al.: Low cost system: GPS/MEMS for Positioning. Proceedings of FIG Working Week 2008, Stockholm, June 2008, 10 pages. [12] Schwieger, V., Gläser, A.: Possibilities of low cost GPS technology for precise geodetic applications. Proceedings of FIG Working Week 2005, Cairo, April 2005, 16 pages. [13] GENEQ Inc.: SXBlue III. Rugged, Bluetooth high accuracy L1/L2 RTK-Mapping receiver. http://www.sxbluegps.com/SXBlue-III-version1.1.pdf, 2011-01-12. [14] PNI Corporation: TCM 5. Tilt compensated 3-axis compass module, http://www.pnicorp.com/files/ TCM5%20Datasheet_05-14-2009.pdf, 2009-05-30. [15] Chandler, J. H., Clark, J. S.: The archival photogrammetric technique. Further applications and development, The Photogrammetric Record 1992, 14(1992)80, 241-247. [16] English Heritage: St. Catherine’s Oratory | English Heritage, http://www.english-heritage.org.uk/ daysout/properties/st-catherines-oratory/, 2011-05-11. [17] HTC Corporation: HTC - Products - HTC Desire – Specification, http://www.htc.com/www/product desire/specification.html, 2011-05-12. [18] Konecny, G.: Geoinformation. Remote sensing, photogrammetry and geographic information systems. London, Taylor & Francis, 248 pages, 2003.