Cultura e Scienza del Colore - Color Culture and Science 59 Cultura e Scienza del Colore - Color Culture and Science | 07 | 2017 ISSN 2384-9568 1Shogo Nishi s-nishi@osakac.ac.jp 1Akira Kimachi kima@osakac.ac.jp 1Motonori Doi doi@osakac.ac.jp 2Shoji Tominaga shojitominaga12@gmail.com 1Faculty of Information and Communication Engineering, Osaka Electro-Communication University 2Graduate School of Advanced Integration Science, Chiba University Image Registration for a Multispectral Imaging System Using Interference Filters and Application to Digital Archiving of Art Paintings ABSTRACT The present paper proposes a calibration method of a multispectral imaging system using interference filters. A multispectral imaging with interference filter is effective to acquire the inherent information of an object surface. However, in the imaging system using interference filters, the problem of misregistration is caused by longitudinal aberration or transverse aberration, where image position is slightly different between each captured image with the interference filters. We present a calibration method for correcting the observed images. The captured images are transformed into frequency domain, where the phase correlation between the Fourier-transformed images for different channels is maximized so as to compensate the misregistration. As a practical use, we describe the application to digital archive of paintings based the present imaging system. The surface properties such as surface-spectral reflectance and surface normal are estimated using the multichannel images. KEYWORDS Interference filter, Spectral imaging, Misregistration, Calibration, Phase-only correlation, Digital archiving. CITATION: Nishi S., Kimachi A., Doi M., Tominaga S. (2017) ‘Image Registration for a Multispectral Imaging System Using Interference Filters and Application to Digital Archiving of Art Paintings’, Cultura e Scienza del Colore - Color Culture and Science Journal, 06, pp. 59-68, DOI: 10.23738/ccsj. i72017.05 Received 31 July 2016; Revised 30 May 2017; Accepted 31 May 2017 Shogo Nishi, is an associate professor in the department of engineering informatics at Osaka Electro-Communication University, Osaka, Japan. He received his B.E. (1998) and Ph.D. (2003) in information engineering from Kagoshima University. His current research interests include spectral imaging, color image analysis, reflection modeling, and color reproduction. He is a member of IS&T and OSA. Akira Kimachi is a professor in the department of engineering informatics at Osaka Electro-Communication University, Osaka, Japan. He received his B.S. (1993) and Ph.D. (1999) in mathematical engineering and information physics from the University of Tokyo. His current research interests include optical measurement, image sensing and computer vision. He is a member of OSA, SPIE and IEEE. Motonori Doi is an associate professor in the department of telecommunication and computer networks at Osaka Electro-Communication University, Osaka, Japan. He received his B.S. in control engineering from Osaka University (1993) and his Ph.D. in information engineering from Nara Advanced Institute of Science and Technology (NAIST) (1998). His current research interests include color/spectral image analysis. He is a member of IS&T and IEEE. Shoji Tominaga received the Ph.D. degree in electrical engineering from Osaka University, Japan in 1975. He was a Professor at Graduate School of Advanced Integration Science, Chiba University, and is now a Specially Appointed Researcher. His research interests include digital color imaging, multispectral imaging, and material appearance. He is a Fellow of IEEE, IS&T, SPIE, OSA, and IEICE. 60 Cultura e Scienza del Colore - Color Culture and Science | 07 | 2017 | 59 - 68 Nishi S., Kimachi A., Doi M., Tominaga S. ISSN 2384-9568 DOI: 10.23738/ccsj.i72017.05 1. INTRODUCTION In recent years, it has attracted attention to a digital archive for preserving the historical and artistic heritage in the digital image information. In the digital archive of art paintings, the key technology is the image acquisition by a digital camera and the image reproduction to render the realistic appearance on a display device. There were many proposals for the fundamental technology. For instance, Miyake et al. [1] developed a multiband camera system to record the reflectance spectra of paintings. The system consisted of a monochrome charge-coupled device (CCD) camera and a rotating filter wheel composed of five color filters. Tominaga et al. [2] proposed a photometric technique with a six-band multispectral camera for estimating the spectral reflectances, surface normals and 3-D light reflection properties of oil paintings. Using this method, these authors reproduced the appearance of paintings under arbitrary conditions of viewpoint and illumination. This approach is called the viewpoint and illumination-independent digital archiving. Moreover, Tominaga et al. [3] proposed a digital archiving method using a multiband scanner as a multiband spectral imaging system for estimating the painting surfaces. To digitally preserve paintings, information on the painting’s surface must be precisely recorded. The technology of multispectral imaging is quite effective. Since a multispectral imaging system has high wavelength resolution compared with color cameras, colorimetric accuracy in the acquired image is certainly improved. We should note that the multispectral system is useful not only for colorimetry, but also for estimation of surface spectral reflectances and surface normal vectors (see [2]-[3]). The typical multiband systems are constructed by (1) using one or two additional color filters to a trichromatic digital camera [4], (2) combining a monochrome camera and color filters with different spectral bands [5], (3) using narrow band interference filters [6], or (4) using a liquid- crystal tunable (LCT) filter [7]. Our present system is a multispectral imaging system that uses interference filters [8]. This system has the advantage of accuracy and stability in spectral estimations, because the filtration of the spectral system is narrow band and the side lobe component included in the camera output signal is reduced. The system also has the advantage of facilitating the design of a desired spectral measurement system, because a number of relatively low-cost interference filters with various transmission band characteristics are available. However, one limitation of the system is image misregistration, which is caused by longitudinal aberrations or transverse aberrations. A number of calibration techniques for the misregistration in spectral images were proposed so far. Mansouri et al. [9] improved the misregistration optically caused by longitudinal aberration by performing the blur modeling and image restoration. Brauers et al. [10] calibrated misregistrations caused by transverse aberration via modeling based on affine transformation. In our measurement system, the misregistration of the acquired images is non-uniform in the image, and the amount and direction of misregistration vary for each filter. Therefore the previous methods are not available for the present misregistration problem. Because this chromatic aberration is a fatal defect in the digital archive of painting, precise registration is required. In this paper, we propose a solution method for calibrating the misregistration caused by using the interference filters. The spectral images acquired from a painting are divided into several areas to correct for non-uniform misregistration. The divided images are then transformed into frequency domain, where the phase correlation between the Fourier-transformed images for different channels is maximized so as to compensate the misregistration. The alignment is performed between the divided spectral images. Since this method does not require a calibration chart, calibration can be performed directly without depending on an object to be calibrated. As a practical use, we describe the application to digital archive of paintings based the present imaging system. 2. MULTISPECTRAL IMAGING SYSTEM 2.1. COMPOSITION The multispectral imaging system is shown in Fig. 1(a). Our multispectral imaging system is composed of a monochrome CCD camera (Toshiba Teli, CS3920), a C mount lens (Tamron, 20HA), an automatic filter changer (Asahi Spectra, FC8-25A), and eight interference filters (Asahi Spectra). The camera has a resolution of 1636 × 1236 and records with a dynamic range of 10-bit quantization. Center transmission wavelengths of the interference filters are 420, 450, 490, 530, 570, 610, 650, and 670 nm, and the full width at half maximum filtration is approximately 10 nm. These interference filters are mounted on the rotary wheel of the filter changer, and the selected filter is placed in front of the camera lens. Multispectral imaging is achieved by capturing images while rotating the wheel. Fig. 1 (b) shows the total spectral sensitivity multiplied by the spectral transmittance of the eight filters and the spectral sensitivity of the CCD sensor. The CCD outputs 61 Cultura e Scienza del Colore - Color Culture and Science | 07 | 2017 | 59 - 68 Image Registration for a Multispectral Imaging System Using Interference Filters and Application to Digital Archiving of Art Paintings ISSN 2384-9568 are band-pass signals in the visible wavelength range, and they are finally integrated to analyze the surface reflection properties of an object. 2.2. CHARACTERISTICS OF THE INTERFERENCE FILTER IN AN IMAGING SYSTEM An interference filter is formed by depositing a dielectric multilayer film on the substrate surface; therefore, a specific wavelength band can be selectively acquired from the spectral distribution of incident light. Wavelength selection is determined by the same principle adopted by the Fabry-Perot interferometer. An image acquisition system using interference filters has the advantage of analyzing the target at a desired wavelength band. However, incorporating an interference filter in the imaging optical system causes focal shifts and misregistrations. Focus adjustment is conducted manually for each filter. We also confirmed that varying the magnification of the lens can be ignored. Therefore, our goal is to calibrate the misregistration in spectral images. In our multiband imaging system, the interference filters are mounted on a filter changer. Consequently, the main factor underlying the misregistrations may be that the interference filters are not parallel to the sensor surface. Because an interference filter is influenced by temperature and humidity, the spectral transmission characteristic may change for each acquired image. Moreover, a transformation matrix cannot be easily prepared for calibration by using a particular calibration subject. Therefore, the misregistration must be calibrated for each acquired image. 3. IMAGE REGISTRATION FOR SPECTRAL IMAGES Image registration is one of the topics in the field of image processing, and various registration methods have been proposed. Although luminance is an important image feature that is utilized in many registration methods, this feature is not available for registration because the luminance of spectral image varies between wavelength bands. Therefore, the misregistration detection technique that does not utilize the luminance value is required. Misregistration of the target image and the reference image is not a uniform process, and local misregistrations can occur in the image, which increases the difficulty of the registration process. Therefore, calibrations cannot be easily performed using a method based on global characteristics. For image registration, both the reference image and target image are divided into several small regions. For non-uniform image misregistration, the images are divided into small regions, which are then calibrated. Although a non-linear image registration method can be used to calibrate the distortion caused by misregistration, the method cannot be easily adapted for spectral images with considerable changes in luminance values. Although our approach does not represent a fundamental solution to misregistration calibration because it is a region-based registration method, misregistrations can be sufficiently reduced to be ignored. Several methods are available for dividing an image into small regions using the spatial frequency characteristics. In this paper, the image was evenly divided into four areas. Next, misregistrations between the reference image and the target image are detected for each small region, and image registration is then performed. To align spectral images, we propose a calibration method that detects misregistrations by using a phase-only correlation method [11], which is a technique for calculating correlations by using a phase image. A phase image is normalized by the amplitude spectrum of an image. Please note that this “spectrum” is not an intensity distribution for each wavelength of electromagnetic waves but a function of frequency. Therefore, the proposed method can perform corrections without the Figure 1 - Multispectral imaging system: (a - left) system overview; and (b - right) total spectral-sensitivities of the present system 62 Cultura e Scienza del Colore - Color Culture and Science | 07 | 2017 | 59 - 68 Nishi S., Kimachi A., Doi M., Tominaga S. ISSN 2384-9568 DOI: 10.23738/ccsj.i72017.05 influence of the amplitude spectrum in each spectral image. Moreover, misregistrations can be detected with high speed and high accuracy compared with processing in the spatial domain. Let both f(n1, n2) and gp(n1, n2) be N1 × N2 images, with image f(n1, n2) representing the reference image and image gp(n1, n2) representing the target image. One of eight spectral images is designated as the reference image. In this research, a spectral image with a center transmission wavelength of 670 nm was used as the reference image, and the remaining seven spectral images were used as the target images. Figure 2 shows the procedure for detecting the misregistration of spectral images using the phase-only correlation. Let F(k1, k2) and Gp(k1, k2) be the 2-D discrete Fourier transforms of the images f(n1, n2) and gp(n1, n2). Therefore, F(k1, k2) and Gp(k1, k2) are defined as follows: where k1 and k2 are discrete Fourier indices; and WN1 = exp[-j(2π / N1)]; WN2 = exp[-j(2π / N2)]; |F(k1, k2)| and |Gp(k1, k2)| are amplitude spectra; and ϴF(k1, k2) and ϴGp(k1, k2) are phase spectra. The cross spectrum C(k1, k2) between F(k1, k2) and Gp(k1, k2) is defined as follows: where represents the complex conjugate of Gp(k1, k2), and ϴ(k1, k2) = ϴF(k1, k2) - ϴGp(k1, k2) represents the phase difference spectrum. Therefore, the normalized cross spectrum Cn(k1, k2) using the phase image is defined as follows: (1) (2) (3) Figure 2 - Flow chart of the calculation process for the phase-only correlation. 63 Cultura e Scienza del Colore - Color Culture and Science | 07 | 2017 | 59 - 68 Image Registration for a Multispectral Imaging System Using Interference Filters and Application to Digital Archiving of Art Paintings ISSN 2384-9568 where S(li) is the spectral reflectance, E(li) is the spectral distribution of illumination, Rk(li) is the spectral sensitivities of the k-th sensor, and nk is noise. Let ρ be an eight-dimensional column vector representing all spectral camera outputs, s be a n-dimensional vector representing the spectral reflectance S(li) and H (≡[hkj]) be an 8 × n matrix with the element hkj = E(li) Rk(li). Then, the above imaging relationships are summarized in the matrix form ρ = Hs + n. When the signal component s and the noise component n are uncorrelated, a solution that minimizes the estimation error on s is the Wiener estimator: where Rss is an n × n matrix that represents the correlations among the surface spectral reflectances. We assume that white noise has a correlation matrix s2I. The surface normal vector at each pixel of an oil painting’s surface is calculated by using a photometric stereo method. If an object’s surface is a perfect diffuser, then the light intensity O reflected from the surface illuminated by a light source is described as O’=aNtL, where O’ is a normalized vector, L is the illumination directional vector, and a is the diffuse reflectance factor. The surface normal vector N is estimated from the above equation. A reflection model is required to render realistic images of oil paintings. In our previous study [12-14], we found that the surface reflection of most oil paintings can be described by the Cook-Torrance model [15]. The parameters of this mathematical model are estimated from the specular reflection component of the observed images. To detect the specular reflection component, we first calculate the output vector ρD for the diffuse component by using the estimated reflectance. Next, the maximum sensor output rM of all observations is selected, and the difference ρS = ρM − ρD is defined. The specular function of the Cook-Torrance model is fitted to the specular data extracted from all pixel points. Because the specular component at any pixel has the same spectrum as the light source, the parameters are estimated based on the statistical distribution of the specular component. Let the intensity of the specular component be ||ρs||; then, the normalized specular component is represented as Here, G represents the geometrical attenuation factor and F represents the Fresnel spectral reflectance of the microfacets, respectively. V is the view vector. This component is calculated (4) The amplitude of the correlation peak represents the similarity between the reference image and the target image, and the coordinates of the correlation peak represent the relative misregistration of the two spectral images. However, if the misregistration correction is performed by dividing the image into small areas, then the overlapping of images near the boundary may becomes sparse. This problem can be resolved by separately preparing a small region that include the boundary and replacing the sparse region after performing the misregistration correction. The image misregistration correction is performed in the spatial domain after the 2-D inverse discrete Fourier transforms. The Fourier indices k1 and k2 that satisfy arg maxk1,k2 Cn(k1, k2) correspond to the coordinates (N1 / 2 + x’, N2 / 2 + y’) of the target image in the spatial domain. The coordinates N1 / 2 and N2 / 2 represent the center coordinates of the reference image and the target image, and x’ and y’ represent the positional difference between the two images. If there is no difference in the image between the reference image and the target image, the values of x’ and y’ become 0, and the positions of the two images coincide. Therefore, by moving the target image in accordance with the amount of x’ and y’ so that the difference between the two images is minimized, the image misregistration correction is completed. 4. APPLICATIONS TO THE DIGITAL ARCHIVING OF PAINTINGS Here, we describe a method for the digital archiving of oil paintings [2] based on the imaging system. The surface reflection of an oil painting includes the specular reflection or gloss. Therefore, the camera output for the painting surface is composed of a diffused reflection and specular reflection. The reflection properties, the surface-normal vector and the surface-spectral reflectance are estimated from the diffuse reflection component, and the reflection model parameters are estimated from the specular component. To estimate the surface-spectral reflectance, the camera output is described as a model: (5) (6) 64 Cultura e Scienza del Colore - Color Culture and Science | 07 | 2017 | 59 - 68 Nishi S., Kimachi A., Doi M., Tominaga S. ISSN 2384-9568 DOI: 10.23738/ccsj.i72017.05 to minimize the squared sum of the fitting error where rSx and φx are the specular intensity and angle at pixel point x, respectively; D is the distribution function representing the micro facet orientation, which we assume is a Gaussian distribution function; and γ and β are parameters that minimize the above fitting error, and they are solved according to the nonlinear fitting problem solution. Image rendering is then performed based on all estimates of the surface reflection, which are the surface-normal vectors, the surface spectral reflectance, and the 3-D light reflection model parameters. 5. EXPERIMENTS 5.1. VERIFICATION OF CALIBRATION ACCURACY First, we performed experiments to validate the proposed correction method. Although calibration charts are not required in this method, we used graph paper with numbers and letters as a calibration chart, which was examined to verify the calibration accuracy. The calibration results are shown in Fig. 3. Fig. 3 (a) shows the results of synthesizing a region of 1024 × 1024 pixels, which was extracted from the eight images acquired in the system. In this figure, the results of the case without the misregistration calibration are shown. A chromatic aberration is observed in Fig. 3 (a), and the synthesized image appears unfocused. The image acquired by the system was difficult to use without calibration. Fig. 3 (b) shows the calibration result. The misregistration of each of the eight images is detected by the phase- only correlation method, and then registration is performed with the shift amount. The reference image is set to the image acquired with the 8th filter (the center wavelength is 670 nm), and the remaining seven images are corrected as calibration target images. The chromatic aberration has been drastically improved in comparison with Fig. 3 (a). The misregistration calibration performed well. However, a slight chromatic aberration occurs, and registration in the horizontal direction is insufficient. Therefore, although the chart is originally monochrome, a color is slightly perceived because of the non- uniform displacement or distortion in the image. Because several regions cannot be calibrated by a global registration method, the image registration process was local. The results are shown in Fig. 3 (c). The eight images in Fig. 3 (a) are divided into small regions, misregistration detection and calibration by the phase-only correlation method is performed on each small region, and then the images are synthesized as a single image. The size of the small region is 256 × 256 pixels. The results show that the local registration can be calibrated with higher accuracy compared with the global registration. The chromatic aberration in Fig. 3 (b) is corrected, and it is not perceptible in Fig. 3 (c). The calibration results are almost identical with the original appearance of the chart. Similarly, Fig. 4 shows the calibration results for a color object. Similar to the results shown in Fig. 3, the local registration can be calibrated with higher accuracy than the global registration. These results show the usefulness of the proposed Figure 3 - Calibration results of a original calibration chart: (a - left) before calibration; (b - centre) calibration without division; and (c - right) calibration with division. Figure 4 - Calibration results of a color object: (a - left) before calibration; (b - centre) calibration without division; and (c - right) calibration with division. (7) 65 Cultura e Scienza del Colore - Color Culture and Science | 07 | 2017 | 59 - 68 Image Registration for a Multispectral Imaging System Using Interference Filters and Application to Digital Archiving of Art Paintings ISSN 2384-9568 No. Color Patch DE*ab 1 Dark skin 1.581 2 Light skin 1.335 3 Blue sky 0.677 4 Foliage 1.479 5 Blue flower 0.913 6 Bluish green 2.342 7 Orange 2.299 8 Purplish blue 1.313 9 Moderate red 0.44 10 Purple 1.081 11 Yellow green 1.131 12 Orange yellow 2.746 13 Blue 1.825 14 Green 0.603 15 Red 2.388 16 Yellow 2.588 17 Magenta 0.654 18 Cyan 2.73 19 White 1.356 20 Neutral 8 1.19 21 Neutral 6.5 0.648 22 Neutral 5 0.397 23 Neutral 3.5 0.561 24 Black 1.629 Average 1.413 Maximum 2.746 0.2 0.4 0.6 0.8 1.0 400 500 600 700 R ef le ct an ce Wavelength [nm] 0.0 Estimate Measured 01.Dark skin 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 02.Light skin 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 03.Blue sky 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 04.Foliage 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 05.Blue flower 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 06.Bluish green 0.2 0.4 0.6 0.8 1.0 400 500 600 700 R ef le ct an ce Wavelength [nm] 0.0 Estimate Measured 07.Orange 400 500 600 700 Wavelength [nm] Estimate Measured 08.Purplish blue 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 09.Moderate red 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 10.Purple 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 11.Yellow green 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 12.Orange yellow 0.2 0.4 0.6 0.8 1.0 400 500 600 700 R ef le ct an ce Wavelength [nm] 0.0 Estimate Measured 13.Blue 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 14.Green 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 15.Red 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 16.Yellow 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 17.Magenta 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 18.Cyan 0.2 0.4 0.6 0.8 1.0 400 500 600 700 R ef le ct an ce Wavelength [nm] 0.0 Estimate Measured 19.White 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 20.Neutral 8 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 21.Neutral 6.5 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 22.Neutral 5 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 23.Neutral 3.5 0.2 0.4 0.6 0.8 1.0 400 500 600 700 Wavelength [nm] 0.0 Estimate Measured 24.Black 0.2 0.4 0.6 0.8 1.0 0.0 method. Next, we show the performance of the imaging system via reflectance estimation results using an X-Rite ColorChecker Classic. The imaging conditions were as follows. The illuminant was an incandescent lamp. The distance between the system and the color checker was approximately 1 m, and the illuminance of the color checker surface was approximately 1000 lx. The color checker was captured with a shutter speed of 1/30 second, and the noise variance was assumed to be 0.01% of the signal component. The surface spectral reflectance of the color checker was estimated by a Wiener filter using the image signals of the central 100 × 100 pixels of each color patch. The results are shown in Fig. 5. The red curves represent the estimation results using the present imaging system, and the black curves represent direct measurements by a spectrometer. The estimation accuracy of the spectral reflectance was evaluated by the CIELAB color difference. The color differences DE*ab under illuminant D65 are shown in Table 1. The average color differences of DE*ab for all 24 color patches was 1.413. The maximum color difference was 2.746 for No. 12 Orange yellow. This experiment shows that the present imaging system has excellent performance. Table 1 - Reflectance estimation performance by the present imaging system Figure 5 - Estimation results of the surface spectral reflectances of a ColorChecker. 66 Cultura e Scienza del Colore - Color Culture and Science | 07 | 2017 | 59 - 68 Nishi S., Kimachi A., Doi M., Tominaga S. ISSN 2384-9568 DOI: 10.23738/ccsj.i72017.05 5.2. ESTIMATION OF THE PAINTING SURFACE FOR DIGITAL ARCHIVING The oil painting shown in Fig. 6 was preserved to a digital archive by using the present system and the proposed method. The vertical size of the oil painting is 15.5 cm, the horizontal size is 11.5 cm, and the surface has a matte appearance. The present system was placed perpendicular to the painting. The elevation angle of the light source was fixed at 45 degrees, and the azimuth angle of the light source illuminated the 8 directions at 45 degree intervals. To estimate the painting surface, the oil painting was initially captured with different shutter speeds (1/60 sec., 1/250 sec., and 1/1000 sec.). Then, image calibration was performed by the proposed registration method, and high-dynamic-range (HDR) images were acquired. The reflection model parameter, the surface normal vector and the surface spectral reflectance were estimated from the acquired HDR images, and these estimates of the surface properties were integrated to render the oil painting. Using this process, the oil painting was preserved as a digital archive. We estimated the surface normal of all pixels of the oil painting. The estimated result of the surface normal in the white dotted line area of Fig. 6 is shown in Fig. 7. Fig. 7 (a) is a measurement of the painting surface with a laser displacement meter, and Fig. 7 (b) is the estimate by the proposed method. In certain areas, the estimated surface shape is not clear compared with the measurement, and this lack of clarity is considered to be dependent on the resolution of the present system. A comparison of the results in Fig. 7 with the shadow direction shows that the results match and the surface shape restoration via the surface normal estimation performed well. The spectral reflectance of the painting surface was estimated, and the experimental results of area 1 through 4 in Fig. 6 are shown in Fig. 8. The black line shows the measurement results obtained by a spectroradiometer, and the red line shows the estimation results. The measured value and the estimated value are substantially equal in all regions; thus, good estimation results were obtained. The three-dimensional light reflection model of the painting surface was estimated. Figure 9 shows the fitting result into a three-dimensional light reflection model. The specular reflection component from the acquired image was extracted and then fitted to the Gaussian function, which was assumed to be a specular reflection function of the Cook- Torrance mode. The reflection model parameters were estimated as γ = 0.063 and β = 200. The γ and β parameters represent the surface roughness and the specular coefficient of the Area 1 Area 2 Area 3 Area 4 Figure 6 - Measured oil painting. 67 Cultura e Scienza del Colore - Color Culture and Science | 07 | 2017 | 59 - 68 Image Registration for a Multispectral Imaging System Using Interference Filters and Application to Digital Archiving of Art Paintings ISSN 2384-9568 oil painting, respectively. The fitting results are shown in Fig. 9, which shows that the reflection model parameters can be estimated with high accuracy. Image rendering was conducted using the estimation results, and the results are shown in Fig. 10. Both light sources illuminated the oil painting at nearly the same position, and the spectral distribution of the light source was illuminant D65. The perceived resolution is inferior compared with that of the real photograph, which is believed to be related to the original 400 500 600 700 0.0 0.2 0.4 0.6 0.8 1.0 Wavelength (nm) N or m al iz ed v al ue Measurement Estimation 400 500 600 700 0.0 0.2 0.4 0.6 0.8 1.0 Wavelength (nm) N or m al iz ed v al ue Measurement Estimation 400 500 600 700 0.0 0.2 0.4 0.6 0.8 1.0 Wavelength (nm) N or m al iz ed v al ue Measurement Estimation 400 500 600 700 0.0 0.2 0.4 0.6 0.8 1.0 Wavelength (nm) N or m al iz ed v al ue Measurement Estimation 0 50 100 150 200 250 -20 -10 0 10 20 Observed data Gaussian fn. S pe cu la r i nt en si ty Specular angle (deg) resolution of the system. However, because of the calibration performed by the proposed method, the chromatic aberration is barely perceived. This result shows that the color reproduction had a high accuracy. Figure 7 - Estimated surface shape in the area with the white dotted line in Fig. 6: (a - left) measurement; and (b . right) estimated result after calibration. Figure 8 - Estimation results of the surface spectral reflectances of oil paintings: from left to right (a) area1; (b) area 2; (c) area 3; and (d) area 4. Figure 9 -Fitting result to the Gaussian function. Figure 10 - Comparison with a photograph of an oil painting: (a-left) photograph of an oil painting; and (b-right) image rendering result 68 Cultura e Scienza del Colore - Color Culture and Science | 07 | 2017 | 59 - 68 Nishi S., Kimachi A., Doi M., Tominaga S. ISSN 2384-9568 DOI: 10.23738/ccsj.i72017.05 6. CONCLUSIONS This paper has proposed a calibration method of a multispectral imaging system using interference filters. Although the presented technology may not represent a tool for preserving cultural heritage artworks, we suggested one solution to the problem of digital archiving applications in multiband imaging using interference filters. The calibration method is based on the correlation technique in the frequency domain using the phase images of an acquired spectral image. By using the phase image to detecte misregistrations in the frequency domain, correlations can be calculated without the influence of changes in the amplitude spectrum. Misregistration detection can be improved and conducted at high speed and with high accuracy. We described a method for applying this calibration method and the multispecral imaging system for the digital archiving of paintings. The feasibility of the proposed method and the present imaging system was examined by preserving an oil painting as a digital archive. The results demonstrated validity of the proposed method and imaging system. FUNDING This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. CONFLICT OF INTEREST The Authors declare that there is no conflict of interest. BIBLIOGRAPHY [1] Y. Miyake, Y. Yokoyama, N. Tsumura, H. Haneishi, K. Miyata and J.Hayashi, “Development of multiband color imaging systems for recording of art paintings,” Proc. SPIE 3648, pp. 218–225, 1999. [2] S. Tominaga and N. Tanaka, “Spectral Image Acquisition, Analysis, and Rendering for Art Paintings,” J. Electron. Imaging, Vol. 17, No. 4 , pp.043022-1–043022-13, 2008. [3] S. Tominaga, S. Nakamoto, H. Keita and T. Horiuchi, “Estimation of Surface Properties for Art Paintings Using a Six-band Scanner”, Journal of the International Colour Association, vol.12, pp.9–21, 2014. [4] F. H. Imai and R. S. Berns, “Spectral estimation using trichromatic digital camera,” Proc. of International Symp. Multispectral Imaging and Color Reproduction for Digital Archives, pp. 42–49, 1999. [5] S. Tominaga, “Multichannel vision system for estimating surface and illuminant functions,” J. Opt. Soc. Am. A, Vol. 13, pp. 2163–2173, 1996. [6] N. Shimano, K. Terai and M. Hironaga, “Recovery of spectral reflectances of objects being imaged by multispectral cameras,” J. Opt. Soc. Am. A, Vol. 24, pp. 3211–3219, 2007. [7] J. Y. Hardeberg, F. Schmitt, and H. Brettel, “Multispectral color image capture using liquid crystal tunable filter,” Opt. Eng., Vol. 41, No. 10, pp. 2532–2548, 2002. [8] S. Nishi and S. Tominaga, “Calibration of a Multispectral Camera System Using Interference Filters”, Proc. of SPIE, Vol. 8011, pp.80119E-1–80119E-7, 2011. [9] A. Mansouri, F. S. Marzani, J. Y. Hardeberg, and P. Gouton, “Optical calibration of a multispectral imaging system based on interference filters,” Opt. Eng., Vol. 44, No. 2, pp.027004-1–027004-12, 2005. [10] J. Brauers, N. Schulte, and T. Aach, “Multispectral Filter-Wheel Cameras: Geometric Distortion Model and Compensation Algorithms,” IEEE Trans. Image Process., Vol. 17, No. 12, pp.2368–2380, 2008. [11] C. D. Kuglin and D. C. Hines, “The phase correlation image alignment method,” Proc. IEEE Int. Conf. Cybernetics and Society, pp.163–165, 1975. [12] S. Nishi, N. Tanaka and S. Tominaga, “Estimation of Reflection Properties of Paintings and its Application to Image Rendering”, Proc. Inter. Congress of Imaging Science, pp. 292–293, 2006. [13] S. Nishi and S. Tominaga “Measurement and Analysis of Spectral Reflection Properties of Oil Paints”, Journal of the Color Science Association of Japan, Vol. 32, No. 4, pp. 260–270, 2008. [in Japanese] [14] S. Tominaga and S. Nishi, “Surface Reflection Properties of Oil Paints under Various Conditions”, Proc. SPIE, Vol.6807, pp.0M1–0M8, 2008. [15] R. Cook and K. Torrance, “A reflection model for computer graphics,” Proc. SIGGRAPH ’81, Vol. 15, No. 3, pp. 307–316, 1981.