Frontiers in Computing and Intelligent Systems ISSN: 2832-6024 | Vol. 7, No. 2, 2024 50 Three Dimensional Reconstruction of Two‐step Moving Objects based on Phase‐shifting Profilometry Yitao Liang, Jiabei Dai, Lei Lu * School of Information Science and Engineering, Henan University of Technology, Zhengzhou Henan, 450001, China * Corresponding author: Lei Lu Abstract: In recent years, 3D object reconstruction based on phase-shifting profilometry has gradually received attention and been widely applied. Domestic and foreign scholars have been continuously researching and exploring the accuracy and speed of three-dimensional measurement, and gradually developing towards dynamic measurement. Most dynamic measurements require projecting multiple stripe patterns to obtain sufficient object phase information, and the more stripes there are, the greater the phase error caused by motion. This article proposes the use of increasing the sampling fringe pattern during the projection period of a fringe pattern to achieve high frame rate dynamic 3D object reconstruction. By combining the intensity values of ambient light; Finally, the phase information of the object is extracted by tracking the motion information obtained from the moving object. This article demonstrates the feasibility of this method through simulation experiments and improves the frame rate of 3D reconstruction of moving objects. Keywords: Phase Shifting Profilometry; 3D Reconstruction; Dynamic Measurement; Double Sampling. 1. Introduction Phase Shifting Profilometry (PSP) is one of the most popular 3D reconstruction techniques, with advantages such as high accuracy and strong robustness [1-4]. Its measurement system consists of a projector and an industrial camera, which projects a set of sine wave fringes containing phase shift onto the object through the projector, and uses the camera to capture images at another angle. The projected fringes undergo deformation due to the height of the object, and the height distribution of the object is included in the fringe deformation information. The fringe deformation is analyzed based on the phase information in the sine wave fringe pattern rather than the intensity of the fringes. After steps such as phase extraction and phase unwrapping, the phase distribution is obtained, and finally combined with system calibration parameters to complete 3D reconstruction. Effective progress has been made in both reconstruction accuracy and efficiency. The research on real-time 3D reconstruction algorithms for dynamic scenes has gradually become a hot and difficult topic of concern for many well-known domestic and foreign researchers, and has made many important contributions. This article studies high frame rate 3D reconstruction for dynamic objects. In recent years, the 3D reconstruction of moving objects based on phase-shifting profilometry has received widespread attention from researchers. Lu et al. [5] reconstructed multiple moving objects based on the dual frequency phase unwrapping method. Dual frequency phase unwrapping correctly unfolds the phase of multiple objects by projecting additional low-frequency fringes. The motion of an object introduces errors between multiple phases shifted fringe patterns and high and low frequency fringe patterns. By tracking object motion and analyzing the impact of motion on phase information, the phase change relationship caused by object motion between the phase shift fringe phase distribution and the low-frequency fringe phase distribution is obtained. Finally, the number of stripe periods is determined through the dual frequency phase unwrapping algorithm. Due to the error caused by the movement of objects during the fringe projection period, Li et al. [6] proposed combining Fourier transform profilometry (FTP) and phase shift profilometry to reconstruct moving objects. First, use the single FTP method and spatial phase unwrapping to extract the continuous relative phase map of each isolated object; Reuse the PSP method to obtain a rough absolute phase map with motion errors for the entire scene; Determine the order of fringes between the absolute phase map and the relative phase map; To generate the final absolute phase map of each isolated object. The motion prediction method reduces phase shift errors caused by moving objects by predicting their motion. Han et al. [7] proposed an effective and fast vibration detection and motion compensation method based on multi frequency phase shift 3D sensors. The proposed method quantitatively indicates the intensity of vibration and compensates for motion errors by correcting the wrapped phase instead of accessing adjacent pixels. The feasibility of this method was verified by simulating different vibration intensities using industrial robot motion. Liu et al. [8] compensated for errors caused by motion by estimating phase shift in the imaging plane of the projector. By comparing the three-dimensional coordinates of the same point in different frames, the motion of an object is estimated. Using the pinhole model of the projector, the relationship between phase and object motion is established to determine the phase shift error caused by motion. Finally, the phase shift error compensation method is used for object 3D reconstruction. In other methods, Wang et al. [9] compensated for errors caused by motion based on the Hilbert transform. The error frequency caused by motion is twice the frequency of the projected fringes. By using Hilbert transform to move the fringe pattern by half a cycle and extract the phase, this phase information is used to compensate for the phase value extracted from the original fringe pattern, thereby reducing phase errors caused by motion. This method is effective for objects with slow changes in surface height. 51 To achieve high frame rate 3D measurement of moving objects, this paper proposes a two-step motion object measurement method based on double sampling. Using the double sampling capture stripe mechanism, a single stripe pattern is projected to continuously capture two images of moving objects. The phase change characteristics of object motion under the same projection stripe is analyzed, and finally, high frame rate 3D reconstruction of moving objects is achieved based on the two stripe patterns. 2. Traditional PSP Principle Assuming that the N-step phase-shifting profilometry reference plane and the stripe pattern captured on the object are [10]: , a cos , 2 1 ⁄ (1) , a bcos , Φ , 2 1 / (2) Among them, , and , are the intensity values of the captured reference plane and object stripe patterns, respectively, 1, … , is the stripe pattern number; a is ambient light, b is intensity modulation, , is the phase value of the reference plane;Φ , is the phase change introduced by the height of the object. Parameters a, b, , , Φ , are unknown parameters, while , and , are known parameters. When the number of stripe patterns exceeds 3, the phase values of the reference plane and object, as well as the ambient light intensity, can be obtained by the following equation: , = arctan -∑ , sin2π(n-1)N n=1 /N ∑ , cos2π(n-1)N n=1 /N (3) , +Φ , =arctan -∑ , sin2π(n-1)N n=1 /N ∑ , cos2π(n-1)N n=1 /N (4) a , = 1+ 2 + 3…..+ N N (5) According to formulas (3)-(4), the phase calculated by the arctangent function is discontinuous and ranges from -π to π. Based on the phase unwrapping algorithm [11], the jumping wrapped phase is unfolded into a monotonically increasing unwrapping phase, and finally combined with system calibration parameters to complete three-dimensional reconstruction. 3. Three Dimensional Reconstruction of Double Sampling and Two-step Moving Objects based on Phase- Shifting Profilometry The traditional stripe capture mechanism projects stripe patterns on a single stripe projection cycle, with the projector projecting the stripe pattern and the camera capturing it once. In terms of capturing frame rate: The commercial projector used in this study typically has a refresh limit of 120Hz for sine fringes with 8-bit grayscale values, while the camera can sample up to several thousand frames. Therefore, the traditional single sampling reconstruction speed is limited by the speed of the projector. If in a dynamic scene, the motion of the measured object will induce phase shift errors, and traditional PSP methods cannot address the problem of measuring dynamic objects. Unlike traditional capture mechanisms, this article uses a dual sampling mechanism to capture the stripes of moving objects, which means that the camera continuously captures the stripe information of moving objects twice during the illumination period of a stripe pattern [9]. The timing diagram is shown in Figure 1. Fig 1. Trigger structure between camera and projector In Figure 1, there is no phase shift caused by the switching of projected stripes in the two images captured under the same projected stripe pattern (such as and ) only phase shift caused by object motion; There is both phase shift caused by object motion and phase shift caused by stripe switching in the two images captured under adjacent projected stripe patterns (such as and ). Analyze the impact of object motion on stripe description, and introduce motion information into stripe description, , and can be represented as [12]: x, y a bcos Φ x, y f x, y , g x, y a bcos Φ x, y f x, y , g x, y a bcos Φ x, y (6) In the above equation x, y , f x, y , g x, y , f x, y , g x, y 2 / ;The rotation and translation matrices that describe the motion of an object are:(f2 x,y ,g2 x,y and (f3 x,y ,g3 x,y . There is only phase shift caused by object motion in and which can be obtained from formula (6): x, y a bcos Φ x, y f x, y , g x, y a bcos Φ x, y (7) Furthermore, it can be concluded that: , , , , = , , (8) When the ambient light a is known, it can be obtained that: Φ x, y = (9) Among them, 52 = x,y -a (10) = f x, y , g x, y a (11) At this point, based on and , the phase value can be obtained, which only includes the phase shift caused by object motion. In summary, when the ambient light is known, the two-step phase-shifting profilometry based on double sampling only requires two stripe patterns to achieve high frame rate 3D reconstruction, as shown in Figure 2. Fig 2. Strategies for fringe pattern projection. (a) The reconstructed frame rate of using traditional PSP; (b) The reconstructed frame rate of the method proposed in this paper 4. Experiments In order to study the feasibility of the above algorithm, the simulation environment for this study is Matlab2016. Firstly, perform a three-dimensional reconstruction of a stationary object as a reference for experimental comparison. Firstly, the traditional three-step phase-shifting profilometry is used to calculate the stationary object. The three stripe patterns in the following image are all 1280 pixels X 1024 pixels. The object in the image is a cube, Figures 3(a), (b), and (c) show the simulated three-step phase-shifting fringe patterns, respectively. Fig 3. Traditional three-step phase-shifting profilometry fringe pattern Figure 4 (a) shows the reconstruction result, Figure 4 (b) shows the calculation of the phase principal value, and the phase distribution on the 400th row of the object surface is shown in Figure 4 (c). Fig 4. Reconstruction results of stationary objects 53 Given ambient light under simulation conditions and a constant value. The object in the image is a cube, moving along the lower right corner, with each movement being 10mm in the x direction and 10mm in the y direction. There is no stripe phase shift between Figure 5(a) and Figure 5(b), only the phase shift generated by motion; There is both a phase shift caused by stripe switching and a phase shift caused by object motion between Figure5 (b) and Figure5(c). Fig 5. Simulated fringe patterns of moving objects under double sampling· Based on the distribution of ambient light intensity and formula 9, the phase of the moving object wrapping is calculated as shown in Figure 6 (a), and the phase distribution on the 400th row of the object surface is shown in the figure 6(b). The 3D reconstruction results of the object can be obtained through phase unwrapping, as shown in Figure 6(c)(d). Fig 6. uses the method proposed in this article to reconstruct moving objects Fig 7. Traditional methods for reconstructing the phase distribution of moving objects 54 To compare the effectiveness of the experimental results, traditional phase-shifting profilometry was used to perform three-dimensional reconstruction of the aforementioned moving object, as shown in Figure 7. Unable to achieve 3D reconstruction. To quantitatively analyze the effectiveness of the method proposed in this article, the measurement results when the object is stationary are used as the true values to evaluate the effectiveness of the proposed method and traditional methods, as shown in Table 1. The mean and root mean square errors of traditional algorithms are 0.2176 millimeters and 6.2172 millimeters, respectively. The method proposed in this paper has mean and root mean square errors of 0.0707 millimeters and 0.0713 millimeters, respectively. Table 1. Measurement error evaluation Method Mean error (mm) RMS error (mm) Tradition PSP 0.2176 6.2172 Our method 0.0707 0.0713 5. Conclusion This article proposes a dual sampling and two-step 3D reconstruction method for moving objects based on phase- shifting profilometry. By controlling the relationship between the projector and camera, two moving object fringe patterns were collected under one projected stripe, and the analysis revealed that only object motion phase shift existed under the same projected stripe. Finally, high frame rate 3D reconstruction of moving objects was achieved based on two stripe images with known ambient light. References [1] Zuo C, Feng S, Huang L, et al. Phase shifting algorithms for fringe projection profilometry: A review[J]. Optics and Lasers in Engineering, 2018, 109: 23-59. doi: 10.1016/ j. optlaseng. 2018.04.019. [2] Zhang, S. High-speed 3D shape measurement with structured light methods: A review[J]. Optics and Lasers in Engineering, 2018, 106,119–131. doi: 10.1016/ j.optlaseng.2018.02.017. [3] Feng S, Zuo C, Zhang L, et al. Calibration of fringe projection profilometry: A comparative review[J]. Optics and lasers in engineering, 2021, 143: 106622. doi:10.1016/j. optlaseng. 2021. 1066. [4] Yin Yongkai, Zhang Zonghua, Liu Xiaoli, et al. Review of the system model and calibration for fringe projection profilometry [J]. Infrared and Laser Engineering, 2020, 49(3):0303008-0303 008-18.doi:10.3788/IRLA202049.0303008. [5] Lu L, Jia Z, Luan Y, et al. Reconstruction of isolated moving objects with high 3D frame rate based on phase shifting profilometry [J].Optics Communications,2019,438:61-66.doi: 10. 1016/j.optcom.2018.12.092. [6] Guo W, Wu Z, Li Y, Liu Y, Zhang Q. Real-time 3d shape measurement with dual-frequency composite grating and motion-induced error reduction[J].Optics Express, 2020, 28 (18): 26882–97.doi: 10.1364/OE.403474. [7] Han L, Li Z , Zhong K , Cheng X , Luo H , Liu G , et al. Vibration detection and motion compensation for multi- frequency phase-shifting-based 3d sensors[J]. Sensors,2019, 19(6): 1368.doi:10.3390/s19061368. [8] Liu Z, Zibley PC , Zhang S . Motion-induced error compensation for phase shifting profilometry[J].Optics Express, 2018, 26(10):12632–7.doi: 10.1364/OE.26.012632. [9] Wang Y, Suresh V , Li B . Motion-induced error reduction for binary defocusing profilometry via additional temporal sampling [J].Optics Express,2019,27(17):23948–58.doi: 10. 1364/ OE.27.023948. [10] Wei Y, Lu L, Xi J. Reconstruction of moving object with single fringe pattern based on phase shifting profilometry[J]. Optical Engineering, 2021, 60(8): 084106. doi: 10.1117/1. OE. 60.8. 084106. [11] Li Z. Research on Structured Light 3D Measurement Technology and System Based on Digital Grating Projection [D]. Huazhong University of Science and Technology,2009. (in Chinese). [12] Lu L, Xi J , Yu Y ,et al.New approach to improve the accuracy of 3-D shape measurement of moving object using phase shifting profilometry[J].Optics Express,2013,21(25):30610- 30622.doi:10.1364/OE.21.030610.