paperISPIV2021_Sciacchitano_LPT_July052021 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 Main results of the first Lagrangian Particle Track ing Challenge A. Sciacchitano 1*, B. Leclaire 2, A. Schröder 3 1 Delft University of Technology, Faculty of Aerospace Engineering, Delft, The Netherlands 2 ONERA, Department of Aerodynamics, Aeroelasticity and Acoustics, Meudon, France 3 DLR, Institute of Aerodynamics and Flow Technology, Göttingen, Germany *corresponding author: a.sciacchitano@tudelft.nl Abstract This work presents the main results of the first Lagrangian Particle Tracking challenge, conducted within the framework of the European Union’s Horizon 2020 project HOMER (Holistic Optical Metrology for Aero-Elastic Research), grant agreement number 769237. The challenge, jointly organised by the research groups of DLR, ONERA and TU Delft, considered a synthetic experiment reproducing the wall-bounded flow in the wake of a cylinder which was simulated by LES. The participants received the calibration images and sets of particle images acquired by four virtual cameras, and were asked to produce as output the particles positions, velocities and accelerations (when possible) at a specific time instant. Four different image acquisition strategies were addressed, namely two-pulse (TP), four-pulse (FP) and time-resolved (TR) acquisitions, each with varying tracer particle concentrations (or number of particles per pixel, ppp). The participants’ outputs were analysed in terms of percentages of correctly reconstructed particles, missed particles, ghost particles, correct tracks and wrong tracks, as well as in terms of position, velocity and acceleration errors, along with their distributions. The analysis of the results showed that the best-performing algorithms allow for a correct reconstruction of more than 99% of the tracer particles with positional errors below 0.1 pixels even at ppp values exceeding 0.15, whereas other algorithms are more prone to the presence of ghost particles already for ppp < 0.1. While the velocity errors remained contained within a small percentage of the bulk velocity, acceleration errors as large as 50% of the actual acceleration magnitude were retrieved. 1 Introduction Since the introduction of tomographic particle image velocimetry (shortly tomo-PIV, Elsinga et al., 2006, Scarano, 2012) for volumetric flow velocity measurements, much research has been conducted aiming at enhancing the measurement accuracy and spatial resolution, as well as reducing the computational cost of the image analysis. In the first years of tomo-PIV development, the three-dimensional distribution of tracer particles in the measurement domain was performed via a voxel- based reconstruction of their intensities, using the MART approach (Herman and Lent 1976) or one of its evolutions (multiplicative first guess MFG, Worth and Nickels, 2008; multiplicative-line of sight MLOS-MART, Atkinson and Soria, 2009; motion tracking enhancement MTE-MART, Novara et al., 2010, among others). The three-dimensional velocity field was then evaluated via cross-correlation analysis (Scarano, 2012), similar to planar PIV. The sparsity of the particles distribution in three-dimensional space has been exploited to enhance the reconstruction accuracy (Champagnat et al., 2014) as well as to increase the computational efficiency (Cornic et al., 2015). For double-frame recordings, Cornic et al. (2020) recently introduced the double-frame tomographic PTV (DF-TPTV) approach that first uses voxel grids to find the possible position of particle candidates and obtain a coarse predictor of their displacements via a correlation analysis, and finally determines the individual particles’ intensities and exact positions via a global optimisation procedure. Wieneke (2013) proposed an iterative particle reconstruction (IPR) algorithm where the distribution of the tracer particles in the volumetric measurement domain is represented from the beginning by three-dimensional positions and intensity values rather than by a voxel-based intensity distribution. The IPR algorithm requires detailed knowledge of the spatially varying optical transfer function (OTF) between the locations in the physical domains and the cameras pixels, which is achieved via the calibration and application of a non-uniform OTF as introduced by Schanz et al. (2012). The IPR approach was compared with the conventional three-dimensional triangulation, MLOS, and MART, exhibiting higher performances than the former two approaches and similar performances to MART up to ppp =0.05. Schanz et al. (2016) combined the iterative particle reconstruction from Wieneke (2013) with the temporal information from time-resolved measurements to predict the particles’ locations at successive time instants and correct their positions by a local image matching (‘shaking’) step, which therefore identifies particles’ tracks thus performing Lagrangian Particle Tracking (LPT). The approach, named Shake-The-Box (STB), enables the evaluation of the velocity and Lagrangian acceleration of individual tracer particles at high particle image densities exceeding 0.1 ppp. Comparison with the conventional correlation-based tomo-PIV processing showed higher performance of the STB approach in terms of number of correctly reconstructed particles, accuracy of the reconstruction, and suppression of ghost particles (Kähler et al., 2016). For high-speed flows, where time-resolved image recording is often unfeasible due to hardware limitations, the applicability of the STB approach has been recently extended via the concept of Multi-Pulse STB employing multiple systems (Novara et al., 2016a) or exposures (Novara et al., 2019). 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 Alternative solutions to the problem of particles reconstruction and flow field evaluation have been proposed over the years. Yang et al. (2018) proposed modification to the “shaking” phase of the STB algorithm and resolved the optimization with the ensemble technique. This technique is further integrated into the Kernelized Lagrangian Particle Tracking (KLPT) approach in Yang and Heitz (2021), in which a sampling-learning-detection strategy is adopted. A dataset describing an ensemble of the possible state of one particle with their back-projections is firstly sampled. Then a function is learned from this sampled dataset that maps the image intensities to the physical coordinates. Finally, the particle position is detected by applying the learned function to image recordings. To improve the initialisation accuracy of STB and KLPT, Khojasteh et al. (2021) proposed using Lagrangian Coherent Structure (LCS) to check if an initialised particle is locally coherent with its neighbour coherent motions. Lasinger et al. (2018) introduced an approach for particle reconstructions based on the joint minimisation of an energy function accounting for the deviation between reconstructed particles and image recordings as well as for the sparsity of the particles in the three-dimensional space. In the same work, the velocity field was evaluated using a variational model that included physical information on the flow, such as incompressibility and viscosity. The approach was further developed in Lasinger et al. (2019), where the tracer particles’ distribution and the velocity field were jointly reconstructed via an integrated energy minimisation process. In the Lagrangian PIV approach proposed by Yang et al. (2019), an Eulerian description of the velocity field is sought which minimises the positional difference between the particle image recordings at time k and the back-projected particles reconstructed from the information at previous time instants. From the discussion above, it emerges that the evaluation of the three-dimensional flow fields from particle image recordings is a topic of active research, and that different research groups have tackled this problem with different approaches. The aim of the first LPT challenge, whose main results are summarised in this work, is to comparatively assess the different approaches in terms of the accuracy of the particles’ reconstruction and of their velocities and accelerations via the use of a dedicated synthetic database. 2 Dataset and test cases 2.1 Dataset description This section provides a brief description of the dataset employed in the LPT challenge. A more detailed description of the simulation parameters is reported in the communication of Leclaire et al. (2021), also presented at the ISPIV 2021 symposium. The synthetic experiment reproduces the turbulent wall-bounded flow in the wake of a cylinder, considered representative of many turbulent flows where large fluctuations both in the velocity and the pressure take place. Whereas the simulation was performed for an air flow in quasi-incompressible conditions, scaling was conducted to transpose it to the virtual experimental context of water at bulk velocity Vꝏ = 0.667 m/s. The cylinder had a diameter D = 0.01 m, and was located at a gap distance G = 0.01 m from the wall, where a turbulent boundary layer was present with thickness δ ≈ 60 mm; the momentum thickness Reynolds number 10 mm upstream of the cylinder was Reθ = 4,150. According to literature (Wang and Tan 2008), these conditions lead to vortex shedding in ground effect, thus yielding large pressure fluctuations on the wall. A sample of the instantaneous flow field is shown in Figure 1. The flow region used for the LPT challenge has dimensions of 0.1 � × 0.05 � × 0.03 � (∆X × ∆Y × ∆Z, being X, Y and Z the streamwise, spanwise and wall-normal directions respectively). Such domain is centred in span, with its upstream face located 0.035 � downstream of the cylinder centre, and its bottom face located at the wall. Figure 1: Snapshot of the instantaneous flow field, illustrating iso-surfaces of the Q-criterion colour-coded by streamwise velocity u, and contours of the static pressure. Synthetic particle images were generated considering four virtual cameras with sensor size of 1920×1200 pixels and 10 μm pitch, using pinhole projection. No Scheimpflug nor image distortion were simulated. The cameras viewed the measurement domain mainly from above, and were located along the X-axis at Y = 0 and height of about 600 mm above the wall, at angles of [-30, -10, 10, 30] degrees respectively. Based on the simulated cameras’ locations and lenses’ focal lengths (100 mm), each 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 back-projected pixel ( ���) (defined with a value close to the average size of pixel back-projections through the volume) corresponded to 60 µm in the object space. The particles were given a poly-disperse intensity distribution; the particle images were produced with a Gaussian (σ = 0.6) point-spread function to model diffraction limited imaging. Thermal and shot noise were also added to the images. 2.2 Test cases Three different image acquisition strategies were considered, namely two-pulse (TP), four-pulse (FP) and time-resolved (TR). The TP case consisted of only one image pair with time separation ∆t = 600 µs, corresponding to a maximum pixel displacement of about 7 pixels in the camera images. In the FP case, the time separations between the four exposures t0, t1, t2, t3 of a sequence were [2∆t, ∆t, 2∆t] (again, with ∆t = 600 µs), with the first two exposures captured in the first image and the last two exposures captured in the second image. Finally, for the TR case, sequences of 50 or 100 images were considered depending of the number of particles per pixels (ppp), with time separation between two successive images of ∆t = 600 µs. The datasets were generated at ppp values of 0.005, 0.025, 0.05, 0.08, 0.12, 0.16 for the TP and FP cases, and also of 0.2 ppp for the TR case. The corresponding tracers’ concentrations was in the range between 0.043 particles/mm3 (ppp = 0.005) and 1.7 particles/mm3 (ppp = 0.2). Calibration data, namely list of calibration points with their coordinates in the physical domain and the projections in the four camera images, were provided to the participants without any error. In the TP case, the participants were asked to produce as output the positions of the tracer particles at the time instants t0 (X0, Y0, Z0) and t1 (X1, Y1, Z1), respectively. In the FP case, the requested output consisted of the raw particles’ positions at the four time instants, plus the three components of the position (XM, YM, ZM), velocity (VX, VY, VZ) and acceleration (AX, AY, AZ) at the intermediate time instant � = ��� + ��� 2⁄ . In the TR case, the participants needed to provide the raw particles’ positions (X, Y, Z) as well as the fitted positions (Xfit, Yfit, Zfit), the velocity (VX, VY, VZ) and the acceleration (AX, AY, AZ) at time instant 40 for ppp ≤ 0.12 and 90 for ppp ≥ 0.16. The results could be submitted for any of the three cases, analysing the datasets from ppp = 0.005 to the maximum ppp value handled by the participants’ algorithms. The datasets were publicly released for download on March 9th, 2020 (https://w3.onera.fr/first_lpt_and_da_challenge/). Participants were then requested to upload their processed results by July 17th, 2020. 2.3 Data analysis The data uploaded by the participants were analysed via the evaluation of the following parameter: - Percentage of correct particles: a reconstructed particle was defined as “correct” if its location fell within 60 µm ( or 1 ���) from a true particle; - Percentage of false particles (or ghost): a reconstructed particle was defined as “false” (or ghost) if its distance from a true particle exceeded 60 µm (or 1 ���); - Percentage of false negatives (or missed particle): a true particle was considered missed when no particle was reconstructed within 60 µm (or 1 ���) from the former; - Percentage of correct tracks: a track was defined correct when composed by all correct particles; - Percentage of wrong tracks: a track was defined wrong when at least one particle of the track was not correct (either missed or ghost); - Error of the particles’ position (for all cases), velocity and acceleration (only for the FP and TR cases), in terms of the mean error magnitude, as well as error distribution to detect the presence of any bias. It should be noted that the errors were evaluated only for the correct particles (not for the ghost particles), as the difference between the reconstructed particle position, velocity or acceleration and the actual values of the corresponding true particles. 3 Participant and approaches A total number of six research groups participated to the LPT challenge, namely the Swiss Federal Institute of Technology in Zürich (ETHZ), the German Aerospace Centre from Göttingen (DLR), the French National Research Institute for Agriculture, Food & Environment (INRAE), the Kutateladze Institute of Thermophysics in Russia (IOT), the French Aerospace Lab (ONERA) and the German instrumentation company LaVision GmbH. A summary of the participating groups and the algorithms used is reported in Table 1. Table 1: List of cases of the LPT challenge, participants and their algorithms. Case Participant Algorithm Reference TP DLR Multi-Pulse Shake-The-Box Novara et al. (2019) ETHZ 3D Fluid Flow Estimation with Integrated Particle Reconstruction Lasinger et al. (2019) INRAE Lagrangian PIV Yang et al. (2019) LaVision Multi-Pulse Shake-The-Box Novara et al. (2019) ONERA Iterative Double Frame Tomographic PTV Cornic et al. (2020) FP DLR Multi-Pulse Shake-The-Box Novara et al. (2019) LaVision Multi-Pulse Shake-The-Box Novara et al. (2019) TR DLR (Time-resolved) Shake-The-Box Schanz et al. (2016) 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 INRAE Kernelized Lagrangian Particle Tracking Yang and Heitz (2021) INRAE Lagrangian PIV Yang et al. (2019) INRAE Lagrangian Coherent Track Initialization Khojasteh et al. (2021) IOT OpenLPT Tan et al. (2020) LaVision (Time-resolved) Shake-The-Box Schanz et al. (2016) 3.1 Two-pulse test case The following groups participated to the TP case of the LPT challenge: DLR, ETHZ, INRAE, LaVision and ONERA. All participants processed the data in the entire data of ppp values from 0.005 to 0.16, with the exception of ONERA and INRAE that processed the data only up to ppp = 0.08. The algorithms used are briefly summarised hereafter. 3.1.1 DLR: Multi-Pulse Shake-The-Box The approach used by the DLR group is the multi-Pulse Shake-The-Box from Novara et al. (2019). The approach first reconstructs the particles in three-dimensional space via the use of an advanced iterative particle reconstruction (IPR) from Jahn et al. (2021), based on Wieneke (2013). Then, a displacement predictor is obtained via the application of the Particle Space Correlation algorithm (PSC, Novara et al., 2016b) between the reconstructed particles of the two frames. For each particle in the first frame IPR reconstruction, a search radius δ2p is established around the predicted location to generate two- pulse track candidates. Possible ambiguities between track candidates are solved by selecting the candidate that exhibits the lowest value of a cost function, which accounts for the variation of the peak intensity along a track and the difference with respect to the predicted position. Two iterations are performed, with δ2p = 1 pixel and 10 pixels, respectively. 3.1.2 EHTZ: integrated particle reconstruction The ETHZ group used a variational approach to jointly estimate the sparse 3D particle locations at the reference time step and the dense flow field on a regular grid. Particles at the reference time step are triangulated iteratively, similar to IPR (Wieneke, 2013). The individual iteration steps are alternated with the optimization of the energy function: ( ) ( ) ( ) ( )1 , , , , 2 2D S SpE P C U E P C U E U E C λ µ≡ + + (1) where P, C and U represent the set of particles positions, the set of particles intensities, and the estimated flow field, respectively. The data term ED penalizes deviations between predicted and observed images by evaluating the 2D reprojection error in all camera views for both time steps. For the second time step, the particles are displaced by the estimated flow field U. The smoothness term ES is derived from the stationary Stokes equations and enforces a divergence-free flow field as well as a quadratic regularization per component of the flow gradient. The sparsity term ESp enforces sparsity of the reconstructed particle set by suppressing low-intensity ghost particles. For camera calibration, the polynomial camera model of Soloff et al. (1997) (38 parameters) is fitted. The output of this approach is a dense flow field on a regular grid. Particle locations for the second time step were, thus, obtained from displacements interpolated from the estimated flow field. For higher seeding densities (ppp from 0.05 to 0.16) a grid resolution of 200×100×60 was used, while for lower seeding densities (0.005, 0.025), a coarser grid of 167×84×50 was employed. Further details on the algorithm are reported in Lasinger et al. (2019). 3.1.3 INRAE LaPIV The INRAE group processed the TP data with the Lagrangian PIV algorithm (LaPIV) from Yang et al. (2019). The algorithm first builds the particle positions in object space from the first frame using an IPR-like method (Wieneke, 2013). Then, the best Eulerian velocity field is sought that minimises a cost function accounting for the discrepancy between the image recording at the second time instant and the back-projected image using the optical transfer function OTF (Schanz et al., 2012). The particle positions in the second frame are reconstructed by integrating the interpolated Eulerian velocity field, thus finding a temporal link between the two frames in accordance with the flow. Finally, the particles positions at the second frame are further optimised using one step of the Kernelized LPT approach (KLPT, Yang et al., 2018) to further increase accuracy. For the two-pulse case, the algorithm is reported to diverge for seeding concentrations beyond ppp = 0.08, because it is unable to accurately reconstruct the initial particles field due to the high seeding density. 3.1.4 LaVision: Two-pulse Shake-The-Box Similarly to the DLR group, LaVision employed the two-pulse Shake-The-Box algorithm (Jahn et al., 2017), which makes use of an iterative combination of IPR and particle tracking. With respect to DLR, LaVision used more iterations, namely from 8 to 20 depending on the ppp value. For ppp > 0.12, an intermediate filtering of the trajectories via the use of a median filter was also performed to remove spurious trajectories. 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 3.1.5 ONERA: Iterative double-frame tomographic PTV The algorithm used by ONERA for processing the TP data is an improved version of the Double Frame Tomographic PTV (DF-TPTV) algorithm from Cornic et al. (2020). The DF-TPTV approach involves three stages. Firstly, particles reconstruction is performed on a fine voxel grid with a sparsity-based algorithm (Needell and Tropp, 2009) for each time step. Secondly, these reconstructions are expanded on a coarser grid, on which 3D correlation is performed (Cheminet et al., 2014), yielding a predictor displacement field that allows to efficiently match particles at the two time instants. As these particles are still located on a voxel grid, the final step achieves particle position refinement to their actual sub-voxel position by a global optimisation process, also accounting for their intensities. Residual images are built based on the unmatched particles; the three stages of the DF-TPTV approach are then iterated on the residual images, using a predictor estimated from the previously optimised particles. 3.2 Four-pulse test case Only DLR and LaVision processed the data of the FP test case, both using their implementations of the Multi-Pulse Shake- The-Box (MP-STB) algorithm from Novara et al. (2019). Similarly to the algorithm described in section 3.1.1 for the TP case, the approaches make use of advanced IPR (Jahn et al. 2021) and IPR (Wieneke, 2013) and a displacement predictor by Particle Space Correlation (PSC, Novara et al., 2016b). As for the TP case, for each particle in the first frame IPR reconstruction, a search radius δ2p is established around the predicted location to generate two-pulse track candidates. The latter are then extrapolated backwards and forwards and an additional search radius (δ4p) is adopted to identify four-pulse track candidates. The cost function used to solve ambiguities between candidates accounts not only for the variation of the particle peak intensity along the track and the distance from the predicted position as in the TP case, but also for the mean acceleration magnitude. While DLR used 21 iterations of the MP-STB algorithm, LaVision used a number of iterations from 10 to 100 depending on the ppp. Both groups made use of a second order polynomial regression on the reconstructed particles positions to extract the position, velocity and acceleration at the intermediate time instant (time tM). Additionally, LaVision employed an intermediate spatial filtering of the trajectories via a median filter for ppp > 0.08 to remove spurious trajectories. Also, for ppp> 0.12 LaVision applied a time-average reference velocity field as a predictor for the initial reconstruction. In addition to the data processed by DLR and LaVision, a “Hacker” participant was also considered, following the approach of the 4th PIV challenge (Kähler et al., 2016). Hacker used the exact particle positions at times t0, t1, t2, and t3, and fitted a second order polynomial through the particles of a track to determine the position, velocity and acceleration at the intermediate time instant. The comparison between the results from Hacker and those from the other participants allows to evaluate to which extent the errors on position, velocity and acceleration are caused by errors in the particles reconstruction and due to the applied image noise, rather than by the temporal resolution of the simulated experiments. 3.3 Time-resolved test case Four research groups processed the data of the TR test case, namely DLR, INRAE, IOT and LaVision. The INRAE group used three different algorithms (Kernelized LPT, Lagrangian PIV, and Lagrangian Coherency-based Tracks Initialisation), which are explained in the reminder of this section. Additionally, as for the FP case, a “Hacker” participant made use of the exact particle positions at three time instants centred around the time instant of interest, and evaluated the velocity and acceleration via a second order least-square polynomial regression and analytical time derivation. The range of datasets processed by each participant are summarised in Table 2. Table 2: Datasets (ppp cases) processed by the participants for the TR case. 0.005 0.025 0.050 0.080 0.120 0.160 0.200 DLR        INRAE KLPT        INRAE LaPIV        INRAE LCTI        IOT        LaVision        Hacker        3.3.1 DLR: Time-resolved Shake-The-Box The DLR group processed the images of the TR case using a three-pass Shake-The-Box algorithm (Schanz et al., 2016). The processing chain was optimised for the highest seeding density (0.2 ppp) and then applied to all other cases. For all seeding densities, the final 25 images of the time series were processed (25-49 for ppp ≤ 0.12; 75-99 for ppp ≥ 0.16). The first five images of the first pass used an extensive particle reconstruction via an advanced IPR approach (Jahn et al., 2021; Wieneke, 2013). The allowed triangulation radius was gradually increased from 0.4 to 1.0 pixels. After each triangulation iteration, 16 sub-iterations of particle shaking using an image matching scheme with analytic cost function (Jahn et al., 2021) were carried 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 out. The tracks selection was based on a Wiener-filter-type fit. Starting from the fifth time step, a predictor for the identification of new tracks was constructed from the average velocity from the six closest neighbouring tracked particles, with a search radius of 4.5 pixels. After the STB processing, the raw particle positions were fitted using the TrackFit approach (Gesemann et al., 2016), which employs a system of 1D cubic-B-splines to yield a continuous and smooth function for each dimension of the track. 3.3.2 INRAE: KLPT, LaPIV and LCTI The INRAE group made use of three algorithms. The Kernelized LPT (KLPT) approach (Yang and Heitz, 2021) initializes the tracer particle distribution in physical space via the IPR method (Wieneke, 2013). Then, KLPT adopts a sampling strategy that firstly forecasts a cloud of one particle’s possible position at time k from the particle’s sampled history up to time k–1, followed by projecting the ensemble prediction into image space using the OTF. Successively, a function mapping the image pixel intensities to the particles’ 3D coordinates is learned by minimizing a regularized empirical risk. This risk measures the discrepancy between the sample-predicted position at time k and the mapping function output value taking the sampled image projection as input. Finally, the learned function is applied to the image recording at time k to yield the best flow variable. Such optimisation problem is solved using kernel methods (Hofmann et al., 2008). The raw particles positions were then regularised using a spline method. The velocity and acceleration were computed via analytical derivation in time of the spline curves. For track lengths smaller than five samples, the finite difference method was used instead of the spline regularisation. At the highest seeding densities (ppp ≥ 0.12), the images were processed also with the Lagrangian PIV approach (LaPIV, Yang et al., 2019), which has been described already in section 3.1.3 for the image analysis of the TP case. Finally, the Lagrangian Coherent Tracks Initialisation (LCTI) approach from Khojasteh et al. (2021) was also employed. The method initialises new tracks at every iteration of the STB or KLPT processing based on the local Lagrangian information of the particles. In particular, the concept of the Finite-Time Lyapunov Exponent (FTLE, Ott, 2002) is exploited locally to identify LCS ridges (e.g. boundaries) in the flow field, and ensure that the newly-added and the recovered lost tracks are coherent with the neighbouring existing tracks. 3.3.3 IOT: Open-LPT The IOT group processed the TR data via the STB algorithm C++ implementation from the OpenLPT project (Tan et al. 2020). The following STB parameters were used: 2D particle image detection with intensity threshold of 24 counts and 3- point Gaussian subpixel interpolation; maximum triangulation error of 0.015 mm (0.255 pixels); 4 outer and inner loop iterations, with the shaking step of 0.04 mm (0.68 pixels). The possible particle’s shift between two consecutive frames was limited to 0.42 mm (7.14 pixels). The search radius for particles detection based on a predictor was 0.1 mm (1.7 pixels). Three initial predictor fields were obtained by particle space correlation on a Cartesian grid with 1.25 mm spacing, using spherical interrogation windows of 2 mm radius. The resulting velocity at the particles’ locations were validated via a spatial moving average filter. The particles positions at successive time instants were predicted using a Wiener filter. Weighted cardinal B-splines (Gesemann et al. 2016) were employed to regularise the particles’ positions and in turn determine their velocities and accelerations. 3.3.4 LaVision: Time-resolved Shake-The-Box LaVision made use of the DaVis 10 implementation of the Shake-The-Box algorithm (Schanz et al., 2016), employing four- frame track search initialisation. For ppp > 0.12, spatial filtering of the trajectories via a median filter was conducted during the STB iterations to remove spurious tracks. For ppp = 0.2, the particles’ reconstruction was carried out marching both forward and backward to enhance its accuracy. The particles’ raw positions were regularised with a second order polynomial least square regression over a kernel of 7 samples, from which the velocity and acceleration were determined via analytical differentiation in time. 4 Results 4.1 Two-pulse test case It is well established that the particles’ reconstruction accuracy is dependent on the particles’ concentration or number of particles per pixels (ppp) (Scarano, 2012), with decreasing quality of reconstruction for increasing ppp. Figure 2 illustrates the true particles (red crosses) and the reconstructed particles (black circles) in sub-domains of the measurement volume, for selected increasing ppp values from 0.005 (top row) to 0.16 (bottom row). At the lowest ppp value, most algorithms succeed in correctly reconstructing almost the totality of the true particles, with no ghost particles or missed particles appearing in the considered sub-volume. The only exception is the result from ETHZ, which presents several ghost particles and a few missed particles. When the ppp value is increased to 0.08, the difference in performance among the processing algorithms becomes more noticeable. The DLR and LaVision implementations of STB enable to reconstruct correctly a very high percentage of the true 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 particles without the appearance of ghost particles. Also, the INRAE LaPIV approach shows good particles reconstruction capabilities, with only a few ghost and missed particles present. The other processing algorithms, instead, are more prone to ghost particles (ETHZ) or missed particles (ONERA). The DLR and LaVision algorithms retain their high performances even at the highest ppp value of 0.16, where only a few ghost and missed particles appear in the domain. Conversely, the result from the ETHZ algorithm maintains a large amount of ghost particles in the analysed sub-domain. Figure 2: Comparison between true particles (red crosses) and reconstructed particles (black circles) in sub-domains of the entire measurement volume, for the two-pulse (TP) case. Top: ppp = 0.005. Middle: ppp = 0.08. Bottom: ppp = 0.16. Notice that, for sake of clarity, smaller sub-volumes are selected for the ppp = 0.08 and ppp = 0.16 cases. The results in terms of percentages of correctly reconstructed particles and false positives (ghosts) are illustrated in Figure 3. The percentages are computed relative to the number of true particles in the measurement domain, averaged over the two time instants. It can be noticed that the results from DLR and LaVision exhibit a close-to-perfect reconstruction, with nearly all particles correctly reconstructed at all ppp values, and no ghost particles. Also, the ETHZ algorithm correctly reconstructs more than 90% of the true particles at all the ppp values; however, the number of ghost particles is equal to or even exceeding that of the true particles. The results of the ONERA and INRAE LaPIV algorithms have been submitted only up to ppp = 0.08; within these range of seeding concentrations, the approaches exhibit good particle reconstruction capabilities, with over 95% of the true particles correctly reconstructed. The INRAE algorithm is found to be more prone to ghost particles, with up to 5% false negatives recorded at ppp = 0.08. The positional mean error magnitude as a function of the ppp is illustrated in Figure 4. At the lowest ppp (equal to 0.005), all algorithms exhibit a positional error of around 0.04 ���, except the ETHZ algorithm which features an error of 0.28 ���. In general, the positional error increases with the seeding concentration, although the slope of the increase varies depending on the algorithm. For the DLR approach, the increase is very mild and the error value reaches 0.06 ��� at the highest ppp of 0.16. The error increase is slightly higher for the LaVision implementation of the two-pulse Shake-The-Box algorithm, with the error reaching 0.10 ��� at ppp = 0.16. The INRAE LaPIV result follows closely that of LaVision up to ppp = 0.08. The ONERA algorithm exhibits a higher error increase with ppp, with error values of 0.17 pixels at ppp = 0.08. The error from the ETHZ approach is in the range 0.2-0.3 ��� at all seeding concentration analysed and shows a non-monotonic variation with the ppp. 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 Figure 3: Left: percentage of the correct particles (relative to the number of true particles), as a function of the number of particles per pixels (ppp) in the images. Right: percentage of false particles (ghosts) as a function of the ppp. The symbol keys apply to all figures. All results are for the two-pulse (TP) test case. Figure 4: Mean positional error magnitude (considering all three position components, and both time instants) as a function of the ppp, for the TP test case. The analysis of the histograms of the positional errors (not shown here) highlights that the positional errors along the Z-direction are two to three times larger than those along the X- and Y-directions due to the given limitation of the camera system aperture (+/- 30 °), and that the random errors dominate over the bias errors. The latter are typically null, except for the X-position component with the ONERA algorithm and the Z-position component with the LaVision algorithm, where they remain within 0.02 ���. Additionally, the errors appear to be uniformly distributed in the entire measurement domain, with no significant spatial variations. 4.2 Four-pulse test case The distribution of true particles (red crosses) and reconstructed particles (black circles) in sub-domains of the measurement volume for two sample ppp values, namely 0.050 and 0.160, are shown in Figure 5. At the lower seeding concentration of the two, both the DLR and the LaVision algorithm correctly reconstruct all the particles in the visualised sub-volume, with neither ghost nor missed particles. At the highest ppp of 0.160, the two algorithms still retain their high performance in particles reconstruction; only a few particles are missed by the LaVision algorithm, whereas the DLR algorithm correctly reconstructs all of them. Even at this high value of the seeding concentration, no ghost particles appear. The quantitative analysis of the percentages of correctly reconstructed particles, false particles (ghosts), correct tracks and wrong tracks is presented in Figure 6. The DLR algorithm exhibits outstanding performances, with nearly all particles and tracks correctly reconstructed at all the analysed seeding concentrations, and percentages of ghost particles and wrong tracks below 0.05% and 0.15%, respectively. The LaVision implementation features a slightly higher sensitivity to the seeding 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 concentration, with a small decrease of the performance at the highest ppp. Nevertheless, the percentages of correctly reconstructed particles and tracks remain above 90% even at the highest ppp, and the amount of ghost particles and wrong tracks does not exceed 0.15% and 0.6%, respectively. Figure 5: Comparison between true particles (red crosses) and reconstructed particles (black circles) in sub-domains of the entire measurement volume, for the four-pulse (FP) case. Top: ppp = 0.05. Bottom: ppp = 0.16. Notice that, for sake of clarity, a smaller sub- volume is selected for the ppp = 0.16 case. Figure 6: Percentages of correct particles (top-left), missed particles (ghosts, top-right), correct tracks (bottom-left) and wrong tracks (bottom-right) as a function of the ppp. The symbol keys apply to all figures. All results are for the four-pulse (FP) test case. The mean error magnitudes of position, velocity and acceleration, all evaluated at the intermediate time instant tM, are illustrated in Figure 7 for varying seeding concentrations. As explained in section 3.2, also the results from a fictitious participant “Hacker” are shown for reference; the latter made use of the exact particles positions and a second-order polynomial regression to retrieve position, velocity and acceleration at the intermediate time instant. Hence, the results from Hacker are regarded to as the minimum errors attainable in case of perfect particles reconstructions. From Figure 7 top-left, it is clear that the minimum positional error of the DLR and LaVision algorithms is almost one order of magnitude larger than that from Hacker, indicating a clear margin for further improvement of the reconstruction accuracy. The positional error 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 increases linearly with the seeding concentration. The error increase is mild with the DLR algorithm, with maximum errors below 0.05 ��� at the highest ppp; a steeper increase occurs with the LaVision algorithm, although the errors remain well below 0.1 ��� even at ppp = 0.16. Instead, the positional errors from Hacker are independent of the seeding concentration and attain a value of about 0.005 ���. The velocity retrieved with the DLR algorithm exhibits an error of about 0.4% of Vꝏ (Figure 7 top-right), with negligible variations with the ppp. Such error is of the same order as that of Hacker (0.3% of Vꝏ), thus indicating that the temporal modulation due to the finite temporal resolution plays a significant role. The velocity error of the LaVision algorithm shows more pronounced variations with the seeding concentration, increasing from 0.4% to about 0.6% of Vꝏ at the highest seeding concentration. The mean acceleration error magnitude (Figure 7 bottom-left) is of the order of 10 m/s2, or 22.5% of the reference acceleration Vꝏ 2/D. The values of the two algorithms are very close to those of Hacker, and show only minor variations with the amount of particles per pixel. As for the velocity, this result suggests that the error is mainly dominated by the temporal resolution effects, rather than by the uncertainty in the particles positions. For reference, the mean acceleration magnitude is plotted as a thick grey line, which attains a value of 15 m/s2. Hence, it can be concluded that the errors of the measured acceleration are as large as 60% to 80% of the mean acceleration magnitude. Figure 7: Mean error magnitude of position (top-left), velocity (top-right) and acceleration (bottom-left) as a function of the ppp. All quantities are computed at the intermediate time instant tM and only for the correct tracks. The symbol keys apply to all figures. The thick grey line in the acceleration error plot represents the mean acceleration magnitude. 4.3 Time-resolved test case The use of temporal information is expected to enhance the quality of the particles’ reconstruction, thus enabling accurate particles’ reconstructions at higher ppp values. Figure 8 illustrates the distribution of true particles (red crosses) and reconstructed particles (black circles) in sub-domains of the measurement volume, for values of the ppp equal to 0.005, 0.08 and 0.2. At the lowest seeding concentration, all algorithms are able to reconstruct the large majority of the particles, with only a few particles missed by the approach submitted by IOT (first row of Figure 8). Similar performances are achieved also at ppp = 0.08, with almost no missed or ghost particles for all algorithms, except for the IOT result. At the highest ppp level of 0.2, larger differences among the algorithms are reported. The IOT submission correctly reconstructs only a small percentage of the true particles, and exhibits a large number of false negatives (missed particles). The INRAE LaPIV algorithm yields large amounts of both false positives (ghost particles) and false negatives. Conversely, the DLR and the LaVision algorithms maintain high reconstruction accuracy even at such high concentration, with a limited number of missed particles (higher for the LaVision algorithm) and no ghost particle. 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 Figure 8: Comparison between true particles (red crosses) and reconstructed particles (black circles) in sub-domains of the entire measurement volume, for the time-resolved (TR) case. Top: ppp = 0.005. Middle: ppp = 0.08. Bottom: ppp = 0.20. Notice that, for sake of clarity, a smaller sub-domain is selected for ppp = 0.08 and 0.20. The quantitative analysis of the particles reconstruction at all the ppp values is reported in Figure 9. The percentage of correctly reconstructed particles is very close to 100% in the entire range of ppp values for both the DLR and LaVision algorithms, as well as for the INRAE KLPT and LCTI algorithms in the range of submitted results (up to ppp = 0.08) and for the INRAE LaPIV result at ppp = 0.12. The latter algorithm shows a significant degradation of its performance for larger ppp values; in particular, at ppp = 0.2, only 10% of the true particles are correctly reconstructed, and the number of ghost particles exceeds 70%. The IOT submission exhibits a high sensitivity to the tracers’ concentration, with the percentage of particles correctly reconstructed decreasing progressively from 95% to 10% in the considered range of ppp values; nevertheless, the number of ghost particles remains always well below 1%. Figure 9: Left: percentage of the correct particles (relative to the number of true particles), as a function of the number of particles per pixels (ppp) in the images. Right: percentage of false particles (ghosts) as a function of the ppp. All results are for the time-resolved (TR) test case, and considered the fitted particles’ positions. The symbol keys apply to both figures. 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 The mean error magnitudes of the fitted position, velocity and acceleration for all ppp values are shown in Figure 10. The velocity and acceleration errors are also compared with the result from Hacker to assess the role of the temporal discretisation. The positional errors range between 0.02 and 0.5 ��� depending on the algorithm and the seeding concentration, with a general trend of increased error values for increasing ppp. Such increase is very mild for the DLR algorithm, where the error remains below 0.05 ��� at ppp = 0.2. The INRAE KLPT, INRAE LCTI and the LaVision algorithms exhibit a slightly larger error increase, with the latter approach yielding an error above 0.1 ��� at the largest ppp. The INRAE LaPIV algorithm features a much higher sensitivity to the ppp value beyond 0.12, indicating that the algorithm diverges for high seeding concentrations. Finally, the IOT approach shows a steep error increase already from the lowest analysed ppp value. The velocity errors range between 0.3% and 2.5% of the bulk velocity Vꝏ for all cases except for the INRAE LaPIV algorithm beyond ppp = 0.12. As already discussed, in the latter case the algorithm diverges thus yielding invalid results. The DLR algorithm provides the lowest errors, always below 0.5% of Vꝏ, which are very close to the minimum attainable values (Hacker errors: 0.25% of Vꝏ). The other algorithms exhibit a slightly more pronounced sensitivity to the seeding density, yielding velocity errors that increase from below 1% of Vꝏ at ppp = 0.005 to above 1.5% of Vꝏ at the highest ppp. The acceleration errors from Hacker, DLR, LaVision and IOT show very little dependence on the ppp value. The Hacker result already exhibits an error of the order of 8 m/s2 or 18% of the reference acceleration Vꝏ 2/D: this is a clear indication that, in the current test case, modulation errors due to the finite temporal resolution are predominant with respect to the random errors. Most of the algorithms provide acceleration errors between 25% and 35% of Vꝏ 2/D, with the lower errors achieved with the DLR algorithm. Nevertheless, these acceleration error values are between 50% and 100% of mean magnitude of the actual acceleration, thus highlighting the inadequacy of the algorithms to correctly resolve the flow acceleration in the current test case. The INRAE algorithms return the highest acceleration errors, which increase with the seeding density well above of the actual acceleration magnitude. Figure 10: Mean error magnitude of position (top-left), velocity (top-right) and acceleration (bottom-left) as a function of the ppp. The position error is evaluated based on the fitted particles positions. The symbol keys apply to all figures. The thick grey line in the acceleration error plot represents the mean acceleration magnitude. 5 Conclusions The main results of the Lagrangian Particle Tracking challenge organised within the framework of the European Union’s Horizon 2020 project HOMER (Holistic Optical Metrology for Aero-Elastic Research) are presented. The challenge employed a synthetic experiment where the wall-bounded flow in the wake of a cylinder was simulated. Particle images were acquired with four synthetic cameras at different seeding concentrations corresponding to a range of particles per pixel between 0.005 and 0.20. Three different image acquisition strategies were considered, namely two-pulse, four-pulse and time- 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 resolved acquisitions. Six research groups participated to the challenge. The submitted results were analysed in terms of accuracy of particles reconstruction (namely percentages of correct particles, false positives and false negatives), as well as based on the errors on position, velocity and acceleration. The analysis of the two-pulse case showed that the best algorithms are capable to reconstruct correctly nearly all the particles in the entire range of considered ppp values, with a negligible percentage of missed or ghost particles (less the 0.1%). For those algorithms, the positional errors were in the range between 0.05 and 0.1 ���. For the other algorithms, the percentage of correctly reconstructed particles remained well above 80%, although some of the algorithms such as that of ETHZ exhibited a large amount of ghost particles, of the same order of the true particles. Positional errors up to 0.3 ��� were recorded. Only two research groups participated to the four-pulse test case of the challenge, namely DLR and LaVision, using slightly different implementations of the same Multi-Pulse Shake-The-Box algorithm. The results showed the very high performance of this algorithm, with over 90% correctly reconstructed particles and tracks at all ppp values (close to 100% for the DLR implementation), and less than 0.2% and 0.6% ghost particles and wrong tracks, respectively. The positional errors were in the range between 0.03 and 0.08 ���, showing a slight increase with the seeding concentration. Velocity errors of about between 0.4% and 0.6% Vꝏ have been retrieved, slightly larger than the minimum admissible ones (0.3%) obtained by the Hacker participant. Conversely, much larger acceleration errors were found, between 60% and 80% of the acceleration magnitude, a large part of them being ascribed to temporal truncation. The results of the time-resolved test case showed that the best algorithms are capable to perform a close-to-perfect particle reconstruction up to ppp = 0.2, with nearly all particles correctly reconstructed and less than 0.1% ghost particles. For those algorithms, the positional errors remain below 0.05 ��� at all ppp values. Other algorithms exhibited larger sensitivity to the seeding density, either already from ppp = 0.005 (IOT submission) or only at the higher ppp values (INRAE LaPIV algorithm). Positional errors exceeding 0.3 ��� were found for these algorithms at the highest ppp. The velocity errors range from below 0.5% Vꝏ for the best algorithm to beyond 4% Vꝏ for the least-performing algorithm. Instead, much larger acceleration errors were retrieved, from 50% to several times the acceleration magnitude, also due to the limited temporal resolution of this test case. Acknowledgments This work has been carried out in the context of the HOMER (Holistic Optical Metrology for Aero-Elastic Research) project, funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 769237. The data processing performed by the research groups who participated in the challenge is kindly acknowledged. References Atkinson C and Soria J (2009) An efficient simultaneous reconstruction technique for tomographic particle image velocimetry. Experiments in Fluids, 47(4), 553-568 Champagnat F, Cornic P, Cheminet A, Leclaire B, Le Besnerais G and Plyer A (2014) Tomographic PIV: particles versus blobs. Measurement Science and Technology, 25(8), 084002 Cheminet A, Leclaire B, Champagnat F, Plyer A, Yegavian R and Le Besnerais G (2014) Accuracy assessment of a Lucas- Kanade based correlation method for 3D PIV. In 17th International Symposium on Applications of Laser Techniques to Fluid Mechanics. Cornic P, Leclaire B, Champagnat F, Le Besnerais G, Cheminet A, Illoul C and Losfeld G (2020) Double-frame tomographic PTV at high seeding densities. Experiments in Fluids, 61(2), 1-24 Elsinga GE, Scarano F, Wieneke B and van Oudheusden BW (2006), Tomographic particle image velocimetry. Experiments in fluids, 41(6), 933-947 Gesemann S, Huhn F, Schanz D and Schröder A (2016) From noisy particle tracks to velocity, acceleration and pressure fields using B-splines and penalties. In 18th international symposium on applications of laser and imaging techniques to fluid mechanics, Lisbon, Portugal (pp. 4-7) Herman GT and Lent A (1976) Iterative reconstruction algorithms. Computers in biology and medicine, 6(4), 273-294 Hofmann T, Schölkopf B and Smola AJ (2008) Kernel methods in machine learning. The annals of statistics, 1171-1220 Jahn T, Schanz D, Gesemann S and Schröder A (2017) 2-pulse STB: 3D particle tracking at high particle image densities. In 12th international symposium on PIV-PIV17, Busan, Korea Jahn T, Schanz D and Schröder A (2021) Advanced Iterative Particle Reconstruction for Lagrangian Particle Tracking. Accepted for publication in Experiments in Fluids. Kähler CJ, Astarita T, Vlachos PP, Sakakibara J, Hain R, Discetti S, ... & Cierpka C (2016) Main results of the 4th International PIV Challenge. Experiments in Fluids, 57(6), 97 Khojasteh AR, Yang Y, Heitz D and Laizet S (2021) Lagrangian Coherent Track Initialisation (LCTI). arXiv:2106.11049. Lasinger K, Vogel C, Pock T and Schindler K (2018) Variational 3D-PIV with sparse descriptors. Measurement Science and Technology, 29(6), 064010 14th International Symposium on Particle Image Velocimetry – ISPIV2021 August 1–4, 2021 Lasinger K, Vogel C, Pock T and Schindler K (2019) 3D fluid flow estimation with integrated particle reconstruction. International Journal of Computer Vision, 1-16 Leclaire B, Mary U, Liazun C et al. (2021) First Lagrangian Particle Tracking and Data Assimilation challenge: datasets description and evolution to an open online benchmark. in 14th International Symposium on Particle Image Velocimetry – ISPIV 2021, August 1–5, 2021 Needell D and Tropp JA (2009) CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Applied and computational harmonic analysis, 26(3), 301-321 Novara M, Batenburg KJ and Scarano F (2010) Motion tracking-enhanced MART for tomographic PIV. Measurement science and technology, 21(3), 035401 Novara M, Schanz D, Gesemann S, Lynch KP and Schröder A (2016b) Lagrangian 3D particle tracking for multi-pulse systems: performance assessment and application of Shake-The-Box. In: 18th international symposium on applications of laser techniques to fluid mechanics, pp 4–7 Novara M, Schanz D, Reuther N, Kähler CJ and Schröder A (2016a) Lagrangian 3D particle tracking in high-speed flows: Shake-The-Box for multi-pulse systems. Experiments in Fluids, 57(8), 1-20 Novara M, Schanz D, Geisler R, Gesemann S, Voss C and Schröder A (2019) Multi-exposed recordings for 3d lagrangian particle tracking with multi-pulse Shake-The-Box. Experiments in Fluids, 60(3), 1-19 Ott E (2002) Chaos in dynamical systems. Cambridge university press Scarano F (2012) Tomographic PIV: principles and practice. Measurement Science and Technology, 24(1), 012001 Schanz D, Gesemann S and Schröder A. (2016) Shake-The-Box: Lagrangian particle tracking at high particle image densities. Experiments in fluids, 57(5), 1-27 Schanz D, Gesemann S, Schröder A, Wieneke B and Novara M (2012) Non-uniform optical transfer functions in particle imaging: calibration and application to tomographic reconstruction. Measurement Science and Technology, 24(2), 024009 Soloff SM, Adrian RJ and Liu ZC (1997) Distortion compensation for generalized stereoscopic particle image velocimetry. Measurement science and technology, 8(12), 1441 Tan S, Salibindla A, Masuk AUM and Ni R (2020) Introducing OpenLPT: new method of removing ghost particles and high- concentration particle shadow tracking. Experiments in Fluids, 61(2), 1-16 Van Gent PL, Michaelis D, Van Oudheusden BW, WeissPÉ, de Kat R, Laskari A, ... and Schrijer, FFJ (2017) Comparative assessment of pressure field reconstructions from particle image velocimetry measurements and Lagrangian particle tracking. Experiments in Fluids, 58(4), 33 Wang XK and Tan SK (2008) Comparison of flow patterns in the near wake of a circular cylinder and a square cylinder placed near a plane wall. Ocean Engineering, 35(5-6), 458-472 Wieneke B (2012) Iterative reconstruction of volumetric particle distribution. Measurement Science and Technology, 24(2), 024008 Worth NA and Nickels TB (2008) Acceleration of Tomo-PIV by estimating the initial volume intensity distribution. Experiments in Fluids, 45(5), 847-856 Yang Y, Heitz D and Mémin E (2018) An ensemble filter estimation scheme for Lagrangian trajectory reconstruction. In 16ème Congrès Francophone de Techniques Laser pour la mécanique des fluides (p. 8) Yang Y, Heitz D and Mémin E (201). Lagrangian particle image velocimetry. In ISPIV2019-13th International Symposium on Particle Image Velocimetry (pp. 1-9) Yang Y and Heitz D (2021) Kernelized Lagrangian Particle Tracking. ⟨hal-03212696⟩