mytitle 14th International Symposium on Particle Image Velocimetry – ISPIV 2021 August 1–5, 2021 Particle Detection by means of Machine Learning in Defocusing PTV Maximilian Dreisbach1∗, Robin Leister1, Matthias Probst2, Pascal Friederich3,4, Alexander Stroh1, Jochen Kriegseis1 1 Institute of Fluids Mechanics, Karlsruhe Institute of Technology, Germany 2 Institute of Thermal Turbomachinery, Karlsruhe Institute of Technology, Germany 3 Institute of Theoretical Informatics, Karlsruhe Institute of Technology, Germany 4 Institute of Nanotechnology, Karlsruhe Institute of Technology, Germany ∗ maximilian.dreisbach@kit.edu The accurate measurement of a fluid flow inside a measurement volume (MV) with limited optical access poses a challenge since the view on the MV is often partially obstructed for all but one viewing angle. Defocusing particle tracking velocimetry (DPTV) can be used to determine the instantaneous three- dimensional velocity field of the flow with a standard PIV setup, requiring only a single optical axis. Current detection algorithms reach an out-of-plane accuracy in an order of magnitude lower than the planar accuracy, on top of a low rate of detected particles in comparison to other PTV approaches. These drawbacks originate from the low image quality due to noise, fluctuations in illumination, reflections and overlapping particle images. It has been shown that Machine Learning (ML) based detection is more robust against these adverse effects, due to the ability to leverage a higher amount of optical features for detection than conventional algorithms (Lecun et al. (1998)). Therefore, the present work addresses the applicability of ML algorithms in the post-processing of DPTV experiments, which will be evaluated on the ground of the DPTV experiments conducted by Leister and Kriegseis (2019). The setup of these experiments can be seen in Figure 1(a) and a section of a raw image recorded during the experiments in Figure 1(b). (a) DPTV setup (b) Raw image 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.6 0.7 0.8 0.9 1 Recall [-] Pr ec is io n [- ] Faster R-CNN Synthetic Faster R-CNN MUNIT Faster R-CNN Real Hough Transform RetinaNet Synthetic RetinaNet MUNIT RetinaNet Real (c) Detection results Figure 1: (a) DPTV setup in an open wet clutch; (b) recorded DPTV raw image above the grooved region of an open wet clutch; (c) precision-recall diagram of the detection on DPTV images. Two established object detection frameworks, Faster R-CNN by Ren et al. (2017) and RetinaNet by Lin et al. (2017) based on convolutional neural networks (CNN) are trained for particle detection by super- vised learning. To fully exploit the capacity of these deep learning algorithms large datasets consisting of O(10000) images with annotated centre point location and diameter for each particle are necessary. Since manual image annotation requires an unfeasible time effort and leads to the introduction of additional un- certainty and bias, two kinds of synthetic image data sets are used. The first training set Da is generated from 2D-Gaussian particle images with additional noise, while the second approach focuses on generating refined synthetic data Db by employing ML algorithms for unsupervised image-to-image translation on the first training data set. The image-to-image translation method MUNIT by Huang et al. (2018) is used to train a mapping from an input domain consisting of the synthetic particle images to an output domain of real particle images via unsupervised learning. The objective of the image translation step is to generate particle images that represent the feature distribution of the real data while preserving the location and diameter information of the individual synthetic input images. mailto:maximilian.dreisbach@kit.edu Overall, we find that the ML-based detection approach outperforms the current state-of-the-art conven- tional detection algorithm used by (Leister and Kriegseis, 2019) which detects particles through a Hough transformation by up to 49 percent in terms of detection rate, depending on the image quality; see Fig- ure 1(c). Especially on images with fluctuations in background illumination (e.g. Fig. 1(b)) the ML ap- proaches surpass the detection performance of the Hough transform. The translation framework MUNIT generates realistic synthetic representations of real particle images by reproducing small scale features. In particular, increasingly wide edges for higher diameters, noisy contours, distortions and intensity variations over the circumference of the particle images are generated, leading to visual similarity as shown in Fig- ure 2(a). As illustrated in Figure 2(b) the radial intensity profile, which is a major characteristic of the real particle images, is approximated more closely by MUNIT in comparison to the 2D-Gaussian synthetic images. The object detection algorithms trained on the feature-rich synthetic data Db from the image trans- (a) Particle images 5 10 15 20 26 30 0 0.2 0.4 0.6 0.8 1 1.2 Radius [px] In te ns ity [- ] Synthetic MUNIT Real (b) Radial intensity distribution - 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0 0.25 0.5 M is s ra te [- ] Faster R-CNN Hough RetinaNet - 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0 1 2 Overlap [IoU] σ D [p x] (c) Results on overlapping particles Figure 2: (a) Random synthetic (Da), real and translation refined particle images (Db) at ascending diam- eters; (b) averaged radial intensity profiles of synthetic (Da) and refined synthetic (Db) and real particle images with a diameter of D = 22.5±0.025px; (c) miss rate and uncertainty in diameter determination σD on synthetic particle images with increasing relative overlap, measured by intersection over union (IoU). lation reach a higher detection rate combined with a lower rate of false positives as compared to the same algorithms trained on synthetic data Da; see Figure 1(c). This shows that the image translation algorithm successfully transformed the distribution of the synthetic dataset closer to the DPTV image distribution, leading to a better generalization and real-world accuracy of the ML model trained on the refined images Db compared to the fully synthetic images Da. In experiments on synthetic images and artificial particle images produced through a pinhole aperture RetinaNet achieves a higher accuracy than the Hough transformation. Both Faster R-CNN and RetinaNet resolve overlapping synthetic particle images with a higher accuracy and a lower miss rate as seen in Figure 2(c). In summary, we showed that a machine learning approach based on synthetic data generation and data refinement using the MUNIT images translation method leads to highly accurate particle detection. The high detection rate and accuracy even on overlapping synthetic particle images also renders the presented approach promising for significantly increased particle seeding densities in DPTV experiments. Further- more, the insight of significant improvements by means of training-data refinement uncovers the impact of small scale features for ML-based particle detection, which in turn emphasizes the need for realistic training data. References Huang X, Liu MY, Belongie S, and Kautz J (2018) Multimodal unsupervised image-to-image translation. in Computer Vision - ECCV 2018. pages 179–196 Lecun Y, Bottou L, Bengio Y, and Haffner P (1998) Gradient-based learning applied to document recognition. Pro- ceedings of the IEEE 86:2278–2324 Leister R and Kriegseis J (2019) 3D-LIF experiments in an open wet clutch by means of defocusing PTV. in 13th International Symposium on Particle Image Velocimetry (ISPIV 2019), July 22-24, 2019, Munich, Germany Lin T, Goyal P, Girshick R, He K, and Dollár P (2017) Focal loss for dense object detection. in 2017 IEEE International Conference on Computer Vision (ICCV). pages 2999–3007 Ren S, He K, Girshick R, and Sun J (2017) Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 39:1137–1149