14th International Symposium on Particle Image Velocimetry – ISPIV 2021
August 1–5, 2021

TrackAER: Real-Time Event-Based Particle Tracking
Alexander Rusch1∗, Thomas Roesgen1

1 ETH Zurich, Institute of Fluid Dynamics, Zurich, Switzerland
∗ arusch@ethz.ch

1 Introduction & Related Work
Event-based cameras (Lichtsteiner et al., 2008; Posch et al., 2010; Gallego et al., 2020) operate fundamen-
tally different from frame-based cameras: Each pixel of the sensor array reacts asynchronously to relative
brightness changes creating a sequential stream of events in address-event representation (AER). Each event
is defined by a microsecond-accurate time stamp, the pixel position and a binary polarity indicating a rel-
ative increase or decrease of light intensity. Thus, event-based cameras only sense changes in a scenery
while effectively suppressing static, redundant information. This renders the camera technology promising
also for flow diagnostics. In established approaches like PIV or PTV vast amounts of data are generated,
only for a large part of redundant information to be eliminated in data post-processing. In contrast, event-
based cameras effectively compress the data stream already at the source. To make full use of this potential,
new data processing algorithms are needed since event-based cameras do not generate conventional frame-
based data. This work utilizes an event-based camera to identify and track flow tracers such as helium-filled
soap bubbles (HFSBs) with real-time visual feedback in measurement volumes of the order of several cubic
meters.

The utility of event-based cameras for flow diagnostics has been demonstrated in a few studies already.
Drazen et al. (2011) used one of the first dynamic vision sensors (DVS) available for a proof-of-concept
study showing that 2D particle tracking velocimetry is possible for pipe flows with solid particles as tracers.
Ni et al. (2012) performed particle tracking with event-based cameras in a microscopic environment. In order
to distinguish microspheres from noise, they applied a circle Hough transform. Borer (2014) and Borer et al.
(2017) extended the idea of event-based particle tracking to a larger domain of interest and used multiple
event-based cameras for volumetric PTV measurements. Data sets were generated in a 3D calibrated setup
of three DVS cameras and processed off-line. Their tracking approach made use of a Kalman filter for track
prediction and relied on triangulating particle tracks after identifying them in each camera’s individual data
stream. Wang et al. (2020) used an event-based stereo camera setup for volumetric PTV measurements
imposing an incompressibility constraint for regularization. Their approach was evaluated using simulated
data and with measurements in a centimeter-sized, water-filled cavity.

2 Real-Time Algorithm
There are two basic ways of processing the asynchronous event data stream: In one approach, events are
accumulated in fixed time windows or fixed-size batches followed by processing steps using pseudo-frames
reconstructed from the events. While this allows the application of well-established frame-based computer
vision algorithms, it also implies a loss of the high temporal accuracy and low latency of event-based cam-
eras. Alternatively, events can be processed in a fully sequential manner, i.e., “event-by-event”, keeping the
precise timing information and ordering of the events. No frames are reconstructed that could interface to
classical computer vision algorithms for further processing. In this case, event-based algorithms are required
that analyze the flow (motion) information contained in the asynchronous event stream. Our work follows
such a per-event approach, preserving the speed and scalability of the technology. The key to identifying
and tracking a moving object from the event stream is to identify coherent spatio-temporal event activity.
Each incoming event may be added to a cluster, updating that cluster’s position and the list of contributing
events. If no assignment can be made, the event information is registered to prepare a trigger for the future
creation of a new cluster. Stale events are pruned from existing clusters using an “age” criterion. This may
happen in response to an occlusion in the scenery or when a moving object leaves the field of view. The al-
gorithm self-supervises its performance and may selectively skip processing steps if necessary for real-time
responsiveness.



(a) TrackAER event snapshot (16 ms time window). Events are
displayed indicating positive (white) and negative (black) polar-
ity. Tracked clusters are overlaid in red.

(b) Pseudo-streak integration from events over 10 s to visualize
particle trajectories. Only every fifth event is visualized and event
polarity is not indicated.

Figure 1: Visualization (field of view approx. 3.5 m × 2.0 m) of the flow caused by an indoor ventilation
system. Helium-filled soap bubbles are introduced at the bottom of the scene and convected.

3 Results & Conclusion
For visualization only, pseudo-frames can be generated from the event stream and fused with the movement
of the cluster trackers. No frames are used at any stage of the tracking algorithm. Figure 1 shows results
from a study of an indoor ventilation system. A room is partially seeded with HFSBs that move due to the
flow induced by the ventilation system. Events shown in the background are measurement noise. TrackAER
successfully ignores these spurious events and does not interpret them as tracer particles. More than 100
HFSBs are successfully tracked simultaneously and in real-time.

Our new real-time event-based detection and tracking algorithm, TrackAER, provides immediate insight
into flow fields during measurements. Tracer particles are successfully detected and tracked in volumes of
several cubic meters, whereas noise from the camera sensor and the scenery are effectively eliminated.
Ongoing research focuses on the actual flow field reconstruction as well as on the extension of the system
to volumetric measurements.

References
Borer D (2014) 4D Flow Visualization with Dynamic Vision Sensors. Ph.D. thesis. ETH Zurich

Borer D, Delbruck T, and Rösgen T (2017) Three-dimensional particle tracking velocimetry using dynamic
vision sensors. Experiments in Fluids 58:165

Drazen D, Lichtsteiner P, Häfliger P, Delbrück T, and Jensen A (2011) Toward real-time particle tracking
using an event-based dynamic vision sensor. Experiments in Fluids 51:1465–1469

Gallego G, Delbruck T, Orchard GM, Bartolozzi C, Taba B, Censi A, Leutenegger S, Davison A, Conradt
J, Daniilidis K, and Scaramuzza D (2020) Event-based vision: A survey. IEEE Transactions on Pattern
Analysis and Machine Intelligence

Lichtsteiner P, Posch C, and Delbruck T (2008) A 128×128 120 db 15 µs latency asynchronous temporal
contrast vision sensor. IEEE Journal of Solid-State Circuits 43:566–576

Ni Z, Pacoret C, Benosman R, Ieng S, and Régnier S (2012) Asynchronous event-based high speed vision
for microparticle tracking. Journal of Microscopy 245:236–244

Posch C, Matolin D, and Wohlgenannt R (2010) A qvga 143 db dynamic range frame-free pwm image sensor
with lossless pixel-level video compression and time-domain cds. IEEE Journal of Solid-State Circuits
46:259–275

Wang Y, Idoughi R, and Heidrich W (2020) Stereo event-based particle tracking velocimetry for 3d fluid
flow reconstruction. in European Conference on Computer Vision. pages 36–53. Springer


	Introduction & Related Work
	Real-Time Algorithm
	Results & Conclusion