Acta Polytechnica CTU Proceedings https://doi.org/10.14311/APP.2023.41.0044 Acta Polytechnica CTU Proceedings 41:44–48, 2023 © 2023 The Author(s). Licensed under a CC-BY 4.0 licence Published by the Czech Technical University in Prague ANALYSIS OF OBJECTIVE EYE TRACKING DATA USED FOR VEHICLE ASSESSMENT Jan Novotný Czech Technical University in Prague, Faculty of Transportation Sciences, Department of Vehicles, Horská 2040/3, 128 00 Prague, Czech Republic correspondence: jan.novotny@cvut.cz Abstract. This paper describes the application of eye tracking technology in the automotive industry in the context of vehicle assessment in customer clinics. The paper presents the mentioned technology, the applied experimental methodology and selected results. In particular, the hypotheses that were expressed during the research of the topic and during the preparation of the experiment are answered. In this way, the innovative technology of eye tracking can be used to reflect the “voice of the customer” in the vehicle design. Keywords: Eye tracking, customer interest, eye fixation, driver distraction. 1. Introduction In the consumer world almost everything is sold to the eyes of the customers. The practicality, functionality and other attributes of a product are obviously impor- tant, but the first opinions are formed by the visual, design appearance of the product. Therefore, it is crucial for vehicle manufacturers to take the customer interest and opinions into consideration. For this rea- son, they often organise the so called customer clinics, where they gather their new models, sometimes also their competitors vehicles, and let participants from the public give them their view. This usually hap- pens in a form of an interview or a group discussion, but there never is the real view of the participant. And that opens an opportunity for the eye tracking technology. The eye tracking technology is able to track pupils and therefore detect the exact location where the participant is looking. This study aims to appropri- ately combine subjective and objective evaluation and develop an innovative methodology about customer qualitative assessments. The subjective evaluation is obtained using a method called “Lautes denken”, a form of interview with the participant in the pres- ence of vehicles. For the objective evaluation, the eye tracking technology is used, which consists of putting on special glasses and tracking and measuring the participant’s eye movements. 2. The eye tracking applications The eye tracking technology is frequently used in mar- keting studies, product or web design assessments. It was confirmed by many studies that the product design strongly influences the buyers decision [1]. An- other example is a study by Lange et al. [2], where a visual demand of in-vehicle touchscreen displays was measured. As is apparent, the eye tracking can be used for distraction levels assessment, which is closely connected with vehicle driving safety. Besides the safety and distraction assessment pur- poses, the eye tracking technology in the field of auto- motive industry is occasionally used for design assess- ments. At least there was an effort to do that based on several studies. Zhaolin et al. [3] propose a car styling based on the eye tracking technology, where eye movement hotspots maps are used to build an in- dex system. They evaluate the data as a classification model in order to prove that this approach is feasible to objectively evaluate vehicles styling. A study by Wang et al. [4] also shows the possi- bilities to effectively evaluate product design process with the application of the eye tracking technology. Because the researchers get the objective data and not only a subjective ranking, score or a narrative from the participants. This opens further possibilities to combine the posi- tive attributes of both types of design assessments. In this paper, a combination of an eye tracking study and a subjective interview design study will be introduced. 3. The eye tracking method The vision is one of the primary senses that tells a lot about person’s behaviour. The input of vision are eyes, more specifically the pupils which process the incoming light. But the perfect image requires a correct angle, therefore the eyes have to move and shift its focus frequently. For the purpose of the eye tracking technology, we recognise three types of movement as shown in Figure 1. A gaze is a very brief stopping of the eyes. If there are multiple gazes in close proximity, it can be detected as a fixation, which can already be considered as a focus on certain object and takes between 100–700 milliseconds. And lastly the saccade is a very fast movement between gazes or fixations [5]. The eye tracking technology Tobii Pro Glasses 2 illustrated in Figure 2 allows to track and detect all of 44 https://doi.org/10.14311/APP.2023.41.0044 https://creativecommons.org/licenses/by/4.0/ https://www.cvut.cz/en vol. 41/2023 Objective Eye Tracking Data for Vehicle Assessment Figure 1. Basic eye movements illustration. Figure 2. The Eye Tracker design. the mentioned eye movements and states. Based on videooculography, the eye tracker tracks the centre of pupils using reflection of infrared light and camera sys- tem in frequency of 50 Hz. The designated algorithm I-VT then classifies what type of movement or eye state is happening and where in the coordinate system the person is fixating with regard to the forward view (captured by FullHD camera). The classification is determined by the velocity of movement and can be modified for a given experiment. In this experiment, the velocity threshold parameter between a fixation and a saccade was set to 30◦ s−1 [6]. 4. The experiment demography and description The group of participants in this experiment contained a total of 25 people. Complete datasets (subjective and objective data) were obtained for 24 of them. One person had to be excluded from the objective assessment due to a significant eye defect. People with an interest in automobiles who were assumed to be able to talk about the issue at hand were selected. The gender representation was 17 men and 8 women participated in the experiment. Participants were divided into three groups according to age. These were those aged 18–29 years (48 %), aged 30–45 years (36 %) and aged 46 years and over (16 %). All participants were brought to a table and asked to provide some personal information. They were also informed about the process of the following measure- ment, in particular to speak as much and as openly as possible. They were also asked to follow the mod- erator to the indicated places and to move with their head down. This was so that the participant’s very first glance at a given part of the vehicle was already recorded by the measuring equipment. The participant’s eye tracking glasses were fitted and calibrated and then they were led to a designated spot 2.5 m from the front of the Škoda Octavia IV Style. The first 5 s were reserved for undisturbed static observation, when initial points or areas of interest can be recorded. The moderator then begins questioning, the intention of which is to obtain as many of the participant’s insights and opinions as possible. Participant is asked (variant for the front of the car): • “Your first impression of the front of the car?” • “How would you characterise it?” • “Does anything characterise the quality of the car/evokes the sporty design?” • “What do you like?” • “What do you dislike?” The participant is then led to a marker 2.5 m from the side of the vehicle where the procedure is repeated. The participant is then brought to a mark 2.5 m from the rear of the vehicle. The procedure is then repeated with the Octavia IV RS variant. 45 Jan Novotný Acta Polytechnica CTU Proceedings Figure 3. Data from the rear of the RS model. 5. Evaluation The data from the eye tracking part of the measure- ment were methodically evaluated using the dedicated Tobii Pro Lab software. Areas of Interest (AOI) were created to generate the data and to obtain exact in- formation on which parts of the car the participant was looking at during the measurement. A large amount of data was measured and analysed. This paper will highlight the interesting points and the answers to the hypotheses presented below. 5.1. Hypotheses 1. Do people perceive the spoiler and diffuser as a sporty feature? 2. Does the standard deviation of the observation of the electric charging lid take on higher values than the adjacent front door? How do participants per- ceive electric charging lit? 3. Does the term quality associate with the car’s face/joints? (Comparison of first view and focus on quality) 4. The ACC adaptive cruise control sensor will attract a large amount of fixation (attention). 5. Does the average fixation time on the front logo of the Style model take higher values than the RS model? 6. The electric charger will be perceived negatively on the RS model. 7. A large number of fixations will be attracted by the additional badges at the rear of the cars. 5.2. Evaluation and data demonstration 1. Do people perceive the spoiler and diffuser as a sporty feature? The sporty design dominates the rear of the model RS, which is confirmed by the measured data. The exhaust pipes, diffuser and black elements (trims and especially the spoiler) and black badges are the most frequently mentioned and observed as shown in Figure 3 proves the spoiler is the most mentioned in terms of sporty design, but the heatmap does not highlight it much – a thin, elongated zone. Therefore, both sources of information – objective eye tracking and subjective opinions of participants – must be taken into account to produce a credible result. 2. Is the standard deviation of the observa- tion of the electric charging lid higher than the adjacent front door? How do participants perceive electric charging lit? The standard deviation was calculated from the mea- sured individual data. The standard deviation of the electric charging lid is 0.69, the front door is only 0.35. The hypothesis suggesting a larger inconsistency and certain controversy of the electric charging lid is therefore confirmed. Subjective opinions also confirm this claim. Almost all participants fixated at the front door zone, with an average fixation duration of 0.50 s, there is not much variation between participants. In contrast, the lid was viewed by half of the participants in the first five s, with an average fixation duration of 0.96 s. The value varied frequently, from a minimum of 0.22 s to a maximum of 2.80 s. 3. Does the term quality associate with the moulding/joints of the car? (Comparison of first view and focus on quality) When looking at a vehicle, one forms an initial opinion in the first few s. He or she forms a first impression, is attracted by various design or practical features, whether positive or negative. A form of conversation, supplementary questions, is then used to distinguish between these positive or negative impressions. Therefore, in this study, we mainly measure the first 5 s of a person’s view of a given part of the car, then directly compare it with the interval in which we ask about the perception of quality. Differences are observable right away for this 46 vol. 41/2023 Objective Eye Tracking Data for Vehicle Assessment Figure 4. Heatmaps – First view / Question about the quality of the STYLE model. Figure 5. Fixations division of Model Style – front view. first view of the front of the Style model, both in the heatmaps in Figure 4 and in the Figure 5. The participants’ first glances are mainly directed at the vehicle logo, the ACC sensor cover, the windscreen area and the shape of the bumper. When asked about perceived quality, the fixation of the glances focuses more on the joints around the radiator grille and also more on the trim and bumper materials. A distinct cluster of fixations can be observed on the small cover on the left side of the bumper, this is probably due to the regular shape of the cover where the quality of the moulding can be easily seen. The subjective data also supports the hypothesis, with the moulding receiving the most responses to evoking quality (one third of all recorded responses), followed by the lights quality. 4. The ACC adaptive cruise control sensor will attract a large number of fixations (attention). The model Style’s ACC sensor attracted 7.19 % of all eye fixations, compared to just 2.37 % for the RS model. This is probably because the RS model has a number of other dominant, sporty features such as the RS badge and a variety of aerodynamic elements. However, considering the small size of the sensor (viewing zone), it is still a feature that attracts a great degree of attention. 5. Is the average fixation time to the front logo of the Style model higher than that of the RS model? The average fixation time on the Style’s front logo at the first five-second observation is 0.24 s. The RS model logo recorded an average of 0.46 s. Thus, the hypothesis was disproved as the RS model achieved a double the attention. For the Style model, attention shifted to the logo only when asked about perceived quality (0.55 s). 6. The RS model will have a negative percep- tion of the electric charger. The side view of the RS model is dominated by the detail of the electric charger lid, which attracted 9.34 % fixation on first glance! The heatmap in Figure 6 confirms this fact with a significant cluster of fixations that also had a long duration. If we look at the subjective opinions of the partici- pants, 13 of them directly expressed that they did 47 Jan Novotný Acta Polytechnica CTU Proceedings Figure 6. Heatmap – First look at the side of the RS. Figure 7. Proportion of fixations on RS model details. not like this element, others that they were not used to it. Confirming again the hypothesis number 2. 7. A large number of fixations are attracted by the additional badges on the rear of the cars. The accessories, badges and small logos truly draw the attention of the participants for both models. For the Style model, the accessories (ŠKODA logo, ex- haust pipes, and the signs “Octavia” and “G-Tech”) attracted 12.01 % of the total fixation duration. As shown in the Figure 7, the RS model even has a fix- ation share of 18.62 % for accessories. In particular, brand and model logos and sports exhausts dominate. 6. Conclusion The experiment answered the basic hypotheses con- cerning the examined models. Six hypotheses were confirmed, one hypothesis (number 5) was disproved as the result was the opposite. The results were pre- sented above. The experiment and the paper demon- strates a new measurement methodology combining the benefits of the objective eye tracking technology and the subjective interview technique. This creates a new possibility of applying the eye tracking tech- nology in the assessment of design features of new vehicles. References [1] T. N. Reid, E. F. MacDonald, P. Du. Impact of product design representation on customer judgment. Journal of Mechanical Design 135(9):091008, 2013. https://doi.org/10.1115/1.4024724 [2] D. R. Large, G. Burnett, E. Crundall, et al. Developing predictive equations to model the visual demand of in-vehicle touchscreen HMIs. International Journal of Human–Computer Interaction 34(1):1–14, 2017. https://doi.org/10.1080/10447318.2017.1306940 [3] L. Zhaolin, L. Shengbo, X. Shaobing, C. Bo. Automobile style evaluation based on eye tracking. Journal of Tsinghua University (Science and Technology) 55(7):775–781, 2015. [4] Z. Y. Wang, H. S. Liu, H. H. Shi, H. Liu. The research of automobile design evaluation method based on the eye tracking system technology. Advanced Materials Research 230-232:654–658, 2011. https://doi.org/10. 4028/www.scientific.net/amr.230-232.654 [5] Eyeware. Understanding Eye Tracking & How it Can Work for You: Definitions, Metrics, and Applications, 2019.[2022-11-5], https: //eyeware.tech/blog/what-is-eye-tracking/. [6] O. Anneli. The Tobii I-VT Fixation Filter 3 / 21. 2012. [2020-10-26], https://www.tobiipro.com/sitea ssets/tobii-pro/learn-and-support/analyze/how -do-we-classify-eye-movements/tobii-pro-i-vt-f ixation-filter.pdf?fbclid=IwAR1_em_u-TGZZLSnDh jbOeMNycUALVdT9KHcbCP1PUG3emHxOQbFSDgvj6Y. 48 https://doi.org/10.1115/1.4024724 https://doi.org/10.1080/10447318.2017.1306940 https://doi.org/10.4028/www.scientific.net/amr.230-232.654 https://doi.org/10.4028/www.scientific.net/amr.230-232.654 https://eyeware.tech/blog/what-is-eye-tracking/ https://eyeware.tech/blog/what-is-eye-tracking/ https://www.tobiipro.com/siteassets/tobii-pro/learn-and-support/analyze/how-do-we-classify-eye-movements/tobii-pro-i-vt-fixation-filter.pdf?fbclid=IwAR1_em_u-TGZZLSnDhjbOeMNycUALVdT9KHcbCP1PUG3emHxOQbFSDgvj6Y https://www.tobiipro.com/siteassets/tobii-pro/learn-and-support/analyze/how-do-we-classify-eye-movements/tobii-pro-i-vt-fixation-filter.pdf?fbclid=IwAR1_em_u-TGZZLSnDhjbOeMNycUALVdT9KHcbCP1PUG3emHxOQbFSDgvj6Y https://www.tobiipro.com/siteassets/tobii-pro/learn-and-support/analyze/how-do-we-classify-eye-movements/tobii-pro-i-vt-fixation-filter.pdf?fbclid=IwAR1_em_u-TGZZLSnDhjbOeMNycUALVdT9KHcbCP1PUG3emHxOQbFSDgvj6Y https://www.tobiipro.com/siteassets/tobii-pro/learn-and-support/analyze/how-do-we-classify-eye-movements/tobii-pro-i-vt-fixation-filter.pdf?fbclid=IwAR1_em_u-TGZZLSnDhjbOeMNycUALVdT9KHcbCP1PUG3emHxOQbFSDgvj6Y https://www.tobiipro.com/siteassets/tobii-pro/learn-and-support/analyze/how-do-we-classify-eye-movements/tobii-pro-i-vt-fixation-filter.pdf?fbclid=IwAR1_em_u-TGZZLSnDhjbOeMNycUALVdT9KHcbCP1PUG3emHxOQbFSDgvj6Y Acta Polytechnica CTU Proceedings 41:44–48, 2023 1 Introduction 2 The eye tracking applications 3 The eye tracking method 4 The experiment demography and description 5 Evaluation 5.1 Hypotheses 5.2 Evaluation and data demonstration 6 Conclusion References