Year PublishedAuthor(s)TitleAbstractFieldsURLPDFJournal/ConferenceKeywordsProducts
2022W. E. Nahal, H. G. Zaini, R. H. Zaini, S. S. M. Ghoneim and A. M. A. HassanRobust and High Accuracy Algorithm for Detection of Pupil ImagesRecently, many researchers have tried to develop a robust, fast, and accurate algorithm. This algorithm is for eye-tracking and detecting pupil position in many applications such as head-mounted eye tracking, gaze-based human-computer interaction, medical applications (such as deaf and diabetes patients), and attention analysis. Many real-world conditions challenge the eye appearance, such as illumination, reflections, and occasions. On the other hand, individual differences in eye physiology and other sources of noise, such as contact lenses or make-up. The present work introduces a robust pupil detection algorithm with and higher accuracy than the previous attempts for real-time analytics applications. The proposed circular hough transform with morphing canny edge detection for Pupillometery (CHMCEP) algorithm can detect even the blurred or noisy images by using different filtering methods in the pre-processing or start phase to remove the blur and noise and finally the second filtering process before the circular Hough transform for the center fitting to make sure better accuracy. The performance of the proposed CHMCEP algorithm was tested against recent pupil detection methods. Simulations and results show that the proposed CHMCEP algorithm achieved detection rates of 87.11, 78.54, 58, and 78 according to Świrski, ExCuSe, Else, and labeled pupils in the wild (LPW) data sets, respectively. These results show that the proposed approach performs better than the other pupil detection methods by a large margin by providing exact and robust pupil positions on challenging ordinary eye pictures.Pupil Detection, Eye Images
Computers, Materials & Continua, vol. 73, no.1, pp. 33–50, 2022
Pupil detection; eye tracking; pupil edge; morphing techniques; eye images datasetinvisible
2022In: Harris, D., Li, WC. (eds) Engineering Psychology and Cognitive Ergonomics. HCII 2022. Lecture Notes in Computer Science(), vol 13307. Springer, Cham.Assessments on Human-Computer Interaction Using Touchscreen as Control Inputs in Flight OperationsThe developing technology on innovative touchscreen applied in the cockpit can integrate control inputs and outputs on the same display in flight operations. Flight systems could be updated by modifying the touchscreen user interface without the complicated processes on reconfiguring cockpit panels. There is a potential risk on touchscreen components constrained by the issues associated with inadvertent touch, which may be defined as any system detectable touch issued to the touch sensors without the pilot’s operational consent. Pilots’ visual behaviours can be explored by using eye trackers to analyze the relationship between eye scan patterns and attention shifts while conducting monitoring tasks in flight operations. This research aims to evaluate human-computer interactions using eye tracker to investigate the safety concerns on implementation of touchscreen in flight operations. The scenario was set to conduct an instrument landing on the final approach using future system simulator. Participants were required to interact with all the control surfaces and checklists using the touchscreens located on different areas in the cockpit. Each participant performed landing scenario as pilot-flying (PF) and pilot-monitoring (PM) in random sequence. Currently PF and PM perform different tasks related to control inputs and control outputs monitoring in the flight deck. The PF’s primary obligation is to fly the aircraft’s flight path, and the PM’s main responsibility is to monitor the aircraft’s flight path and cross-check to the PF’s operational behaviours. By analyzing participants’ visual behaviours and scanning patterns, the findings on HCI related to applying touchscreen for future flight deck design would be applicable. There are some benefits on the implementation touchscreen for future flight deck design if the human-centred design principle can be integrated in the early stage.
Flight Operations, HCI, Usability, Ergonomics
In: Harris, D., Li, WC. (eds) Engineering Psychology and Cognitive Ergonomics. HCII 2022. Lecture Notes in Computer Science(), vol 13307. Springer, Cham.
Attention distribution,
Flight deck touchscreen,
Human-computer interaction,
System usability,
Visual behaviours
2022Novacek, T., & Jirina, M.Overview of Controllers of User Interface for Virtual Reality. Virtual reality has been with us for several decades already, but we are still trying to find the right ways to control it. There are a lot of controllers with various purposes and means of input, each with its advantages and disadvantages, but also with specific ways to be handled. Our hands were the primary
means of input for human-computer interaction for a long time. However, now we can use movements of our eyes, our feet or even our whole body to control the virtual environment, interact with it, or move from one place to another. We can achieve this with various controllers and wearable interfaces, like eye tracking, haptic suits or treadmills. There are numerous devices that we can choose from for every category, but sometimes it can be hard to pick the one that suits our intentions best. This article summarises all types of user interface controllers for virtual reality, with their main pros and cons and their comparison.
Virtual Reality
PRESENCE: Virtual and Augmented Reality, 1-100.
2022Ktistakis, E., Skaramagkas, V., Manousos, D., Tachos, N. S., Tripoliti, E., Fotiadis, D. I., & Tsiknakis, M.COLET: A Dataset for Cognitive WorkLoad Estimation Based on Eye-TrackingThe cognitive workload is an important component in performance psychology, ergonomics, and human factors. Unfortunately, publicly available datasets are scarce, making it difficult to establish new approaches and comparative studies. In this work, COLET-COgnitive workLoad estimation based on Eye-Tracking dataset is presented. Forty-seven (47) individuals’ eye movements were monitored as they solved puzzles involving visual search tasks of varying complexity and duration. The authors give an indepth study of the participants’ performance during the experiments while eye and gaze features were derived from low-level eye recorded metrics, and their relationships with the experiment tasks were investigated. The results from the classification of cognitive workload levels solely based on eye data, by employing and testing a set of machine learning algorithms are also provided. The dataset is available to the academic community.Cognitive Workload
Computer Methods and Programs in Biomedicine 00 (2022) 1–12
Cognitive workload, Workload classification, Eye movements, Machine Learning, Eye-tracking, Affective computingcore
2022Vortmann, L. M., Schult, M., & Putze, F.Differentiating Endogenous and Exogenous Attention Shifts Based on Fixation-Related Potentials.Attentional shifts can occur voluntarily (endogenous control) or reflexively (exogenous control). Previous studies have shown that the neural mechanisms underlying these shifts produce different activity patterns in the brain. Changes in visual-spatial attention are usually accompanied by eye movements and a fixation on the new center of attention. In this study, we analyze the fixation-related potentials in electroencephalographic recordings of 10 participants during computer screen-based viewing tasks. During task performance, we presented salient visual distractors to evoke reflexive attention shifts. Surrounding each fixation, 0.7-second data windows were extracted and labeled as “endogenous” or “exogenous”. Averaged over all participants, the balanced classification accuracy using a person-dependent Linear Discriminant Analysis reached 59.84%. In a leave-one-participant-out approach, the average classification accuracy reached 58.48%. Differentiating attention shifts, based on fixation-related potentials, could be used to deepen the understanding of human viewing behavior or as a Brain-Computer Interface for attention-aware user interface adaptations.Attention
27th International Conference on Intelligent User Interfaces (pp. 243-257).
fixation related potential, EEG, Attention, endogenous, exogenouscore
2022Feldman, A., Patou, F., Baumann, M., Stockmarr, A., Waldemar, G., Maier, A. M., & Vogel, A.Protocol: Listen Carefully protocol: an exploratory case–control study of the association between listening effort and cognitive functionIntroduction: A growing body of evidence suggests that hearing loss is a significant and potentially modifiable risk factor for cognitive impairment. Although the mechanisms underlying the associations between cognitive decline and hearing loss are unclear, listening effort has been posited as one of the mechanisms involved with cognitive decline in older age. To date, there has been a lack of research investigating this association, particularly among adults with mild cognitive impairment (MCI).Methods and analysis: 15–25 cognitively healthy participants and 15–25 patients with MCI (age 40–85 years) will be recruited to participate in an exploratory study investigating the association between cognitive functioning and listening effort. Both behavioural and objective measures of listening effort will be investigated. The sentence-final word identification and recall (SWIR) test will be administered with single talker non-intelligible speech background noise while monitoring pupil dilation. Evaluation of cognitive function will be carried out in a clinical setting using a battery of neuropsychological tests. This study is considered exploratory and proof of concept, with information taken to help decide the validity of larger-scale trials.Linguistics Open, 12(3).gaze detection, linear discriminant analysivr
2022Shemesh, A., Leisman, G., Bar, M., & Grobman, Y. J.The emotional influence of different geometries in virtual spaces: A neurocognitive examinationIn this paper, a multidisciplinary approach to examining the connection between visual perception, human emotions and architectural space is presented. It details a study in which emotional reactions to architectural space geometry are empirically measured and quantified. Using various sensors, including EEG (Electroencephalography), GSR (Galvanic Skin Response), and eye-tracking (ET), we collected data from 112 individuals experiencing virtual environments (VEs), characterized by a variance of geometric manipulations. Diffusion map algorithms, as well as other statistical methods were used to analyze the data. Findings suggest that criteria of protrusion, curvature, scale and proportion of space influence the user's emotional state. Indices of ET, GSR, electrical brain activity, as well as dwelling duration and self-report liking ranks, show both “negative” and “positive” interest changes.Neuropsychology, emotion
Journal of Environmental Psychology, 81, 101802.
Virtual environment, Space perception, Cognitive neuroscience, Affective responsevr
2022Brouwer, V. H., Stuit, S., Hoogerbrugge, A., Ten Brink, A. F., Gosselt, I. K., Van der Stigchel, S., & Nijboer, T. CApplying machine learning to dissociate between stroke patients and healthy controls using eye movement features obtained from a virtual reality taskConventional neuropsychological tests do not represent the complex and dynamic situations encountered in daily life. Immersive virtual reality simulations can be used to simulate dynamic and interactive situations in a controlled setting. Adding eye tracking to such simulations may provide highly detailed outcome measures, and has great potential for neuropsychological assessment. Here, participants (83 stroke patients and 103 healthy controls) we instructed to find either 3 or 7 items from a shopping list in a virtual super market environment while eye movements were being recorded. Using Logistic Regression and Support Vector Machine models, we aimed to predict the task of the participant and whether they belonged to the stroke or the control group. With a limited number of eye movement features, our models achieved an average Area Under the Curve (AUC) of .76 in predicting whether each participant was assigned a short or long shopping list (3 or 7 items). Identifying participant as either stroke patients and controls led to an AUC of .64. In both classification tasks, the frequency with which aisles were revisited was the most dissociating feature. As such, eye movement data obtained from a virtual reality simulation contain a rich set of signatures for detecting cognitive deficits, opening the door to potential clinical applications.Neuropsychology, 8(4), e09207.Emotions, Parametric design, Evidence-based design, Space geometry, Neuroaesthetics, Neuroarchitecturevr
2022Kay, L., Keogh, R., Andrillon, T., & Pearson, J.The pupillary light response as a physiological index of aphantasia, sensory and phenomenological imagery strength.The pupillary light response is an important automatic physiological response which optimizes light reaching the retina. Recent work has shown that the pupil also adjusts in response to illusory brightness and a range of cognitive functions, however, it remains unclear what exactly drives these endogenous changes. Here, we show that the imagery pupillary light response correlates with objective measures of sensory imagery strength. Further, the trial-by-trial phenomenological vividness of visual imagery is tracked by the imagery pupillary light response. We also demonstrated that a group of individuals without visual imagery (aphantasia) do not show any significant evidence of an imagery pupillary light response, however they do show perceptual pupil light responses and pupil dilation with larger cognitive load. Our results provide evidence that the pupillary light response indexes the sensory strength of visual imagery. This work also provides the first physiological validation of aphantasia.Mental Imagery, Pupillometry
Elife 11 (2022): e72484.
aphantasia, pupillary light reflexcore
2022Hirzle, T., Fischbach, F., Karlbauer, J., Jansen, P., Gugenheimer, J., Rukzio, E., & Bulling, A.Understanding, addressing, and analysing digital eye strain in virtual reality head-mounted displays.Digital eye strain (DES), caused by prolonged exposure to digital screens, stresses the visual system and negatively affects users’ well-being and productivity. While DES is well-studied in computer displays, its impact on users of virtual reality (VR) head-mounted displays (HMDs) is largely unexplored—despite that some of their key properties (e.g., the vergence-accommodation conflict) make VR-HMDs particularly prone. This work provides the first comprehensive investigation into DES in VR HMDs. We present results from a survey with 68 experienced users to understand DES symptoms in VR-HMDs. To help address DES, we investigate eye exercises resulting from survey answers and blue light filtering in three user studies (N = 71). Results demonstrate that eye exercises, but not blue light filtering, can effectively reduce DES. We conclude with an extensive analysis of the user studies and condense our findings in 10 key challenges that guide future work in this emerging research area.Digital eye strain
ACM Transactions on Computer-Human Interaction (TOCHI), 29(4), 1-80.
Virtual reality, digital eye strain, well-beingvr
2022Wright, G. A., Patel, M. R., Perez-Edgar, M. K., Fu, X., Brown, K., & Adhikary, S.Eye-Tracking Technology to Determine Procedural Proficiency in Ultrasound-Guided Regional AnesthesiaAnesthesiology
Perioperative Medicine journal
2022Nonaka, S., Nobuhara, S., & Nishino, KDynamic 3D Gaze from Afar: Deep Gaze Estimation from Temporal Eye-Head-Body Coordination.We introduce a novel method and dataset for 3D gaze estimation of a freely moving person from a distance, typically in surveillance views. Eyes cannot be clearly seen in such cases due to occlusion and lacking resolution. Existing gaze estimation methods suffer or fall back to approximating gaze with head pose as they primarily rely on clear, close-up views of the eyes. Our key idea is to instead leverage the intrinsic gaze, head, and body coordination of people. Our method formulates gaze estimation as Bayesian prediction given temporal estimates of head and body orientations which can be reliably estimated from a far. We model the head and body orientation likelihoods and the conditional prior of gaze direction on those with separate neural networks which are then cascaded to output the 3D gaze direction. We introduce an extensive new dataset that consists of surveillance videos annotated with 3D gaze directions captured in 5 indoor and outdoor scenes. Experimental results on this and other datasets validate the accuracy of our method and demonstrate that gaze can be accurately estimated from a typical surveillance distance even when the person’s face is not visible to the camera.3D Gaze Estimation of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)core
2022Wang, Q. I., Yongmei Ding, and Shiqian Wu.Human Implicit Intention Inference Based on Gaze InformationIn artificial intelligence, correct intention inference is conducive to the natural and effective interaction between humans and robots. This paper proposes a gaze-based implicit intention inference model. Firstly, we extract the gaze coordinates from the eye movement data and obtain the gaze center coordinates through clustering. Then the visual gaze center coordinates are registered with the object center coordinates in the scene map to complete the recognition of the human gaze object sequence Finally, the sequence of objects being watched is matched with a set of trained single intention models. Through the matching score, the implicit intention behind the sequence of human gaze objects is inferred. The experimental results show that in the case of interference, the accuracy of implicit intention inference is more than 93%, which ensures the reliability of implicit intention inference of restricted people in human-robot interaction.human-robot interaction
Available at SSRN 4086566.
Implicit intention inference, Gaze information, human-robot interactioncore
2022Zumkeller, Z. ZEffect of head-mounted displays on the tear-filmPurpose: The use of computers spread among the population, both for work and leisure purposes. This poses an increased risk factor for dry eye disease, a multifactorial disease influencing the surface of the eye. With the recent increasing usage of head-mounted displays, it is critical to determine whether or not they have the same impact on the tear film as conventional screens. Methods: The experiment was divided into three parts, where ten subjects participated. A baseline measurement was taken on the first day, and two additional measurements were retaken after 90 minutes of usage of a head-mounted display or a conventional screen. Results: No statistically significant differences were found in the impact on the tear film, neither in blink frequency nor duration while using a head-mounted display. Humidity and temperature in the periocular environment increased in the first 30 minutes of HMD usage, then remained stable while the oxygen level decreased. In comparison, temperature and humidity were lower, with oxygen levels increasing while using a conventional screen to become stable over 90 minutes. Subjects perceive subjectively less dry eye after using
an HMD. Conclusions: Although no final conclusions can be drawn from the results, there is a tendency toward HMDs having a slightly better impact on the tear film than conventional screens.
Digital eye strain
Doctoral dissertation, Hochschule Aalen
Virtual reality, dry eye, tear film, Blinkingcore
2022Alexeeva, Svetlana, Vladislav Zubov, and Yana NikonovaLexiad, the first dyslexia-specific cyrillic font compared to the popular times new roman and roboto fonts when read by adults.The LexiaD font was developed for Russian-speaking people with reading disorders (dyslexia) (Alexeeva et al., 2020). LexiaD demonstrated an advantage in letter feature extraction and information integration over other modern Cyrillic fonts (PT Sans and PT Serif) while reading by primary school dyslexic and non-dyslexic children. However, for dyslexic and non-dyslexic adolescents, the familiar Arial font wasLinguistics
Psychological Applications and Trends (2022): 464.
Dyslexia, font, mobile eye-tracker, printed text, Russian.core
2022Mitchell, D., & Choi, HAssessing the Spatial Distribution of Visual Attention in a Virtual Environment: Development and Validation of a Novel VR-based Attentional Visual Field (AVF) Task.more effective (Alexeeva, Zubov, 2020).In this study, we tested two possible reasons for the advantages of Arial: familiarity or its structure. LexiaD was compared to Times New Roman (TNR; another familiar font) and Roboto (a font similar to Arial, but less familiar than TNR) when reading texts printed on a paper page. The study involved 42 adults without reading disorders. The previous studies did not show that the font effect interacts with the participant group (with/without dyslexia).The participants read silently three parts of the text about Easter Island and answered comprehensionAttention, virtual reality, field of view assessment
In CHI Conference on Human Factors in Computing Systems Extended Abstracts (pp. 1-7)
Virtual reality, Head-mounted display, Visual attention, Attentionalvr
2022Koch, M., Weiskopf, D., & Kurzhals, K.A Spiral into the Mind: Gaze Spiral Visualization for Mobile Eye Trackingquestions. The texts and tasks were borrowed from The Program for International Student Assessment (PISA). During the reading, eye movements were recorded using a mobile tracker (PupilCore) with a sampling frequency of 200 Hz. The mean word reading rate (reading speed) and the mean number ofscanpaths, visualisations
arXiv preprint arXiv:2204.13494.
visual fieldinvisible
2022Giaretta, A.Security and Privacy in Virtual Reality--A Literature Surveyfixations per word were analyzed. Mixed-design ANOVA showed a significant difference between the fonts in reading speed (p=0.05) and the number of fixations (p=0.03). LexiaD was inferior to Roboto in both measures. There was no evidence that the control fonts differed from each other or LexiaD differed from TNR. Thus, it could be assumed that the design made Arial a facilitating font in the previous study. A longitudinal study of LexiaD is required to test how it will perform when it becomes more familiar to readers.Security
arXiv preprint arXiv:2205.00208.
Virtual Reality, VR, privacy, security, cybersecurity, authenticationvr
2022Chavant, M., & Kapoula, Z.Audiovisual Integration for Saccade and Vergence Eye Movements Increases with Presbycusis and Loss of Selective Attention on the Stroop Test.Multisensory integration is a capacity allowing us to merge information from different sensory modalities in order to improve the salience of the signal. Audiovisual integration is one of the most used kinds of multisensory integration, as vision and hearing are two senses used very frequently in humans. However, the literature regarding age-related hearing loss (presbycusis) on audiovisual integration abilities is almost nonexistent, despite the growing prevalence of presbycusis in the population. In that context, the study aims to assess the relationship between presbycusis and audiovisual integration using tests of saccade and vergence eye movements to visual vs. audiovisual targets, with a pure tone as an auditory signal. Tests were run with the REMOBI and AIDEAL technologies coupled with the pupil core eye tracker. Hearing abilities, eye movement characteristics (latency, peak velocity, average velocity, amplitude) for saccade and vergence eye movements, and the Stroop Victoria test were measured in 69 elderly and 30 young participants. The results indicated (i) a dual pattern of aging effect on audiovisual integration for convergence (a decrease in the aged group relative to the young one, but an increase with age within the elderly group) and (ii) an improvement of audiovisual integration for saccades for people with presbycusis associated with lower scores of selective attention in the Stroop test, regardless of age. These results bring new insight on an unknown topic, that of audio visuomotor integration in normal aging and in presbycusis. They highlight the potential interest of using eye movement targets in the 3D space and pure tone sound to objectively evaluate audio visuomotor integration capacities.hearing, multi sensory processing, audio-visual integration
Brain Sciences, 12(5), 591.
audiovisual integration, presbycusis, aging, saccade, vergencecore
2022Silva-Gago, M., Fedato, A., Hodgson, T., Terradillos-Bernal, M., Alonso-Alcalde, R., & Bruner, E.The Influence of Tool Morphology on Visual Attention During the Interaction with Lower Palaeolithic Stone ToolsHumans are specialized in eye-hand coordination through a complex visuospatial system. When a tool is observed, the motor areas of the brain are activated and, when grasped, it is sensed as a part of the body. One approach to understanding the underlying mechanisms behind this process regards the analysis of visual attention. Vision influences the spatial interaction with tools and plays a crucial role in the perception of an object’s affordances. In this study, we employ eye-tracking technology to investigate whether Lower Palaeolithic stone tool morphology influences visual attention during visual exploration and manipulation. Our results suggest that the handaxe morphology has a moderate influence on the visual scanning of the tool. In contrast, visual exploration of the chopper is only influenced by the weight of the tool. The different visual behaviours exerted by these two technologies suggest divergences in the visuospatial process underlying the interaction with these tools.Hand-eye coordination, motor control, evolution, cognitive archaeology
Lithic Technology, 1-12.
Handaxe, choppers, affordances, cognitive archaeology, eye trackingcore
2022Hicken, L. K., & Duke, R. A.Differences in Attention Allocation in Relation to Music Teacher Experience and Expertise.To assess allocation of attention by music teachers with different levels of experience and expertise, we recruited five participant flautists: an artist teacher, two graduate students, and two undergraduates, all of whom observed nine brief video recordings of flute, clarinet, and saxophone players; a juggler; a baseball batter; and a ballerina. We tracked participants’ gaze using wearable eye-tracking hardware and software, and we analyzed the targets and durations of over 1,300 visual fixations and the paths of participants’ eye movements while observing the videos. The gaze behavior of the artist teacher and one of the graduate students, when they observed flute playing, was much like that of the experts in other domains of human experience. These two participants’ fixations were longer than those of the other three participants, and the sequence of fixation targets reflected a hierarchical prioritization of the fundamentals of flute playing. These same features were not apparent when these same two participants observed the other videos, and they were not observed in the gaze behavior of the other three participants in any of the videos observed. The results of this study demonstrate that allocation of attention, as indicated by gaze behavior, is reflective of expertise in music teaching as it is in other domains.Music, Expertise
Journal of Research in Music Education, 00224294221096701
music-teacher expertise, attention, decision-making, eye trackingcore
2022Drouot, M., Le Bigot, N., Bricard, E., de Bougrenet, J. L., & Nourrit, V.Augmented reality on industrial assembly line: Impact on effectiveness and mental workload.Studies examining the potential of augmented reality (AR) to improve assembly tasks are often unrepresentative of real assembly line conditions and assess mental workload only through subjective measurements and leads to conflicting results. We proposed a study directly carried out in industrial settings, to compare the impact of AR-based instructions to computerized instructions, on assembly effectiveness (completion time and errors) and mental workload using objective (eye tracking), subjective (NASA-TLX) and behavioral measurements (dual task paradigm). According to our results, AR did not improve effectiveness (increased assembly times and no decrease in assembly errors). Two out of three measurements indicated that AR led to more mental workload for simple assembly workstation, but equated computer instructions for complex workstation. Our data also suggest that, AR users were less able to detect external events (danger, alert), which may play an important role in the occurrence of work accidents.Industrial Assembly Line, Mental Workload
Applied Ergonomics, 103, 103793.
Augmented realityIndustrial assemblyMental workloadvr
2022Shahini, F., Park, J., Welch, K., & Zahabi, M.Effects of unreliable automation, non-driving related task, and takeover time budget on drivers’ takeover performance and workloadThe objective of this study was to assess the effects of unreliable automation, non-driving related tasks (NDRTs), and takeover time budget (TOTB) on drivers’ takeover performance and cognitive workload when faced with critical incidents. Automated vehicles are expected to improve traffic safety. However, there are still some concerns about the effects of automation failures on driver performance and workload. Twenty-eight drivers participated in a driving simulation study. The findings suggested that drivers require at least 8 s of TOTB to safely take over the control of the vehicle. In addition, drivers exhibited safer takeover performance under the conditionally automated driving situation than negotiating the critical incident in the manual driving condition. The results of drivers’ cognitive workload were inconclusive, which might be due to the individual and recall biases in subjective measures that could not capture subtle differences in workload during takeover requests.Driving, 1-35Driving performance;core
2021Anna Lena Scherger, Gianna Urbanczik, Timon Ludwigs, Jasmin M. KizilirmakThe Bilingual Native Speaker Competence: Evidence From Explicit and Implicit Language Knowledge Using Elicited Production, Sentence-Picture Matching, and PupillometryThe present pilot study investigated potential effects of early and late child bilingualism in highly proficient adult bilinguals. It has been shown that some early second language (eL2) speakers stagnate when it comes to complex linguistic phenomena and that they display subtle difficulties in adulthood. Therefore, we have chosen the complex structure of double object constructions. We investigate the long-term achievement in a combined-method approach using elicited production, explicit comprehension by sentence-picture matching and a measure of implicit linguistic knowledge, namely pupillometry. This eye tracking method is suitable for measuring implicit reactions of the pupils to unexpected or ungrammatical stimuli. For production, ditransitive structures were elicited by means of a game. For comprehension, a sentence-picture matching task was conducted. Two pictures were shown on a monitor that were equal with respect to the involved objects, but the thematic roles of direct and indirect objects were interchanged. Items were controlled for length, gender, animacy, semantic likelihood and word order. Reaction times and accuracy scores were analyzed. To this end, N = 18 bilingual adult speakers of German (+ another language, mean age: 26.5) with different ages of onset participated in this study and were compared to N = 26 monolingual German adult speakers (mean age 23.9). All participants had a proficiency of German above 89% correct in placement and cloze tests. Results show fully comparable productive and comprehensive competencies in monolinguals and bilinguals including the reaction times in the sentence-picture matching task and a word order effect on the reaction times in both groups. In the pupillometry task, we found monolinguals and bilinguals to be sensitive to differing conditions with respect to grammatical and ungrammatical utterances. However, we find between group differences in pupil dilations in that bilinguals react differently to strong grammatical violations than monolinguals. These results are discussed with respect to the term of native speaker competence and the variation within both groups.Psycholinguistics
Frontiers in psychology, 12, 717379.
automation; cognitivecore
2022B Provost, D VonderheideEye Tracking for Seizure DetectionTreatment plans for epilepsy depend on seizure frequency, duration, and type. Seizure monitoring in non-clinical settings relies on unreliable self-reports. Our sponsors have developed a suite of non-cerebral sensors which detects seizures with high specificity, though personalization of the suite may optimize results. Eye and head motion, not currently measured in the sensor suite, are indicative of certain seizure types, though state-of-the-art eye tracking devices have high power consumption and may not be suitable for extended monitoring. To inform future development of a low-power prototype for potential addition to the sensor suite, we gathered data from 8 subjects using a commercial eye-tracker (Pupil Core) and Inertial Measurement Unit (IMU), for non-seizure behaviors (technology use, eating, and conversation) and simulated seizure activity. Data processing using linear discriminant analysis (LDA) showed high separability between seizure and non-seizure data with features derived from head accelerometry and low-resolution eye position classification (area under the receiver operating characteristic curve was .98 and F1 score was .86 for eye position mapped to a 3x3 grid). The experimental protocol and data processing scripts we have provided allow for easy replication of our methodology to evaluate future hardware against these benchmark results. Additionally, adding artificial noise to Pupil Core data revealed that tracking the pupil within 2.75 mm was tolerable before separability scores significantly decreased. These results are promising for the use of eye and head motion data to identify certain seizure presentations, without requiring the high precision of current technology.Seizure Detection
Bachelor of Engineering, Thesis Dissertation
workload; takeovercore
2022Zhang, Q., Huang, Y., Chernyshov, G., Li, J., Pai, Y.S. and Kunze, K.GazeSync: Eye Movement Transfer Using an Optical Eye Tracker and Monochrome Liquid Crystal DisplaysCan we see the world through the eyes of somebody else? We present an early work to transfer eye gaze from one person to another. Imagine you can follow the eye gaze of an instructor while explaining a complex work step, or you can experience a painting like an expert would: Your gaze is directed to the important parts and follows the appropriate steps. In this work, we explore the possibility to transmit eye-gaze information in a subtle, unobtrusive fashion between two individuals. We present an early prototype consisting of an optical eye-tracker for the leader (person who shares the eye gaze) and two monochrome see-through displays for the follower (person who follows the eye gaze of the leader). We report the results of an initial user test and discuss future works.User Interfaces
In 27th International Conference on Intelligent User Interfaces (pp. 54-57).
time budget core
2022Ljubenovic A, Said S, Braun J, Grande B, Kolbe M, Spahn DR, Nöthiger CB, Tscholl DW, Roche TRVisual Attention of Anesthesia Providers in Simulated Anesthesia Emergencies Using Conventional Number-Based and Avatar-Based Patient Monitoring: Prospective Eye-Tracking StudyBackground:
Inadequate situational awareness accounts for two-thirds of preventable complications in anesthesia. An essential tool for situational awareness in the perioperative setting is the patient monitor. However, the conventional monitor has several weaknesses. Avatar-based patient monitoring may address these shortcomings and promote situation awareness, a prerequisite for good decision making.

The spatial distribution of visual attention is a fundamental process for achieving adequate situation awareness and thus a potential quantifiable surrogate for situation awareness. Moreover, measuring visual attention with a head-mounted eye-tracker may provide insights into usage and acceptance of the new avatar-based patient monitoring modality.

This prospective eye-tracking study compared anesthesia providers' visual attention on conventional and avatar-based patient monitors during simulated critical anesthesia events. We defined visual attention, measured as fixation count and dwell time, as our primary outcome. We correlated visual attention with the potential confounders: performance in managing simulated critical anesthesia events (task performance), work experience, and profession. We used mixed linear models to analyze the results.

Fifty-two teams performed 156 simulations. After a manual quality check of the eye-tracking footage, we excluded 57 simulations due to technical problems and quality issues. Participants had a median of 198 (IQR 92.5-317.5) fixations on the patient monitor with a median dwell time of 30.2 (IQR 14.9-51.3) seconds. We found no significant difference in participants' visual attention when using avatar-based patient monitoring or conventional patient monitoring. However, we found that with each percentage point of better task performance, the number of fixations decreased by about 1.39 (coefficient –1.39; 95% CI –2.44 to –0.34; P=.02), and the dwell time diminished by 0.23 seconds (coefficient –0.23; 95% CI: –0.4 to –0.06; P=.01).

Using eye tracking, we found no significant difference in visual attention when anesthesia providers used avatar-based monitoring or conventional patient monitoring in simulated critical anesthesia events. However, we identified visual attention in conjunction with task performance as a surrogate for situational awareness.
JMIR Serious Games 2022;10(1):e35642
Anesthesia; eye-tracking technology; patient monitoring; patient simulation; situation awareness; task performance; visual attention; avatar based model; simulated anesthesia; perioperative
2022Inhyeok Jeong, Kento Nakagawa, Rieko Osu, Kazuyuki KanosueDifference in gaze control ability between low and high skill players of a real-time strategy game in esportsThis research investigated the difference in aspects of gaze control between esports experts (Expert) and players with lower skills (Low Skill) while playing the real-time strategy game called StarCraft. Three versions of this game at different difficulty levels were made with the StarCraft Editor, and the gaze movements of seven Expert and nine Low Skill players were analyzed while they played the games. The gaze of Expert players covered a significantly larger area in the horizontal direction than the gaze of Low Skill players. Furthermore, the magnitude and number of saccadic eye movements were greater, and saccade velocity was faster in the Expert than in the Low Skill players. In conclusion, StarCraft experts have a specific gaze control ability that enables them to quickly and widely take visual information from all over the monitor. This could be one of the factors enabling StarCraft experts to perform better than players with lower skills when playing games that require task-switching ability.esports ONEsaccadic eye movements, saccade velocitycore
2022P Sanz Diez, A Bosco, P Fattori, S WahlHorizontal target size perturbations during grasping movements are described by subsequent size perception and saccade amplitudePerception and action are essential in our day-to-day interactions with the environment. Despite the dual-stream theory of action and perception, it is now accepted that action and perception processes interact with each other. However, little is known about the impact of unpredicted changes of target size during grasping actions on perception. We assessed whether size perception and saccade amplitude were affected before and after grasping a target that changed its horizontal size during the action execution under the presence or absence of tactile feedback. We have tested twenty-one participants in 4 blocks of 30 trials. Blocks were divided into two experimental tactile feedback paradigms: tactile and non-tactile. Trials consisted of 3 sequential phases: pre-grasping size perception, grasping, and post-grasping size perception. During pre- and post-phases, participants executed a saccade towards a horizontal bar and performed a manual size estimation of the bar size. During grasping phase, participants were asked to execute a saccade towards the bar and to make a grasping action towards the screen. While grasping, 3 horizontal size perturbation conditions were applied: non-perturbation, shortening, and lengthening. 30% of the trials presented perturbation, meaning a symmetrically shortened or lengthened by 33% of the original size. Participants’ hand and eye positions were assessed by a motion capture system and a mobile eye-tracker, respectively. After grasping, in both tactile and non-tactile feedback paradigms, size estimation was significantly reduced in lengthening (p = 0.002) and non-perturbation (p<0.001), whereas shortening did not induce significant adjustments (p = 0.86). After grasping, saccade amplitude became significantly longer in shortening (p<0.001) and significantly shorter in lengthening (p<0.001). Non-perturbation condition did not display adjustments (p = 0.95). Tactile feedback did not generate changes in the collected perceptual responses, but horizontal size perturbations did so, suggesting that all relevant target information used in the movement can be extracted from the post-action target perception.Motor Control ONEreach, grasp, saccadescore
2022Martin Chavant & Zoï KapoulaLoss of audiovisual facilitation with age occurs for vergence eye movements but not for saccadesThough saccade and vergence eye movements are fundamental for everyday life, the way these movements change as we age has not been sufficiently studied. The present study examines the effect of age on vergence and saccade eye movement characteristics (latency, peak and average velocity, amplitude) and on audiovisual facilitation. We compare the results for horizontal saccades and vergence movements toward visual and audiovisual targets in a young group of 22 participants (mean age 25 ± 2.5) and an elderly group of 45 participants (mean age 65 + 6.9). The results show that, with increased age, latency of all eye movements increases, average velocity decreases, amplitude of vergence decreases, and audiovisual facilitation collapses for vergence eye movements in depth but is preserved for saccades. There is no effect on peak velocity, suggesting that, although the sensory and attentional mechanisms controlling the motor system does age, the motor system itself does not age. The loss of audiovisual facilitation along the depth axis can be attributed to a physiologic decrease in the capacity for sound localization in depth with age, while left/right sound localization coupled with saccades is preserved. The results bring new insight for the effects of aging on multisensory control and attention.Ageing Reportsvergence, saccades, audiovisual, ageingcore
2022Zheng, L. J., Mountstephens, J., & Teo, JEmotion Classification in Eye-Tracking-Based Affective VR: Eye Fixation Versus Pupil DiameterBackground: The usage of eye-tracking technology is becoming increasingly popular in machine learning applications, particularly in the area of affective computing and emotion recognition. Typically, emotion recognition studies utilize the combination of multiple physiological signals such as electroencephalography,electrocardiography, and electrodermography while the research on emotion detection that relies solely on eyetracking data is limited. In this study, an empirical comparison of the accuracy of eye-tracking-based emotion recognition in a virtual reality (VR) environment using eye fixation versus pupil diameter as the classification feature is performed. We classified emotions into four distinct classes according to Russell’s four-quadrant Circumplex Model of Affect. 3600 videos are presented as emotional stimuli to participants in a VR environment via a head-mounted display (HMD) to evoke the user’s emotions. Three separate experiments were conducted using Support Vector Machines (SVMs) as the classification algorithm for the two chosen eye features. Findings: The results showed that emotion classification using fixation position obtained an accuracy of 75% while pupil diameter obtained an accuracy of 57%. Conclusions: For four-quadrant emotion recognition, eye fixation as a learning feature produces better classification accuracy compared to pupil diameter. Therefore, this empirical study has shown that eye-tracking-based emotion recognition systems would benefit from using features based on eye fixation data rather than pupil size.Emotion Classification recognition, eye-tracking, fixation, pupil diameter, virtual reality, support vector machinesvr
2022Roche, T. R., Maas, E. J., Said, S., Braun, J., Machado, C., Spahn, D. R., ... & Tscholl, D. W.Anesthesia personnel’s visual attention regarding patient monitoring in simulated routine and critical incidents. An eye-tracking study.Background: Cognitive ergonomics design of patient monitoring may reduce human factor errors in highstress environments. Eye-tracking is a suitable tool to gain insight into the visual interaction of healthcare professionals with patient monitors, which may facilitate their further development.

Methods: This prospective, randomized, high-fidelity simulation study compared anesthesia personnel's visual attention (fixation count and dwell time) to 15 areas of interest on the patient monitor during critical and non-critical anesthesia scenarios. Furthermore, we examined the extent to which participants' experience influenced visual attention and which vital signs displayed on the patient monitor received the most visual attention. We used mixed zero-inflated Poisson regression and mixed linear models to analyze the results.

Results: Analyzing 23 ten-minute scenarios, we found significantly more fixations to the areas of interest on the patient monitor during critical situations (rate ratio of 1.45; 95% CI 1.33 to 1.59; p < 0.001). However, the dwell-time on the areas of interest did not significantly differ between the critical and noncritical scenarios (coefficient of –1.667; 95% CI –4.549 to 1.229; p = 0.27). The professional experience did not significantly influence the visual attention (fixation: rate ratio of 0.88; 95% CI 0.54 to 1.43; p = 0.61 and dwell-time: coefficient of 0.889; 95% CI –1.465 to 3.229; p = 0.27). Regarding vital signs, anesthesia personnel paid the most attention to blood pressure (fixation: mean [SD] of 108 [74.83]; dwell-time: mean [SD] of 27 [15.90] seconds), end-expiratory carbon dioxide (fixation: mean [SD] of 59 [47.39]; dwell-time:
mean [SD] of 30 [21.51] seconds), and the electrocardiogram (fixation: mean [SD] of 58 [64.70]; dwelltime: mean [SD] of 15 [14.95] seconds).

Conclusions: Critical anesthesia situations increased anesthesia personnel’s visual interaction with the patient monitor. Furthermore, we found that their visual attention focused mainly on a few vital signs. To assist clinicians in critical situations, manufacturers should optimize monitors to convey necessary information as easily and quickly as possible and optimize the visibility of less frequently observed but equally critical vital signs, especially when they are in an abnormal range.
Medical, Anesthesiology
Research Square (pre-print)
Anesthesia, general, eye-tracking technology, patient monitoring, patient simulation, situation
awareness, visual attention
2022Suada Dacić, Amel Kosovac, Adnan OmerhodžićImpact of Roadside Advertising
Structures on Traffic Safety
The focus of research presented in this
paper is quantification of the relationship between the
traffic safety and roadside advertising structures along
a selected road in urban environment. In order to
determine correlation, statistical analysis of traffic
safety indicators was performed. Then identificcation
of characteristics of advertising structures and analysis
was done by using multi-criteria evaluation in order to
determine their impact on driver distraction.
Determination of driver distraction was performed by
testing the drivers in real traffic conditions, using
specialized measuring equipment. The indicators
derived from the research were then analyzed using
the methods of regression analysis, and safety
Road Safety JournalTraffic safety, driver distraction,
advertising structures
2022Joel Troya, Daniel Fitting, Markus Brand, Boban Sudarevic, Jakob N Kather, Alexander Meining, Alexander HannThe influence of computer-aided polyp detection systems on reaction time for
polyp detection and eye gaze
Background and study aims: Multiple computer-aided systems for polyp detection (CADe) are currently introduced into clini-
cal practice, with an unclear effect on examiner behavior. In this work, we aimed to measure the influence of a CADe system on
reaction time, mucosa misinterpretations, and changes in visual gaze pattern.
Patients and methods: Participants with variable levels of experience in colonoscopy examined video sequences while eye
movement was tracked. Using a crossover design, videos were presented in two assessments with and without CADe (GI Ge-
nius, Medtronic) support. Reaction time for polyp detection and eye-tracking metrics were evaluated.
Results: 21 Participants performed 1218 experiments. CADe was with a median of 1.16sec significantly faster in detecting
polyps compared to the users with 2.97sec (99%CI;0.40-3.43 and 2.53-3.77sec, respectively). However, the reaction time of the
user with the use of CADe with a median of 2.9sec (99%CI;2.55-3.38sec) was similar than without its use. CADe increased the
misinterpretations of normal mucosa and reduced the eye travel distance.
Conclusions: This work confirms that CADe systems detect polyps faster than humans. In addition, they led to increased misin-
terpretations of normal mucosa and decreased eye travel distance. Possible consequences of these findings might be prolonged
examination time and deskilling
2022Kim Buchmüller, Chengyan Xu, Angela Bearth, Michael SiegristConsumers’ decision-making process when choosing potentially risky, frequently used chemical household products: The case of laundry detergentsChemical household products are a common cause of accidents in the domestic sphere. Despite such products being associated with certain risks in the event of swallowing or contact with the skin or eyes, they are used in nearly every household worldwide for hygiene purposes. In most European countries, chemical household products feature warnings of the Globally Harmonized System (GHS) as well as other warnings. In this eye-tracking study (N = 147), which was conducted in a virtual environment, we examined (i) whether consumers use such warnings when choosing a laundry detergent, (ii) whether they consider information irrelevant to risk assessment and (iii) whether they make use of this information for their final product choice. For this, the participants were split randomly into three experimental groups (a risk priming group, an effectiveness priming group, and a control group) that each received different tasks while purchasing a laundry detergent. The results indicate that the warnings found on laundry detergents are effective when they are used, although the majority of consumers do not look at the warnings. Therefore, we suggest that the alternative placement of warnings or the use of simplified warnings should be considered to improve consumers’ awareness of potential risks.Risk Perception
Environmental Research
Warning; Chemical household product; Risk perception; Eye tracking; Virtual environment (VR); Packagingvr
2022Javier Marina-Miranda, V. Javier TraverHead and eye egocentric gesture recognition for human-robot interaction using eyewear camerasNon-verbal communication plays a particularly important role in a wide range of scenarios in Human-Robot Interaction (HRI). Accordingly, this work addresses the problem of human gesture recognition. In particular, we focus on head and eye gestures, and adopt an egocentric (first-person) perspective using eyewear cameras. We argue that this egocentric view offers a number of conceptual and technical benefits over scene- or robot-centric perspectives.
A motion-based recognition approach is proposed, which operates at two temporal granularities. Locally, frame-to-frame homographies are estimated with a convolutional neural network (CNN). The output of this CNN is input to a long short-term memory (LSTM) to capture longer-term temporal visual relationships, which are relevant to characterize gestures.
Regarding the configuration of the network architecture, one particularly interesting finding is that using the output of an internal layer of the homography CNN increases the recognition rate with respect to using the homography matrix itself. While this work focuses on action recognition, and no robot or user study has been conducted yet, the system has been de signed to meet real-time constraints. The encouraging results suggest that the proposed egocentric perspective is viable, and this proof-of-concept work provides novel and useful contributions to the exciting area of HRI.
Gesture recognition
arxiv: Computer Science
Gesture recognitioncore
2022Wolfgang Fuhl, Daniel Weber, Enkelejda KasneciPistol: Pupil Invisible Supportive Tool to extract Pupil, Iris, Eye Opening, Eye Movements, Pupil and Iris Gaze Vector, and 2D as well as 3D GazeThis paper describes a feature extraction and gaze estimation software, named Pistol that can be used with Pupil Invisible projects and other eye trackers in the future. In offline mode, our software extracts multiple features from the eye including, the pupil and iris ellipse, eye aperture, pupil vector, iris vector, eye movement types from pupil and iris velocities, marker detection, marker distance, 2D gaze estimation for the pupil center, iris center, pupil vector, and iris vector using Levenberg Marquart fitting and neural networks. The gaze signal is computed in 2D for each eye and each feature separately and for both eyes in 3D also for each feature separately. We hope this software helps other researchers to extract state-of-the-art features for their research out of their recordings. Software
arxiv: Computer Science
2021Ye, Yuyan, Yongmei Ding, and Hongping Fang.Bayesian Intention Inference Based on Human Visual SignalIn the robot-assisted technology designed for the disabled, it is very challenging to infer the potential intentions based on accurate eye movements. This paper proposes a framework to capture the user's eye movement characteristics, infer their implicit intentions, and realize the interaction between humans and assistive robots. The framework uses the I-DT algorithm to extract the intentional gaze points and uses the DBSCAN algorithm to cluster their intentional gaze points to identify the subject's area of interest and the corresponding objects under the area of interest, thereby building a small intent knowledge base. Using this as a priori information, when the intention is unknown, the Naive Bayes model is used to infer the intention of the eye movement event. The empirical results of 9 subjects using the Pupil Core eye tracker show that the frame accuracy rate can reach 86.7%, indicating that the proposed eye movement event recognition and intention inference have certain validity and accuracy.Robot assisted technology
4th International Conference on Algorithms, Computing and Artificial Intelligence
Robotics, Disabilities, Assistive technology, DBSCAN algorithmcore
2022Chavant, Martin and Kapoula, ZoïPresbycusis and the Aging of Eye Movement: Common Attention MechanismsPresbycusis, physiological age-related hearing loss, is a major health problem because it is the most common cause of hearing impairment, and its impact will grow in the coming years with the aging population. Besides auditory consequences, the literature recently found an association between hearing loss and cognitive decline over the last two decades, emphasizing the importance of the early detection of presbycusis. However, the current hearing tests are not sufficient to detect presbycusis in some cases. Furthermore, the underlying mechanisms of this association are still under discussion, calling for a new field of research on that topic. In that context, this study investigates for the first time the interaction between presbycusis, eye movement latency and Stroop scores for a normal aging population. Hearing abilities, eye movement latency and the Stroop Victoria test were measured for 69 elderly (mean 66.7 ± 8.4) and 30 young (mean 25.3 ± 2.7) participants. The results indicated a significant relationship between saccade latency and speech audiometry in the silence score, independently from age. These promising results suggest common attentional mechanisms between speech processing and saccade latency. The results are discussed regarding the relationship between hearing and cognition, and regarding the perspective of expanding new tools for presbycusis diagnosis.Saccades, Vergence, Aging Brain Sciencespresbycusis; saccade and vergence latency; attention; agingcore
2022Pereira, D., De Pra, Y., Tiberi, E. et al. Flipping food during grilling tasks, a dataset of utensils kinematics and dynamics, food pose and subject gazeThis paper presents a multivariate dataset of 2866 food flipping movements, performed by 4 chefs and 5 home cooks, with different grilled food and two utensils (spatula and tweezers). The 3D trajectories of strategic points in the utensils were tracked using optoelectronic motion capture. The pinching force of the tweezers, the bending force and torsion torque of the spatula were also recorded, as well as videos and the subject gaze. These data were collected using a custom experimental setup that allowed the execution of flipping movements with freshly cooked food, without having the sensors near the dangerous cooking area. Complementary, the 2D position of food was computed from the videos. The action of flipping food is, indeed, gaining the attention of both researchers and manufacturers of foodservice technology. The reported dataset contains valuable measurements (1) to characterize and model flipping movements as performed by humans, (2) to develop bio-inspired methods to control a cooking robot, or (3) to study new algorithms for human actions recognition.Food, Research Data, Biomedical Engineering
Nature: Scientific Data
Food, Research Data, Biomedical Engineeringcore
2022Murali, S., Händel, BMotor restrictions impair divergent thinking during walking and during sittingCreativity, specifically divergent thinking, has been shown to benefit from unrestrained walking. Despite these findings, it is not clear if it is the lack of restriction that leads to the improvement. Our goal was to explore the effects of motor restrictions on divergent thinking for different movement states. In addition, we assessed whether spontaneous eye blinks, which are linked to motor execution, also predict performance. In experiment 1, we compared the performance in Guilford’s alternate uses task (AUT) during walking vs. sitting, and analysed eye blink rates during both conditions. We found that AUT scores were higher during walking than sitting. Albeit eye blinks differed significantly between movement conditions (walking vs. sitting) and task phase (baseline vs. thinking vs. responding), they did not correlate with task performance. In experiment 2 and 3, participants either walked freely or in a restricted path, or sat freely or fixated on a screen. When the factor restriction was explicitly modulated, the effect of walking was reduced, while restriction showed a significant influence on the fluency scores. Importantly, we found a significant correlation between the rate of eye blinks and creativity scores between subjects, depending on the restriction condition. Our study shows a movement state-independent effect of restriction on divergent thinking. In other words, similar to unrestrained walking, unrestrained sitting also improves divergent thinking. Importantly, we discuss a mechanistic explanation of the effect of restriction on divergent thinking based on the increased size of the focus of attention and the consequent bias towards flexibility.Divergent thinking, Walking
Psychological Research
Divergent thinking, walkingcore
2022Tammi, Tuisku and Pekkanen, Jami and Tuhkanen, Samuel and Oksama, Lauri and Lappi, OttoTracking an occluded visual target with sequences of saccadesGaze behavior during visual tracking consists of a combination of pursuit and saccadic movements. When the tracked object is intermittently occluded, the role of smooth pursuit is reduced, with a corresponding increase in the role of saccades. However, studies of visual tracking during occlusion have focused only on the first few saccades, usually with occlusion periods of less than 1 second in duration. We investigated tracking on a circular trajectory with random occlusions and found that an occluded object can be tracked reliably for up to several seconds with mainly anticipatory saccades and very little smooth pursuit. Furthermore, we investigated the accumulation of uncertainty in prediction and found that prediction errors seem to accumulate faster when an absolute reference frame is not available during tracking. We suggest that the observed saccadic tracking reflects the use of a time-based internal estimate of object position that is anchored to the environment via fixations. Saccades, Smooth Pursuits of Visionvision, saccades, smooth pursuitscore
2022Barbara Arfé, Gaia Spicciarelli, Flavia Gheller, Massimiliano Facca, Nadina Gómez-Merino, Patrizia Trevisi, Alessandro MartiniThe cognitive effects of noise on the memory performance of children with cochlear implantsConcentrating to perform cognitive tasks in a noisy environment requires to re-allocate mental resources to overcome the interference of noise. This process and the resulting fatigue, i.e., cognitive effort, can be detrimental to hearing children’s cognitive performance and ultimately to their learning. However, we know little of how background noise affects the cognitive performance of children with hearing loss. In a pilot trial, we addressed this research question. Eight cochlear implanted (CI) children and 5 age-matched normally hearing children (NH) (7-12 years) carried out an auditory attention task and a digit span task in quiet and babble noise. Behavioral (accuracy), self-report, and psychophysiological (pupil dilation) measures were used to assess children’s cognitive performance and cognitive effort. CI children performed worse than NH children in both acoustic conditions. However, no significant effects of acoustic condition (quiet/noise) were observed. Although CI children efficiently compensated for noise in performing the cognitive tasks, their pupil dilation revealed greater cognitive effort in noise than in quietAuditory 2021noise, cognitive effort, working memory, cochlear implants, children.core
2022Topalovic, Uros and Barclay, Sam and Ling, Chenkai and Alzuhair, Ahmed and Yu, Wenhao and Hokhikyan, Vahagn and Chandrakumar, Hariprasad and Rozgic, Dejan and Jiang, Wenlong and Basir-Kazeruni, Sina and Maoz, Sabrina L. and Inman, Cory S. and Gill, Jay and Bari, Ausaf and Fallah, Aria and Eliashiv, Dawn and Pouratian, Nader and Fried, Itzhak and Suthana, Nanthia and Markovic, DejanA wearable platform for closed-loop stimulation and recording of single-neuron and local field potential activity in freely-moving humansAdvances in technologies that can record and stimulate deep-brain activity in humans have led to impactful discoveries within the field of neuroscience and contributed to the development of novel therapies for neurological and psychiatric disorders. Further progress, however, has been hindered by device limitations in that recording of single-neuron activity during freely-moving behaviors in humans has not been possible. Additionally, implantable neurostimulation devices, currently approved for human use, have limited stimulation programmability and lack full-duplex bi-directional capability. Here, we developed a wearable bi-directional closed-loop neuromodulation system (Neuro-stack) and used it to record single-neuron and local field potential activity during stationary and ambulatory behavior in humans. Together with a highly flexible and customizable stimulation capability, the Neuro-stack provides an opportunity to investigate the neurophysiological basis of disease, develop improved responsive neuromodulation therapies, explore brain function during naturalistic behaviors in humans, and consequently, bridge decades of neuroscientific findings across species.Competing Interest StatementThe authors have declared no competing interest.Neuroscience
2022Ogihara, Takahiko and Tanioka, Kensuke and Hiroyasu, Tomoyuki and Hiwa, SatoruPredicting the Degree of Distracted Driving Based on fNIRS Functional Connectivity: A Pilot StudyDistracted driving is one of the main causes of traffic accidents. By predicting the attentional state of drivers, it is possible to prevent distractions and promote safe driving. In this study, we developed a model that could predict the degree of distracted driving based on brain activity. Changes in oxyhemoglobin concentrations were measured in drivers while driving a real car using functional near-infrared spectroscopy (fNIRS). A regression model was constructed for each participant using functional connectivity as an explanatory variable and break reaction time to random beeps while driving as an objective variable. As a result, we were able to construct a prediction model for 11 of the 12 participants. Furthermore, the regression model with the highest prediction accuracy for each participant was analyzed to gain a better understanding of the neural basis of distracted driving. The 11 models were classified into five clusters by hierarchical clustering based on their functional connectivity edges used in each cluster. The results showed that the combinations of the dorsal attention network (DAN)-sensory motor network (SMN) and DAN-ventral attention network (VAN) connections were common in all clusters and that these networks were essential to predict the degree of distraction in complex multitask driving. They also confirmed the existence of multiple types of prediction models with different within- and between-network connectivity patterns. These results indicate that it is possible to predict the degree of distracted driving based on the brain activity of the driver during actual driving. These results are expected to contribute to the development of safe driving systems and elucidation of the neural basis of distracted driving.Competing Interest StatementThe authors have declared no competing interest.AALAutomated anatomical labelingBPBootstrap percentageBRTBrake reaction timesCANController area networkDANDorsal attention networkFCFunctional connectivityFPFrequency percentageLSLLab-streaming layerMFGMiddle frontal gyrusMNIMontreal Neurological InstituteNBNumber of bootstrappingOLSOrdinary least squaresSDDShort-distance detectorSMNSensory motor networkSNSalience networkSSRShort separation regressionUPGMAUnweighted pair group method with arithmeticVANVentral attention networkDLPFCDorsolateral prefrontal cortexDMNDefault mode networkECGElectrocardiographyFPNfrontoparietal networkNIRSNear-infrared spectroscopyDriving
2022Melisa Stevanovic, Samuel Tuhkanen, Milla Järvensivu, Emmi Koskinen, Camilla Lindholm, Jenny Paananen, Enikö Savander, Taina Valkeapää, Kaisa ValkiarantaMaking Food Decisions Together: Physiological and Affective Underpinnings of Relinquishing Preferences and Reaching Decisions We used a novel interdisciplinary experimental paradigm where two types of dyads—15 dyads with one depressed and one non-depressed participant and 15 dyads with two non-depressed participants—engaged in a series of food-decision-making tasks. We examined how different communicative events during the decision-making process were reflected in the affective responses of the interacting participants, as indicated in their skin conductance (SC) response rates. The participants’ SC response rates were found to be higher during the emergence of the final decision, compared to the other segments during the process. Furthermore, relinquishing one’s initially expressed preferences was associated with SC response rates higher than the baseline. However, during the relinquishment segments, there was a negative interaction between depression diagnosis and SC response rates, which suggests that, compared to their non-depressed comparisons, it is affectively less arousing for the participants with depression to give up their previously expressed preferences.Food Openjoint decision making, skin conductance, affective arousal, relinquishment, depressioncore
2022Zhou, Y., Feng, T., Shuai, S. et al.DVAM: a 3D eye-tracking dataset for visual attention modeling in a virtual museumPredicting visual attention facilitates an adaptive virtual museum environment and provides a context-aware and interactive user experience. Explorations toward development of a visual attention mechanism using eye-tracking data have so far been limited to 2D cases, and researchers are yet to approach this topic in a 3D virtual environment and from a spatiotemporal perspective. We present the first 3D Eye-tracking Dataset for Visual Attention modeling in a virtual Museum, known as the EDVAM. In addition, a deep learning model is devised and tested with the EDVAM to predict a user’s subsequent visual attention from previous eye movements. This work provides a reference for visual attention modeling and context-aware interaction in the context of virtual museums.Museum
Front Inform Technol Electron Eng
Visual attention, Virtual museums, Eye-tracking datasets, Gaze detection, Deep learningvr
2022Ma, Jun and Li, Jiateng and Huang, HongweiEvaluation of Multimodal and Multi-Staged Alerting Strategies for Forward Collision Warning SystemsV2X is used for communication between the surrounding pedestrians, vehicles, and roadside units. In the Forward Collision Warning (FCW) of Phase One scenarios in V2X, multimodal modalities and multiple warning stages are the two main warning strategies of FCW. In this study, three warning modalities were introduced, namely auditory warning, visual warning, and haptic warning. Moreover, a multimodal warning and a novel multi-staged HUD warning were established. Then, the above warning strategies were evaluated in objective utility, driving performance, visual workload, and subjective evaluation. As for the driving simulator of the experiment, SCANeR was adopted to develop the driving scenario and an open-cab simulator was built based on Fanatec hardware. Kinematic parameters, location-related data and eye-tracking data were then collected. The results of the Analysis of Variance (ANOVA) indicate that the multimodal warning is significantly better than that of every single modality in utility and longitudinal car-following performance, and there is no significant difference in visual workload between multimodal warning and the baseline. The utility and longitudinal driving performance of multi-staged warning are also better than those of single-stage warning. Finally, the results provide a reference for the warning strategy design of the FCW in Intelligent Connected Vehicles.Driving connected vehicle; active safety; multimodal warning; multi-staged warning; driving simulator; eye-tracking analysiscore
2022Lan, G., Scargill, T., & Gorlatova, M.EyeSyn: Psychology-inspired Eye Movement Synthesis for Gaze-based Activity Recognition.Recent advances in eye tracking have given birth to a new genre of gaze-based context sensing applications, ranging from cognitive load estimation to emotion recognition. To achieve state-ofthe-art recognition accuracy, a large-scale, labeled eye movement dataset is needed to train deep learning-based classifiers. However, due to the heterogeneity in human visual behavior, as well as the labor-intensive and privacy-compromising data collection process, datasets for gaze-based activity recognition are scarce and hard to collect. To alleviate the sparse gaze data problem, we present EyeSyn, a novel suite of psychology-inspired generative models that leverages only publicly available images and videos to synthesize a
realistic and arbitrarily large eye movement dataset. Taking gazebased museum activity recognition as a case study, our evaluation demonstrates that EyeSyn can not only replicate the distinct patterns in the actual gaze signals that are captured by an eye tracking device, but also simulate the signal diversity that results from different measurement setups and subject heterogeneity. Moreover, in the few-shot learning scenario, EyeSyn can be readily incorporated with either transfer learning or meta-learning to achieve 90% accuracy, without the need for a large-scale dataset for training.
Machine Learning, Gaze Prediction
Proceedings of ACM/IEEE IPSN.
Eye tracking, eye movement synthesis, activity recognitioninvisible
2022N Kraus, M Niedeggen, G HesselmannNegative affect impedes perceptual filling-in in the uniformity illusionThe notion of cognitive penetrability, i.e., whether perceptual contents can in principle be influenced by non-perceptual factors, has sparked a significant debate over methodological concerns and the correct interpretation of existing findings. In this study, we combined predictive processing models of visual perception and affective states to investigate influences of affective valence on perceptual filling-in in extrafoveal vision. We tested how experimentally induced affect would influence the probability of perceptual filling-in occurring in the uniformity illusion (N = 50). Negative affect led to reduced occurrence rates and increased onset times of visual uniformity. This effect was selectively observed in illusionary trials, requiring perceptual filling-in, and not in control trials, where uniformity was the veridical percept, ruling out biased motor responses or deliberate judgments as confounding variables. This suggests an influential role of affective status on subsequent perceptual processing, specifically on how much weight is ascribed to priors as opposed to sensory evidence.Illusion, Perception
Consciousness and Cognition
Predictive processing
Precision weighting
Perceptual filling-in
Affective valence
Cognitive penetrability
2022P.Puerta, L. Laguna, A. Tárrega, E.CarrilloRelevant elements on biscuits purchasing decision for coeliac children and their parents in a supermarket contextThe aim of this work was to study the behaviour and motivations of coeliac children and their parents when purchasing biscuits. Four groups (n = 30) of participants differing in coeliac condition (coeliac and non-coeliac) and age (children and parents) were studied. Participants were asked to “purchase” biscuits, either for themselves (children) or for their children (parents), in a simulated supermarket aisle that included twelve commercial biscuits (six gluten-free and six regular ones). Eye-tracking technique was used to register visual attention during the purchasing exercise and laddering interviews were used to obtain the self-reported reasons for their choice. The number of fixations received by biscuits and label elements were analysed and most of them varied depending on the coeliac condition, the age or both. In comparison with the non-coeliac children, coeliac children fixated more on the ingredients, gluten-free words and symbols, and fixated less on the biscuit image. Parents of coeliac children put more attention on the ingredients and the certified gluten-free symbol, and less attention on the biscuit image, product name, cartoon, and nutritional information than non-coeliac parents. According to the chains of reasons (attribute-consequence-value), all children looked for pleasure as the final value, but only coeliac children showed interest in the brand and in unknown products they want to try. Parents differed on the attributes linked to health that were certification logo and a short ingredient list for coeliac group, and low sugar or fat contents for non-coeliac one. Trust and economy were relevant only for parents of coeliac children.Children, Purchasing Decision
Food Quality and Preference
Coeliac children
Purchasing decision
Simulated context
2022Luise Reitstätte, Karolin Galter & Flora BakondiLooking to Read: How Visitors Use Exhibit Labels in the Art Museum“Do they read? Oh, yes, they do,” was the conclusion of a paper identifying the proof of label use in visitors’ in-gallery conversations versus the difficulties of observing them reading. This paper ­methodologically refines this research question by asking how exactly exhibit labels are used. Answers are derived from an empirical study that analyzed viewing behavior both before and after the reinstallation of a museum’s collection through mobile eye tracking (MET), subjective mapping, and questionnaires. As the introduction of interpretive labels was one of the major changes implemented, the paper demonstrates differences in visitors’ responses to the artworks with or without contextual information. Analytical emphasis rests on the exploration of patterns in the process of decision making (differentiating between visitors’ reading affinities); visual engagement (analyzing the combined activities of looking and reading); and memory (echoing label texts in visitors’ artwork reflections). Our findings show that all visitors read, albeit to very different extents, the majority being medium-affinity readers; that the basic viewing pattern “art-label-art” becomes more complex with more text and more ­visitors on-site; and that art interpretations deepen and differ through additional information. The power of labels to guide eyes and thoughts suggests their intentional use in museum and curatorial practice.Art, Museum, UX StudiesArt museum, exhibit labels, interpretation, memory, reading, affinity, viewing patterns, visual engagementcore
2022Davide Maggi, Richard Romano, Oliver CarstenHanding control back to drivers: Exploring the effects of handover procedure during transitions from Highly Automated DrivingThe operational capabilities of automated driving features are limited and sometimes require drivers’ intervention through a transition of control. Assistance at an operational level might be extremely beneficial during transitions but the literature lacks evidence on the topic. A simulator study was conducted to investigate the potential impacts that lateral assistance systems might have while the Automated Driving System (ADS) hands back control to the driver. Results showed that drivers benefitted from a strong Lane Keeping Assist during the first phase of the transfer, helping them to keep the lane centre. However, assisting the drivers at an operational level did not enhance their capability of addressing a more complex task, presented as a lane change. In fact, it was more task-specific assistance (Blind-spot assist) that allowed drivers to better cope with the tactical decision that the lane change required. Moreover, longer exposure to lane-keeping assist systems helped them in gaining awareness of the surrounding traffic and improved the way drivers interacted with the Blind-spot assist.Driver behavior
Vehicle automation
Intelligent vehicle systems
Autonomous driving Research Part F: Traffic Psychology and BehaviourDriver behavior
Vehicle automation
Intelligent vehicle systems
Autonomous driving
2022García Cena, C.; Costa, M.C.; Saltarén Pazmiño, R.; Santos, C.P.; Gómez-Andrés, D.; Benito-León, JEye Movement Alterations in Post-COVID-19 Condition: A Proof-of-Concept StudyThere is much evidence pointing out eye movement alterations in several neurological diseases. To the best of our knowledge, this is the first video-oculography study describing potential alterations of eye movements in the post-COVID-19 condition. Visually guided saccades, memory-guided saccades, and antisaccades in horizontal axis were measured. In all visual tests, the stimulus was deployed with a gap condition. The duration of the test was between 5 and 7 min per participant. A group of n=9 patients with the post-COVID-19 condition was included in this study. Values were compared with a group (n=9) of healthy volunteers whom the SARS-CoV-2 virus had not infected. Features such as centripetal and centrifugal latencies, success rates in memory saccades, antisaccades, and blinks were computed. We found that patients with the post-COVID-19 condition had eye movement alterations mainly in centripetal latency in visually guided saccades, the success rate in memory-guided saccade test, latency in antisaccades, and its standard deviation, which suggests the involvement of frontoparietal networks. Further work is required to understand these eye movements’ alterations and their functional consequencesMedical, Neuropsychology, COVID
saccadic movement
2021Roche, T. R., Said, S., Braun, J., Maas, E. J., Machado, C., Grande, B., ... & Tscholl, D. W.Avatar-based patient monitoring in critical anaesthesia events: a randomised high-fidelity simulation study.Background
Failures in situation awareness cause two-thirds of anaesthesia complications. Avatar-based patient monitoring may promote situation awareness in critical situations.

We conducted a prospective, randomised, high-fidelity simulation study powered for non-inferiority. We used video analysis to grade anaesthesia teams managing three 10 min emergency scenarios using three randomly assigned monitoring modalities: only conventional, only avatar, and split-screen showing both modalities side by side. The primary outcome was time to performance of critical tasks. Secondary outcomes were time to verbalisation of vital sign deviations and the correct cause of the emergency, perceived workload, and usability. We used mixed Cox and linear regression models adjusted for various potential confounders. The non-inferiority margin was 10%, or hazard ratio (HR) 0.9.

We analysed 52 teams performing 154 simulations. For performance of critical tasks during a scenario, split-screen was non-inferior to conventional (HR=1.13; 95% confidence interval [CI], 0.96–1.33; not significant in test for superiority); the result for avatar was inconclusive (HR=0.98; 95% CI, 0.83–1.15). Avatar was associated with a higher probability for verbalisation of the cause of the emergency (HR=1.78; 95% CI, 1.13–2.81; P=0.012). We found no evidence for a monitor effect on perceived workload. Perceived usability was lower for avatar (coefficient=–23.0; 95% CI, –27.2 to –18.8; P<0.0001) and split-screen (–6.7; 95% CI, –10.9 to –2.4; P=0.002) compared with conventional.

This study showed non-inferiority of split-screen compared with conventional monitoring for performance of critical tasks during anaesthesia crisis situations. The patient avatar improved verbalisation of the correct cause of the emergency. These results should be interpreted considering participants' minimal avatar but extensive conventional monitoring experience.
Medical, Anesthesiology, HCI
British Journal of Anaesthesia
avatar, critical event, high-fidelity simulation, patient monitoring, situation awareness, visual patientinvisible
2021Meyer, Johannes and Frank, Adrian and Schlebusch, Thomas and Kasneci, EnkeljedaA CNN-Based Human Activity Recognition System Combining a Laser Feedback Interferometry Eye Movement Sensor and an IMU for Context-Aware Smart GlassesSmart glasses are considered the next breakthrough in wearables. As the successor of smart watches and smart ear wear, they promise to extend reality by immersive embedding of content in the user's field of view. While advancements in display technology seems to fulfill this promises, interaction concepts are derived from established wearable concepts like touch interaction or voice interaction, preventing full immersion as they require the user to frequently interact with the glasses. To minimize interactions, we propose to add context-awareness to smart glasses through human activity recognition (HAR) by combining head- and eye movement features to recognize a wide range of activities. To measure eye movements in unobtrusive way, we propose laser feedback interferometry (LFI) sensors. These tiny low power sensors are highly robust to ambient light. We combine LFI sensors and an IMU to collect eye and head movement features from 15 participants performing 7 cognitive and physical activities, leading to a unique data set. To recognize activities we propose a 1D-CNN model and apply transfer learning to personalize the classification, leading to an outstanding macro-F1 score of 88.15 % which outperforms state of the art methods. Finally, we discuss the applicability of the proposed system in a smart glasses setup.Human activity recognition
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.
Human activity recognition, head and eye movement, context awarness smart glasses, Laser Feedback Interferometrycore
2021Haifeng Zhao; Petra Karlsson; Omid Kavehei; Alistair McEwanAugmentative and Alternative Communication with Eye-gaze Technology and Augmented Reality: Reflections from Engineers, People with Cerebral Palsy and CaregiversThis paper reports on the outcome of a survey in which the perceived satisfaction and workload of a novel augmented reality (AR) device for Augmentative and Alternative Communication (AAC) was explored among engineers, people with Cerebral Palsy (CP) and/or their caregivers. The current design of a novel device using AR glasses, Microsoft HoloLens, and an eye-tracking system from Pupil Labs HoloLens Binocular Add-on was explored. The survey was based on QUEST 2.0 and NASA-TLX. The average result for the QUEST 2.0 was 3.52, 3.65, and 3.04 out of 5 for all participants, engineers, and people with cerebral palsy and/or their guardians, respectively. Findings from the NASA-TLX showed that people with cerebral palsy and/or their caregivers perceived a lower workload than the engineers. Therefore, it is believed that, although the current design still has some challenges, it holds the potential as a novel AAC solution for people with cerebral palsy, with complex communication needs.Communication SensorsCerebral Paulsy, Communicationar
2022Ye, B., Fujimoto, Y., Uchimine, Y., Sawabe, T., Kanbara, M., & Kato, H.Cross-talk elimination for lenslet array near eye display based on eye-gaze tracking.Lenslet array (LA) near-eye displays (NEDs) are a recent technical development that creates a virtual image in the field of view of one or both eyes. A problem occurs when the user’s pupil moves out of the LA-NED eye box (i.e., cross-talk) making the image look doubled or ghosted. It negatively impacts the user experience. Although eye-gaze tracking can mitigate this problem, the effect of the solution has not been studied to understand the impact of pupil size and human perception. In this paper, we redefine the cross-talk region as the practical pupil movable region (PPMR50), which differs from eye box size because it considers pupil size and human visual perception. To evaluate the effect of eye-gaze tracking on subjective image quality, three user studies were conducted. From the results, PPMR50 was found to be consistent with human perception, and cross-talk elimination via eye-gaze tracking was better understood in a static gaze scenario. Although the system latency prevented the complete elimination of cross-talk for fast movements or large pupil changes, the problem was greatly alleviated. We also analyzed system delays based on PPMR50, which we newly defined in this paper and provided an optimization scheme to meet the maximum eyeball rotation speed.Near eye displays
Optics Express, 30(10), 16196-16216.
Lenslet array, near-eye displays, corss-talkcore
2021Benjamin A. Newman, Kris M. Kitani, Henny AdmoniHand-Eye Coordination Primitives for Assistive Robotic Co-ManipulationRobots can help augment human performance in
teleoperation tasks that are difficult to complete, for example,
assisting a user with motor impairment to eat independently.
To do so, robots must intelligently recognize the user’s activity
state and offer the appropriate type of assistance. Prior work has
shown that a user’s teleoperation input (such as a joystick control
signal) can be used to predict their activity. However, basing such
assessments only on direct control signals misses the opportunity to
use rich human behavior signals that can further reveal user state,
specifically user intent. For example, eye gaze is tightly linked to the
target and timing of manipulation, and has been shown to predict
a users actions and identify errors in teleoperation. We propose
a semantically meaningful dictionary of hand-eye coordination
primitives to characterize a user’s state during co-manipulation.
We perform extensive analysis of a human-robot collaboration
dataset, HARMONIC, to extract frequently occurring hand-eye
coordination primitives, and identify a set of five action primitives
(exploration, pursuit, correction, mode switching, and toggle) that
characterize the user’s state. Additionally, we design data-driven
models to automatically classify these primitives. Preliminary
experiments with both synthetic and real data reveal the potential
and limitations of state-of-the-art learning approaches.
Hand-eye coordination, Robotics
2021Jia Zheng Lim, James Mountstephens, and Jason TeoExploring Pupil Position as An Eye-Tracking Feature for
Four-Class Emotion Classification In VR
This paper presented a preliminary investigation of a novel approach on emotion
recognition using pupil position in Virtual Reality (VR). We explore pupil position as an eye-tracking feature for four-class emotion classification according to the four-quadrant model of emotions via a presentation of 360 0 videos in VR. A total of ten subjects participated in this emotional experiment. A 360 0 video with four sessions of stimulation of emotions will be presented in VR to evoke the user’s emotions. The eye data were recorded and collected using Pupil Labs eye-tracker and the emotion classification was done by using pupil position solely. The classifier used in this investigation is the Support Vector Machine (SVM) machine learning algorithm. The results showed that the best accuracy achieved from this four-class random classification was 59.19%
Emotion classification
Journal of Physics: Conference Series
Emotion classificationvr
2021Gabriel Kerekes, Volker SchwiegerTowards Perceived Space Representation using
Brain Activity, Eye-Tracking and Terrestrial La-
ser Scanning
Deciphering how humans perceive physical spaces can lead to
new reality capture methods and eventually replace measurement instru-
ments like terrestrial laser scanners (TLSs). The current approach pursues
this goal using non-invasive wearable sensors like Brain Computer Inter-
faces (BCIs) and mobile eye-trackers. Work in progress from static experi-
ments shows that positions of targets on a wall at 4 m can be determined
with differences up to several dm and better w.r.t. ground truth using
Space perception, TLS, human perception 2021Space perception, TLS, human perceptioncore
2021Z. Wang, J. Epps and S. ChenAn Investigation of Automatic Saccade and Fixation Detection from Wearable Infrared CamerasEye movement plays an important role in cognition and perception, and the detection of saccade and fixation has been studied for human-computer applications, however often under conditions where head movement is constrained, and often using calibration-dependent gaze information rather than the raw pupil position. In order to investigate the performance of saccade and fixation detection using gaze and pupil data, three representative saccade detection algorithms are applied to both pupil data and gaze data collected with and without head movement, and their performance is evaluated against a stimulus-induced ground truth under different measures. Results indicate that saccade/fixation detection using pupil data generally provides better performance than using gaze data with an 8.6% improvement in Cohen’s Kappa (averaged across the three algorithms), even when moderate head movement is involved. Hence, pupil data can be used as an alternative to gaze data for saccade and fixation detection in wearable contexts with less effort in calibration and higher accuracy.Saccades, Fixations, Fixationscore
2021C. Lam, J. Epps and S. ChenWearable Fatigue Detection Based on Blink-Saccade SynchronisationAutomatic detection of fatigue based on remote cameras and computer vision has been investigated for many years, and is a viable solution in many contexts, but not for mobile contexts. This paper investigates eye activity extraction methods for wearable fatigue detection using low-cost hardware that is becoming ubiquitous in glasses form-factors, and proposes a new measure for fatigue based on the synchronization between blink and saccade. Evaluation on a novel dataset shows that the novel blink-saccade synchronization measure achieves statistically significant separation of control and fatigued participants, and provides automatic fatigue detection accuracy improvements of 10% relative to existing saccade measures based on velocity and amplitude.Fatigue, Driving
2021Martin, Joel T.
Pinto, Joana
Bulte, Daniel
Spitschan, Manuel
PyPlr: A versatile, integrated system of hardware and software for researching the human pupillary light reflexWe introduce PyPlr —a versatile, integrated system of hardware and software to support a broad spectrum of research applications concerning the human pupillary light reflex (PLR). PyPlr is a custom Python library for integrating a research-grade video-based eye-tracker system with a light source and streamlining stimulus design, optimisation and delivery, device synchronisation, and extraction, cleaning, and analysis of pupil data. We additionally describe how full-field, homogenous stimulation of the retina can be realised with a low-cost integrating sphere that serves as an alternative to a more complex Maxwellian view setup. Users can integrate their own light source, but we provide full native software support for a high-end, commercial research-grade 10-primary light engine that offers advanced control over the temporal and spectral properties of light stimuli as well as spectral calibration utilities. Here, we describe the hardware and software in detail and demonstrate its capabilities with two example applications: (1) pupillometer-style measurement and parametrisation of the PLR to flashes of white light, and (2) comparing the post-illumination pupil response (PIPR) to flashes of long and short-wavelength light. The system holds promise for researchers who would favour a flexible approach to studying the PLR and the ability to employ a wide range of temporally and spectrally varying stimuli, including simple narrowband stimuli.Pupillometry, PLR, Neurology Research MethodsPupillometry, Pupillary light reflex, PLR, Ganzfeld, Melanopsincore
2021Hougaard, B. I., Knoche, H., Jensen, J., & Evald, LSpatial neglect midline diagnostics from virtual reality and eye tracking in a free-viewing environmentPurpose: Virtual reality (VR) and eye tracking may provide detailed insights into spatial cognition. We hypothesized that virtual reality and eye tracking may be used to assess sub-types of spatial neglect in stroke patients not readily available from conventional assessments. Method: Eighteen stroke patients with spatial neglect and 16 age and gender matched healthy subjects wearing VR headsets were asked to look around freely in a symmetric 3D museum scene with three pictures. Asymmetry of performance was analyzed to reveal group-level differences and possible neglect sub-types on an individual level.Results: Four out of six VR and eye tracking measures revealed significant differences between patients and controls in this free-viewing task. Gaze-asymmetry between-pictures (including fixation time and count) and head orientation were most sensitive to spatial neglect behavior on a group level analysis. Gaze-asymmetry and head orientation each identified 10 out of 18 (56%), compared to 12 out of 18 (67%) for the best conventional test. Two neglect patients without deviant performance on conventional measures were captured by the VR and eyetracking measures. On the individual level, five stroke patients revealed deviant gaze-asymmetry within-pictures and six patients revealed deviant eye orientation in either direction that were not captured by the group-level analysis.Conclusion: This study is a first step in using VR in combination with eye tracking measures as individual differential neglect subtype diagnostics. This may pave the way for more sensitive and elaborate sub-type diagnostics of spatial neglect that may respond differently to various treatment approaches.Neuropsychology in Psychologyhemispatial neglect, virtual reality immersion therapy, diagnostic techniques and procedures, unilateral spatial neglect, eye tracking, head rotation, stroke, acquired brain injuryvr
2021Nicole Altvater-MackensenWhat Are You Looking At? Using Eye Tracking Glasses to Monitor Toddler Attention in Natural Learning Situations.Eye movements are taken to reflect cognitive processes. Because eye tracking is non-invasive and does not require an overt behavioural response, it is a suitable method to use with infants and toddlers. Most eye tracking studies to date use screen-based remote systems which allow high experimental control but limit the possibilities to test participants in more naturalistic, interactive settings. In this paper, we present a prototype of mobile eye tracking glasses suitable for toddlers and describe its use in a set-up that allows for automatic gaze coding. We present the components and set-up of the eye tracking glasses, as well as the basic principles and routines of data acquisition and analysis. Data from a pilot study testing 2- to 5-year-olds in a shared reading interaction and a preferential looking task is used to illustrate data quality and potential pitfalls in data collection.Developmental Psychology of the 45th annual
Boston University Conference
on Language Development
2021Graves, J. E., Egré, P., Pressnitzer, D., & de Gardelle, V.An implicit representation of stimulus ambiguity in pupil sizeTo guide behavior, perceptual systems must operate on intrinsically ambiguous sensory input. Observers are usually able to acknowledge the uncertainty of their perception, but in some cases, they critically fail to do so. Here, we show that a physiological correlate of ambiguity can be found in pupil dilation even when the observer is not aware of such ambiguity. We used a well-known auditory ambiguous stimulus, known as the tritone paradox, which can induce the perception of an upward or downward pitch shift within the same individual. In two experiments, behavioral responses showed that listeners could not explicitly access the ambiguity in this stimulus, even though their responses varied from trial to trial. However, pupil dilation was larger for the more ambiguous cases. The ambiguity of the stimulus for each listener was indexed by the entropy of behavioral responses, and this entropy was also a significant predictor of pupil size. In particular, entropy explained additional variation in pupil size independent of the explicit judgment of confidence in the specific situation that we investigated, in which the two measures were decoupled. Our data thus suggest that stimulus ambiguity is implicitly represented in the brain even without explicit awareness of this, pupilometry, cognition of the national academy of sciencesAmbiguity, Uncertainty, Confidence, auditory perception, pupilometrycore
2021Kasapakis, V., Dzardanova, E., Nikolakopoulou, V., Vosinakis, S., Xenakis, I., & Gavalas, D.Social Virtual Reality: Implementing Non-verbal Cues in Remote Synchronous CommunicationSocial Virtual Reality (SVR) platforms allow remote, synchronous interaction and communication between individuals immersed in shared virtual worlds. Such platforms commonly implement full-body motion and real-time voice communication, but often lack complete non-verbal cues support. This work presents the development process and preliminary usability evaluation results of an SVR platform, incorporating non-verbal cues such as finger motion, gaze direction, and facial expressions, while allowing inter-communication between remotely located interlocutors.VR Conference on Virtual Reality and Mixed RealityVirtual Reality, Social Platforms, Non-verbal Cues, Remote Communicationvr
2021Bonnen, K., Matthis, J. S., Gibaldi, A., Banks, M. S., Levi, D. M., & Hayhoe, M.Binocular vision and the control of foot placement during walking in natural terrain.Coordination between visual and motor processes is critical for the selection of stable footholds when walking in uneven terrains. While recent work (Matthis et al. in Curr Biol 8(28):1224–1233, 2018) demonstrates a tight link between gaze (visual) and gait (motor), it remains unclear which aspects of visual information play a role in this visuomotor control loop, and how the loss of this information affects that relationship. Here we examine the role of binocular information in the visuomotor control of walking over complex terrain. We recorded eye and body movements while normally-sighted participants walked over terrains of varying difficulty, with intact vision or with vision in one eye blurred to disrupt binocular vision. Gaze strategy was highly sensitive to the complexity of the terrain, with more fixations dedicated to foothold selection as the terrain became more difficult. The primary effect of increased sensory uncertainty due to disrupted binocular vision was a small bias in gaze towards closer footholds, indicating greater pressure on the visuomotor control process. Participants with binocular vision losses due to developmental disorders (i.e., amblyopia, strabismus), who have had the opportunity to develop alternative strategies, also biased their gaze towards closer footholds. Across all participants, we observed a relationship between an individual’s typical level of binocular visual function and the degree to which gaze is shifted toward the body. Thus the gaze–gait relationship is sensitive to the level of sensory uncertainty, and deficits in binocular visual function (whether transient or long-standing) have systematic effects on gaze strategy in complex terrains. We conclude that binocular vision provides useful information for locating footholds during locomotion. Furthermore, we have demonstrated that combined eye/body tracking in natural environments can be used to provide a more detailed understanding of the impact of a type of vision loss on the visuomotor control process of walking, a vital everyday task.Navigation, Walking, Motor Control Reports (Nature)core
2021Shimizu, Y., Ohnishi, A., Terada, T., & Tsukamoto, M.Gaze-Adaptive Subtitles Considering the Balance among Vertical/Horizontal and Depth of Eye Movement.Subtitles (captions displayed on the screen) are important in 3D content, such as virtual reality (VR) and 3D movies, to help users understand the content. However, an optimal displaying method and framework for subtitles have not been established for 3D content because 3D has a depth factor. To determine how to place text in 3D content, we propose four methods of moving subtitles dynamically considering the balance between the vertical/horizontal and depth of gaze shift. These methods are used to reduce the difference in depth or distance between the gaze position and subtitles. Additionally, we evaluate the readability of the text and participants’ fatigue. The results show that aligning the text horizontally and vertically to eye movements improves visibility and readability. It is also shown that the eyestrain is related to the distance between the object and subtitles. This evaluation provides basic knowledge for presenting text in 3D content.AR International Symposium on Mixed and Augmented Reality AdjunctThree-dimensional displays, Tracking, Games, Motion pictures, Fatigue,
Augmented reality, Videos
2021Cho, D. Y., & Kang, M. K.Human gaze-aware attentive object detection for ambient intelligence.Understanding human behavior and the surrounding environment is essential for realizing ambient intelligence (AmI), for which eye gaze and object information are reliable cues. In this study, the authors propose a novel human gaze-aware attentive object detection framework as an elemental technology for AmI. The proposed framework detects users’ attentive objects and shows more precise and robust performance against object-scale variations. A novel Adaptive-3D-Region-of-Interest (Ada-3D-RoI) scheme is designed as a front-end module, and scalable detection network structures are proposed to maximize cost-efficiency. The experiments show that the detection rate is improved up to 97.6% on small objects (14.1% on average), and it is selectively tunable with a tradeoff between accuracy and computational complexity. In addition, the qualitative results demonstrate that the proposed framework detects a user’s single object-of-interest only, even when the target object is occluded or extremely small. Complementary matters for follow-up study are presented as suggestions to extend the results of the proposed framework to further practical AmI applications. This study will help develop advanced AmI applications that demand a higher-level understanding of scene context and human behavior such as human–robot symbiosis, remote-/autonomous control, and augmented/mixed reality.HCI, AR Applications of Artificial Intelligencecore
2021Eugene HanIntegrating Mobile Eye-Tracking and VSLAM for Recording Spatial Gaze in Works of Art and ArchitectureThis paper proposes a method for spatial
eye-tracking for use with three-dimensional
objects and environments. By integrating mobile eye-tracking (MET) with Visual
Simultaneous Localization and Mapping
(VSLAM) technologies, the study provides
an unobtrusive technique for capturing
an individual’s gaze across an open space
and through an unprescribed viewing path.
The included proof-of-concept is tested
against three scales, from a large sculpture
to a set of towering cement kilns and an
exterior building passage. Demonstrations
show that the integration of MET and
VSLAM provides a useful tool for testing
scenarios without predefined viewing
conditions and allowing insight into how
others view works of art and architecture.
Eye Tracking, Attention, Visualization, Architecture|Architecture + DesignTrackinginvisible
2021Tianyi Zhang, Abdallah El Ali, Chen Wang, Alan Hanjalic and Pablo CesarCorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological SensorsRecognizing user emotions while they watch short-form videos anytime and anywhere is essential for facilitating video content customization and personalization. However, most works either classify a single emotion per video stimuli, or are restricted to static, desktop environments. To address this, we propose a correlation-based emotion recognition algorithm (CorrNet) to recognize the valence and arousal (V-A) of each instance (fine-grained segment of signals) using only wearable, physiological signals (e.g., electrodermal activity, heart rate). CorrNet takes advantage of features both inside each instance (intra-modality features) and between different instances for the same video stimuli (correlation-based features). We first test our approach on an indoor-desktop affect dataset (CASE), and thereafter on an outdoor-mobile affect dataset (MERCA) which we collected using a smart wristband and wearable eyetracker. Results show that for subject-independent binary classification (high-low), CorrNet yields promising recognition accuracies: 76.37% and 74.03% for V-A on CASE, and 70.29% and 68.15% for V-A on MERCA. Our findings show: (1) instance segment lengths between 1–4 s result in highest recognition accuracies (2) accuracies between laboratory-grade and wearable sensors are comparable, even under low sampling rates (≤64 Hz) (3) large amounts of neutral V-A labels, an artifact of continuous affect annotation, result in varied recognition performance. Psychology Sensorscore
2021Silvia Arias, Axel Mossberg, Daniel Nilsson & Jonathan WahlqvistA Study on Evacuation Behavior in Physical and Virtual Reality ExperimentsComparing results obtained in Virtual Reality to those obtained in physical experiments is key for validation of Virtual Reality as a research method in the field of Human Behavior in Fire. A series of experiments based on similar evacuation scenarios in a high-rise building with evacuation elevators was conducted. The experiments consisted of a physical experiment in a building, and two Virtual Reality experiments in a virtual representation of the same building: one using a Cave Automatic Virtual Environment (CAVE), and one using a head-mounted display (HMD). The data obtained in the HMD experiment is compared to data obtained in the CAVE and physical experiment. The three datasets were compared in terms of pre-evacuation time, noticing escape routes, walking paths, exit choice, waiting times for the elevators and eye-tracking data related to emergency signage. The HMD experiment was able to reproduce the data obtained in the physical experiment in terms of pre-evacuation time and exit choice, but there were large differences with the results from the CAVE experiment. Possible factors affecting the data produced using Virtual Reality are identified, such as spatial orientation and movement in the virtual environment.VR, Eye Tracking TechnologyGamesvr
2021Kelley E. Gunther, Xiaoxue Fu, Leigha MacNeill, Alicia Vallorani, Briana Ermanni, & Koraly Pérez-EdgarProfiles of naturalistic attentional trajectories associated with internalizing behaviors in school-age children: A mobile eye tracking studyThe temperament profile Behavioral Inhibition (BI) is a strong predictor of internalizing behavior in childhood. Patterns of attention towards or away from threat are a commonality of both BI and internalizing behaviors. Attention biases are traditionally measured with computer tasks presenting affective stimuli, which can lack ecological validity. Recent studies suggest that naturalistic visual attention need not mirror findings from computer tasks, and, more specifically, children high in BI may attend less to threats in naturalistic tasks. Here, we characterized latent trajectories of naturalistic visual attention over time to a female stranger, measured with mobile eye tracking, among kindergarteners oversampled for BI. Group-based trajectory modeling (GBTM) revealed two latent trajectories: 1) high initial orienting to the stranger, gradual decay, and recovery, and 2) low initial orienting and continued avoidance. Higher probability of membership to the “avoidant” group was linked to greater report of internalizing behaviors. We demonstrate the efficacy of mobile eye tracking in quantifying naturalistic patterns of visual attention to social novelty, as well as the importance of naturalistic measures of attention in characterizing socioemotional risk factors.Behavioral Inhibition, Attentional Trajectoriescore
2021Kaluarachchi, Tharindu Indrajith and Sapkota, Shardul and Taradel, Jules and Thevenon, Arist\'{e}e and Matthies, Denys J.C. and Nanayakkara, SurangaEyeKnowYou: A DIY Toolkit to Support Monitoring Cognitive Load and Actual Screen Time Using a Head-Mounted WebcamStudies show that frequent screen exposure and increased cognitive load can cause mental-health issues. Although expensive systems capable of detecting cognitive load and timers counting on-screen time exist, literature has yet to demonstrate measuring both factors across devices. To address this, we propose an inexpensive DIY-approach using a single head-mounted webcam capturing the user’s eye. By classifying camera feed using a 3D Convolutional Neural Network, we can determine increased cognitive load and actual screen time. This works because the camera feed contains corneal surface reflection, as well as physiological parameters that contain information on cognitive load. Even with a small data set, we were able to develop generalised models showing 70% accuracy. To increase the models’ accuracy, we seek the community’s help by contributing more raw data. Therefore, we provide an opensource software and a DIY-guide to make our toolkit accessible to human factors researchers without an engineering background.HCI, Cognitive Science '21 Adjunct: Adjunct Publication of the 23rd International Conference on Mobile Human-Computer InteractionMotion picturescore
2021Eric A Kaiser; Harrison McAdams; Aleksandra Igdalova; Edda B Haggerty;
Brett L Cucchiara; David H Brainard; Geoffrey K Aguirre
Reflexive Eye Closure in Response to Cone and Melanopsin Stimulation: A Study of Implicit Measures of Light Sensitivity in MigraineObjective: To quantify interictal photophobia in migraine with and without aura using
reflexive eye closure as an implicit measure of light sensitivity, and to assess the
contribution of melanopsin and cone signals to these responses.
Methods: Participants were screened to meet criteria for one of three groups:
headache-free (HAf) controls, migraine without aura (MwoA), and migraine with visual
aura (MwA). MwoA and MwA participants were included if they endorsed ictal and
interictal photophobia. Exclusion criteria included impaired vision, inability to collect usable pupillometry, and history of either head trauma or seizure. Participants viewed light pulses that selectively targeted melanopsin, the cones, or their combination during recording of orbicularis oculi electromyography (OO-EMG) and blinking activity.
Results: We studied twenty participants in each group. MwA and MwoA groups reported increased visual discomfort to light stimuli (Discomfort rating, 400% contrast, MwA: 4.84 [95% CI: 0.33, 9.35]; MwoA: 5.23 [0.96, 9.50]) as compared to HAf controls (2.71 [0, 6.47]). Time course analysis of OO-EMG and blinking activity demonstrated that reflexive eye closure was tightly coupled to the light pulses. The MwA group had greater OO-EMG and blinking activity in response to these stimuli (EMG activity, 400% contrast: 42.9%D [28.4, 57.4]; Blink activity, 400% contrast: 11.2% [8.8, 13.6]) as compared to the MwoA (EMG activity, 400% contrast: 9.9%D [5.8, 14.0]; Blink activity, 400% contrast: 4.7% [3.5, 5.9]) and HAf control (EMG activity, 400% contrast: 13.2%D [7.1, 19.3]; Blink activity, 400% contrast: 4.5% [3.1, 5.9]) groups. Conclusions: Our findings suggest that the intrinsically-photosensitive retinal ganglion cells (ipRGCs), which integrate melanopsin and cone signals, provide the afferent input for light-induced reflexive eye closure in a photophobic state. Moreover, we find a
dissociation between implicit and explicit measures of interictal photophobia depending on a history of visual aura in migraine. This implies distinct pathophysiology in forms ofmigraine, interacting with separate neural pathways by which the amplification of ipRGC signals elicit implicit and explicit signs of visual discomfort.
Light Sensitivity, Blinks
2021Lise Aubin and Ghilès Mostafaoui and Richard Schmidt and Hélène Serré and Ludovic MarinEffects of unintentional coordination on attentional loadThe goal of this study was to evaluate the impact of unintentional (spontaneous) coordination on high attentional visual load. More precisely, we wondered whether such coordination could free up some attentional resources and help improve performance in other more demanding attentional tasks. An experiment was performed in which participant attentional allocation was challenged by performing three tasks simultaneously while simultaneously being induced to unintentional entrain to an environmental rhythm. The first task was an interception task associated with a Stroop test to increase their attentional load. The second task was a reaction time test to alarms in different modalities (auditory, visual and bimodal) which was used to assess participant attentional load. The third task was a motor task in which participants were asked to swing their legs at a preferred frequency. The interface background brightness intensity was either synchronized in real time using a bidirectional coupling to participant leg movement or the background brightness was not changing at all. Our results on the reaction time task demonstrated that participants exhibited better reaction times for alarms in the bimodal condition than in the auditory condition and lastly for the visual condition. Also, participants exhibited a lower reaction time to alarms when the background brightness was synchronizing with their leg regardless the alarm modality. Overall, our study suggests a beneficial effect of unintentional environmental coordination on attentional resource allocation and highlights the importance of bidirectionality in interaction.Motor Behavior, Coordination Movement ScienceFatiguecore
2021Dae-Yong and ChoMin-KooKangHuman gaze-aware attentive object detection for ambient intelligenceUnderstanding human behavior and the surrounding environment is essential for realizing ambient intelligence (AmI), for which eye gaze and object information are reliable cues. In this study, the authors propose a novel human gaze-aware attentive object detection framework as an elemental technology for AmI. The proposed framework detects users’ attentive objects and shows more precise and robust performance against object-scale variations. A novel Adaptive-3D-Region-of-Interest (Ada-3D-RoI) scheme is designed as a front-end module, and scalable detection network structures are proposed to maximize cost-efficiency. The experiments show that the detection rate is improved up to 97.6% on small objects (14.1% on average), and it is selectively tunable with a tradeoff between accuracy and computational complexity. In addition, the qualitative results demonstrate that the proposed framework detects a user’s single object-of-interest only, even when the target object is occluded or extremely small. Complementary matters for follow-up study are presented as suggestions to extend the results of the proposed framework to further practical AmI applications. This study will help develop advanced AmI applications that demand a higher-level understanding of scene context and human behavior such as human–robot symbiosis, remote-/autonomous control, and augmented/mixed reality.HCI, Computer Science, AI Applications of Artificial Intelligencecore
2021Zhonghua Wan; Cai-Hua Xiong; Wenbin Chen; Hanyuan Zhang; Shiqian WuPupil-Contour-Based Gaze Estimation with Real Pupil Axes for Head-Mounted Eye TrackingAccurate gaze estimation that frees from glints and the slippage problem is challenging. Pupil-contour-based gaze estimation methods can meet this challenge, except that the gaze accuracy is low due to neglecting the pupil's corneal refrac-tion. This paper proposes a refraction-aware gaze estimation approach using the real pupil axis, which is calculated from the virtual pupil image based on the derived function between the real pupil and the refracted virtual pupil. We present a 2D gaze estimation method that regresses the real pupil normal's spherical coordinates to the gaze point. The noise and outliers of calibration data are removed by aggregation filtering and Random Sample Consensus, respectively. Moreover, we propose a 3D gaze estimation method that transforms the real pupil axis to the gaze direction. Experimental results show that the proposed gaze estimation approach has comparable accuracy to state-of-the-art pupil-center-based gaze estimation methods, which suffer from the slippage problem.Pupil Detection, Eye Tracking Transactions on Industrial InformaticsAugmented realitycore
2021Michael Winter, Harald Baumeister, Ulrich Frick, Miles Tallon, Manfred Reicher, and R¨udiger PryssExploring the Usability of the German COVID-19 Contact Tracing App in a Combined Eye Tracking and Retrospective Think Aloud StudyIn the course of the corona virus (COVID-19)
pandemic, many digital solutions for mobile devices (e.g.,
apps) were presented in order to provide additional resources
supporting the control of the pandemic. Contact tracing apps
(i.e., identify persons who may have been in contact with a
COVID-19 infected) constitute one of the most popular as
well as promising solutions. However, as a prerequisite for
an effective application, such apps highly depend on being
used by large numbers of the population. Consequently, it is
important that these apps offer a high usability for everyone.
We therefore conducted an exploratory study to learn more
about the usability of the German COVID-19 contact tracing
app Corona-Warn-App (CWA). More specifically, N = 15
participants assessed the CWA, relying on a combined eye
tracking and retrospective think aloud approach. The results
indicate, on the one hand, that the CWA leaves a promising
impression for pandemic control, as essential functions are
easily recognized. However, on the other hand, issues were
revealed (e.g., privacy policy) that could be addressed in future
updates more properly.
Think Aloud, Contract Tracing, Covid-19 Annual International Conference of the IEEE Engineering in Medicine and Biology Societycore
2021Kai Krabben and David Mann and Maria Lojanica and Daniel Mueller and Nadia Dominici and John van der Kamp and Geert SavelsberghHow wide should you view to fight? Establishing the size of the visual field necessary for grip fighting in judoPeripheral vision is often considered vital in (combat) sports, yet most experimental paradigms (e.g., eye tracking) ignore peripheral information or struggle to make inferences about the role of peripheral vision in an in-situ performance environment. This study aimed to determine where visual information is located in the peripheral field during an in-situ combat sports task. Eight advanced judokas competed in grip-fighting exchanges while wearing a mobile eye-tracker to locate gaze direction. Three-dimensional position data of the head and hands were tracked using a VICON motion capture system. Gaze analysis through automatic feature detection showed that participants predominantly fixated on their opponent’s chest. Kinematic data were used to calculate the angles between the opponent’s hands and the gaze-anchor point on the chest of the opponent. Results revealed a nonlinear relationship between visual field (VF) size and visibility of the hands, with athletes needing a VF of at least 30–40 degrees radius to simultaneously monitor both hands of the opponent most of the time. These findings hold implications for the regulation of Paralympic judo for athletes with vision impairment, suggesting that a less severe degree of impairment should be required to qualify than the current criterion of 20 degrees radius.Paralympic, Sports Science, Visual Field Loss of Sports SciencesVideoscore
2021El Hmimdi, Alae Eddine and Ward, Lindsey M and Palpanas, Themis and Kapoula, ZoïPredicting Dyslexia and Reading Speed in Adolescents from Eye Movements in Reading and Non-Reading Tasks: A Machine Learning ApproachThere is evidence that abnormalities in eye movements exist during reading in dyslexic individuals. A few recent studies applied Machine Learning (ML) classifiers to such eye movement data to predict dyslexia. A general problem with these studies is that eye movement data sets are limited to reading saccades and fixations that are confounded by reading difficulty, e.g., it is unclear whether abnormalities are the consequence or the cause of reading difficulty. Recently, Ward and Kapoula used LED targets (with the REMOBI &amp; AIDEAL method) to demonstrate abnormalities of large saccades and vergence eye movements in depth demonstrating intrinsic eye movement problems independent from reading in dyslexia. In another study, binocular eye movements were studied while reading two texts: one using the “Alouette” text, which has no meaning and requires word decoding, the other using a meaningful text. It was found the Alouette text exacerbates eye movement abnormalities in dyslexics. In this paper, we more precisely quantify the quality of such eye movement descriptors for dyslexia detection. We use the descriptors produced in the four different setups as input to multiple classifiers and compare their generalization performances. Our results demonstrate that eye movement data from the Alouette test predicts dyslexia with an accuracy of 81.25%; similarly, we were able to predict dyslexia with an accuracy of 81.25% when using data from saccades to LED targets on the Remobi device and 77.3% when using vergence movements to LED targets. Noticeably, eye movement data from the meaningful text produced the lowest accuracy (70.2%). In a subsequent analysis, ML algorithms were applied to predict reading speed based on eye movement descriptors extracted from the meaningful reading, then from Remobi saccade and vergence tests. Remobi vergence eye movement descriptors can predict reading speed even better than eye movement descriptors from the meaningful reading test.
Dyslexia, Reading Sciencessaccades; vergence; reading eye movements; Machine Learning; dyslexia; reading speedcore
2021William Irvin, Claire Goldie, Christopher O'Brien, Christopher Aura, Leonard Temme, Michael Wilson A virtual reality aviation emergency procedure (EP) testbedTo ensure safe mission completion, Army aviators must be prepared to execute appropriate emergency procedures (EPs) in a range of situations. Augmented and Virtual Reality (AR/VR) technologies provide novel opportunities to enhance the performance of emergency procedures by presenting them in powerful simulations of the operational environment. Currently, USAARL is developing a VR EP research simulator to support systematic, human performance research, development, test, and evaluation (RDT&E) programs to investigate and thereby enhance the effective, safe execution of EPs. Factors to be investigated include workload, flight maneuver, displays, multisensory stimuli, and aircrew coordination, as well psychophysiological and operational stressors known to potentially impact EP execution. The USAARL EP simulator is being developed and operates within the Unity Real-Time Development Platform currently instantiated with HTC Vive Pro hardware. In order to maximize immersion, virtual reality gloves are the user's method of controlling the simulation. Pupil Labs hardware and software has been integrated with the HTC Vive Pro to record, archive, and analyze synchronous, time stamped oculometric data. Engine fire and a single-engine failure are the emergencies that have been implemented to date in the EP simulator. One mode of operation permits the user to passively experience the EP with no required input. Other modes require the participant to execute the EP with predetermined cueing stimuli ranging from substantial step-by-step cueing to no cueing. Future work will incorporate additional EPs into standard maneuvers with defined physiological stressors.Aviation, Aerospace, Commercial Sensing Defense + Commercial SensingVirtual reality, Computer simulation, data archive systems, Failure analysisvr
2021Bharath Shankar, Christian Sinnott, Kamran Binaee, Mark D. Lescroart, Paul MacNeilageErgonomic Design Development of the Visual Experience Database HeadsetHead-mounted devices allow recording of eye movements, head movements, and scene video outside of the traditional laboratory setting. A key challenge for recording comprehensive first-person stimuli and behavior outside the lab is the form factor of the head-mounted assembly. It should be mounted stably on the head to minimize slippage and maximize accuracy of the data; it should be as unobtrusive and comfortable as possible to allow for natural behaviors and enable longer duration recordings; and it should be able to fit a diverse user population. Here, we survey preliminary design iterations of the Visual Experience Database headset, an assembly consisting of the Pupil Core eye tracker, the Intel RealSense T265 ™ (T265) tracking camera, and the FLIR Chameleon™3 (FLIR) world camera. Strengths and weaknesses of each iteration are explored and documented with the goal of informing future ergonomic design efforts for similar head-mounted systems.HCI, UI/UX, Computer Science, Ergonomics '21 Adjunct: ACM Symposium on Eye Tracking Research and Applicationsergonomics, head-mounted data recording, eye tracking, head trackingcore
2021Sanjana Ramanujam, Christian Sinnott, Bharath Shankar, Savannah Jo Halow, Brian Szekely, Paul MacNeilage, Kamran BinaeeVEDBViz: The Visual Experience Database Visualization and Interaction ToolMobile, simultaneous tracking of both the head and eyes is typically achieved through integration of separate head and eye tracking systems because off-the-shelf solutions do not yet exist. Similarly, joint visualization and analysis of head and eye movement data is not possible with standard software packages because these were designed to support either head or eye tracking in isolation. Thus, there is a need for software that supports joint analysis of head and eye data to characterize and investigate topics including head-eye coordination and reconstruction of how the eye is moving in space. To address this need, we have begun developing VEDBViz which supports simultaneous graphing and animation of head and eye movement data recorded with the Intel RealSense T265 and Pupil Core, respectively. We describe current functionality as well as features and applications that are still in development.HCI, UI/UX, Visualization, Computer Science '21 Adjunct: ACM Symposium on Eye Tracking Research and Applicationseye-tracking, head-tracking, data visualization, eye-movements,
head-movements, gaze
2021Jiaying Chen; Ngozi Charity Chidi-Egboka; Isabelle Jalbert; Peter Wagner; Blanka Golebiowski
Real-life in situ measurement of blinking using a wearable device: establishing a gold standard for spontaneous blinkingPurpose : The study of blinking has been hampered in part by the lack of a gold standard condition for measurement. This study measured blinking in situ in a real-life setting during various reading and non-reading tasks and also examined repeatability.

Methods : Ten healthy adults (30.7±3.6 years; 4M:6F) completed a randomised cross-over intervention study. Participants wore an eye-tracking headset (Pupil Labs GmbH) during 8 tasks (15 min each): A) conversation, B) reading from printed text, C) laptop screen, D) smart TV at 6 m, E) smartphone, F) smartphone at 50% brightness, G) smartphone (more complex text), H) walking indoors. To determine repeatability, task E was completed twice. Symptoms (Instant Ocular Symptom Survey) were measured before and after each task. Spontaneous blink rate and interblink interval were recorded using the headset. Blink parameters were compared between tasks using repeated measures ANOVA and post hoc comparisons with Bonferroni correction. Ocular symptoms pre- and post-task were compared using the Paired t-test. Repeatability was examined using the Bland & Altman method (Coefficient of Repeatability, CoR).

Results : Blink rate was reduced during all reading tasks (B to G) compared to conversation (p≤0.003) and walking (p≤0.04). There was no significant difference in blink rate between conversation and walking, nor between any of the reading tasks. There were no significant differences in interblink interval between tasks. Ocular symptoms worsened after reading from a smartphone at reduced brightness (p=0.02), more complex text (p=0.04) and from a distant TV screen (p=0.01). CoR was ±14.5 blinks/min for blink rate and ±22.4 s for interblink interval while reading from a smartphone.

Conclusions : The wearable eye tracker can reliably measure blink rate and interblink interval in situ. In a real-life setting, blink rate was reduced during reading compared to conversation or walking, irrespective of reading task complexity, working distance or device used.
Blinking, Reading, UI/UX, Locomotion Annual Meeting AbstractBlinking, Reading, UI/UX, Locomotioncore
2021Ngozi Charity Chidi-Egboka; Isabelle Jalbert; Blanka Golebiowski
One hour of smartphone use induces ocular discomfort and reduces blinking in childrenPurpose : Smartphone use by children is increasing rapidly, but the ocular impacts are unknown. This study examined the effect of 1 hour of smartphone use on symptoms, tear film and blinking in children.

Methods : Forty-five children aged 6–15 years (mean 10.1±2.6 years; 20M:25F) with healthy eyes and no binocular vision problems completed this prospective study. Children played games on a smartphone continuously for 1 hour and were masked to the study purpose. Symptoms were measured before and after using Symptoms Assessment in Dry Eye (SANDE), Instant Ocular Symptoms Survey (IOSS), Numerical Rating Scale (NRS). Tear film lipid layer thickness (LLT) (LipiView® interferometer.), tear meniscus height (TMH) and non-invasive tear break-up time (NIBUT) (Oculus® Keratograph 5) were assessed at the same timepoints. Spontaneous blink rate (blinks per minute) and interblink interval (time of full eyelid opening, seconds) were assessed in situ, at baseline (during conversation) and throughout the 1 hour of smartphone gaming, using a novel wearable eye tracking headset (Pupil Labs GmbH). Blink rate and interblink interval were compared between baseline and at 10 minutes intervals during smartphone use with repeated measures ANOVA and post hoc comparisons with Dunnett’s test and Bonferroni correction. Associations were examined using Pearson correlation.

Results : Ocular surface symptoms increased following 1 hour of smartphone use (SANDE +7.3 units, p=0.01; NRS overall +6.6, p=0.01; NRS comfort +8.5, p=0.01; NRS tiredness +11.2, p=0.004; IOSS +1.3, p<0.001), but tear film (LLT, TMH, and NIBUT) remained unchanged. Mean group blink rate reduced from 20.7±9.9 blinks/min during conversation to 7.4±5.2 blinks/min within 10 minutes of smartphone use (p<0.001), and interblink interval increased (3.1 s vs 8.4 s; p<0.001). Blink rate and interblink interval remained unchanged throughout 1 hour of smartphone use. There were no significant associations between changes in blink rate, interblink interval and symptoms.

Conclusions : Blinking in children can be successfully assessed in situ using a wearable eye tracking device. Smartphone use quickly resulted in dry eye symptoms, slowed the blink rate to one-third, with much longer open eye periods between blinks. In the short-term, this was not accompanied by disturbances to the tear film.
Children, Smartphone Use, UI/UX Annual Meeting AbstractChildren, Smartphone Use, UI/UXcore
2021Karl Muller; Dan Panfili; Jonathan Matthis; Mary Hayhoe
Foothold selection during locomotion over rocky terrainVery little is known about how visual information is used to guide foot placement during natural locomotion. Previous work studying the role of vision in locomotion has largely relied on constrained environments like treadmills or visual stimuli presented to head-fixed subjects. While these types of experimental controls are helpful for understanding such a complex behavior, there is also much to be learned from studying the behavior as it unfolds naturally. Using an apparatus that combines mobile eye tracking (Pupil Labs), IMU based motion capture (Motion Shadow), and photogrammetric software (Meshroom), we collected a novel dataset that features measurements of gaze direction, body position, and 3D environmental structure as subjects navigate across various outdoor terrains. The dataset is spatially and temporally aligned, so that the gaze direction, body position, and environmental structure information is all in the same coordinate frame, represented as a triangle mesh or a point cloud. Use of Meshroom on the Pupil Labs scene camera images allows correction for IMU drift by fixing the scene relative to the head. The median distance between foothold estimates across 12 different walks and subjects is 3cm. This is much more accurate than previous estimates which assumed a flat ground plane. Using this method, we have examined the distribution of terrain smoothness at locations where subjects fixated, and placed their feet. Differences in smoothness statistics between foothold locations and comparable control locations are modest. However, a convolutional neural network (CNN) trained to distinguish an actual from pseudo randomly selected foothold locations can do so at 65% accuracy. This suggests that there are indeed visual features that differentiate suitable from unsuitable footholds, but the differences are small. Since we found high replicability between paths chosen for different walks and different subjects, a stronger determinant of where subjects walk may be path feasibility.Locomotion, Motor Control, Navigation Sciences Society Annual Meeting AbstractLocomotion, Motor Control, Navigationinvisible
2021Nathaniel V. Powell; Xavier Marshall; Scott T. Steinmetz; Gabriel J. Diaz; Oliver W. Layton; Brett R. Fajen
The coordination of gaze and steering behavior in drone-racing pilots during high-speed flight While locomoting through natural environments, humans coordinate their gaze and steering to efficiently sample the visual information needed to guide movement. The coordination of gaze and steering during high-speed movement has been extensively studied in the context of automobile driving. Theoretical accounts that have emerged from this work, such as the waypoint fixation hypothesis (Lappi & Mole, 2018), capture behavior during self-motion along an explicit, well-defined path over a flat, obstacle-free ground plane. However, humans are also capable of visually guiding self-motion in all three dimensions through cluttered environments that lack an explicit path, as demonstrated during drone racing. The aim of the present study was to explore the gaze and steering behavior of drone pilots as they maneuvered at high speeds through a dense forest. Subjects were instructed to fly a simulated quadcopter along a race course embedded within a forest-like virtual environment built in Unity. The environment was viewed through an HTC Vive Pro head-mounted display while gaze behavior was recorded using a Pupil-Labs VR/AR extension. Drone position, orientation, and controller outputs were recorded by Microsoft AirSim. In the control condition, the race course was defined by an explicit path and there were no obstacles that impeded movement along the path. The task in this condition was similar to steering an automobile along a winding road and allowed for the fixation of waypoints. We compared gaze and steering behavior in the control condition to other conditions in which the waypoint fixation strategy was less suitable, such as when the course was defined by a series of gates rather than a path and when obstacles (trees and overhanging branches) were present that had to be avoided. Discussion focuses on how gaze and steering behavior are adapted to task demands during high-speed steering through cluttered environments.Motor behavior, Drone Racing Sciences Society Annual Meeting AbstractDrone Racing, Gaze and Motor Coordinationvr
2021Helen Lindner and Gry Sandlund LarssonEye-tracking technology to investigate eye contact and attention during music therapy in a patient of vegetative state -a single case studyMusic therapists use music as a therapy tool to stimulate sensory systems, such as visual, auditory, and proprioceptive system, and thus music therapy can be used to stimulate arousal and non-verbal communication in patients with vegetative state (Schnakers et al., 2016). One objective way to measure arousal and non-verbal communication is to quantify eye contact and attention using eye-tracking technology during therapy sessionsMusic Therapy 28-31 JULY 2021 Connectivity and diversity in music cognitionEye Tracking, Eye Contact, Attention, Vegetative stateinvisible
2021Anjali KishoreJogeshwar and Jeff B.PelzGazeEnViz4D: 4-D Gaze-in-Environment Visualization PipelineEye-tracking data visualization and analysis is often performed in three dimensions (x,y,t). It involves overlaying the gaze point (x,y) on scene-camera images and creating a pipeline to process the gaze-overlaid spatio-temporal data. In this project, we present a pipeline (called GazeEnViz4D) to extract 3D data from raw 2D eye-tracking data, and we have developed a custom ENvironment VIsualiZer (EnViz4D§) to allow researchers to visualize the pipeline-processed data in four dimensions (x,y,z,t) for extensive, interactive analysis. GazeEnViz4D consists of creating a 3D point cloud of the environment, calculating the observer motion, locating the 2D gaze obtained from the eye-tracker in the 3D model, and visualizing the data over time using EnViz4D. EnViz4D allows a researcher to zoom in the environment at any instance, pause or play the 3D data, speed up or slow down, forward or reverse, essentially recreating the data collection episode in four dimensions from an arbitrary modifiable viewpoint.Eye tracking, Environmental Model, Motion Tracking Computer ScienceEnvironmental model
gaze visualization
motion tracking
interactive tool
4-dimensional analysis
2021Lanini-Maggi, Sara; Ruginski, Ian Tanner; Fabrikant, Sara Irina Improving pedestrians' spatial learning during landmark-based navigation with auditory emotional cues and narrativeEven if we are not aware, our emotions can influence and interplay with our navigation and use of mobile navigation aids. A given map display can make us feel good by reminding us of pleasant past experiences, or it can make us feel frustrated because we are not able to understand the information provided. Navigation aids could also make a given landmark emotionally charged, and thus more salient and memorable for a navigator, for example, by using an auditory narrative containing emotional cues. By storytelling, it would also be possible to provide details about a given landmark and connect proximal landmarks to each other. But how do navigational instructions in the form of emotional storytelling affect spatial memory and map use? Results from a preliminary study indicated that a video presentation viewed from a first person perspective is looked at more often than an abstract map, and this evidence becomes even stronger when instructions are emotionally laden. We discuss results in the context of place meaning and how emotions’ role in navigation should be further assessed, in particular to increase spatial learning from navigation aids.Pedestrians, Navigation, Emotional Cues Science Short Paper ProceedingsPedestrians, Navigation, Emotional Cuesinvisible
2021Gopika Ajaykumar and Chien-Ming HuangMultimodal Robot Programming by Demonstration:
A Preliminary Exploration
Recent years have seen a growth in the number of industrial robots working closely with end-users such as factory workers. This growing use of collaborative robots has been enabled in part due to the availability of end-user robot programming methods that allow users who are not robot programmers to teach robots task actions. Programming by Demonstration (PbD) is one such end-user programming method that enables users to bypass the complexities of specifying robot motions using programming languages by instead demonstrating the desired robot behavior. Demonstrations are often provided by physically guiding the robot through the motions required for a task action in a process known as kinesthetic teaching.Robotics Hopkins (pre print)invisible
2021Christos Fidas, Marios Belk, Christodoulos Constantinides, Argyris Constantinides, Andreas PitsillidesA Field Dependence-Independence Perspective on Eye Gaze Behavior within Affective ActivitiesEvidence suggests that human cognitive differences affect users’ visual behavior within various tasks and activities. However, a human cognitive processing perspective on the interplay between visual and affective aspects remains up-to-date understudied. In this paper, we aim to investigate this relationship by adopting an accredited cognitive style framework (Field Dependence-Independence – FD-I) and provide empirical evidence on main interaction effects between human cognition and emotional processing towards eye gaze behavior. For doing so, we designed and implemented an eye tracking study (n = 22) in which participants were initially classified according to their FD-I cognitive processing characteristics, and were further exposed to a series of images, which triggered specific emotional valence. Analysis of results yield that affective images had a different effect on FD and FI users in terms of visual information exploration time and comprehension, which was reflected on eye gaze metrics. Findings highlight a hidden and rather unexplored effect between human cognition and emotions towards eye gaze behavior, which could lead to a more holistic and comprehensive approach in affective computing.Cognitive Science, Eye Tracking, Psychology Conference on Human-Computer InteractionIndividual differences, Cognitive processing styles, Global and local processing, Human emotions, Eye tracking, User study core
2021Marie Yamamoto; Yue Hu; Enrique Coronado; Gentiane VentureImpression evaluation of robot’s behavior when assisting human in a cooking taskStudies have shown that the appearance and movements of home robots may play key roles in the impression and engagement of users, opposed to recent rises in the smart speaker market. In this research, we conduct a user experiment with the aim of clarifying the elements required to evaluate the human impression of a robot's movements, based on the hypothesis that adequate movements may lead to better impressions and engagement. We compare the impressions of participants who interacted with a robot with movements (behavior robot) and a robot without movements (non-behavior robot). Results show that when using the behavior robot, participants showed significantly higher values in their impressions of cheerfulness and sociability. Questionnaires about interaction revealed that personalization is also an important function for robots to make a good impression on humans.Robotics, Psychology, Pupillometry IEEE International Conference on Robot & Human Interactive Communication (RO-MAN)Task analysis, Robots
2021María Silva-Gago, Annapaola Fedato, Timothy Hodgson, Marcos Terradillos-Bernal, Rodrigo Alonso-Alcalde & Emiliano BrunerVisual attention reveals affordances during Lower Palaeolithic stone tool explorationTools, which have a cognitive background rooted in our phylogenetic history, are essential for humans to interact with their environment. One of the characteristics of human beings is the coordination between the eyes and hands, which is associated with a skilled visuospatial system. Vision is the first input of an action that influences interaction with tools, and tools have affordances, known as behavioural possibilities, which indicate their possible uses and potentialities. The aim of the present study is to investigate body–tool interaction from a cognitive perspective, focusing on visual affordances during interaction with the early stone tools. We analyse visual attention, applying eye tracking technology, during a free visual exploration and during haptic manipulation of the Lower Palaeolithic stone tools. The central area of the tool is the most observed region, followed by the top and the base, while knapped areas trigger more attention than the cortex. There are differences between stone tool types, but visual exploration does not differ when aided by haptic exploration. The results suggest that visual behaviour is associated with the perception of affordances, possibly from the beginning of the brain–body–tool interaction, associated with the Lower Palaeolithic culture.Cognitive Science, Eye Tracking, Archaeology and Anthropological SciencesEye tracking, Cognitive archaeology, Choppers, Handaxes, Vision, Manipulationcore
2021Petersch, B., Dierkes, K.Gaze-angle dependency of pupil-size measurements in head-mounted eye trackingPupillometry - the study of temporal changes in pupil diameter as a function of external light stimuli or cognitive processing - requires the accurate and gaze-angle independent measurement of pupil dilation. Expected response amplitudes often are only a few percent relative to a pre-stimulus baseline, thus demanding for sub-millimeter accuracy. Video-based approaches to pupil-size measurement aim at inferring pupil dilation from eye images alone. Eyeball rotation in relation to the recording camera as well as optical effects due to refraction at corneal interfaces can, however, induce so-called pupil foreshortening errors (PFE), i.e. systematic gaze-angle dependent changes of apparent pupil size that are on a par with typical response amplitudes. While PFE and options for its correction have been discussed for remote eye trackers, for head-mounted eye trackers such an assessment is still lacking. In this work, we therefore gauge the extent of PFE in three measurement techniques, all based on eye images recorded with a single near-eye camera. We present both real world experimental data as well as results obtained on synthetically generated eye images. We discuss PFE effects at three different levels of data aggregation: the sample, subject, and population level. In particular, we show that a recently proposed refraction-aware approach employing a mathematical 3D eye model is successful in providing pupil-size measurements which are gaze-angle independent at the population level.Pupillometry, Cognitive Science Research MethodsOupillometry, Pupil foreshortening error, PFE, Eye tracking, 3D eye model, Corneal refractioncore
2021Ward LM, Kapoula ZDyslexics’ Fragile Oculomotor Control Is Further Destabilized by Increased Text DifficultyDyslexic adolescents demonstrate deficits in word decoding, recognition, and oculomotor coordination as compared to healthy controls. Our lab recently showed intrinsic deficits in large saccades and vergence movements with a Remobi device independent from reading. This shed new light on the field of dyslexia, as it has been debated in the literature whether the deficits in eye movements are a cause or consequence of reading difficulty. The present study investigates how these oculomotor problems are compensated for or aggravated by text difficulty. A total of 46 dyslexic and 41 non-dyslexic adolescents’ eye movements were analyzed while reading L’Alouette, a dyslexia screening test, and 35 Kilos D’Espoir, a children’s book with a reading age of 10 years. While reading the more difficult text, dyslexics made more mistakes, read slower, and made more regressive saccades; moreover, they made smaller amplitude saccades with abnormal velocity profiles (e.g., higher peak velocity but lower average velocity) and significantly higher saccade disconjugacy. While reading the simpler text, these differences persisted; however, the difference in saccade disconjugacy, although present, was no longer significant, nor was there a significant difference in the percentage of regressive saccades. We propose that intrinsic eye movement abnormalities in dyslexics such as saccade disconjugacy, abnormal velocity profiles, and cognitively associated regressive saccades can be particularly exacerbated if the reading text relies heavily on word decoding to extract meaning; increased number of regressive saccades are a manifestation of reading difficulty and not a problem of eye movement per se. These interpretations are in line with the motor theory of visual attention and our previous research describing the relationship between binocular motor control, attention, and cognition that exists outside of the field of dyslexia.Reading, Dyslexia Brain Sciencesdyslexia; reading; oculomotor system; saccades; vergencecore