1 of 21

Beyond Gaze Overlap: Analyzing Joint Visual Attention Dynamics Using Egocentric Data

Kumushini Thennakoon, Yasasi Abeysinghe, Bhanuka Mahanama, Vikas Ashok, Sampath Jayarathna

Presented by:

Kumushini Thennakoon

PhD Student

Department of Computer Science

Old Dominion University, Norfolk, VA, USA

@KumushiniT @NirdsLab @WebSciDL

​

Advisor: Dr. Sampath Jayarathna

WSDL Research Expo,

May 7, 2025

NIRDS Lab

2 of 21

Joint Visual Attention (JVA) is Shared Focus�

2

WSDL Research Expo 2025

@KumushiniT @NirdsLab @WebSciDL

  • Joint Visual Attention (JVA) is the shared focus of two or more individuals on the same object.

​

  • Developmental psychologists use JVA to understand how children acquire language and identify Autism Spectrum Disorder.

​

  • Learning scientists use JVA to understand how small groups of learners work together.

​

​

https://en.wikipedia.org/wiki/Joint_attention#/media/File:Wpm02_05.JPG

​

3 of 21

How Do We Identify JVA?�

3

@KumushiniT @NirdsLab @WebSciDL

Behavioral indicators:

      • Pointing with hand
      • Gaze following
      • Spontaneous looks to a social partner's face

​

​

Eye gaze indicators:

      • Tracking gaze positions of two or more individuals at the same area of interest simultaneously
      • Observing gaze data with social partners voice ques to look at the same area of interest within a very short period of time

​

​

WSDL Research Expo 2025

4 of 21

Eye Tracking Metrics Used to Analyze JVA�

4

@KumushiniT @NirdsLab @WebSciDL

​

  • Fixation count
  • Fixation duration
  • Saccade amplitude
  • Blink rate

​

  • Ambient/focal attention with coefficient K [1]
  • Gaze transition entropy
  • Low/high index of pupillary activity (LHIPA)
  • Real-time index of pupillary activity (RIPA)

[1] Krzysztof Krejtz, Andrew Duchowski, Izabela Krejtz, Agnieszka Szarkowska, and Agata Kopacz. 2016. Discerning Ambient/Focal Attention with Coefficient K. ACM Transactions on Applied Perception 13

Traditional gaze metrics

Advance Gaze metrics

WSDL Research Expo 2025

K>0 Focal attention behavior | K<0 Scanning attention behavior

5 of 21

Project Aria Glasses for Studying JVA�

5

@KumushiniT @NirdsLab @WebSciDL

WSDL Research Expo 2025

J. Engel et al., “Project ArIA: A new tool for Egocentric Multi-Modal AI Research,” arXiv.org, Aug. 24, 2023. https://arxiv.org/abs/2308.13561

  • Multi modal data collection.

​

  • Contains three types of cameras (wearer’s view, wearer’s peripheral view, and eye-tracking cameras).

​

  • Provide tools for data handling, data synchronization, data streaming, and visualization.

​

  • Data storage capacity (128 GB).

​

​

​

Project Aria glasses by Meta

6 of 21

Aria Everyday Activities Dataset

6

@KumushiniT @NirdsLab @WebSciDL

WSDL Research Expo 2025

Z. Lv et al., “ArIA Everyday Activities Dataset,” arXiv.org, Feb. 20, 2024. https://arxiv.org/abs/2402.13349

  • Recordings of everyday activities

​

  • Contains single user recordings

​

  • Contains multi user recordings

https://explorer.projectaria.com/aea

7 of 21

Approach and Tools - Step 1

7

@KumushiniT @NirdsLab @WebSciDL

Gavindya Jayawardena. 2020. Raemap: Real-time advanced eye movements analysis pipeline. In ACM Symposium on Eye Tracking Research and Applications. 1–4.

​

WSDL Research Expo 2025

Processing pipeline for analyzing JVA between dyads in an egocentric setting

8 of 21

Extracting Regions of Interest from Video Frames

8

@KumushiniT @NirdsLab @WebSciDL

WSDL Research Expo 2025

Extracting regions of interest (ROIs) around the gaze position from video frames.

9 of 21

Approach and Tools - Step 2

9

@KumushiniT @NirdsLab @WebSciDL

Gavindya Jayawardena. 2020. Raemap: Real-time advanced eye movements analysis pipeline. In ACM Symposium on Eye Tracking Research and Applications. 1–4.

​

WSDL Research Expo 2025

Processing pipeline for analyzing JVA between dyads in an egocentric setting

10 of 21

Calculate Visual Similarity between Cropped Video Frames (Regions of Interest)

10

@KumushiniT @NirdsLab @WebSciDL

WSDL Research Expo 2025

11 of 21

Approach and Tools - Step 3

11

@KumushiniT @NirdsLab @WebSciDL

Gavindya Jayawardena. 2020. Raemap: Real-time advanced eye movements analysis pipeline. In ACM Symposium on Eye Tracking Research and Applications. 1–4.

​

WSDL Research Expo 2025

Processing pipeline for analyzing JVA between dyads in an egocentric setting

12 of 21

Percentage of JVA in Different Everyday Activities

12

@KumushiniT @NirdsLab @WebSciDL

WSDL Research Expo 2025

Activities with less movements have higher percentage of JVA

13 of 21

Event Annotations for Time Epochs

13

@KumushiniT @NirdsLab @WebSciDL

WSDL Research Expo 2025

14 of 21

Dynamics of Coefficient K

14

@KumushiniT @NirdsLab @WebSciDL

WSDL Research Expo 2025

A Synchronous attention pattern

An Asynchronous attention pattern

15 of 21

Limitations and Future Work

15

@KumushiniT @NirdsLab @WebSciDL

  • Movements of research subjects negatively affected in identifying JVA frames.

​

  • Poor resolution and lighting conditions of the videos affected in calculating similarity score.

​

  • Further analysis of relationship between JVA and ambient and focal attention coefficient K with a substantial dataset is required.

​

  • Incorporate advanced gaze measures, such as gaze transition entropy to further understand the patterns of gaze shifts during a collaborative activities.

​

​

WSDL Research Expo 2025

16 of 21

Summary

16

@KumushiniT @NirdsLab @WebSciDL

  • Wearable eye-tracking makes it convenient to study JVA.

​

  • Ambient/Focal attention with coefficient K can provides more detailed insights into JVA dynamics compared to traditional gaze position overlap methods.

​

  • JVA with coefficient ambient/focal attention coefficient K can be used to analyze collaborative activity in shared tasks.

​

​

​

WSDL Research Expo 2025

Percentage of JVA = 44.16%

17 of 21

Screen-based Experimental Setup

and Stimuli�

17

@KumushiniT @NirdsLab @WebSciDL

https://play.google.com/store/apps/details?id=ru.iprado.spot

WSDL Research Expo 2025

18 of 21

Better JVA and Poor JVA from Cross- Recurrence Graphs

18

@KumushiniT @NirdsLab @WebSciDL

Absence of a dark diagonal suggests no JVA occurred.

​

​

Black pixels on the diagonal Indicates continuous joint attention, where both participants consistently focus on the same area.

WSDL Research Expo 2025

Bertrand Schneider, Kshitij Sharma, Sebastien Cuendet, Guillaume Zufferey, Pierre Dillenbourg, and Roy Pea. 2018. Leveraging mobile eye-trackers to capture joint visual attention in co-located collaborative learning groups. International Journal of Computer-Supported Collaborative Learning 13, 3 (01 Sep 2018), 241–261. https://doi.org/10.1007/s11412-018-9281-2

Exp #1

Exp #5

19 of 21

Dynamics of Coefficient K

19

@KumushiniT @NirdsLab @WebSciDL

WSDL Research Expo 2025

20 of 21

Approach and Tools for Conducting Analysis

20

@KumushiniT @NirdsLab @WebSciDL

Gavindya Jayawardena. 2020. Raemap: Real-time advanced eye movements analysis pipeline. In ACM Symposium on Eye Tracking Research and Applications. 1–4.

​

WSDL Research Expo 2025

21 of 21

Approach and Tools

21

@KumushiniT @NirdsLab @WebSciDL

Gavindya Jayawardena. 2020. Raemap: Real-time advanced eye movements analysis pipeline. In ACM Symposium on Eye Tracking Research and Applications. 1–4.

​

WSDL Research Expo 2025

Processing pipeline for analyzing JVA between dyads in an egocentric setting