1 of 98

GOAL

To learn about how hardware and software can combine to solve the problem of people who can only communicate with their eyes

Biomechanical Systems

Assistive Technology with Eye Tracking

2025-08-18_v1.0

2 of 98

2

Lab development team

ALLIE SACK

College: Duke University (‘26)

Major: Biomedical Engineering

GRACE MARKOVICH

College: University of Notre Dame (‘25)

Major: Chemical Engineering

DIXON CLEVELAND

College: MIT & George Washington University

Major: Electrical Engineering

Industry Experience: Eyegaze Inc., Greenbriar Systems Inc, Adaptronics Inc.

BETSY FORTMAN

College: Purdue University

Major: Aerospace Engineering & MBA

Industry Experience: NASA, Sophic Synergistics

JOHN SEXTON

College: University of Notre Dame

Major: Electrical Engineering (‘23)

Industry Experience: Epic Systems, LifeDrive

3 of 98

3

Part 1

Using Hardware and Software to Communicate

As you go through the workbook check in the speaker notes for additional information!

4 of 98

4

Identifying the Problem

  • There are many medical conditions that can lead to a loss of motor control or speech
    • As a result, many lose the ability to effectively communicate, control their own body, and carry out basic tasks by themselves
  • The problem engineers must solve: identify a solution that would grant people the ability to communicate, operate their own wheelchair, and complete tasks using their eyes

A primary goal of engineering is to contribute to the common good by improving people’s lives.

What Would You Fight For?

5 of 98

5

Medical Conditions Affecting Mobility and Speech

While these conditions affect muscular function, the movement of the eyes often remain accurate! How can we use this help these people communicate?

Amyotrophic Lateral Sclerosis (ALS)

  • a progressive neurodegenerative disease that affects nerve cells in the brain and spinal cord

Cerebral palsy (CP)

  • a disease typically caused by abnormal brain development that affects a person’s ability to move

Multiple sclerosis (MS)

  • a disease that occurs when the immune system attacks the central nervous system, which can affect one’s ability to move/walk

Rett syndrome

  • a rare genetic neurological condition that results in the degeneration of motor function and speech (diagnosis-tools)

Spinal Muscular Atrophy (SMA)

  • a genetic disease that affects motor neurons in the spinal cord and weakens muscles

Spinal Cord Injuries

  • a traumatic injury that damages the spinal cord, which can result in paralysis

6 of 98

6

Designing for Others

Engineers always need to consider the needs of their users when designing their solution

When you’re designing this type of technology, it’s crucial to remember the motivation for it’s creation

The technology is being created to improve the lives of real people

It’s important to get input from people who will actually be using this technology in order to produce the highest quality product

7 of 98

Thinking Like an Engineer

ANSWER ME! How can you obtain input commands from the user when physical movement is limited? What considerations need to be made when you are designing technology for someone with physical disabilities? (Timestamp 7:45 in the into video)

Type your answer here.

8 of 98

8

Developing a Solution: Eyegaze

Eyegaze uses eye tracking technology to allow users to control a device with their eyes

Components of the Eyegaze System:

  • Mounted computer, camera, on screen keyboard

How the system works:

  • To types with their eyes, users look at the desired key for a specified dwell time until the software sounds an audio click and the character is added

What can people do with Eyegaze?

  • Eyegaze allows users to communicate with others and regain a sense of independence

9 of 98

9

9

Using Eyegaze

What are three tasks that Eyegaze allows users to carry out?

How does this technology account for users with varying needs and abilities?

Type your answer here.

Type your answer here.

Watch the video on the right and answer the following questions

10 of 98

10

Eyegaze in Action

Watch as a person uses the Eyegaze system to communicate!

11 of 98

11

Part 2

Eye Tracking Technology

12 of 98

12

Look through this timeline of major achievements in eye tracking technology!

History of Eye Tracking

13 of 98

13

The Basic Terminology

  • Gaze line: the imaginary line that emerges from the center of the eye’s pupil and points in the direction that the eye is looking

  • Gazepoint: the 3D position in space where the gaze line intersects the screen (where the person is looking on the screen)

  • Video camera: located below the screen and observes the user’s eyes as they looks around the screen

Typical eye tracking device

The basic terminology of the eye gaze system

14 of 98

14

Light-emitting-diode (LED)

  • Where: mounted on the front of the camera lens
  • Purpose: illuminates the eye and generates a reflection on the cornea that is critical to determining where the eye is looking

Aperture

  • What/Where: the opening in front of the camera that lets the light in
  • Purpose: shaped in a triangular pattern to allow the camera to measure how far away the eye is

Special Features of the Eye Tracking Camera

15 of 98

ANSWER ME!

Drag the terms in the word bank below to their correct label on the image.

Label the Parts of the Eye Tracking System!

Gaze Vector

Gaze Line

Eye Location

LED

Gaze Point

Video Camera

16 of 98

The Eye

The Pupil

  • Bright Pupil Effect’: the eye’s pupil in the image to the right is bright as opposed to its usual dark color (similar to the ‘red-eye effect’ in photography)
  • Around the pupil is the iris

Corneal Reflection

  • Corneal reflection (or glint spot): the direct reflection of the LED off the corneal surface of the eye
    • The location of the corneal reflection relative to the pupil is critical in determining the gaze point

17 of 98

Terminology & Parts of the Eye Tracking System!

Glint Spot

18 of 98

Terminology & Parts of the Eye Tracking System!

Glint Spot

Pupil Center

19 of 98

Terminology & Parts of the Eye Tracking System!

What is a Vector?

An object that has both a magnitude and a direction.

Geometrically, we can picture a vector as a directed line segment, whose length is the magnitude of the vector and with an arrow indicating the direction. The direction of the vector is from its tail to its head.

20 of 98

Terminology & Parts of the Eye Tracking System!

Glint Spot

Pupil Center

Glint Pupil

Vector

21 of 98

Determining Where the Eye is Looking

21

  1. Identify the pupil center and the corneal reflection

  • Draw a vector from the center of the corneal reflection to the center of the pupil, known as the glint pupil vector
    1. The head of the vector (the arrow head) should be pointing at the pupil center while the tail originates at the corneal reflection

  • The glint pupil vector tells you where the user is looking with respect to the camera
    • This vector, complex image analysis, and geometry can be used to find the exact gaze point

  • The magnitude of the vector indicates how far away from the camera you are looking

Pupil-Center Corneal-Reflection (PCCR) Method

A pattern to note: As the eye rotates from the left of the camera to the right, the pupil location shifts from the left to right of the corneal reflection.

22 of 98

22

Try it Yourself!

Using the “insert” tool, place a colored-vector from the center of the glint spot to the center of the pupil. Draw the vector directly on the eye image.

Gaze: Type your answer here.

Gaze: Type your answer here.

Describe the direction the user is looking with respect to the camera (up, down, left, right, up-left, down-right, up-right, down-left, etc…)

23 of 98

23

Determining Gaze Point

How does a computer calculate the gaze point?

The image to the left is a plot of the brightness of a cross-section of an image of the eye

  • Think of an image as a 3D mountain range with varying heights across the xy plane
  • In an image, brighter points are taller

The bright pupil effect revisited:

  • With this effect, the eye image shows the iris as a dark, circular, low-level plane
  • The bright pupil is a uniformly bright disk region that is higher that the iris
  • The corneal reflection is like a pencil riding around on top of the either the iris or pupil.

Engineers use image analysis and complex algorithms to finding data peaks and calculate the gaze point

24 of 98

Determining Gaze Point

ANSWER ME! How can you account for the different geometries of people’s eyes? What are other examples from your everyday life of things that need to be calibrated? (Timestamp 14:10 in intro video)

Type your answer here.

25 of 98

25

Activity #1:

Creating your own 3D Eye Geometry Model

26 of 98

26

Visualizing the Eye and Eye Tracker Camera

Look at the images below to get a better understanding of the eye and eye tracker camera that you will be modeling

Anatomical Eye Cross Section

Eye Tracker Camera Cross Section

Camera Image of the Eye

27 of 98

27

Geometry Involved in Eye Tracking

Gaze Offset Angle (B):

  • Angle showing Up/Down

28 of 98

28

3D Model

Thinking about the eye geometry in 3D space

Click the link to view the 3D geometry of the eye. https://www.geogebra.org/3d/bfgaeqgy

Move the pupil around to at least 2 different positions.

Take a screenshot of the geometry.

What do you notice about the glint vector? Did it get bigger or smaller from your first pupil placement to your second?

Place your first 3D diagram here

Place your second 3D diagram here

Insert your answer here:

ANSWER ME!

29 of 98

29

Your Task Today: Create a 3D Foam Model of the Eye Geometry

The instructions for this activity are intentionally vague. Be creative and use your resources to gain a firm understanding of eye geometry. There is no one correct way to construct your model!

  1. Define the eye geometry
  2. Determine the color of each vector
  3. Build your eye model including �the gaze point, glint spot, and �glint vector
  4. Analyze your geometry to determine the accuracy of interpretation
    • www.geogebra.org/3d/bfgaeqgy

INSTRUCTIONS

30 of 98

STUDENT KIT ITEMS

1 kit: 3 students

Item/Link

Quantity

Photo

1

5

2

1

1

TEACHER’S KIT

Materials will be distributed throughout the class.

Item/Link

Quantity

Photo

1

Materials You’ll Need

30

31 of 98

31

Building Your Model

  1. Gather your materials
  2. Determine the colors of your vectors, horizontal/vertical reference lines, angles, and glint spot
  3. Use the black magic marker to draw a circle ~1” in diameter on the medium styrofoam ball to represent the pupil
  4. Using your earlier sketches and the examples on the next slide, construct your 3D models by sticking the pipecleaners in the foam
    1. You can cut one medium foam ball in half and construct the 2 cross sections as shown in your drawings before putting everything together in one model
  5. Attach the plastic sheet and small foam ball as shown to represent the tablet screen and camera, respectively

INSTRUCTIONS

  • Make sure you understand what each line or vector represents
  • Pay attention to the 3D relationships of the vectors
  • While constructing the model, split it up into 2 main sections:
    • The head on view
    • The side view (containing the gaze offset angle)
  • To make the ‘screen’ sturdier, use hot glue to make a rectangular outer border (be careful because the plastic may fold in on itself as you apply the hot glue

TIPS/TRICKS

32 of 98

Video instruction

*Please remember that this is one of many ways to create an eye model. Create a model the makes sense to you; the goal is to gain a deeper understanding of the geometry of an eye.

33 of 98

Other examples!

34 of 98

34

Your Final 3D Foam Models

Insert images of your final models here!

35 of 98

35

The Big Picture

Understanding eye geometry is essential to Eyegaze technology

Geometry forms the basis for modeling and algorithms:

  • The algorithms that are used to interpret eye movement and calculate gaze point rely on the mathematical models of the eye’s shape and structure

Everyone’s eyes are unique

  • Each person has some variation in the shape ans size of their eye. Eyetracking devices need to account for these individual differences, so it is important that the technology has been designed to be calibrated for each user

36 of 98

36

Applying the Glint Pupil Vector to Commands on the Screen

37 of 98

Workbook Activity: Motion

  1. What order of motions does this show?

2. Draw the corresponding

wheelchair motion (path)

The computer stores a vector for each point on the screen. Imagine that the following vectors are stored along with the following commands.

Type your answers here:

Draw the expected wheelchair path here

ANSWER ME!

38 of 98

Workbook

Activity: Words

Type your answer here

What do these eye movements spell?

Now, imagine the vectors are tied to a letter instead of a movement.

Insert your answer here:

ANSWER ME!

39 of 98

39

Further Calculations

Now you’ve formulated the fundamental PCCR equations for computing the compound angle of the gaze line with respect to the camera’s sight line to the eye!

There are still many steps to compute the person’s gaze point on a computer screen:

How do you process an eye image to identify the glint and pupil?

How do you map out the pupil perimeter in the image and compute its center point?

Based on the image data, how do you compute the millimeter location of the eyeball in real space?

How do you compute the gaze line intercept on the computer screen?

40 of 98

40

Activity #2

Use Eye-Gaze to Determine Wheelchair Movements

IT’S YOUR TURN:

Now that you understand the technology, it’s time to interpret eye movements yourself in order to maneuver your wheelchair along a path

41 of 98

LifeDrive

Founder: John Sexton

Company’s Mission

  • “LifeDrive creates adaptive technology for people with disabilities, empowering them to greater independence through power wheelchair control for a lifetime”

LifeDrive Technology

  • EyeDrive: This allows people with disabilities to operate their wheelchairs using eye tracking technology

42 of 98

42

Translating Eye Gaze into

Wheelchair Movements

Objective: To help a user interact with their surroundings, create a controller-template to be used to translate a user’s eye movements into actual outputs (wheelchair movements)

What you will be doing:

  1. Create a controller-template to move a wheelchair a certain distance in any given direction
  2. Determine how many different directions will be on the template as well as the optimal spacing to ensure accuracy of identifying the user’s intent
  3. Consider how to determine the number of units to travel (i.e. This could be based on #s placed on the template or possibly the duration of a user’s eye gaze)

Materials

  • Clear Sheet Protectors

  • Dry-Erase markers (to create your template for controlling the wheelchair)

  • 4 different mazes

In this exercise, you will use a more primitive method of reading where someone is looking.

43 of 98

  • Create groups of 3-4 people and define roles: user, engineer, observer(s)
  • User: looks at the floor plan (maze) and makes eye movements towards template
  • Engineer: interprets the user’s eye gaze through the template (follows the eye movements of the user by drawing their path on a blank piece of graph paper)
  • Observer/Recorder: sees both the user & maze along with the engineer - records how many correct vs. incorrect movements – thinks about possible improvements to the set up

Experiment Set-up & Examples

User

Engineer

12 in

12 in

Recorder

  • The user sits on one side of the template and the engineer on the other
  • User is shown a maze and must use eye movements to communicate how to maneuver through the maze

44 of 98

Holding maze

(behind engineer)

Engineer

User

Template/

Controller

Maze

  1. The User Holds the Template at Arm’s Length

2) The Engineer reads the Eye Movements

3) Holds the Maze and Counts Correct & Missed Steps

45 of 98

45

Independent

Variables

As you work through this activity, consider the following variables that you can change

Distance between the user, engineer, and controller

Design of the controller template

  • 4 directions, 8 directions, rotation button, stop/pause button

Placement of the maze

  • Next to the controller, behind the engineer, etc

Difficulty of the maze

  • Easy, medium, hard, expert

CONSIDER

As you design various iterations of your set-up, how can you change each of these variables to get the best result?

46 of 98

46

Additional Considerations

Before you start this activity discuss the following

Dwell Time

  • How long should your dwell time be?
  • How quickly can the engineer react?
  • Troubleshoot (or “calibrate”) on an easy maze to determine an appropriate dwell time

Determine a ‘step’

  • Do you want to draw blocks/arrows on the maze to counts as steps?
  • Do you want to set a duration of time to count as a step?
  • Will you use blinks to communicate when to stop?

What is your dwell time?

How are you determining a ‘step’/when to stop?

47 of 98

TEMPLATE

EXAMPLES

1

2

3

4

48 of 98

MAZE

EXAMPLES

49 of 98

50 of 98

51 of 98

52 of 98

53 of 98

54 of 98

55 of 98

Insert pictures of each of your templates here:

ANSWER ME!

56 of 98

User Distance to the Template

Engineer Distance to the Template

Total Moves

Total Correct Moves

Time through maze

Accuracy (%)

(# correct moves) x 100

# of total steps

Notes

- Type of Maze (easy, medium, hard)

- Template description, number of buttons

- How to know how many steps to move

12”

12”

10

9

45sec

9/10 x 100 = 90%

- Easy Maze

- Eye Template had up/down/left/right/up-left/up-right/down-left/down-right/stop

- Used dwell time to say go to more than 1 step in a given direction

Experiment Results

57 of 98

Interpreting Results

  1. Which distance combination provided the best accuracy?

  • How did you determine the distance to travel in a given direction?

  • Did you change anything in your templates (i.e. number of commands, types of commands, size of font, space of commands, etc…)? If so, what worked best?

  • What would you change to improve your template?

Insert your answer here.

Insert your answer here.

Insert your answer here.

Insert your answer here.

ANSWER ME!

58 of 98

58

The Big Picture

Considerations when creating Eyegaze technology

When developing eyegaze technology for a specific user, it is important to take all of the results from your experiment into account

  • How far away should the camera be
  • What should the dwell time be
  • How should the design be adjusted based on user feedback

In the real world, the typical parameters are as follows:

  • Distance between camera and eye: 16-30 in
  • Dwell Time: 600 ms (about half of a second!)

* These are averages for each parameter. It is important to remember that these values may vary for each user

59 of 98

ANSWER ME!

What are some possible limitations of eye tracking technology? How could they be addressed?

Type your answer here.

Limitations of Eye Tracking Technology

60 of 98

60

Additional Applications of Eye Tracking Technology

61 of 98

61

Additional Eye Tracking Uses

Eye tracking is not only used for communicating and driving, but can also be used for research, vision therapy, and diagnosing vision impairments.

EyE TRACKING GLASSES

RIGHT EYE

VISION THERAPY

62 of 98

62

Internet of Things

The Internet of Things (IoT) describes the network of physical items with embedded systems consisting of software and sensors that connect over the internet with other devices

Some examples of the IoT in everyday life are…

TO LEARN MORE…

Watch this short video to learn more about the Internet of Things!

Smart homes utilize the IoT by having various appliances connect to the internet so homeowners can control their homes using a smartphone or computer. If an eye gaze user can connect to the internet, and if they have smart devices in their home, then they can control the temperature, lights, and more using their eyes!

SMART WATCHES

SELF DRIVING CARS

TRACKING DEVICES

63 of 98

Using Eye Tracking Technology

ANSWER ME! List at least two possible other applications of eye tracking technology.

Type your answer here.

64 of 98

64

Reflect on Your Design and Results

ANSWER ME!

Write your answer here

What do you think went well when completing this activity?

Write your answer here

What is something you would do differently if you were to do this again?

Complete the mandatory 5-minute Exit Ticket by clicking HERE!

65 of 98

65

Continue to Explore

IF YOU LIKED TODAY’S BREAKOUT, �YOU MAY BE INTERESTED IN THESE TOPICS:

  • Image Processing
  • Algorithms
  • Software Development
  • Geometric Optics
  • Eye Tracking
  • Robotics & Embedded Systems
  • GazeRecorder- a website that shows you your eye gaze as you look at an article.

  • Biomedical Engineering
  • Software Engineering
  • Electrical Engineering
  • Mechanical Engineering

TYPES OF ENGINEERING RELEVANT TO TODAY’S EYEGAZE BREAKOUT:

66 of 98

66

Extension Activities

This section will provide an overview of the optional extension activities. These activities are an opportunity for students to dive deeper and ideate. The materials associated with the extension labs may not provide as many detailed instructions as the main lab activity.

67 of 98

67

Eye Geometry

Calculations

3D Cornea Model & Flashlight

Biomechanical:

Assistive Technology & Devices

E-TRAN Board

68 of 98

68

Extension #1:

Creating an E-TRAN Board

We are being the software algorithm today by taking a visual image and analyzing it for a given pattern to determine the necessary "letter/command" intended by the user's gaze.

What you’ll be doing:

  • Design an E-Tran Board and use it to interpret what a user is saying

Ideal option if you want to:

  • Better understand how a user interacts with eye tracking technology and how the computer may process the data and images

69 of 98

69

“Writing” With Your Eyes

Try this activity!

To the right is an actual keyboard from Eye gaze. Try writing out a sentence (“How are you doing today?”) with your eyes by looking at the letters, separating words using the spacebar, and then speaking it with the “SPK” key.

Time yourself to see how fast you can say the sentence, but be sure to pause briefly on each key so that the ‘computer’ knows you are pressing the key.

What did you learn when doing this? What could go wrong with the process? What does it feel like to have limited function?

70 of 98

70

Your Task Today: Create an E-TRAN Board

  • Define your problem
  • Research possible keyboard configurations and layouts
  • Design the layout of your E-TRAN board
  • Build your E-TRAN board
  • Test your design and try to interpret what your partner is ‘saying’ using just your eyes and a laser pointer
  • Analyze your results to determine the accuracy of interpretation

INSTRUCTIONS

What is an E-TRAN Board?

  • Eye Transfer boards are simple tools used to facilitate communication that don’t require high tech computer systems

Materials

  • 3 clear acrylic sheets of different sizes

  • 3 different colored erasable markers

71 of 98

71

Creating Your Board

What you will be doing: use a clear plastic sheet and write out a keyboard (with an erasable marker), then create a procedure for interpreting what someone is saying

Things to consider when designing your board

  • Think about size of letters, positioning of letters, and special instructions (blinking?)
  • What letters are most commonly used in writing? How does that go into your keyboard design?
  • Can you create a ‘calibration’ exercise to help? Start by reading a known word or sentence, and then try reading a new unknown one.

72 of 98

72

Examples of E-TRAN Boards

73 of 98

73

Design and Build Your Board

1. Sketch a design for your board and insert a photo here.

2. Build your board and insert a photo here.

74 of 98

74

Testing Your Design

You will assign one person to be the ‘reader’ and one person to be the ‘writer’

  • Writer: the writer will face the E-TRAN board and use their eyes and the keyboard to either answer questions or write out a phrase (depending on the layout of your keyboard)
    • Try to only move your eyes and not your head!
  • Reader: the reader is on the opposite side of the board and will attempt to interpret what the reader is saying with their eyes
  • Keep track of the accuracy of interpretation using the table on the next slide

Clear acrylic sheet & erasable markers

A third student can observe the interaction, monitor mistakes, and think of possible improvements

75 of 98

75

Collect Data

Have one partner use their eyes to spell out a phrase using your E-TRAN board while the other interprets what they’re saying. As you go through each word/phrase, record the number of guesses to correctly interpret each character.

Test #

Word or sentence being interpreted

Record the number of guesses for each character

Accuracy

# of Correct / Total x 100

Notes/observations

Sample

Mary is happy

M A R Y I S H A P P Q

2 1 3 1 4 2 2 1 3 1 3 1 __

12/13 x100 = 92.3%

At end of each word, reader could guess next letter, middle letters were harder

Test 1

Test 2

Test 3

Test 4

76 of 98

Analyze Your Results

ANSWER ME! Was it difficult to understand what your partner was trying to say/communicate? Was it harder than you thought it’d be? Easier?

Type your answer here.

Type your answer here.

ANSWER ME! Were your interpretations accurate? How could you improve the design of your E-TRAN board?

ANSWER ME! Did the reader find themself trying to guess the word before the writer finished? How could this be frustrating to a user if someone is looking over their shoulder?

Type your answer here.

77 of 98

77

Eye Geometry

Calculations

3D Cornea Model & Flashlight

Biomechanical:

Assistive Technology & Devices

E-TRAN Board

78 of 98

78

Extension #2:

PCCR Activity, Eye Tracker Model

What you’ll be doing:

  • Building a mechanical model of the front surface of an eye to observe what it looks like when an eye rotates around

Ideal option if you want to:

  • Better understand how the camera sees the eye and the changes in the glint vector

79 of 98

79

Experimenting with an

Eye Tracker Model

Objective: uncover a critical piece of an eye tracker’s optics geometry by modeling how the camera sees the eye

What you will be doing:

  • Build a mechanical model of the front surface of an eye to observe what it looks like when an eye rotates around

  • Sketch drawings of the eye tracker’s/eye’s geometry

  • Use 3D geometry model to observe changes in glint vector magnitude and direction

80 of 98

80

Materials:

Activity #1

  • Small Ornament or Ping Pong Ball

(simulates corneal sphere)

  • Black magic marker (to draw pupil on ornament)

  • Flash light (simulates LED in eye tracker camera)

81 of 98

Step 1: Build Your Model

81

  1. Using the ornament to simulate the corneal sphere of an eye, draw a pupil on it using the black marker
    1. The pupil should be a circle with a diameter ~0.5 cm

  • With one hand, hold the ornament ‘eye’ in front of you as if it were looking at you. Using the other hand, hold the flashlight close to your eye and point the ‘LED’ toward the ‘eye’
    • See where the ‘LED’ glint spot reflects off the ‘corneal surface’

  • Move the ‘eye’ around and watch how the ‘pupil’ moves around the ‘glint spot’ as you translate and rotate the ‘eye’

82 of 98

82

For Reference:

Geometry Involved in Eye Tracking

Gaze Offset Angle (B):

  • Angle showing Up/Down

83 of 98

83

Consider the following while testing your model eye.

In real life eye tracker hardware, the LED is mounted at the center of the camera lens

If the eye is too close to the camera, the glint spot will appear to be in the same spot no matter where the person is looking

Similarly, if you hold the flashlight too close to your ‘eye,’ the glint spot will always align with the center of the corneal sphere, no matter how you rotate it

Distance Between the Eye

and the Eye Tracker

The ‘eye’ should be looking directly at your eye when the glint spot is in the center of the pupil.

See if you should adjust the ‘eye’ model by rotating the ornament. When the glint is at the center of the pupil, your gaze line aligns with the eye model’s gaze line.

Thinking About Line of Sight

Can you begin to predict the eye’s gaze direction from the position of the pupil with respect to the glint?

Question to Consider

84 of 98

Reflect on Your Model

ANSWER ME! As you were experimenting with your eye model, what did you notice? Did anything stick out to you? Did you have any questions as you moved your eye around?

Type your answer here.

ANSWER ME! Insert a picture or drawing of what you observed with the cornea eye model (ornament & light). Be sure to label the center of the pupil, the center of the glint spot (reflection spot), and draw in the glint pupil vector.

Type your answer here.

85 of 98

85

Biomechanical:

Assistive Technology & Devices

E-TRAN Board

Eye Geometry

Calculations

3D Cornea Model & Flashlight

86 of 98

86

Extension #3:

Geometry Involved in Eye Tracking

What you’ll be doing:

  • Using geometrical relationships and mathematical equations to determine the specific values of the glint vector

Ideal option if you want to:

  • Better understand how geometry and math are used in the algorithm to determine the glint vector

87 of 98

87

Determining where the gaze point is

Eye tracking technology uses complex geometric algorithms to determine exactly where the user is looking on the screen

Geometry helps relate the spatial orientation and location of the pupil center, the glint spot, the LED on the camera, and the gaze point using various angles and vectors

Information from the video camera central processing unit (CPU) processes the eye images, calculates the vectors/angles, and computes the gazepoint geometry

Geometry Involved in Eye Tracking

88 of 98

88

B

*

*

*

Gaze Offset Angle (B):

  • Angle showing Up or Down

Angle (A):

  • Angle showing Left or Right

Geometry Involved in Eye Tracking

89 of 98

89

Review of Right Triangles

In eye tracking, it is crucial to determine angles between vectors relating the eye, the camera, and the computer screen to each other

Pythagorean Theorem: a² + b² = c²

a = side of right triangle

b = side of right triangle

c = hypotenuse

These formulas are extremely helpful in determining missing side lengths or angles in right triangles

SOH CAH TOA: a mnemonic that helps one remember the meaning of the three common trigonometric functions - sine, cosine, and tangent functions

a

b

c

90 of 98

90

Inverse Trigonometric Functions

Using inverse notation will allow us to derive a formula to find the measurement of the angles in a triangle. The inverse equations can be seen to the right.

The inverse is the opposite of taking the sine, cosine, or tangent of an angle. The input is the side lengths, and the output is the angle.

91 of 98

91

Geometry Activity

Show your work here!

Show your work here!

Show your work here!

For each of the following triangles, determine the indicated angle or side length value

Find the values of d and θ

Find the value of θ and the missing side length

Find the 2 missing side lengths

92 of 98

92

Compound Angles

Thinking about geometry in 3D spaces

A compound angle describes a direction pointing in 3-dimensional space

It’s like pointing at something in the sky where you first specify a north/south/east/west ‘heading’ angle and then designate an ‘elevation’ angle specifying how high up to point above the horizon

In the sky pointing case, when you point straight up along a gravity line, (perpendicular to the ground) the ‘elevation’ angle (also known as the gaze offset angle) is 90°, and as you point further away from the gravity line, it gets smaller

This is important in eye tracking because the eyegaze system and the eye itself are 3D

CLICK HERE and watch this video to visualize compound angles

93 of 98

Find Glint Vector

& Calculate Magnitude

94 of 98

ANSWER ME!

Given two coordinates

  • Glint spot: (1,3)
  • Pupil center point: (4,6)

Determine the Lengths

Δx = Horizontal Distance = (x₂-x₁)

Δz = Vertical Distance = (z₂-z₁)

Calculate

(4, 6)

(1, 3)

95 of 98

A =

Recall that:

tan(A) = value

A = tan⁻¹ (value)

A = tan⁻¹

ANSWER ME!

Tangent Equation: To Find Angle (A)

Enter values for the Δx & Δz on the diagram (from the previous page).

a = _____

b =_____

c

tan (A) =

3

3

3

3

3

3

(

(

45

Angle (A)

CLICK HERE to see how the equation was derived!

96 of 98

Gaze Offset Angle (B) =

7.8mm

B

ANSWER ME!

Given : Corneal sphere radius: 7.8 mm

4. Enter Δz on the diagram (from the previous pages).

b =

Enter here

Cosine Equation: To Find Gaze Offset Angle (B)

Enter & Calculate

sin (B) =

Enter here

Enter here

B = sin

-1

Enter here

Enter here

(

(

Enter value here

CLICK HERE

to see how the equation was derived!

97 of 98

97

Continue to Explore

IF YOU LIKED TODAY’S BREAKOUT, �YOU MAY BE INTERESTED IN THESE TOPICS:

  • Image Processing
  • Algorithms
  • Software Development
  • Geometric Optics
  • Eye Tracking
  • Robotics & Embedded Systems
  • I Have Voice- an app that allows you to use eye gaze to communicate.
  • GazeRecorder- a website that shows you your eye gaze as you look at an article.

  • Biomedical Engineering
  • Software Engineering
  • Electrical Engineering
  • Mechanical Engineering

TYPES OF ENGINEERING RELEVANT TO TODAY’S BREAKOUT:

98 of 98

Thank you!

Any text here?

Follow up info here?