1 of 29

Controlling Cortical Implants for Partial Recovery of Vision

Ashkan Nejad

Noordelijke Oogheelkundige Nascholing

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the

Marie Sklodowska-Curie grant agreement No 955590

2 of 29

Loss of Vision

2

5/7/2024

3 of 29

Stimulating Visual Receptive Field

3

5/7/2024

Neuralink Show & Tell, Fall 2022

4 of 29

Receptive Field Mapping

4

5/7/2024

Cheng, Feng, Fernandez, Roelfsema, 2020

5 of 29

Receptive Field Mapping

5

5/7/2024

Cheng, Feng, Fernandez, Roelfsema, 2020

6 of 29

Shape Recognition

6

5/7/2024

Cheng, Feng, Fernandez, Roelfsema, 2020

7 of 29

Shape Recognition

7

5/7/2024

Cheng, Feng, Fernandez, Roelfsema, 2020

8 of 29

AI Controlling the Implants

Goal:

8

5/7/2024

9 of 29

AI Controlling the Implants

9

5/7/2024

10 of 29

AI Controlling the Implants

10

5/7/2024

11 of 29

AI Controlling the Implants

11

5/7/2024

12 of 29

AI Controlling the Implants

12

5/7/2024

Neuralink Show & Tell, Fall 2022

13 of 29

Goal:

13

5/7/2024

14 of 29

The Challenge

There are no ground truth

Thus, no right or wrong way

14

5/7/2024

15 of 29

Team

15

5/7/2024

Marcel van Gerven

Burcu Kucukoglu

Jaap de Ruyter

van Steveninck

Frans Cornelissen

16 of 29

Previous work

The training strategy

Gaze not considered

16

5/7/2024

17 of 29

Our Contribution

  • For Object Recognition
  • Including gaze

17

5/7/2024

This is house

This is house

18 of 29

Our Contribution

  • For Object Recognition
  • Including gaze

18

5/7/2024

19 of 29

Our Contribution

  • For Object Recognition
  • Including gaze

19

5/7/2024

20 of 29

Our Contribution

  • For Object Recognition
  • Including gaze

20

5/7/2024

21 of 29

Our Contribution

  • For Object Recognition
  • Including gaze

21

5/7/2024

Is it a book?

No, it’s a house

22 of 29

Our Contribution

  • For Object Recognition
  • Including gaze

22

5/7/2024

I’ll try to understand you better

I’ll do better next time

23 of 29

Training Outputs

23

5/7/2024

24 of 29

Results on Eye-tracking Data

24

5/7/2024

25 of 29

Case of Higher Resolution

25

5/7/2024

26 of 29

Behavioral Experiments

26

5/7/2024

27 of 29

Behavioral Experiments

27

5/7/2024

28 of 29

Behavioral Experiments

  • Behavioral results are expected to be ready soon

28

5/7/2024

29 of 29

29

5/7/2024

?

nejad.info