1 of 4

Using eye-tracking �to examine language processing in real time

Psycholinguistics:

“...investigates and describes the psychological processes that make it possible for humans to master and use language”

Arts and Social Sciences · EyeAthsi(yezobuGcisa) neNzululwazi ngeNtlalo · Lettere en Sosiale Wetenskappe

- I am Robyn Berghoff, from Dept of General Linguistics at SU. The title of my talk today is “Using eye-tracking to examine language processing in real time”.

- Together with my colleague Prof E Bylund, I am responsible for the Multilingualism and Cognition Lab, a research laboratory that, to our knowledge, is the only one of its kind on the African continent

- The MultiCog lab is a research lab for psycholinguistics - the discipline that "investigates and describes the psychological processes that make it possible for humans to master and use language"

- These psychological processes, because they happen in the mind, are not directly observable, but we have various methods that allow us to examine them.

One such method, the focus of our discussion today, is eye-tracking.
The eye-tracker that we have in our lab looks something like this. It sits below a computer screen, and allows us to precisely measure the movement of people’s eyes as they either read a text or view a visual display while listening to auditory input.
As far as these kinds of methods go, eye-tracking is fairly non-intrusive, and is thus a good means of gaining insight into what happens when people comprehend language in real time.

- An example output from an eye-tracking task would be something like this, where the circles reflect where the participant looked, and bigger circles reflect more time spent looking at a particular point in the image. The lines joining the circles reflect how the participant’s eyes moved over the image.

- When we interpret this kind of data, we assume that where people looked at a particular point in time reflects what they were processing at that point. This also means that a longer time spent looking at a particular point is assumed to reflect more effort spent processing that particular thing.

2 of 4

An illustrative example

In a monolingual context, do bilinguals activate only the relevant language?

Arts and Social Sciences · EyeAthsi(yezobuGcisa) neNzululwazi ngeNtlalo · Lettere en Sosiale Wetenskappe

A lot of my work is about how bilingual people manage their two languages. In this field, a question of interest is how bilinguals' two languages are stored and activated. We can imagine two possibilities: one where both languages are completely separate, and we can turn them on and off as the context demands. If this were the case, we would hypothesize that in a monolingual context, only the relevant language would active.
An alternative possibility is that the two languages are intertwined, and both are constantly active to a greater or lesser extent. If this were the case, we would expect to see activation of both languages even in a context where only one is relevant.

- In our lab, we did an experiment to explore this aspect of bilingualism, paying particular attention to “lexical access”, or word processing. We asked whether, in a context where only English was used, bilingual participants would also activate their other language.

- We recruited two groups of participants: one group of first language Afrikaans speakers with English as a second language, and one group of English speakers. Importantly, when we recruited the Afrikaans speakers, we took measures to downplay the relevance of their bilingualism, thus preventing them from knowing that the experiment had anything to do with their knowledge of two languages.

3 of 4

Click on the lion.

Berghoff, R., McLoughlin, J., & Bylund, E. (2021). L1 activation during L2 processing is modulated by both age of acquisition and proficiency. Journal of Neurolinguistics, 58(2), 100979. doi:10.1016/j.jneuroling.2020.100979

target

competitor

Arts and Social Sciences · EyeAthsi(yezobuGcisa) neNzululwazi ngeNtlalo · Lettere en Sosiale Wetenskappe

We had participants come into the lab. The lab environment and the testing session was entirely English, and so the participants were never exposed to anything in Afrikaans during the session.
Participants performed the experiment on a computer, where the eye-tracker was set up to capture their eye movements. The task was very simple: they viewed a display screen with four images on it, and they heard an instruction sentence telling them to click on a target image. We measured their eye movements while they completed this task.

- In the experiment, the crucial visual stimuli were pairs of images, where the name of the target noun in English began with the same sound as the name of the competitor noun in Afrikaans. For example, in this trial, the target noun was "Lion". The competitor was this image. In English, it is called a "drawer". In Afrikaans, it is called a "laai" - which begins with the same sounds as "lion".

- In addition to the crucial target-competitor pair, each trial display also included two distractor nouns, whose names in English and Afrikaans were not similar to the name of the target noun.

- For this trial, participants heard the sentence "Click on the lion". We focused on their eye movements as they heard "lion", in order to see whether they looked at the competitor item - "laai" in Afrikaans – more than the unrelated distractor items.

4 of 4

Want to learn more?

berghoff@sun.ac.za

APsA.africa

Thank you�Enkosi�Dankie

competitor

distractor

Arts and Social Sciences · EyeAthsi(yezobuGcisa) neNzululwazi ngeNtlalo · Lettere en Sosiale Wetenskappe

What we found was that Afrikaans participants - and not English participants - looked at the competitor item more than the unrelated distractor items.
The figure illustrates this pattern. Here, we are plotting looks to the competitor item versus to the unrelated distractor items. The two bottom lines represent the gaze behavior of the English group. They looked at the competitor item, for example “laai”, no more than they looked at the unrelated distractor items. The top two lines represent the gaze behavior of the Afrikaans group, and show that they looked more at the competitor item than the distractor items.
This suggests that their Afrikaans knowledge had also become activated when they heard the English target nouns. Importantly, this happened in a setting where everything was done in English, reflecting that activation of both languages happens even in a setting where only one language is relevant.
This was one example of the kinds of research on language processing that can be done using digital tools such as eye-tracking. Thank you for listening. If you want to find out more, please feel free to contact me, or to check out the website of the African Psycholinguistics Association, which aims to promote psycholinguistic research on the African continent.