Process measures and data analysis��PhD Summer School on Translation Processes Research�CBS, August 2011

1

Kristian Tangsgaard Hvelplund�

^{kthj.isv@cbs.dk}

• Tutorials and hands-on sessions�
• Groups of 4�
• Come up with a research project
• Design an experiment
• Consider relevant process measures and how to analyse the data
• Run the experiment
• Data analysis�����

2

• Translation as information processing�����������Based on Baddeley and Hitch (1974), Baddeley (2007), Bennaroch (2006), Eysenck and Keane (2010), Jaekl and Harris (2007)���

3

�Working memory

7 items (+/- 2)�< 18 seconds��

​

​

​

​

Long-term memory�∞

Sensory memory�< 500 ms�-> 60 ms

Motor system�-> 200 ms

Attentional control

• Methods of data elicitation�
• Eye tracking
• Cognitive sciences
• Psycholinguistics
• Psychology
• Human-computer interaction
• Marketing research
• Etc.

​

• Key logging
• Translation process studies
• Writing process studies�����

4

• Advantages of eye tracking and key logging�
• Reliability
• We can be fairly certain that eye-tracking data and key-logging data are manifestations of ongoing cognitive processing

​

• Nonintrusive
• The reliability of the data as reflections of the participant’s translation process is not compromised by the research process

​

• Completeness and level of detail
• Eye-tracking data and key-logging data in combination offer a highly complete and detailed representation of the translation process����

5

• Typical key-logging based measures�
• Pauses�Character count�Revision behaviour�Editing�Etc.��

6

• Typical eye-movement based measures�
• Fixation duration
• The time the eye fixates on a single locale – often measured in milliseconds (ms)

• Fixation count
• More fixations reflects more cognitive effort / fewer fixations reflects less cognitive effort

​

• Pupil size
• Pupil size reflects cognitive processing intensity

​

• Total gaze time
• More time spent in a region reflects more cognitive effort / less time spent in a region reflects less cognitive effort

7

• Typical eye-movement based measures�
• First pass gaze duration / fixation count / pupil size
• Reflects the cognitive effort / intensity during the initial reading of a word or segment�Often character count is introduced as control variable

​

• Second pass gaze duration / fixation count / pupil size
• Reflects the cognitive effort / intensity during the subsequent reading of a word or segment�Is often compared with first pass

​

• Regressions
• May indicate uncertainty on the part of the reader in comprehending the text�More regressions = problems with comprehension

​

• Transitions
• Reflects the number of times attention shifts between two tasks��

8

• Types of eye movements�
• Reading consists of two types of eye movements + pupillary movement:�
• (Visual) fixations
• Saccades
• Pupil dilation and pupil constriction

9

• Visual fixation�
• The continued maintenance of visual gaze at a specific location so that the retina is stabilised over an object of interest (Duchowski 2007: 46).�
• Eye-mind and immediacy assumptions (Just and Carpenter 1980: 331)��“there is no appreciable lag between what is being fixated and what is being processed”, and��“the interpretations at all levels of processing are not deferred; they occur as soon as possible”����

10

• Interpretation and problems�
• Visual focus of a word = cognitive resources allocation to this word
• Short fixations = less cognitive effort
• Long fixations = more cognitive effort�
• Covert attention
• Attention can shift independently from eye movement

​

• Saccades
• The eye is blinded during saccadic eye movements����

11

• Visual fixation – durations based on task�
• Silent reading = 225 ms
• Reading aloud = 275 ms
• Reading emerging text (reading while typing) = 400 ms (Rayner 1998: 373)
• Reading during translation (Jakobsen and Jensen 2008)
• Source text reading = 218 ms
• Target text reading = 259 ms

����

12

• Saccades�
• Rapid eye movements between actual fixations. No visual information transmitted to the cognitive system. �
• Saccade speed -> 500 degrees per second�Typical saccade length during reading -> 2 degrees (8 letter characters)�Typical saccade duration -> 30 ms�
• Saccades account for around 5-15 percent of all eye movements during reading. �����

13

• Pupillary movement�
• Small adjustable opening in the centre of the eye’s iris that allows light to enter the eye’s retina.

​

• Interpretation of pupillary movement relies also on the eye-mind assumption (Just and Carpenter 1980: 331); �
• Pupillary movement when fixation = relative change in cognitive resources allocated��-> Smaller pupils = relatively less cognitive effort�-> Larger pupils = relatively more cognitive effort�����

14

• Three types of eye trackers�

Head-supported

Head-mounted

Remote�����

15

• Comparison chart�
• Spatial resolution�-> The smallest change in eye position that can be measured
• Temporal resolution�-> Number of recorded eye positions per second�����������

16

• Implications for research design �
• Use large fonts (20 pitch tahoma or larger)
• Use short texts (no longer than 200 source text words)
• Consider a design in which no online or offline translation aids are available
• Consider having objects of interest that cover large areas of the screen

17

• Analysing eye-tracking data��with ClearView and Tobii Studio

​

• Recording scenes
• Temporal object of interest

​

• Areas of interest (AOIs)
• Spatial object of interest�����������

18

• Eye-tracking measures��with ClearView and Tobii Studio�����������

19

• Hot spot visualisation

�Reading

experiment

20

• Gaze plot visualisation��Translation of the�Spielberg text�(EN->DA)

21

• Gaze replay �����������

22

• Limitations of these measures�
• Qualitative measures are relevant when
• Getting an initial impression of processing pattern
• Visualising processing patterns

​

• Quantitative measures are few ...
• Average fixation duration across one participant / task
• Fixation count
• What about pupil size information?
• What about first/second pass information? Regression data?
• What about key-logging information?

​

• ... and potentially misleading
• Average fixation duration doesn’t consider variance between individual fixation durations

​

• Solution
• Raw data

���

23

• Raw eye-tracking data �����������

24

• Raw data – time stamp�����������

25

• Raw data – fixations�����������

26

• Raw data – fixation coordinates�����������

27

• Raw data – pupil diameter (mm)�����������

28

• Raw data – key-logging�����������

29

• Raw data – fixation annotation�����������

30

• Advantages of analysing raw data�
• The process data set is much richer / many more items
• Pupil size values
• Individual fixation durations
• Simultaneous reading and typing
• First / second pass gaze duration

​

• Disadvantages of analysing raw data�
• Very labour intensive

31

• Eye-tracking data quality�
• The quality of the eye-tracking data is sensitive to various factors, including: (cf. e.g. O’Brien 2009, Hvelplund 2011)

​

• Lighting conditions
• Distance to the eye tracker
• Too much head movement
• Eye colour
• Optical aids�
• Precautionary steps:

​

• Maintain the same dimmed lighting conditions (preferably articifial light)
• Distance no more than 55-65 cm from the eye tracker
• Avoid too much movement
• Avoid participants who have very dark eyes*
• Avoid participants who wear glasses or contact lenses

​

32

• Measuring eye-tracking data quality�
• Still risk of poor eye-tracking data. Three ways to see if the data quality is acceptable:�
• Mean Fixation Duration
• Gaze Time on Screen as percentage of total production time
• Gaze sample to Fixation Percentage����������

33

• Mean fixation duration�
• Typical fixation duration during reading is 225 to 275 ms (Rayner 1998)

​

• If fixation duration around 175-200 ms or shorter, perhaps flawed data�
• Example = Participant A, Task 1: Mean fixation duration = 146 ms

34

• Gaze time on screen�
• Written translation involves some amount of ST reading and TT reading.

​

• If very limited ST and TT reading during the task, perhaps flawed data�
• Example = Participant A, Task 1: Total gaze time on screen = 24 seconds / 8 percent �����������

35

• Gaze sample to fixation percentage�
• Fixations account for 85-95 percent of all eye movements
• Saccades account for 15-5 percent of all eye movements

​

• If the raw data reflects a distorted distribution, perhaps flawed data�
• Example = Participant A, Task 1: Fixations = 50 percent, saccades = 50 percent �����������

36

• Data quality comparison�����������

37

• Example – reading for different purposes

​

• Eye movement behaviour across four different types of reading task (Jakobsen and Jensen 2008)

​

• Six professional translators & six student translators
• L2 English -> L1 Danish

​

������������

38

• Example – translation directionality

​

• Eye tracking translation directionality (Pavlovic and Jensen 2009)

​

• Four professional translators & four student translators�
• L2 English -> L1 Danish (~250 words)�L1 Danish -> L2 English (~250 words)

​

​

​

​

​

​

​

​

������������

39

• Example – translation directionality

​

​

​

​

​

​

​

​

​

​

​

• Both hypotheses confirmed. All comparisons p < 0.05.

​

������������

40

• Example – translation directionality

​

​

​

​

​

​

​

​

​

​

​

• Partially (tentatively) confirmed. No statistical test performed.

​

������������

41

• Allocation of cognitive resources in translation ��Investigate how translators allocate cognitive resources during translation.�Two indicators are employed to investigate allocation of cognitive resources to the source text and the target text:�
• Cognitive resource management
• Cognitive load��How are these indicators affected by:�
• Different types of processing
• Differences in translational expertise
• Differences in time conditions

​

42

• The processes of translation ����

43

• Sequential and parallel processing�����

44

ST

TT

ST

TT

ST

TT

ST

TT

ST

TT

ST

TT

ST

TT

• Data collection and analysis��

45

• Independent variables�

46

• Dependent variables�

���

47

• Hypotheses����

48

• Findings – cognitive resource management��

49

• Findings – cognitive resource management��

50

• Findings – cognitive load��

51

• Findings – cognitive load��

52

• Findings summary��

53

• What does the quantitive data not tell us?

​

• Correlation between more cognitive effort and translation quality?
• Better translation product if more time is available?
• Can we readily assume that an ’optimal’ translation process will lead to better translation quality?
• How did the translators experience the time comstraints?
• How did the translators experience text complexity? ������������

54

• Quantitative data needs to be supplemented with ...

​

• Questionnaire data
• Translation quality assessment
• Verbal protocols which do not interfere with the translation process
• Etc.

55