Polar questions�in Sign Language of the Netherlands

Logic and Conversation 2023, Lecture 13, Floris Roelofsen

� Based on slides for a seminar at UPF, Barcelona, March 2023�� Joint work with Marloes Oomen, Lyke Esselink & Tobias de Ronde

Research question:

How are polar questions marked in �Sign Language of the Netherlands (NGT)?

​

2

Received view

� Polar questions in sign languages are marked by raised eyebrows

​

Standard references for NGT: Schermer and Koolhof 2009 and Klomp 2021

Broader typological claim: Zeshan 2004

​

Other non-manual markers sometimes cited as appearing in polar questions across sign languages:

• wide-opened eyes,
• addressee eye contact,
• forward position of the body and/or head

3

Data challenging the received view

Both in corpora and in experimentally elicited data researchers have found a lot of variation in the marking of polar questions:��For NGT: Coerts 1992, De Vos et al. 2009, Gaasbeek 2023

For LSC: Cañas Peña 2019

4

Hypothesis:

This variation is (at least partly) due to the fact that there may be different question forms, conveying different kinds of biases.

​

5

Two types of bias

In English and other spoken languages, two types of bias are well-known to affect which polar question forms can be used in a given context:

• Prior speaker belief (SB)
• Immediate contextual evidence (CE)

​

6

Illustration (1)

Context: Sue is going out for dinner with her friends. So far, Sue, Ann, Mary, and Jill have arrived at the restaurant. Sue has no idea whether anyone else was invited.

​

Ann: Let’s not order until Olivia is here. Neutral SB, Positive CE

�Sue: Is Olivia coming as well?� Olivia is coming as well?� #Isn’t Olivia coming as well?� #Olivia is coming as well, isn’t she?�

7

Illustration (2)

Context: Sue is going out for dinner with her friends. So far, Sue, Ann, Mary, and Jill have arrived at the restaurant. Sue expects that Olivia will come as well.

​

Ann: I think we’re all here, so let’s order. Positive SB, Negative CE

�Sue: #Is Olivia coming as well?� #Olivia is coming as well?� Isn’t Olivia coming as well?� Olivia is coming as well, isn’t she?

​

8

Bias

• We know that speaker belief and contextual evidence affect the form of polar questions in many spoken languages
• We hypothesize that NGT and other sign languages are no different in this respect�
• In our experiment, SB and CE are established through dialogues with confederates
• Not via description (“You believe that…”)
• Not by manipulating the situational context (e.g. speaker gets some visual evidence)

9

Hypothesis elaborated

• We hypothesize that there are different polar question forms in NGT, �possibly distinguished in terms of
• Sentence structure
• Non-manual markers
• Presence of specific manual markers�
• Different polar question forms are associated with different types of bias
• The range of contexts in which a given form is used may be constrained
• The mapping between context and form need not be one-to-one

10

Previous work on biased polar questions in sign languages

• Cañas Peña (2019) shows that different combinations of (mostly) non-manual markers in Catalan Sign Language are associated with particular combinations of epistemic (SB) and evidential (CE) bias:�
• Note: some contexts not covered yet
• Neutral SB + Positive CE
• Neutral SB + Negative CE
• Positive SB + Negative CE

11

• Gökgöz & Wilbur (2017) and Sze & Lee (2022) report on ‘high negation’ questions in Turkish Sign Language and Hong Kong Sign Language, respectively.

Visual cues in polar questions in spoken languages

• Borràs-Comes et al. (2013) show that in neutral polar questions (Neutral SB, Neutral CE) in Catalan and Dutch, raised eyebrows and addressee-directed eye gaze are more frequent than in statements.�
• Borràs-Comes et al. (2011) show that in ‘counter-expectational questions’ (Negative SB, Positive CE) in Catalan, furrowed eyebrows, squinted eyes, and head backward are important visual cues

12

Data collection for NGT

• Adapted from Domaneschi et al. (2017) on English and German
• Participants: 6 deaf native NGT signers (+ 1 trial participant)
• 7 experimental conditions x 5 scenarios = 35 target questions per participant

​

​

​

• SB and CE provided by 2 confederates in role-play setting
• Recording: regular + 3D camera for face

13

How it went in practice:

​

​

14

0/0

Two data sets

• Video data with manual annotations
• manual signs
• facial features
• body and head position/movement�
• 3D data
• facial features only

15

Annotation

• Target questions annotated in ELAN for NMM and MM
• Procedure:
• Annotation manual drafted based on Gaasbeek (2022)
• Test phase: 18 target questions annotated by 2 annotators & evaluated
• Inter-rater reliability phase: 20% of data annotated by 2 annotators
• Full annotation phase: remaining data annotated by 1 annotator

16

Annotations in ELAN – An example

17

0/+

Facial data from depth-sensing camera

18

19

Hypothesis

• There are different polar question forms in NGT,�possibly distinguished in terms of
• Non-manual markers
• Sentence structure
• Presence of specific manual markers�
• Different polar question forms are associated with different types of bias
• The range of contexts in which a given form is used may be constrained
• The mapping between context and form need not be one-to-one

20

Analysis of annotated video data

• We focus today on all question forms containing headshake (N=82 out of 201):���

​

​

​

​

• Headshake is the main marker of negation in NGT; it can negate a proposition on its own (Coerts 1992)
• Across spoken languages, we know that biased polar question forms often involve negation

21

Results

1. Non-manual marker patterns
2. General sentence structure
3. Function and place of occurrence of headshake
4. Distributional constraints

​

22

1. Non-manual marker patterns

Three key observations:

• Head/body forward position is the most consistently attested non-manual marker across conditions

​

• Eyebrows: often lowered or inner brow raised, often with squint, attested across conditions
• Note: eyebrows hardly ever raised (7/82)!

​

• Combination head up and nose wrinkle, in addition to brow lowering and eye squint, common in contrastive conditions [-SB,+CE] and [+SB,-CE]

​

23

1. Non-manual marker patterns

​

What we think they signal:

​

• Head/body forward position
• Seems to signal inquisitiveness (the ? operator), independent of bias
• Prediction: occurs across all experimental conditions and baseline questions, but not in statements

​

24

1. Non-manual marker patterns

​

What we think they signal:

​

• Lowered brows / inner brow raise + squint
• May be a marker of uncertainty, but may also be connected to negation (see Zeshan 2004)
• Prediction: does not occur as often in items without headshake, but does occur in statements of disagreement; does not occur in affirmative declaratives

​

25

1. Non-manual marker patterns

​

What we think they signal:

​

• Head up + nose wrinkle, in addition to lowered brows + squint
• Signals unexpectedness of the radical / a refusal to accept the interlocutor’s suggestion �(cf. Borràs-Comes 2011)
• Prediction: occurs in the conditions [+SB,-CE] and [-SB,+CE], independent of the presence of headshake

26

2. Sentence structure: Five common patterns

(1) Negative sentence radical with inquisitive NMM (not glossed):

​

​

​

27

hs

index₃ money^free.of.charge index_up

‘Is it not free of charge there?’

0/-

2. Sentence structure: Five common patterns

(2) Negative sentence radical + right:

​

​

28

hs

kim home, right palm-up

‘Kim isn’t home, right?’

-/+

2. Sentence structure: Five common patterns

(3) Positive sentence radical + mouthed ‘or not’:

29

‘or not’, hs

kim home index₃

‘But is Kim home or not?’

0/0

2. Sentence structure: Five common patterns

(4) Positive sentence radical + inquisitive headshake:

30

hs

kim index₃ person vegetarian index₃ palm-up

‘Is Kim a vegetarian?’

0/0

2. Sentence structure: Five common patterns

(5) Negative sentence radical + inquisitive headshake:

​

​

31

hs hs

so not home hesitate

‘So [Kim] is not home?’

+/-

3. Headshake: Two main functions (1)

• Forms (1), (2), (3), (5): negation

​

​

​

​

​

​

​

• In (1)-(2), headshake negates the sentence radical, while in (3) it does not negate the sentence radical but rather introduces a negative alternative (‘or not’).

32

hs

(2) kim home, right palm-up

‘Kim isn’t home, right?’

hs

(1) index₃ money^free.of.charge index_up

‘Is it not free of charge there?’

‘or not’, hs

(3) kim index₃ vegetarian index₃

‘But is Kim a vegetarian or not?’

hs hs

(5) so not home hesitate

‘So Kim is not home?’

3. Headshake: Two main functions (2)

• Forms (4) and (5): headshake does not express negation but functions as�(part of) a question marker

​

​

​

• In this use, headshake occurs sentence-finally, typically in combination with the manual signs PALM-UP or HESITATE or a held sentence-final sign.

33

hs hs

(5) so not home hesitate

‘So [Kim] is not home?’

hs

(4) kim index₃ person vegetarian index₃ palm-up

‘Is Kim a vegetarian?’

4. Distributional constraints (1)

(i) Questions involving a negative sentence radical with inquisitive headshake or other inquisitive NMM, (1) and (5), require negative CE or SB, and are incompatible with positive CE:

34

(1)

(5)

Negative sentence radical with inquisitive NMM

Negative sentence radical + inquisitive headshake

4. Distributional constraints (2)

(ii) Questions with a negative sentence radical and RIGHT (2) require negative SB:

35

(2)

Negative sentence radical + RIGHT

4. Distributional constraints (3)

(iii) Questions with ‘or not’ (3) require neutral CE:

36

(3)

Positive sentence radical + or not

4. Distributional constraints (4)

(iv) Questions with an inquisitive headshake, (4) and (5), require that the CE does not contradict the sentence radical, which means that if the sentence radical is positive, CE cannot be −, while if the sentence radical is negative, CE cannot be +:

37

(5)

(4)

Negative sentence radical + inquisitive headshake

Positive sentence radical + inquisitive headshake

Back to facial 3D data: initial analysis

38

Back to facial 3D data: initial analysis

39

Results of clustering

Three main clusters:

1. Cluster ‘UP’: high values of
• BrowInnerUp (0.75)
• BrowOuterUp (0.67)
• EyeWide (0.82)
2. Cluster ‘DOWN’: high/moderately high values of
• BrowDown (0.60)
• EyeSquint (0.39)
3. Cluster ‘NEUTRAL’: low values (≤0.20) for all blendshapes => neutral face�

Sub-clusters of UP and DOWN differ most prominently in the value of MouthFrown.

40

41

Some effects of context

• Context has an effect on which facial expressions are used.

​

For instance:

• While in general UP is much less frequent than DOWN, 19% vs 39%, �in NeutPos contexts UP is more frequent, 35% vs 33%. �
• On the other hand, in PosNeg contexts UP does not occur at all�while DOWN is very frequent, 60%.

42

Research question:

How are polar questions marked in �Sign Language of the Netherlands (NGT)?

​

43

Partial answer so far:

1. By non-manual markers connected to
• Inquisitiveness (interactive, targets future move by interlocutor) → head/body forward
• Uncertainty (speaker state, not interactive) → brow frown or inner raise, eye squint
• Unexpectedness / refusal to accept (interactive, triggered by previous move/event) → head up, nose wrinkle, brow frown (eye squint)

​

In our dataset, markers of inquisitiveness and uncertainty occur across conditions, but markers of unexpectedness occur only in contrastive conditions.

​

44

Partial answer so far:

1. By non-manual markers connected to
• Inquisitiveness → head/body forward
• Uncertainty → brow frown or inner raise, eye squint
• Unexpectedness / refusal to accept → head up, nose wrinkle, brow frown (eye squint)

​

In our dataset, markers of inquisitiveness and uncertainty occur across conditions, but markers of unexpectedness occur only in contrastive conditions.

​

45

Partial answer so far:

2. Through particular sentence-final inquisitive elements:
• Sentence radical + ‘or not’
• Sentence radical + inquisitive headshake (+ HESITATE / PALM UP)
• Sentence radical + RIGHT

​

In our dataset these are all restricted to particular conditions.

​

46

Next steps

• Analysis of remaining video data, not involving headshake

​

• Analysis of interaction between non-manual markers and sentence structure (combining video and 3D data)

47

Some plans for future work

• Further develop annotation guidelines for non-manual features to be used across languages, both signed and spoken
• Exploring automatic annotation of non-manual facial features based on 3D data

​

• Same production study with speakers and signers of other languages �(current MOL thesis project Elynn Weijland on spoken Dutch)
• Comprehension study

​

48

​

Thank you!

49

References

• Borràs-Comes, Pugliesi, Prieto (2011). Audiovisual perception of counter-expectational questions. Auditory-Visual Speech Processing.
• Borràs-Comes, Kaland, Prieto, Swerts (2013). Audiovisual Correlates of Interrogativity:

A Comparative Analysis of Catalan and Dutch. Journal of Non-verbal Behavior.

• Coerts (1992). Nonmanual grammatical markers: an analysis of interrogatives, negations and topicalisations in Sign Language of the Netherlands. Ph.D. thesis, University of Amsterdam.
• de Vos, van der Kooij, and Crasborn (2009). Mixed signals: Combining linguistic and affective functions of eyebrows in questions in Sign Language of the Netherlands. Language and Speech, 52(2-3), 315-339.

50

References (continued)

• Domaneschi, Romero, and Braun (2017). Bias in polar questions: Evidence from English and German production experiments. Glossa: a journal of general linguistics, 2(1).
• Gaasbeek (2022). (Non)manual marker annotation guideline. Unpublished document.
• Zeshan (2004). Interrogative constructions in signed languages: Crosslinguistic perspectives. Language, 80(1), 7-39.

51

An example scenario (1)

Context video 1: You want to go to the Efteling this weekend, but you’re not sure it’s open. You meet Ria, who has a subscription to the park. Ask her.

​

Participant: “Is the Efteling open this weekend?”

Confederate 1: + “Yes, the Efteling is open this weekend.”

0 “It’s open on Saturday but I don’t know about Sunday.� I never go on Sunday.”

— “It’s open on Saturday but I think I read in the newspaper that it’s not open on Sunday.”

​

52

An example scenario (2)

Context video 2: Later that day, you meet Tom. He works at the Efteling.

You know he has the weekend off. Ask him if he’d like to

come to the Efteling with you this weekend.

​

Participant: “Do you want to go to the Efteling with me?”

Confederate 1: + “Fun! Shall we go on Sunday?”

0 “I can’t this weekend.”

— “The Efteling is only open on Saturday. I’m available then.”

​

53

An example scenario (3)

Picture prompt: Participant:

“Is the Efteling open on Sunday?”

“Is the Efteling not open on Sunday?”

“The Efteling is open on Sunday right?”

“The Efteling isn’t open on Sunday?!”

…

​

54

Overview

55