A Longitudinal Study Investigating How Students’ Perceptions of Teacher Fear Appeals Relate to Academic Outcomes

Dr Laura Nicholson

Department of Psychology

Edge Hill University

1

Overview

• My research area: Psychological predictors of student motivation and achievement
• Longitudinal project investigating a range of motivational constructs with a main focus on teacher fear appeals
• Lots of research questions around how the constructs are related
• Both quantitative and qualitative data collected from Y10-11 students and GCSE maths teachers
• Five time points of data collection over two years including collection of achievement data (but affected by the closure of schools ☹)
• Today: Two studies based on this research…

​

​

​

2

Study 1

Longitudinal Relationships Between Teacher Fear Appeals, Student Engagement and Achievement

3

Teacher Fear Appeals

• Persuasive teacher messages that highlight consequences of success/failure and actions required to achieve success/avoid failure
• “If you do not work hard, you will not pass your maths GCSE and will not be able to go to college or 6th form”
• Designed to motivate students to work hard and do well in exams

​

​

• Challenge evaluations (mastery, hope, excitement)
• Threat evaluations (loss, worry, anxiety)
• Irrelevant evaluations (indifference)

​

4

What Determines Fear Appeal Evaluations?

1. How much students value the outcome (e.g., passing their GCSEs)
1.  
1. Attainment value: How much they value getting good marks on tests
2. Utility value: How much they value the utility of the subject outside of the classroom

​

2. How much students believe they have the skills to achieve the outcome (confidence)
1.  
2. Self-efficacy: How much they believe that they are capable of learning the subject
3. Expectancy of success: How much they believe they are capable of achieving the outcome (e.g., doing well in an exam)

​

5

High value AND High confidence

= Challenge evaluation

​

​

High value AND Low confidence

= Threat evaluation

​

​

Very low value OR Very high confidence

= Irrelevant evaluation

​

6

I know this exam is important but I have belief in my ability

I know this exam is important but I do not think I have the ability to pass it

I do not need to listen because this exam is not important / I know I will definitely pass

NB: Student evaluations do not need to be accurate. A student may have high ability but believe themselves incapable of passing an exam.

Relations with Educational Outcomes

Challenge evaluation Adaptive outcomes

Threat evaluation Maladaptive outcomes

7

Academic

Achievement

Previous Research

• Challenge evaluations are positively, and threat evaluations, negatively related to engagement and achievement (Nicholson & Putwain, 2020, Putwain et al., 2016, 2017; Putwain & Symes, 2011)
• Existing research limited…
• No temporal separation between fear appeal evaluations and engagement (Putwain et al., 2016) so we cannot determine directionality
• Not measuring frequency of fear appeals (Nicholson & Putwain, 2019)
• Not measuring achievement / Measuring achievement but measuring predictors and mediators at same wave (Putwain et al., 2017)

8

The Present Study

• Addressed weaknesses of prior research
• Used a robust design and analytical approach
• Aim: Examine indirect relationships between fear appeals to achievement, mediated by fear appeal evaluation and student engagement
• Two wave longitudinal design
• Multilevel approach
• Frequency of fear appeals is naturally a classroom climate variable
• Measuring frequency of fear appeals at the student level: How students differ from the rest of the class in their fear appeal reporting; an indicator of student attention to fear appeals
• Only findings at student level reported here

9

Hypotheses

• H1: T1 frequency of fear appeals will be positively related to T2 challenge and threat evaluation
• H2: T1 challenge evaluation will be positively, and T1 threat evaluation negatively, related to T2 student engagement
• H3: T2 student engagement will be positively related to subsequent GCSE grade

10

Figure 1: Hypothesised Model

11

Indirect Path 1:

Fear Appeals to Engagement via Fear Appeal Evaluations

Indirect Path 2:

Fear Appeal Evaluations to GCSE Grade via Engagement

Method

• 1,530 students (45.4% male, 50.7% female, 4.0% unknown) from 14 schools in NW of England
• Two wave design
• T1: Students in Y10 (M age = 14.6 years), spring term 2019
• T2: Students in Y11 (M age = 15.6 years), autumn term 2020
• Measures
• Fear appeal frequency and evaluations
• Behavioural engagement
• GCSE maths grade (grades 1-9; teacher assessments)
• Y9 maths grade, gender, age, FSM

​

​

12

Fear Appeals Questionnaire: Teachers Use of Fear �Appeals Questionnaire (Putwain et al., 2019)

13

Behavioural Engagement Questionnaire: Engagement versus Disaffection with Learning Questionnaire (Skinner et al., 2009)

14

Analytical Strategy

• Conducted using Mplus 8.0
• Preliminary analysis
• Checking of assumptions / Descriptive statistics / Correlations
• Multilevel confirmatory factor analysis to examine measurement models
• Testing longitudinal measurement invariance
• Multilevel structural equation modelling to test hypotheses
• Controlled for Y9 maths grade, gender, age and FSM
• Criteria for good model fit:
• RMSEA < .06, SRMR < .08, and CFI/TLI > .95 (Hu & Bentler, 1999)
• SRMR_B >.08 may not necessarily indicate model misspecification when the number of L2 units is <100 (Pagett & Morgan, 2021)

15

Analytical Strategy

Measurement Models

• Students reported that their teachers used fear appeals ‘sometimes’
• ML-CFA of fear appeal frequency, challenge evaluation, threat evaluation and behavioural engagement at T1 and T2 showed an acceptable fit to the data

​

​

​

​

• Sufficient measurement invariance over time

​

16

ML-SEM

• ML-SEM tested the hypothesised paths (shown in Figure 1)
• It showed an acceptable fit to the data

​

​

​

​

​

• We proceeded to inspect path coefficients…

​

​

​

​

17

Path Coefficients

18

Indirect Paths

​

​

​

​

​

​

​

​

19

Discussion

• H1: T1 frequency of fear appeals will be positively related to T2 challenge and threat evaluation
• Partially supported
• H2: T1 challenge evaluation will be positively, and T1 threat evaluation negatively, related to T2 student engagement
• Partially supported
• H3: T2 student engagement will be positively related to subsequent maths grade
• Supported

​

20

Fear Appeal Relations with Achievement

Higher perceived frequency of fear appeals

​

​

Higher challenge evaluation

​

​

Higher engagement in class

Higher GCSE maths grade

​

21

Also Higher threat evaluation

Lower GCSE maths grade

Limitations

• Measured only one form of engagement
• Two waves of data collection only
• Teacher assessments used (not exams)
• Student reports of fear appeals and self-perceptions of fear appeal evaluations and engagement

​

​

​

​

​

22

Study 2

Achievement Emotions as a Mediator of the Fear Appeal Evaluation and Student Engagement Relationship

23

The Role of Emotions

• Gaps in the literature concerning the relationship between fear evaluations and student engagement / unexpected findings…
• Nicholson et al. (2019)
• Person-centred approach
• Students can simultaneously evaluate fear appeals at a high level of both threat and challenge
• These students experienced high levels of behavioural engagement but also high levels of emotional disaffection
• Measuring discrete emotions may help to disentangle these findings

24

The Role of Emotions cont.

• Nicholson & Putwain (2020)
• Cross-lagged panel design
• Challenge evaluations prospectively (negatively) predicted behavioural disaffection but were not related to behavioural engagement or emotional engagement/ disaffection
• Threat evaluations did not prospectively predict either behavioural engagement or disaffection, but did predict both EMOTIONAL engagement (negatively) and disaffection (positively)
• Study 1
• Challenge evaluations prospectively predicted behavioural engagement
• Threat evaluation did not prospectively predict behavioural engagement and was directly related to achievement
• Emotions may be a more proximal outcome following the evaluation of fear appeals as a threat

​

25

• Emotions linked to academic learning, classroom instruction and achievement activities and outcomes (Pekrun, 2017)

​

• 4 categories
• Positive activating (enjoyment, hope)
• Positive deactivating (relief, relaxation)
• Negative activating (anxiety, anger)
• Negative deactivating (hopelessness, boredom)

​

• Differential effects on engagement and performance

​

26

Achievement Emotions

The Present Study

• Previous research suggests that achievement emotions may play an important role in how fear appeal evaluations relate to behavioural engagement
• Research question
• Do achievement emotions partially mediate relations between threat and challenge evaluations of fear appeals and behavioural engagement?
• Four emotions
• Enjoyment, hope, anxiety, hopelessness

​

27

Focusing in on…

28

Academic

Achievement

Student

Engagement

Hypothesised Mediational Model

29

+ve

Hypotheses

• H1 Challenge and threat evaluations will relate positively and negatively to behavioural engagement respectively
• H2 Challenge evaluations will relate positively, and threat appraisals negatively, to enjoyment and hope
• H3 Challenge evaluations will relate negatively, and threat evaluations positively, to anxiety and hopelessness
• H4 Enjoyment and hope will relate positively to engagement
• H5 Anxiety and hopelessness will relate negatively to engagement
• H6 Achievement emotions will partially mediate relations between fear appeal evaluations and engagement

​

30

Method

• 1,488 Y10 students (45.4% male, 50.7% female, 4.0% unknown; M age: 14.60 years) from 14 schools in NW of England
• Cross-sectional design: Data collected during spring term 2019 (T1)
• Measures
• Fear appeal frequency and evaluations
• Achievement emotions: Enjoyment, anxiety, hope, hopelessness
• Behavioural engagement
• Y9 maths grade, gender

​

31

Achievement Emotions Questionnaire: Test Emotions Questionnaire (Pekrun et al., 2004)

32

Analytical Strategy

• Conducted using Mplus 8.0
• Preliminary analysis
• Checking of assumptions / Descriptive statistics
• 4 Confirmatory factor analyses to examine measurement models (4 emotions)
• 4 structural equation models to test hypotheses (4 emotions)
• Controlled for Y9 maths grade and gender
• Students nested within classroom statistically accounted for
• Criteria for good model fit:
• RMSEA < .06, SRMR < .08, and CFI/TLI > .95 (Hu & Bentler, 1999)

33

Measurement Models

​

​

34

SEM for Enjoyment

35

Challenge to Engagement via Enjoyment: β = 0.16, SE = 0.05, 95% CIs [0.09, 0.24]

Threat to Engagement via Enjoyment: β = -0.04, SE = 0.01, 95% CIs [-0.06, −0.02]

Fit: χ2(180) = 752.47, p <.001, RMSEA = .050, SRMR = .054, CFI = .942, TLI = .925

SEM for Hope

36

Fit: χ2(198) = 804.96, p <.001, RMSEA = .049, SRMR = .058, CFI = .941, TLI = .925

Challenge to Engagement via Hope: β = 0.11, SE = 0.04, 95% CIs [0.04, 0.18]

Threat to Engagement via Hope: β = -0.06, SE = 0.02, 95% CIs [-0.10, −0.02]

SEM for Anxiety

37

Fit: χ2(180) = 885.99, p <.001, RMSEA = .055, SRMR = .083, CFI = .931, TLI = .911

Mediation was not tested due to suppression effect of threat appraisal, and challenge appraisal and anxiety were not related.

SEM for Hopelessness

38

Fit: χ2(180) = 601.14, p <.001, RMSEA = .043, SRMR = .054, CFI = .960, TLI = .949

Mediation was not tested as hopelessness was not related to engagement.

Discussion 1

• H1 Challenge and threat evaluations will relate positively and negatively to behavioural engagement respectively
• Partially supported
• H2 Challenge evaluations will relate positively, and threat evaluations negatively, to enjoyment and hope
• Fully supported
• H3 Challenge evaluations will relate negatively, and threat evaluations positively, to anxiety and hopelessness
• Partially supported

​

​

39

Discussion 2

• H4 Enjoyment and hope will relate positively to engagement
• Fully supported
• H5 Anxiety and hopelessness will relate negatively to engagement
• Not supported
• H6 Academic emotions will partially mediate relations between fear appeal evaluations and engagement
• Partially supported

​

​

​

40

Key Messages

• Challenge evaluation and behavioural engagement robustly positively related
• Enjoyment and hope fully accounted for the relationship between threat evaluation and behavioural engagement
• These positive emotions may buffer against the negative effect of threat evaluations
• Hopelessness not related to behavioural engagement
• The relationship between threat evaluation, anxiety and engagement needs probing further with longitudinal data

​

​

​

41

Limitations

• Cross-sectional design
• Achievement not measured
• Self-report data

​

​

​

​

​

​

42

Educational Implications

• Fear appeals are neither negative or positive
• The critical point is how they are evaluated by the student
• Students who evaluated fear appeals as a challenge showed greater behavioural engagement
• Students who evaluated fear appeals as a threat showed lower achievement
• Promotion of challenge evaluation via increasing confidence
• Mastery and vicarious experiences
• Strategy-focused feedback
• Effort reinforcement

​

• Teachers need to be educated…

​

43

What do educators need to know?

1. What teachers say to students matters
2. Messages may not be interpreted as intended
3. Impact depends on how the student evaluates the message
4. What is the optimal message?
5. Messages may be better directed to individual students rather than a whole class or year group

44

See also Putwain, Symes, Nicholson, & Remedios (2021)

Future Directions

• Will continue to analyse all the data…
• Repeat the cross-sectional analyses regarding achievement emotions using longitudinal data
• Perhaps the role of emotions is particularly strong for those reporting high threat evaluations and their achievement is low regardless of behavioural engagement
• Cross-lagged panel/mediation/moderation models to further identify the direction and nature of relationships between
• Fear appeal frequency, Fear appeal evaluations, Achievement emotions, Student-teacher relationships, Behavioural engagement, and how these relate to GCSE grade
• How demographic data relate to these constructs and relationships
• Qualitative interview data from students about their experiences of fear appeals
• Qualitative written data from teachers about their use of fear appeals

​

​

​

​

45

Acknowledgements

• Thank you to Prof Dave Putwain and Dr Gulsah Kutuk
• Funding for this research programme was received from the Edge Hill University Research Investment Fund

46

References 1

• Pekrun, R. (2017). Achievement emotions. In A. J. Elliot, C. S. Dweck & D. S. Yeager (Eds.), Handbook of competence and motivation (2nd ed., pp. 251-271).
• Pekrun, R., Goetz, T., Perry, R. P., Kramer, K., Hochstadt, M., & Molfenter, S. (2004). Beyond test anxiety: Development and validation of the test emotions questionnaire (TEQ). Anxiety, Stress & Coping, 17(3), 287-316. 10.1080/10615800412331303847
• Nicholson, L. J., & Putwain, D. W. (2020). A cross‐lagged panel analysis of fear appeal appraisal and student engagement. British Journal of Educational Psychology, 90(3), 830-847.
• Nicholson, L.J., Putwain, D.W., Nakhla, G., Porter, B., Liversidge, A., & Reece, M. (2019). A person-centered approach to students’ evaluations of perceived fear appeals and their association with engagement. Journal of Experimental Education, 87(1), 139-160.
• Putwain, D.W., Nakhla, G., Liversidge, A., Nicholson, L.J., Porter, B., & Reece, M. (2017). Teachers use of fear appeals prior to a high-stakes examination: Is frequency linked to perceived student engagement and how do students respond? Teaching and Teacher Education, 61, 73–83.

​

47

References 1

• Putwain, D.W., Nicholson, L.J., Nakhla, G., Reece, M., Porter, B., & Liversidge, A. (2016). Fear appeals prior to a high-stakes examination can have a positive or negative impact on engagement depending on how the message is appraised. Contemporary Educational Psychology 44-45, 21–31.
• Putwain, D. W., Symes, W., & McCaldin, T. (2019). Teacher use of loss-focused, utility value messages, prior to high-stakes examinations, and their appraisal by students. Journal of Psychoeducational Assessment, 37(2), 169-180.
• Putwain, D.W., Symes, W., Nicholson, L.J., & Remedios, R. (2021). Teacher motivational messages prior to examinations: What are they, how are they evaluated, and what are their educational outcomes? In A.J. Elliot (Ed.), Advances in motivational science (pp. 63 – 103). Academic Press.
• Putwain, D.W., Symes, W., & Remedios, R. (2016). The impact of fear appeals on subjective-task value and academic self-efficacy: The role of appraisal. Learning and Individual Differences 51, 307–313.
• Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009). A motivational perspective on engagement and disaffection: Conceptualization and assessment of children's behavioral and emotional participation in academic activities in the classroom. Educational and Psychological Measurement, 69(3), 493-525.

​

​

48

Thank you for listening!

49

Laura.Nicholson@edgehill.ac.uk