1 of 70

Assessment and feedback in training and teaching

ELIXIR-GOBLET Train the Trainer - Session 4

2 of 70

Teaching Goal

Session 4

  • Develop an understanding of different types of assessment in training, when to receive and give feedback, and for which purpose.

3 of 70

Learning outcomes

Session 4

  • Describe the differences between formative and summative assessment�
  • Explain why frequent feedback is important

  • Describe and Exemplify few techniques for formative assessment�
  • List a few techniques to cope with feedback on your teaching efforts

4 of 70

Curriculum Design

- Nicholl’s cycle -

Inspired by: Via A, Palagi PM, Lindvall JM et al. Course design: Considerations for trainers  – a Professional Guide. F1000Research 2020, 9:1377 doi: 10.7490/f1000research.1118395.1

Aligned with LEs?

Support progress towards LOs?

Supports LEs?

Promotes LOs?

LOs achievable by most learners?�Content, LEs & Assessments support LOs?

SMART

LEs aligned with LOs?

LOs

LEs

Content

Assessment*

Evaluation*

1

2

3

4

5

Legend

Confirmed

Denied

Proposition to move forward

Phase

Success

5 of 70

6 of 70

Key concepts:

ASSESSMENT

FEEDBACK

7 of 70

What is assessment?

8 of 70

Assessment is estimation of:

  • Current knowledge
  • Performance
  • Quality

recolored image by @vectorjuice at Freepik

9 of 70

Assessment is bidirectional

It can be used both by instructors and learners

10 of 70

Assessment is estimation of:

  • By the instructors:
    • Learning current state
    • Assess output of a learning gain �
  • By the learners:
    • Quality, Pertinence, Adequacy of
      • Training session
      • Interaction with trainer
      • Course material

recolored image by @vectorjuice at Freepik

11 of 70

There are different types of assessment and different ways that we can categorize them

12 of 70

When?

ASSESSMENT

BEFORE

DURING

AFTER

Pre-course

At the beginning of the course

related to the time of the course

13 of 70

What?

When?

ASSESSMENT

BEFORE

DURING

AFTER

Pre-course

At the beginning of the course

related to the time of the course

Discuss…

14 of 70

What?

When?

ASSESSMENT

BEFORE

DURING

AFTER

Pre-course

At the beginning of the course

related to the time of the course

Assess prerequisites of target audience

Assess if learning has occurred

Assess prior knowledge�mental model(s)

Assess whether learning is occurring

15 of 70

What?

Type

When?

ASSESSMENT

BEFORE

DURING

AFTER

Pre-course

At the beginning of the course

related to the time of the course

Assess prerequisites of target audience

Assess if learning has occurred

Assess prior knowledge�mental model(s)

Assess whether learning is occurring

Formative assessment

Summative assessment

16 of 70

Formative assessment

Summative assessment

Assess prerequisites of target audience

Assess if learning has occurred

Assess prior knowledge

mental model(s)

Assess whether learning is occurring

ASSESSMENT

BEFORE

DURING

AFTER

Pre-course

At the beginning of the course

related to the time of the course

Diagnosis assessment

17 of 70

What is Feedback?

18 of 70

Feedback

is information meant for improvement, such as:

  • Person's performance
  • Task
  • Knowledge
  • Skills
  • Competencies
  • Abilities
  • etc.

_Feedback is bi-directional_

19 of 70

One categorization: related to the time of the course

Long time after

Measure impact of the course

At the end

Measure the quality & �adequacy of the course & trainer

Short term feedback

Long term feedback

TRAINER

LEARNER

ASSESSMENT

During course

After course

Feedback

20 of 70

Challenge 4.1

Diagnostic Assessment

Group Activity (in pairs) �8 min

  1. Discuss HOW you could collect information about prior experience from learners �
    1. before
    2. at the beginning of a course �
  2. how you can use it?

Write it in the Shared notes

21 of 70

What is the difference?

vs.

ASSESSMENT

FEEDBACK

22 of 70

How and what to assess

23 of 70

Assessment process flowchart

ASSESSMENT

WHAT

HOW

PRODUCT

PROCESS

SUMMATIVE

FORMATIVE

24 of 70

What to assess

PRODUCT

ASSESSMENT

PROCESS

ASSESSMENT

Model

Gymnastics routine

Project

Assignment

Exercise

Painting

Attitudes

Social-relational skills

Problem-solving ability

Creativity

Method use

Progress

Instructor’s didactic process

25 of 70

Why to do each assessment

FORMATIVE

SUMMATIVE

TIMING

Throughout the�learning process

At the end of an�instructional period

PURPOSE

Gauge student learning and provide instant feedback

Evaluate overall achievement

(LOs achievement)

26 of 70

vs.

What is the difference?

FORMATIVE

SUMMATIVE

27 of 70

Comparing Formative & Summative

Formative �During teaching & learning. �

  • Helps becoming aware of the focus �
  • Improves quality of students’ learning

Does not:

  • Provide evidence for grading

Summative �At the end of teaching and learning.�

  • Evaluate students’ learning
    • compares against� standards � benchmarks�
  • Provides evidence for grading

recolored exam icon by Freepik

recolored Q&A icon by Freepik

28 of 70

Summative assessment �Methods

  • The most common method
  • Any other types?

29 of 70

Summative assessment �Methods

  1. Questionnaires
  2. Interviews
  3. Observations
  4. Testing
  5. Projects

_For training, the focus is __formative assessment _

30 of 70

Formative assessment�Collect information on their learning progress

  • Goals and objectives
  • Prior knowledge
  • Mental models
  • Frequent mistakes
  • Ability to perform a task

Image from: Ken Whytock, Flickr

31 of 70

Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement (Hattie, 2008)

Reversal

Developmental

Average teacher effects

Zone of enhanced

achievement

1.2

-0.2

-0

0.2

0.4

Treatment: Feedback

d=0.73

32 of 70

Break

10 min

33 of 70

Challenge 4.2�Formative Assessment

Group Activity �⏰ 5 min

Now, discuss how you could collect information on learners’�

    • Goals and objectives
    • Mental models
    • Frequent mistakes

�Write it in the Shared notes

34 of 70

Challenge 4.3

Multiple Choice Questions and mental model

⏰ 3-5 min - individual

Choose each wrong answer and write down on the shares doc which misconception is associated with that wrong answer.

Q: what is 27 + 15 ?

  1. 42
  2. 32
  3. 312
  4. 33

https://carpentries.github.io/instructor-training/02-practice-learning/index.html

35 of 70

Multiple Choice Questions and mental model

The correct answer is 42,

but each of the other answers provides valuable insight.

Q: what is 27 + 15 ?

  1. 42
  2. 32
  3. 312
  4. 33

36 of 70

Formative assessment & feedback can be done in many ways

37 of 70

Tools for Assessment

Google Forms

Mentimeter

Socrative

38 of 70

in-class Assessment approach

Sticky notes

Zoom Reactions

39 of 70

Shared notes

  • Output of group activities
  • Exercises
  • Opinions & Ideas & Comments & Thoughts
  • Questions & Answers

GDocs

Etherpad

40 of 70

Asking questions & discussions

41 of 70

Group activities

  • solve problems in groups
  • do exercises in front of the class
  • asking to describe the strategy they would adopt to solve a problem

42 of 70

_Asking questions & discussions_

43 of 70

_Self assessment_

_Peer evaluation_

44 of 70

Challenge 4.4

How to integrate the result of formative assessment / feedback in your course (5 min)

�Think about this course:

  • How we (instructors) assessed your learning - write it in the Shared notes
  • Choose one of the techniques to collect feedback given in the previous slides, and describe in the shared document how you can integrate the result in your lesson, on the fly

45 of 70

Challenge 4.5

How frequent should formative assessment be? (2 min)

Take some notes for the short review afterwards.�

  • Think about this course: how many feedback opportunities have you had so far?
  • How frequent do you think formative feedback should be?

46 of 70

FORMATIVE

SUMMATIVE

PURPOSE

Gauge student learning and provide feedback

Evaluate overall achievement

TIMING

Throughout the learning process

At the end of an instructional period (unit, semester, year, etc)

FREQUENCY

Frequent and ongoing (daily, weekly, etc)

Infrequent, usually a few times a year

EXAMPLES

  • In-class discussions
  • Weekly quizzes
  • Polls
  • Surveys
  • Bell ringers
  • Exit tickets
  • Homework
  • Group work
  • Chat with students
  • Self-assessments
  • Peer assessments
  • Instructor-created exams
  • Final essays
  • Final presentations
  • Final reports
  • Final projects
  • Portfolios
  • State standardized tests
  • National standardized tests (i.e. ACT/SAT)

47 of 70

Feedback mechanismsTeacher ⇄ Students

48 of 70

Triangle - feedback about teacher’s efforts

Model developed by Alexia Cardona (2021)

49 of 70

Learn and teach others how to give good feedback

50 of 70

Giving good feedback

  • Be specific and constructive
  • Focus on behaviours and outcomes, not personal traits
  • Encourage reflection and action
  • Support shared goals and understanding

51 of 70

Feedback types

FEEDFORWARD

FEEDBACK

FEEDUP

CURRENT LEVEL

TARGET LEVEL

Strategy to reach the target level

Gap between current performance and target level

52 of 70

What and how?

Effective feedback

Conditions

Feedback type:

      • Feed UP
      • Feed BACK
      • Feed FORWARD

Feedback levels:

      • Task level
      • Process level
      • Metacognitive level
      • Personal level

Frequent

Clear goals

Immediate

Measurable goals

Specific

Achievable and challenging goals

Detailed

Quality instructions

What does not work:

Personal level feedback.

Example: you are really smart

What does work:

  1. Task level feedback
  2. Process level feedback
  3. Metacognitive level feedback

53 of 70

Feedback is hard

54 of 70

Dealing with (bad) feedback �(received from trainees)

Humans focus more on negative feedback than on positive

you are not alone!

55 of 70

Dealing with (bad) feedback �(received from trainees)

  • 🌬️ Breathe deeply�
  • 🧐 Try to see the point in the criticism;� Learn from it�
  • 🚷Don’t take it personally �
  • ℹ️ Information is key;� Take time to consider it along other information

56 of 70

Good trainer & Effective training

(Complementing session 3)

57 of 70

Good practices

  • Monitor what is happening to students’ learning
      • Constantly
      • Provide feedback on how to progress

  • Act upon to improve learning�
  • Collect feedback from students’
    • Learning
    • Course structure
    • etc�
  • Investigate when things do not proceed as planned or expected

58 of 70

Other ways of course assessment:

  • Look at
    • Feedback from past sessions
    • Counter examples - contradictions
    • Repetitive patterns �(not at only one single answer)
  • Review what they have effectively learned
      • Formative assessment
      • Summative assessment�
  • Consider your own experience of teaching�(Self-assessment)�
  • Discuss with colleagues and friends

59 of 70

Combining the 4 sessions

60 of 70

Curriculum Design

- Nicholl’s cycle -

Legend

Inspired by: Via A, Palagi PM, Lindvall JM et al. Course design: Considerations for trainers  – a Professional Guide. F1000Research 2020, 9:1377 doi: 10.7490/f1000research.1118395.1

Aligned with LEs?

Support progress towards LOs?

Supports LEs?

Promotes LOs?

LOs achievable by most learners?�Content, LEs & Assessments support LOs?

SMART

LEs aligned with LOs?

LOs

LEs

Content

Assessment

Evaluation

Confirmed

Denied

Proposition to move forward

Phase

Success

61 of 70

8 Learning Principles

  • Principle P1: �Identities and stage of development matter for learning.�
  • Principle P2: �Students' prior knowledge can help or hinder learning.�
  • Principle P3:�How students organize knowledge influences how they learn.�
  • Principle P4: �Students motivation determines, directs and sustains what they learn.

62 of 70

8 Learning Principles

  • Principle P5: �Acquire, integrate and apply new skills to develop mastery�
  • Principle P6: �Enhance quality of learning combining�
  • Principle P7: �Learning is impacted by course climate �
  • Principle P8: �Students must learn to monitor and adjust�their approaches to learning

Goal-directed practice �+Targeted feedback

  • Social
  • Emotional
  • Intellectual

Become self-directed

63 of 70

6 Learning Strategies

Spaced practice

Spreads activities out over time

Interleaving

Switching between topics while teaching

Elaboration

Asking & explaining why and how

Concrete examples

Illustrate abstract concepts with specific examples

Dual coding

Combining words with visuals

Retrieval practice

Activating memory bringing learned information back

64 of 70

7 Evidence Based Learning Principles

  • Principle P1: Students' prior knowledge can help or hinder learning.
  • Principle P2: How students organize knowledge influences how they learn and apply what they know.
  • Principle P3: Students motivation determines, directs and sustains what they do learn.
  • Principle P4: To develop mastery, students must acquire component skills, practice integrating them, and know when to apply what they have learned
  • Principle P5: Goal-directed practice coupled with targeted feedback enhances the quality of students' learning
  • Principle P6: Students' current level of development interacts with the social, emotional, and intellectual climate of the course to impact learning
  • Principle P7: To become self-directed learners, students must learn to monitor and adjust their approaches to learning

65 of 70

Resources to learn more about how learning works

  • Principle P6: Students' current level of development interacts with the social, emotional, and intellectual climate of the course to impact learning

  • Split into Principles P1 and P7Identities and stage of development matter for learning

Learning is impacted by course climate

New

66 of 70

Challenge 4.6

Example

Principle 7: �To become self-directed learners, students must learn to assess the demands of the task, evaluate their own knowledge and skills, plan their approach, monitor their progress, and adjust their strategies as needed.

  • Be explicit in describing the task
  • Have students create their own plan
    • make it the central goal of the assignment
  • Have students do guided self-assessment
  • Guide students into their metacognitive processes

67 of 70

Challenge 4.6

How do you see the 8 Principles of Learning now? (15 min)

Work in 3 groups of 3-4�

Carefully read the principles as a group

Each group will work on 2-3 principles (see Shared Google doc)

Discuss

  • what you would do as a teacher/instructor to facilitate learning according to the principle
  • which learning strategies you can use to facilitate learning

68 of 70

Challenge 4.6

Example

Principle 2:�How students organise knowledge influences how they learn and apply what they know.

  • Possible teaching practices:
    • Teach learners about concept maps
      • How to build
      • How to use
      • how new concepts or facts fits in it
        • Explain it every time a new concept or fact is introduced
        • Reinforce the general overview

69 of 70

Conclusion

  • The first step to become a better trainer is to attend this type of courses, most of all because it means you are asking yourself how to.

70 of 70

Thank you

elixir-europe.org

@elixir-europe.org

/company/elixir-europe

Follow us on social media