Speaker(s)

Evaluation & Assessment

Speaker(s)

Assessment vs. Evaluation

What does it mean?

• Assessment:
• Ongoing
• Collection of data
• Evaluation:
• “To judge the value”
• Analysis of data

​

Speaker(s)

Assessment vs. Evaluation

Individual Learners

• Assessment of individual learning
• Pre-class surveys
• Knowledge checks
• K-W-L Charts
• Evaluation of effectiveness of training
• Final Exams
• Performance indicators

​

Instructional Design

• Assessment of audience and objectives
• Entry and exit surveys
• Final Exam scores
• Performance indicators
• Evaluation of effectiveness of training
• Results – Behavior – Learning – Reaction

Speaker(s)

WHO/WHAT ARE WE EVALUATING?

Speaker(s)

Group Discussion

Who and What do we evaluate?

Speaker(s)

Assessment

Data Gathering

Speaker(s)

Types of Classroom Assessment

Formative

• Pre-Assessment
• Knowledge Checks
• Often ungraded

Summative

• End of Program (typically)
• Usually Graded
• “Final Exams”

Diagnostic: Polling, Self-assessments, discussions, observations

Speaker(s)

What are “Authentic” Assessments?

“a(n) … assessment task that involves the student deeply, both in terms of cognitive complexity and intrinsic interest, and are meant to develop or evaluate skills and abilities that have value beyond the assessment itself.”

Frey, Bruce B., Schmitt, Vicki L., & Justin P. Allen (2012). Defining Authentic Classroom Assessment. Practical

Assessment, Research & Evaluation, 17(2). Available online: http://pareonline.net/getvn.asp?v=17&n=2

Speaker(s)

Traditional vs. Authentic Summative Assessments

Traditional:

• Requires “Correctness”
• Unknown prior to assessment date
• Disconnected
• Knowledge is measured in an isolated way
• “One Shot”
• Scored in, an often, numerical way

Authentic:

• Require a “high quality” product
• Should be known prior to assessment
• Tied to “real-world” contexts
• Knowledge must be used in coordination
• Contain recurring or ongoing tasks
• Scored in a diagnostic way

https://citl.indiana.edu/teaching-resources/assessing-student-learning/authentic-assessment/

Speaker(s)

Group Discussion

What are some examples of Authentic Assessment you have utilized? What are the pros and cons of Authentic Assessments, in your experience/opinion?

Speaker(s)

Evaluation

Data Utilization

Speaker(s)

KIRKPATRICK’S MODEL

https://www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model

Speaker(s)

The Basic Model:

Level 1: Reaction

• The degree to which participants find the training favorable, engaging and relevant to their jobs

Level 2: Learning

• The degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training

Level 3: Behavior

• The degree to which participants apply what they learned during training when they are back on the job

Level 4: Results

• The degree to which targeted outcomes occur as a result of the training and the support and accountability package

​

https://www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model

Speaker(s)

HOW CAN WE EVALUATE THAT??

This Photo by Unknown Author is licensed under CC BY-SA

Speaker(s)

Speaker(s)

A few more ideas from the field…

Level Three:

• Observation: 360 Self/Peer Evaluations
• Alumni Groups
• Post-training networking
• Involve past learners in training evaluation

Level Four:

• Observation: 360 Supervisor Evaluations
• Include measurable learning objectives
• Data gathering: Audit records
• Big Picture post-training surveys

Speaker(s)

Group Activity

Case Study Activity

Speaker(s)

Instructions:

Develop an initial program/course evaluation plan given the data in your case study. Answer the following questions with your plan:

1. What is the problem or perceived problem with the training program?
2. What assessments have been utilized?
3. How will you evaluate the program based on the data you’ve been given?
4. What questions will you be attempting to answer with your evaluation?
5. What changes would you see, ideally, if your evaluation and resulting revisions are successful?

Speaker(s)

Low performance on assessment; �high experience level

Case Study #1

Speaker(s)

High performance on assessment; reporting “useless” training

Case Study #2

Speaker(s)

Few issues in assessment; needed change not being seen

​

​

Case Study #3

Speaker(s)

Training viewed as “not relevant” to target audience- �low motivation

Case Study #4

Speaker(s)

Training giving short term positive effect with little lasting effect on behavior

Case Study #5

Speaker(s)