1 of 21

Matching Outcomes

to Indicators

Module 8

1

2 of 21

Introduction

2

3 of 21

By the end of this module, you will be able to...

Choose appropriate indicators for given outcomes.

Align outcomes to possible indicators.

Understand the extent to which outcomes are achieved.

3

4 of 21

What is an indicator?

Criteria for measuring if/how your targeted outcomes are being achieved

4

Outcomes:

What learning outcomes are you trying to achieve?

Goals:

What aspects of learning does your project address?

Indicators:

What measures will show progress towards your goals?

CAISE 2011

5 of 21

What Does a Good Indicator Do?

A good indicator:

Aligns with program goals and outcome
Reflects the reason why you are doing the evaluation

5

6 of 21

Good Indicators should:

Align to the outcomes�
Be measurable�
Be unbiased�
Sensitive to change

6

7 of 21

Align to outcomes

7

Outcome:

Gaining knowledge about the local ecosystem.

Indicator:

Measuring knowledge gained about the local ecosystem.

8 of 21

Be Measurable

Surveys
Pre/post tests
Interviews
Polls
Questionnaires
Observations

8

9 of 21

Be Unbiased

9

Biased	Neutral
How much did you enjoy your class today?	Share with us your class experience today.
You were satisfied with our exhibit, weren’t you?	How would you rate the exhibit?
Do you always come to the museum?	How often do you visit the museum?

10 of 21

Be Sensitive to Change

Target a specific behavior or attitude�
Measure the level or amount of change

10

11 of 21

Mapping Outcomes to Indicators

11

Outcome	Indicator
Visitors will increase their knowledge about local plants	Visitors will indicate that they learned about local plants
Visitors will increase their willingness to engage in public or private action	Visitors indicate interest and willingness to engage in personal or private action

So, what does this look like in practice? �

We suggest making a t-chart like this one with your outcomes on the left side and the indicators on the right. This will help to make sure that you are matching outcomes to specific indicators.

In this example, the outcome is that the visitors will increase their knowledge about local plants. One possible indicator would be to ask visitors how much (or if) they learned about the local plants. If they report that they had learned about the plants in the local ecosystem, then you have evidence for success!

Or, if you hope that the program inspires participants to take action on a topic, your indicator would measure if, or to what extent, people say their interest and willingness to take action increased after the program.

Since it would be difficult to measure a future action, you would measure people’s INTENT for that action. Notice the direct connection between the indicator and the targeted program outcome.

12 of 21

Activity

How could we “measure” success for each of these outcomes?

12

Outcome	Indicator
Visitors will increase their comfort in nature
Visitors will increase their interest in STEM related careers
Visitors will take part in and promote citizen science projects

13 of 21

Activity

How could we “measure” success for each of these outcomes?

13

Outcome	Indicator
Visitors will increase their comfort in nature	Visitors will indicate (pre/post survey or interview) that they feel more comfortable in nature.
Visitors will increase their interest in STEM related careers	Visitors will indicate (Pre/post survey or interview) that they are more interested in STEM related careers
Visitors will take part in and promote citizen science projects	Visitors sign up or or indicate (survey or interview) that they will engage in and tell a friend about upcoming citizen science projects

14 of 21

What else should I consider?

Your target audience : Which visitors? All or some? Ages?

Data collection method: Surveys, interviews, pre/post tests, focus groups?

Timeline: How long will it take to complete?

Resources: Who will collect the data? What materials are needed?

14

When you are developing indicators, there are a few other things that are important to consider in terms of practicality:

Who is your target audience? When choosing indicators, you may want to consider who the participants are. Are they young? Do you need to consider the reading level of your indicators? Do you need to assess every participant or just a few?

How will you collect the data to provide the evidence you need? In the next module, we will discuss different data collection methods. Think about if your indicator would make sense as a survey or an interview question.

How long will it take you to collect these data? We have to be reasonable about the amount of time we have to do this. How much data do we really need?

What resources do you need to do this? Another thing to consider are the resources you need to measure this indicator. Do you need people or special materials to help you?

You may have a great idea to collect data, but not enough staff or access to materials, or access to participants to do it. So you may need to rethink your ideas so it is more practical for your situation but still can provide evidence to measure the success of your program.

15 of 21

Wrap up/Highlights

Indicators provide a way to measure and give evidence for your goals.

Good indicators match the intent of the outcome.

It is important to consider:

Who your audience is

Data collection method

What kind of data you need

What resources you need

15

16 of 21

Conclusion

Thinking about what you have learned in this module,

How might you create indicators

for your evaluation project?

16

17 of 21

For further information

Center for Advancement of Informal Science Education (CASIE). (2011). Principal Investigator's Guide: Managing Evaluation in Informal STEM Education Projects. [online] Available online at: https://www.informalscience.org/principal-investigators-guide-managing-evaluation-informal-stem-education-projects-pi-guide

17

18 of 21

In the next module you will be able to…

Describe the difference between quantitative and qualitative data

Identify a data collection plan for your project

18

19 of 21

Stay connected

Visit us: https://research.ced.ncsu.edu/impactnc/

Dr. K.C. Busch

Assistant Professor of STEM Education

North Carolina State University | College of Education

kbusch@ncsu.edu

19

20 of 21

Credits

K.C. Busch

Lynn Chesnut

Regina Ayala Chávez

Aimee Fraulo

Kathryn T. Stevenson

Katy May

Lincoln Larson

Madeline Hinckle

20

21 of 21

21