Scaling Brigade Impact Pilots
Measuring Network Impact
Brigade Congress
October 20, 2019
This deck can be found @
bit.ly/congress_cascades
Who
Are We?
Mary Joyce
Founder
Likes measuring social change outcomes that are hard to measure
(she/her)
Kathryn Shroyer
Sr. Design Advisor
Likes engaging newbies in collaborative design processes
(she/her)
Do Big Good
Social Impact Consultancy
Seattle, WA | #DoBigGood
Social Impact = Benefit - Harm
...in this case, of a civic tech project
Measuring Brigade Impact
14 Pilot
Projects
Why Impact Measurement?
Marie-Aimée Brajeux
Hack for LA: Santa Monica
2019 - 2020 Community Fellow
Session
Overview
Pilot
Reports
Shared Housing
LA Family Housing
Bonnie Wolfe
Owen O’Malley
Hack for LA: Downtown LA
Brigade Volunteers
GetHousing.org
Gregory Johnson
Code for Miami
2019 - 2020 Community Fellow
Re-entry
Roadmap
Jill Bjers
Open Charlotte Brigade
2019 - 2020 Community Fellow
Affordable
Water
Harshita Girase
Code for Buffalo
2019 - 2020 Community Fellow
About Impact
Cascades
Impact Cascade
Template v4.0
An impact cascade is a map of the goals and metrics for each phase of a civic tech project.
Areas in pink are to be filled in by you.
System
Effects
System Change Goal
# of ____ per _____
Actions Time Unit
# of ____ per _____
Actions Time Unit
Tool Design
Design Goal
Anti-Goal (counter-measure)
# of ____ per _____
Actions Time Unit
Tool Use
Use Goal
Anti-Goal (counter- measure)
# of ____ per _____
Actions Time Unit
Individual
Effects
Goal for Individuals
Anti-Goal (counter- measure)
Anti-Goal (counter- measure)
Stake-
holders
Responsible: ____________
Accountable to: ____________
Supporting:
____________
Consulted:
____________
Informed:
____________
Impact Cascade
Template v4.0
First, identify your stakeholders in creating impact. (Help is here.)
System
Effects
System Change Goal
# of ____ per _____
Actions Time Unit
# of ____ per _____
Actions Time Unit
Tool Design
Design Goal
Anti-Goal (counter-measure)
# of ____ per _____
Actions Time Unit
Tool Use
Use Goal
Anti-Goal (counter- measure)
# of ____ per _____
Actions Time Unit
Individual
Effects
Goal for Individuals
Anti-Goal (counter- measure)
Anti-Goal (counter- measure)
Stake-
holders
Responsible: ____________
Accountable to: ____________
Supporting:
____________
Consulted:
____________
Informed:
____________
1.
Impact Cascade
Template v4.0
System
Effects
System Change Goal
# of ____ per _____
Actions Time Unit
# of ____ per _____
Actions Time Unit
Tool Design
Design Goal
Anti-Goal (counter-measure)
# of ____ per _____
Actions Time Unit
Tool Use
Use Goal
Anti-Goal (counter- measure)
# of ____ per _____
Actions Time Unit
Individual
Effects
Goal for Individuals
Anti-Goal (counter- measure)
Anti-Goal (counter- measure)
Stake-
holders
Responsible: ____________
Accountable to: ____________
Supporting:
____________
Consulted:
____________
Informed:
____________
Then, fill in the goals of each phase of your project.
2.
Impact Cascade
Template v4.0
System
Effects
System Change Goal
# of ____ per _____
Actions Time Unit
# of ____ per _____
Actions Time Unit
Tool Design
Design Goal
Anti-Goal (counter-measure)
# of ____ per _____
Actions Time Unit
Tool Use
Use Goal
Anti-Goal (counter- measure)
# of ____ per _____
Actions Time Unit
Individual
Effects
Goal for Individuals
Anti-Goal (counter- measure)
Anti-Goal (counter- measure)
Stake-
holders
Responsible: ____________
Accountable to: ____________
Supporting:
____________
Consulted:
____________
Informed:
____________
Then, fill in the goals of each phase of your project...by starting at the end and working backwards.
2.
or
Impact Cascade
Template v4.0
System
Effects
System Change Goal
# of ____ per _____
Actions Time Unit
# of ____ per _____
Actions Time Unit
Tool Design
Design Goal
Anti-Goal (counter-measure)
# of ____ per _____
Actions Time Unit
Tool Use
Use Goal
Anti-Goal (counter- measure)
# of ____ per _____
Actions Time Unit
Individual
Effects
Goal for Individuals
Anti-Goal (counter- measure)
Anti-Goal (counter- measure)
Stake-
holders
Responsible: ____________
Accountable to: ____________
Supporting:
____________
Consulted:
____________
Informed:
____________
Consider unintended harms (anti-goals) at each phase and counter-measures to prevent them.
3.
Impact Cascade
Template v4.0
System
Effects
System Change Goal
# of ____ per _____
Actions Time Unit
# of ____ per _____
Actions Time Unit
Tool Design
Design Goal
Anti-Goal (counter-measure)
# of ____ per _____
Actions Time Unit
Tool Use
Use Goal
Anti-Goal (counter- measure)
# of ____ per _____
Actions Time Unit
Individual
Effects
Goal for Individuals
Anti-Goal (counter- measure)
Anti-Goal (counter- measure)
Stake-
holders
Responsible: ____________
Accountable to: ____________
Supporting:
____________
Consulted:
____________
Informed:
____________
Finally, come up with a metric to measure progress toward your goal at each phase. (Help is here.)
4.
Impact Cascade
Template v4.0
System
Effects
System Change Goal
# of ____ per _____
Actions Time Unit
# of ____ per _____
Actions Time Unit
Tool Design
Design Goal
Anti-Goal (counter-measure)
# of ____ per _____
Actions Time Unit
Tool Use
Use Goal
Anti-Goal (counter- measure)
# of ____ per _____
Actions Time Unit
Individual
Effects
Goal for Individuals
Anti-Goal (counter- measure)
Anti-Goal (counter- measure)
Stake-
holders
Responsible: ____________
Accountable to: ____________
Supporting:
____________
Consulted:
____________
Informed:
____________
And determine when and how often you will collect metric data and where you will put it.
5.
Impact Cascade
Template v4.0
System
Effects
System Change Goal
# of ____ per _____
Actions Time Unit
# of ____ per _____
Actions Time Unit
Tool Design
Design Goal
Anti-Goal (counter-measure)
# of ____ per _____
Actions Time Unit
Tool Use
Use Goal
Anti-Goal (counter- measure)
# of ____ per _____
Actions Time Unit
Individual
Effects
Goal for Individuals
Anti-Goal (counter- measure)
Anti-Goal (counter- measure)
Stake-
holders
Responsible: ____________
Accountable to: ____________
Supporting:
____________
Consulted:
____________
Informed:
____________
And determine when and
how often you will collect metric data and where you will put it. (There’s a Google Spreadsheet for that.)
5.
Impact Cascade
Deep Dive
✱ This example uses an older version of the cascade.
Charlotte
Impact Cascade
Target Tool Effect: Reducing re-entry anxiety through design and launch Re-entry Roadmap
- Total people giving input:
- Total partners signed on for tool creation and adoption:
- Community members doing user testing:
- Total number of user tests completed:
- Total resources provided in tool:
- Page in process per user
- Total pages visits
- Total completions (print result)
Tool Use
Measuring User Problem Areas and Areas of Interest
Tool Creation
Measuring Community Participation
Beneficiary Effect
Measuring Change in Re-Entry Anxiety
[Try to use same scoring system as Indianapolis]
Data Collection
Start Date: June 26, 2019
Data Collection
Start Date:
Data Collection
Start Date:
Structural Effect
Measuring Institutional Adoption
- Number of re-entry centers where tool is used
Charlotte Data Repository
Now You
Try!
This Deck: bit.ly/congress_cascades
Beginning a
Cascade
Wrap-Up
Feedback
Challenge:
Write what did not work well for you about the cascade here.
Side 2
Success:
Write what worked well for you about the cascade here.
Side 1
Create An
Impact Cascade
Use This Separate Deck: bit.ly/post_cong_cascadess
Appendix
Of Help
Metric Identification Help
Our goal is that ___ receives the benefit of ____________.
We’ll know this goal happened when we see________.
To express this action as a metric, we write:
# of ____ per _____
actions time unit
Phase Goal Action Metric
Stakeholder Identification Help
Type | Description | Examples |
Responsible | Who will make final decisions on cascade content and ensure that data is collected? | Project Lead, Product Manager, Project Manager, |
Accountable To | Who are we trying to benefit? | Individual Beneficiary, Institution, Community Partner, Client, User |
Support | Who will help carrying out cascade tasks (e.g. data collection)? | Project volunteers, |
Consulted | Whose opinion should we ask, though they have no decision-making role? | Community Partner, Client, User |
Informed | Who will receive impact information, but have no decision-making role? | CfA Brigade Team, Brigade Members |
More Cascade Examples