Navigating Impact Measurement: A Data-Driven Approach to the SDGs

Impact Measurement Learning Series

Wednesday, 6 November 2024, 4:00 pm CET | 10:00 am EDT | 8:30 pm IST

​

Share in the chat:

​

🙋 Who you are (Name, Organization, Role)

🌍 Where you are logging in from around the world?

🧑‍💻 Which tools are you using to track data currently?

Housekeeping

Please …

🎥 Keep your video on if possible

❓ Show your name & organisation - click on your video, then "rename"

🔇 Unmute when you speak

✋ Technical questions: zoom.us/support or raise your hand ✋

💚 Enjoy!

Moderator

• What comes to your mind when thinking about impact measurement?
• Which tools are you and your organization using to measure impact?

Why Measure Impact?

​

• Tracking progress toward Global Goals

• Attracting funding and partnerships

• Demonstrating transparency and accountability

​

Challenges in Impact Measurement

​

• Complexity of social change

• Data collection and reporting difficulties

• Resource constraints

​

​

​

Case Study - IMPACT NFT Project

​

​

​

Problem: Every 15 days, a native dialect, tradition, or cultural expression is lost, contributing to the erosion of cultural heritage. Indigenous communities, and particularly women within them, often struggle with limited economic opportunities and digital exclusion, making it difficult to preserve their heritage while participating in the global economy. Without intervention, these traditions risk extinction.

Theory of Change: Framework for Impact Measurement

​

A Theory of change provides a structured approach to measuring impact, helping organizations track progress, align efforts, and communicate outcomes effectively. It also ensures transparency and accountability by connecting project outputs to broader outcomes.

​

It serves as a valuable tool to:

• Clarify the Path to Impact.
• Align Activities with Outcomes
• Guide Data Collection for Impact Measurement
• Support Evaluation and Adaptation
• Enhance Transparency and Accountability
• Facilitate Communication of Impact

Case Study - IMPACT NFT Project

​

Case Study - IMPACT NFT Project

​

Case Study - IMPACT NFT Project

TRAINING PROGRAMS

CONDUCTED 20 DIGITAL LITERACY WORKSHOPS

Participants demonstrate increased digital skills.

Indigenous women use digital skills to create and sell art online

Improved economic resilience and cultural preservation among indigenous communities.

INPUT

OUTPUT

OUTCOMES

Access Now defends and extends the digital rights of people and communities at risk.

​

​

​

As a grassroots-to-global organization, we partner with local actors to bring a human rights agenda to the use, development, and governance of digital technologies, and to intervene where technologies adversely impact our human rights. �

​

​

By combining direct technical support, strategic advocacy, grassroots grantmaking, and convenings such as RightsCon, we fight for human rights in the digital age.

• Definition: The direct products or services generated by Access Now’s activities.
• Characteristics: Immediate, measurable, and often tangible actions that support our advocacy or technical support.
• Examples in Access Now’s Work:
• Launching a campaign or advocacy initiative.
• Publishing reports or open letters (e.g., letters to U.S. lawmakers on spyware abuse)
• Providing material or financial assistance to at-risk communities.

1. What is an Output?

• Definition: The changes or benefits that result from Access Now’s outputs, reflected in shifts in policy, protection, or security.
• Characteristics: Observable results that show progress toward digital rights objectives.
• Examples in Access Now’s Work:
• Legislative victories, such as passing data protection laws
• Policy reversals (e.g., Apple reversing client-side scanning)
• Increased safety for at-risk communities (e.g., improved security for Helpline clients)
• The demise or limitation of bad actors (e.g., spyware firms facing legal or financial setbacks)

2. What is an Outcome?

• Definition: The long-term, systemic change Access Now aims to achieve, often influencing broader human rights protections and digital freedom worldwide.
• Characteristics: Lasting effects, often harder to measure directly but visible over time in society and policy shifts.
• Examples in Access Now’s Work:
• Strengthened digital rights frameworks globally
• Significant curbs on abusive spyware use
• Enhanced freedoms and protections for vulnerable communities across multiple regions

3. What is an Impact?

Background: On February 5, 2024, following advocacy by Access Now and partners, the U.S. announced a policy imposing visa restrictions on those involved in or benefiting from commercial spyware misuse.

Outputs:

• Publication of a joint letter urging U.S. officials to sanction spyware companies and responsible individuals.
• Meetings organized by Access Now to influence policymakers directly.
• A press release with recommendations for improving the visa restriction policy.

4. Case Study: U.S. Imposes Visa Sanctions on Global Spyware Abusers

Outcomes:

• New U.S. policy restricting visas for spyware abusers.
• Sanctions and actions against companies like NSO Group and others listed by Access Now.
• Feedback from U.S. policymakers citing Access Now’s influence on their understanding and decisions.

Impact:

• Long-term curtailing of spyware misuse on a global scale.
• Strengthened policy framework to protect against human rights abuses enabled by commercial spyware.
• Progress toward transparency and accountability in the regulation of surveillance technologies.

4. Case Study: U.S. Imposes Visa Sanctions (cont.)

• Outputs: What we produce to drive our mission forward.
• Outcomes: The shifts and protections resulting from our efforts.
• Impact: The ultimate change in the global digital rights landscape we strive to achieve.

All three elements are crucial to understanding Access Now’s effectiveness and long-term vision.

5. Key Takeaways

https://rightscon.org

accessnow.org

@accessnow

@accessnow

info@accessnow.org

Q&A and panel discussion

Breakout room

1. As you speak, introduce yourself in 30 seconds and
2. Give examples of how you measure your organisations impact (outputs and outcomes).
1. Co-create: Help those with challenges with moving from simple outputs to turn them into outcomes/impact.

Catalyst 2030 Impact Report 2022-2024

www.catalyst2030.net

Catalyst 2030 Impact Report & Impact of Member-led Collaborations

Launched at the 2020 World Economic Forum in Davos, Catalyst 2030 has exceeded its initial goals

To catalyse cross-sector collaboration for global systems change through knowledge sharing, peer learning, and effective practices.

Our collective journey

Our reach extends across 140 countries, with 75 country, 7 regional and 2 city chapters with 64% of our members hailing from the Global South. These chapters partner with local communities, businesses, and governments to drive grassroots change. Our councils and working groups enhance diversity, promote collaborative leadership, and foster engagement, impacting billions globally.

Our collective journey

Key Takeaways

Key Takeaways

• Outputs are easier to track and measure (e.g. # of workshops, # attendees etc.)

→ Outcomes are difficult to track. Think behavioural, knowledge, and shifts in conditions.

→ Example outcomes: increased skills in coping with adversity, change in behaviour, number of offenders employed, increased family income post programme engagement, etc.

• Outcomes are the the long term impact you seek to achieve.
• Using the theory of change and drawing your roadmap will make it easier to measure the impact.

• ​

Key Takeaways

• Feedback after programme launch
• Traditional measures: Programme surveys, interviews and focus group discussions to understand what has changed.
• Challenges: We do not know if people are telling the truth. People tend to be positive on survey responses.
• Solution: Identify another type of measurement and see if what is being perceived by people is the actual reality.

​

​

​

• ​

We want to hear your opinion!

Upcoming Impact Measurement session

Social Change Innovators (SCI)

​

​

​

​

​

Catalyst 2030 Medium page

Find out more

https://catalyst2030.net

Follow Catalyst on Social Media

@Catalyst2030

@Catalyst_2030

@Catalyst_2030

Catalyst 2030

Catalyst 2030

Output vs. Outcomes

Impact Measures

Outputs

Outcomes

🡩 number of participants

🡩number of clients served

🡩number of members

• create an app for clients
• host a policy seminar
• launch a social media campaign
• number of products sold

​

​

​

​

​

🡩revenue/budget

🡩staff size

🡩geographic reach of services.

🡫 in unemployment in community

🡩 external investment leveraged.

🡩government funding

🡩government support

Taking an Output → to a Quantifiable Outcome

• Output: Training workshops for 500 teachers on innovative teaching methods. → Outcome: Improved teaching quality resulting in higher national ratings.
• Output: Distribution of 1,000 textbooks to under-resourced students. → Outcome: Increased student literacy rates and performance in standardized exams.
• Output: Launch of an online learning platform with courses on digital skills. → Outcome: Improved digital literacy, leading to increase increase in employment.
• Output: Vaccination campaigns across 50 rural villages, reaching 5,000 people. → Outcome: xx% reduction in preventable diseases among children <five.
• Output: Creation of a mobile health app that offers maternal health advice. → Outcome: Improved maternal health outcomes, % reduced rates of complications.
• Output: Construction of sanitation facilities in 20 underserved communities. → Outcome: Lower % of waterborne diseases and improved overall community health.
• Output: Microloans disbursed to 500 small business owners. → Outcome: Increased household income and local economic growth through the expansion of small enterprises.
• Output: Vocational training programs provided to 200 unemployed youth. → Outcome: Higher employment rates and improved livelihoods for participants.
• Output: Financial literacy workshops conducted for low-income households. → Outcome: Better financial management, leading to increased savings and reduced debt.
• ​

Adapted these GPTchat generated responses

Taking an Output → to a Quantifiable Outcome

• Output: Infrastructure development projects completed in three rural areas. → Outcome: Enhanced market access for local farmers, leading to agricultural income.
• Output: 10,000 trees planted as part of a reforestation project. → Outcome: Restoration of degraded ecosystems and improved biodiversity in the region.
• Output: Installation of solar panels in 100 homes. → Outcome: Increased use of renewable energy, reducing carbon emissions and lowering energy costs for households.
• Output: Environmental awareness campaigns conducted in urban areas. → Outcome: Greater community engagement in recycling programs and % reduced waste generation.
• Output: Clean water access provided to 20 remote communities through water filtration systems. → Outcome: Improved health outcomes and increased access to safe water.
• Output: Legal aid services provided to 200 people facing discrimination or injustice.→ Outcome: Greater access to justice, more cases resolved in favor of marginalized groups.
• Output: Gender equality workshops held in 50 schools and community centers. → Outcome: Increased awareness and reduction of gender-based violence /discrimination.
• Output: Partnerships established with 30 governments to promote inclusive policies. → Outcome: Policy changes that create equitable access for marginalized communities.
• Output: Social media campaign launched to raise awareness about human trafficking. → Outcome: Increased reporting of trafficking incidents, leading to successful rescues.