Measuring the Business Impact

of Open Source & OSPOs

​

The Study of OSPOs

​

The Study of OSPOs

​

Follow-up: Measuring the Business Impact of Open Source & OSPOs

https://github.com/todogroup/ospology/discussions/63

​

​

​

​

​

The Study of OSPOs

​

> Introduction (use the chat and say hi!)

> OSPO & OS Management News (10 mins)

> Topic of the day (20 mins)

> Group Discussion (15 mins)

​

​

⏱ Full OSPOlogy session: 45 mins

The Study of OSPOs

​

​

🙋 New OSPO Discussions

​

​

​

• How OSPOs are dealing with Log4j vulnerability issue?
• Tools for Detecting Open source code snippets
• How OSPOs do metrics on public repos to share upwards

​

​

​

​

​

​

​

The Study of OSPOs

​

https://landscape.todogroup.org/

​

✅ Join OSPONews!

The Study of OSPOs

​

​

Welcoming New Contributors 👋

The Study of OSPOs

​

Contribute to upcoming OSPOlogy sessions!

• Present WIP or finished projects being done at your OSPO or open source initiative
• Share OSPO journey history behind your OSPO
• Share OSPO or Open Source Management tooling that help the main activities of an OSPO
• Present research studies and/ or whitepapers to the community
• Present a proposal for a potential research or whitepaper

More info: https://github.com/todogroup/ospology/tree/main/meetings#what-are-ospology-monthly-meetings

​

​

​

​

​

​

​

The Study of OSPOs

​

​

New Contributing Guidelines

​

​

​

​

​

​

​

​

📝 Work on new or existing TODO Guides

​

🧩 Propose a new OSPO Case Study

​

📚 Work with open source peers on Whitepapers

​

🙋 Submit OSPOlogy CFP

​

​

​

​

​

The Study of OSPOs

​

🌄 Topic of the month 🌄

The Study of OSPOs

​

Measuring the Business Impact of Open Source + OSPOs

@amcasari

open source researcher + engineer

google_logo Open Source

bit.ly/ospology-2022-business-impact

google_logo Open Source

hello, i'm amanda casari

Open Source Scientist @ Google Open Source Programs Office

Google lead for Project OCEAN

External Faculty @ Vermont Complex Systems Center

Advisor for Open Source Community Africa

Co-author with Alice Zheng, “Feature Engineering for Machine Learning” [O’Reilly, 2018]

♥️ Mama Mama + US Navy Veteran

google_logo Open Source

@amcasari

What is open source?

The open source definition standardized by the Open Source Initiative focuses mostly on how software is distributed and licensed.

google_logo Open Source

@amcasari

"Open" is about more than just licenses + code.

The "open" in open source is also a collective (but not always standardized) framework that shapes how software is created, released, shared, and distributed

— as well as the community and how it forms around the projects.

google_logo Open Source

@amcasari

When open source becomes essential for your organization, you need new tools.

google_logo Open Source

@amcasari

Organizations need common maps to navigate open source.

Does everyone in your organization:

• know what licenses they can use to create new products?
• know if you have policies on releasing what they have created?
• feel comfortable with what software they can use to create something new?
• have someone to ask "open" questions to?

google_logo Open Source

@amcasari

Welcome, open source programs offices

google_logo Open Source

@amcasari

OSPOs come in many shapes and sizes.

They may focus on:

• navigating open source licenses
• funding open source projects your organization relies on
• focusing on internal compliance
• developing and documenting organizational best practices
• synthesizing decentralized efforts into scalable programs
• tracking organizational investments in open source ecosystems
• <and many, many more!>

google_logo Open Source

@amcasari

opensource.google

Bringing all the value of open source to Google and

all the resources of Google to open source.

How do we know which "all" to focus on and for whom?

google_logo Open Source

@amcasari

With great open source programs there must also come

great program management.

google_logo Open Source

@amcasari

Questions we ask about our open source programs

1. Why are we doing this?
2. Who are we doing this for?
3. Who are the stakeholders?
4. What is the impact on our business?

google_logo Open Source

@amcasari

You need to know why you are doing this.

• Why were you created?
• What are the problems you are scoped to focus on?
• What is your current mission?
• How do you measure success of your organization?
• Who do you regularly report to?
• Who funds your organization and collective efforts?

google_logo Open Source

@amcasari

You need to know who are you doing this for.

• Who does this program include?
• Who does this program leave out?
• Who benefits from this?
• Who may be impacted by this program?
• What other work would this program touch or change?

google_logo Open Source

@amcasari

You need to know your stakeholders.

How does your organization define "stakeholder"?

A stakeholder can be:

• A group without whose support the organization would cease to exist
• Any person or organization that has a legitimate interest in your organization or projects
• Any person who has an interest ("stake") or concern in your organization
• Any person or organization who can affect or be affected by your organization's actions, objectives, and policies.

Ref: Wikipedia

google_logo Open Source

@amcasari

Identifying your organization's stakeholders for open source

Have you considered someone who would be a stakeholder:

• by position (e.g. your manager, the Engineering Director, CTO)
• by role (e.g. Product Managers, open source contributors at my company)
• by team (e.g. Legal, Privacy, Marketing)
• by organization (e.g. open source organizations you sponsor, projects you are dependent upon)
• by other business partnerships (e.g. developer platform partners, foundation partners)

google_logo Open Source

@amcasari

Openly communicate with these stakeholders across your organization.

Formally documenting, regularly reviewing, and sharing your stakeholders helps to define and clarify:

• expectations
• needs
• shared outcomes
• potential future work

google_logo Open Source

@amcasari

Avoid estimating ROI for your open source efforts.

• Return on Investment is a very specific economic model
• Created as a way to relate profits to capital invested
• Generally used when economic modeling is simplified to monetary equivalents
• Many investments in open source do not readily simplify to standardized monetary equivalents
• Code reviews for strategic open source projects
• Board memberships for standards bodies
• Employees volunteering as mentors + sponsors in their communities

google_logo Open Source

@amcasari

Focus on the business impact of your programs.

• Identify + track your investments
• Project funding
• Event sponsorships
• Employee contributor programs
• Identify + track these impacts
• How do you + your stakeholders track success?

We've shifted to an input-output model of "Investment : Impact"

google_logo Open Source

@amcasari

You can demonstrate business impact with useful metrics and KPIs.

google_logo Open Source

@amcasari

Metrics are measurements.

Measurements are recorded observations — in and of themselves, there is no judgement we should make.

google_logo Open Source

@amcasari

Observations are NOT unbiased.

• What you choose to observe, the lens you use to observe it, what you choose to leave out — all of these decisions bias the outcomes, regardless of your intention.
• You must record design decisions and be clear as to your motives for observing and measuring.

google_logo Open Source

@amcasari

Think of a scientific experiment.

• When you have a question, you want observations to understand what is happening.
• Collecting data from your observations paints a picture/tells a story/gives insight into what is happening.

google_logo Open Source

@amcasari

Focus on useful metrics.

Identify objectives

Define your desired trend

Consider the processes, outcomes, and feedbacks.

Determine signal phenomena

More on this: bit.ly/foss-backstage-pram

google_logo Open Source

@amcasari

Use case: Measuring Alphabet's contributions to open source

• Author + project lead: Sophia Vargas
• Annual analysis examining Google's contributions to open source
• Centered on Google Open Source's mission - "resources of Google for open source."
• Explicitly outlines data samples + evolving methodology
• Quantitative + qualitative methods
• More at: bit.ly/google-opensource-2020-metrics

google_logo Open Source

@amcasari

KPIs are a specific kind of metric — they measure the pulse points that correlate to a process, program, or product being considered to be "successful".

google_logo Open Source

@amcasari

You need to define what success is, or looks like, for you and all of your stakeholders to create useful KPIs.

google_logo Open Source

@amcasari

What defines success for you?

• How do you introduce people to your work?
• Have you written a mission statement?
• How do you talk about your team to other people in your organization?
• How do other people introduce your team's work?
• What do you point back to when you want to show The Why?

google_logo Open Source

@amcasari

What defines success for your stakeholders?

• For each stakeholder, map out what they care about to identify additional metrics which will help you explain your vision of success
• How a stakeholder defines success may be listed in their —
• Organizational/team mission or vision
• Business goals (such as OKRs or quarterly goals)
• Shared roadmaps
• Shared dashboards (what do they track and why?)

google_logo Open Source

@amcasari

How often and when to measure KPIs depends on how quickly things change that you are measuring, not how often someone wants a report on them.

google_logo Open Source

@amcasari

Considerations for how + when to measure KPIs include:

• Aggregation buckets
• how to group or compare metrics
• Collection cycles
• when to observe and how often
• Reporting cycles
• when to make public your observations
• Privacy + security risks

google_logo Open Source

@amcasari

Use case: Defining KPIs for open source events

• Project Team: Amanda Casari, Kassandra Dhillon, Teresa Terasaki, María Cruz
• Collaborative effort to improve success metrics for all Google Open Source events
• Multiple programs support different kinds of events
• Identified overlapping stakeholders + success metrics
• Clarified individual program missions to define program- specific KPIs

google_logo Open Source

@amcasari

We are all on this open source raft together.

• Open source is more than code or a specific license.
• Managing open source in organizations requires good program management practices.
• Avoid an "ROI" model to show the value of your investments.
• Metrics + KPIs are a reflection of what matters to you — focus on useful ones.

google_logo Open Source

@amcasari

Thank you!

Slides (PDF) + Transcript: bit.ly/ospology-2022-business-impact

google_logo Open Source

The Study of OSPOs

​

FYI: Group discussion is not recorded and is held under Chatham House Rule

​

​

https://community.linuxfoundation.org/events/details/lfhq-todo-group-presents-ospology-the-state-of-ospos-2021/

​

https://github.com/todogroup/ospology/tree/main/meetings

https://github.com/todogroup/ospology/discussions/50

​

The Study of OSPOs

​

Group discussion is not recorded and is held under Chatham House Rule

​