The Apache Way and collaborative development

Ismaël Mejía. @iemejia

​

​

Who am I ?

​

@iemejia�Software Engineer�Apache Beam PMC / Committer�Apache Software Foundation (ASF) member�Open and free knowledge enthusiast

2

Disclaimer

​

Very often, the Apache Software Foundation does not have an official view on things - we are a group of individuals with varying opinions.

​

What follows is my own personal view.

​

​

​

3

Open-source software

​

​

​

​

​

​

​

​

​

​

​

​

Open source success is not only about the source

4

Software is easy. People are HARD

It is easier than ever today to create and collaborate online.

'Free' infrastructure: github, gmail, google groups, � google docs, skype, slack, IRC, etc.

​

but, the health of an open source project is not only about�technical requirements or the source:

​

• Users: Experience, Documentation, Feedback
• Developers: Maintainers (Bus factor)
• Policies / Project Governance / Decision Making
• Documentation
• Legal matters: IP, Corporate Influence, branding
• Awareness: Communication / Marketing

​

5

What would you expect from a project where...

• Multiple people must agree to take important decisions
• People work from different parts in the world in different languages and with independent agendas
• We deal with people with really different level of knowledge and background (professionals and beginners)
• Institutions involved have different levels of resources / funding
• Everything can be viewed, judged and modified by others
• Somebody can engage in a task and just disappear

6

7

Not necesarily…

​

The Apache Software Foundation (ASF)

​

​

​

​

​

​

�19 years building open source communities

8

The Apache Software Foundation (ASF)

• Non-profit corporation
• 501(c)3 charity (USA)
• Volunteer organization (member-based)
• Virtual world-wide organization
• Exists to provide organizational, legal and financial support for various OSS projects

9

The ASF is the open source software foundation with the largest number of projects and contributors.

ASF origin

• Apache group
• Informal corporate structure
• 8 members
• Resumed the work on NCSA httpd in Feb 1995 (“a patchy” server)
• Chose permissive licensing

​

10

Mission

​

Provide open source software to public free of charge

​

11

Evolution of the ASF

Before

• Incorporated in 1999
• 21 members
• 2 projects
• All servers and services donated

​

​

​

​

Now

• ~730 members
• ~320 projects (190 TLPs)
• ~6700 committers
• ~8000 signed ICLAs

Large and growing infrastructure

12

Some Apache projects

13

More Apache projects

14

ASF growth - committers

15

ASF growth - code

16

How to achieve the mission?

Provide open source software to public free of charge

�Let the coders code - foundation�deals with the rest

​

​

Infrastructure for open source development

Legal entity for donations, intellectual property (IP) and branding

Project governance and collaboration guidance

17

Infrastructure

ASF infrastructure is composed mainly by the following services:

• Mailing lists
• Web servers and wikis
• Source code repositories
• Issue / bug tracking
• Communication tools
• Distribution system (mirrors)

18

Legal

• Legal entity for donation purposes (charity)
• Apache license based copyright agreement (includes clauses to protect from patent litigation)
• The project intellectual property (IP) is donated to the foundation which protects it against lawsuits, patent trolls, etc
• Apache branding protection

​

19

Understanding governance and open collaboration following the life of a project in the ASF

​

20

ASF structure

21

Project Incubation

• Entrypoint for all projects.
• A project in incubation is called a podling
• A committee of ASF members mentors the podling (PPMC)
• Goals of incubation:
• Ensure and track Intellectual Property
• Learn about technical tools and procedures
• Learn the Apache Way and open collaborative development (ASF principles)
• Create first releases
• A project with a healthy community gets promoted into a Top Level Project (TLP)

​

Note: Multiple projects working on a similar domain may be ok (not ideal but this evolves naturally).

22

ASF principles

​

Meritocracy

The Apache Way

Open collaboration

Community over code

23

Meritocracy

• Govern by merit
• Merit is based on what you do
• Merit never expires
• Those with merit get privileges but more responsibilities too
• Merit does not buy you authority
• Any constructive contribution earns merit
• Not just code (bug reports, new feature ideas, documentation, evangelism, …)

​

24

The Apache Way

The Zen of Apache

Charity: Apache’s mission is providing software for the public good.

Community: Many of us are more effective than all of us.

Consensus: Getting good enough consensus is often better than voting.

Merit: Those that have proven they can do, get to do more.

Open: Technical decisions are discussed, decided, and available publicly.

Pragmatic: Apache projects use the broadly permissive Apache license.

​

​

25

* This is based in Shane Curcuru's interpretation of The Apache Way

Open collaboration

• Making open source is not equal to open development
• Open communication matters
• "If it didn't happen in the mailing list, it didn't happen"
• Strong incentives to design and develop in the open
• Peer based contribution and review
• Shared vision
• Self-service archives

26

Community over code

• Contributors represent themselves - individuals
• Trust and mutual respect
• Every vote has the same value
• A healthy community produces healthy code
• A healthy community is “warm & inviting” and encourages a continued influx of developers
• Final result: better code and better project in the long term

​

27

Code of conduct

1. Be open.

2. Be empathetic, welcoming, friendly, and patient.

3. Be collaborative.

4. Be inquisitive.

5. Be careful in the words that we choose.

6. Be concise.

7. Step down considerately.

​

​

​

​

28

Apache welcomes and encourages participation by everyone. We are committed to being a community that everyone feels good about joining.

Project normal lifecycle

• Iterative development
• Discussion on Mailing lists, issue tracking
• Other tools (code review, version control systems, continuous integration, etc)
• Commit Then Review (CTR) vs Review Then Commit (RTC)
• Each project is free to choose its best practices and technical direction

​

​

​

Key point: Make governance only important for what it should be and to those who care

​

29

Vote and reaching consensus

• Goal: The best ideas win and we agree on most of them
• Coexistence rules (from experience and discussion)
• Vote to reach consensus on new procedures, releases, important changes
• Votes on releases (72h window). PMC votes are binding
• Vote process. Express opinions and reach consensus
• +1
• -1 requires a clear explanation of why
• [-1,1] different levels of agreement:
• +0: 'I don't feel strongly about it, but I'm okay with this.'
• +0.5: 'I agree but I am not totally enthusiast.'
• -0: 'I won't get in the way, but I'd rather we didn't do this.'

​

30

ASF organization

31

Project

Foundation/Administrative

PMC Members

Committers

Contributors

Users

Members

Officers

Board

Project�People

report quarterly

elect

delegates

Project Management Committee (PMC)

• Project Management Committee (PMC)
• Vote for new PMC members and committers
• Vote to release new versions
• Establish project specific rules and report to the board
• Oversees legal compliance and community health
• Each PMC has a chair (appointed by the board) - eyes and ears of the board (oversight only)

32

PMC governance

• Projects submit quarterly reports to the Board
• Oversight focuses on avoiding anti-patterns
• “Umbrella” projects
• Undue commercial influence
• Stagnating community
• ...and encouraging good things
• Adding new committers and PMC members
• Cutting releases
• NOT concerned with technical issues

33

Projects fail or end and that's ok

• Contributors change interests/priorities
• Companies stop funding and no other people cares.
• Projects do 'finish' or are considered 'complete'.
• Apache projects get retired into the Attic for archive purposes.

​

34

Board and ASF Members

• Remember only volunteers
• Member-based corporation - individuals only
• Members nominate and elect new members
• Members elect the board - 9 seats
• Manage and oversees the corporation actives and resources
• Board may intervene to resolve community conflicts
• Annual online meeting
• The board appoints 10 officers
• Oversee day-by-day affairs, e.g. legal, fundraising, marketing, infrastructure, etc.

35

Some key lessons on community and software building the Apache way

Organisations can try and promote some of the Apache practices:

• Light governance: facilitator role towards objectives.
• Open communication: with archives.
• Decentralization and participative consensus.
• Meritocracy brings engagement.
• Recognition is not only nice, it brings engagement.
• A good ecosystem promotes naturally cross-project collaboration

​

36

Thriving to create living systems

​

​

A "Living System" is one that grows into its environment, by self-organizing around opportunities. Living systems can last for a long time, adapt well to change, and thus be highly successful. By contrast, "Planned Systems" tend to be fragile, poor at coping with change, and thus short-lived.*

37

* Social architecture by Pieter Hintjens

Learn more and participate

The Apache Software Foundation (ASF)

The ASF blogs - ASF FAQ

Apache Community Development - Mentor program

@TheASF on Twitter (or Facebook or LinkedIn)

​

If you have doubts or are just curious, please contact me. I will be around, or send me an email or twitter message.

Thanks!

39