Gerrit Community Survey

Gerrit Team at Google

​

​

Jan 2020

Results from H2-2019

Research objectives

​

Understand key properties of Gerrit use in the community:

• which version they use,
• the size of the installation,
• plugin use,
• relevant integrations,
• overall satisfaction,
• pain points, and
• attitudes towards contributing back to the community.

​

​

Study design

Survey

​

Invited from two sources:

• People registered for the two Gerrit User Summits in Gothenburg and Sunnyvale (a total of 127 people across 30 organisations).
• Afterwards, invited all subscribers of the repo-discuss list (4,210 people) to participate.

Asked respondents to only fill one survey per organisation.

N=50 participants/organisations total

​

• 9 from the Gothenburg invitation, 10 from the Sunnyvale invitation and 31 after inviting folks from repo-discuss.

​

Caveat: We do not have a good idea of how many organisations are using Gerrit overall, so we cannot know to what extend the overall external user base is represented in these results.

Method

Participants

Results

Gerrit Use

• Almost half of respondent organisations have been using Gerrit for 1 - 5 years
• They chose Gerrit over GitHub/Gitlab after those products became fairly popular.

​

• Only 2 organisations started using Gerrit within the last year.
• Is growth slowing? Do these organisations not know about the community yet?
• Almost half of organisations are long-term Gerrit users.
• They are likely Gerrit experts by now.

Gerrit Versions

• 6 organisations use multiple versions at the same time (purpose unclear)
• 15/50 organizations use the latest release (3.0.x), 30/50 use a supported 2.x release (from 2.15.x)
• Backporting bug-fixes is important for the community.
• Only 15/50 orgs use NoteDb to some extent and another 21 possibly do so (having a 2.16.X release)

Hosts, Repositories, Users

62 %�of organisations have more than one Gerrit host.

20 %�of organisations have more than 10 Gerrit hosts.

25 %�of organisations have more than 5,000 repos on largest Gerrit host.

> 66.5k�Gerrit-hosted repositories on external hosts.

57 %�of organisations have less than 500 Gerrit users.

16 %�of organisations have more than 5,000 Gerrit users.

> 46.5k�Gerrit users �on external hosts.

56 %�of organisations have at most one full-time Gerrit admin.

*

*

* These are very conservative lower bounds derived from the bucketed answers in the survey, simply adding up the lower end of the responses participants chose (i.e. counting 50 when respondent chose 50-500 repositories).

Integrations, Code Review

94 %�of organisations report having some form of CI/CD integration.

80 %�of organisations report having some form of issue tracker integration.

63 %�of organisations report having mandatory code review.

33 %�of organisations report deciding whether or not to require code review per project.

• Gerrit core has only very basic issue tracker integration. Maybe we need better support for this?
• Gerrit is used for actual review as opposed to git hosting in most cases
• Having mandatory code review in many orgs means code review needs to be fast, otherwise it can become a productivity blocker for many people

Plugin Adoption

Plugin Adoption

Top plugins are all core plugins

Plugin Adoption

34% of orgs report using custom plugins

→ Making these easier to create is likely important

Plugin Adoption

80% of orgs report issue tracker integration, but only 28% report using its-* plugins.

92 %�of respondents report being somewhat or very satisfied with Gerrit overall.*��* this is a biased number: unsatisfied people are less likely to continue to use Gerrit and may also be less likely to respond to the survey.

4 % �of organisations report feel somewhat dissatisfied with Gerrit overall.

Reasons for feeling dissatisfied

“Upgrades are difficult, and Gerrit often makes breaking API changes [...] No built-in repository browser. ACL system is way too complicated, should assume some simple, role-based defaults rather than throwing 1000 buttons and dials at admins with a complicated inheritance hierarchy. Wish there was an option to use native Git instead of JGit. [...] If we started from scratch today, we would probably use self-hosted GitLab. Supports everything we want out of the box:”

​

​

“Lack of documentation or out of date documentation. NoteDB and PolyGerrit have management questioning replacing Gerrit.”

General Satisfaction

Areas of Improvement

• All areas of improvement were rated as most important or somewhat important by most respondents.
• Code-Review and Reliability were rated as most important compared to other areas.

Barriers to Contribution

• Knowledge gaps relates to lack of time to some extent. Given sufficient motivation, people could likely make time to contribute and/or learn about prerequisites first.
• We can try to decrease the time it takes to learn about most important aspects, such as creating REST APIs

What do you like most / least about Gerrit?

• Community and commit-based reviews most appreciated -- main differentiator over other code review systems?
• UI, upgrading and the plugin experience are most important to improve

Like Most

11 - per commit reviews workflow

7 - open source community

4 - simple UI

3 - reliable

3 - labels & voting

3 - scalability

Like Least

11 - non-intuitive UI

6 - upgrades

6 - issues with plugins (versioning, need to install, plugin, dev documentation)

4 - ACLs

4 - scalability

3 - documentation

3 - NoteDb

Open-ended answers labeled manually, reporting only those with at least three mentions (>5%)

Summary

Summary and Recommendations

Half of the organisations responding to our survey have been using Gerrit for 1-5 years. There is a significant amount of “newcomers” to the community.

​

Only 30% of respondents use version 3.0 and later. Backporting patches is important for the community. Upgrading seems not straightforward and NoteDb is mentioned repeatedly as a concern.

​

Large-scale Gerrit installations (multiple hosts, > 5000 repos per host, > 5000 users) are not uncommon. Cross-host pains (e.g. different account IDs, different user prefs, no cross-host-dashboards, etc.) may also exist outside of Google.

​

Issue Tracking is a very common integration that core Gerrit only has very basic support for.

​

Current areas of focus (performance, reliability, review features) mirrors top community desires. Fixing the plugin experience (creating and using) should be another relevant area of focus.

1.

​

2.

​

3.

​

4.

​

5.

When running a community survey again, we should improve the following aspects of the survey based on what we learned (or didn’t learn):

• True ranking of possible improvements (consider using MaxDiff)
• Find out purposes of custom plugins
• Find out more about issue tracking integration
• Drill into satisfaction more to understand what aspects they are (not) satisfied with
• Investigate need for features spanning hosts

Improving Future Community Surveys