| A | B | C | D | E | F | G | H | I | J | K | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Importance | Drupal.org | Github | Gitlab | Verdict | |||||||
2 | OH HAI! When leaving comments, please LEAVE COMMENTS (rather than add more columns to the spreadsheet), so we can see who said what. :) | |||||||||||
3 | ||||||||||||
4 | Cost | xxx | Free (for now, could change) | Same as d.o. | Github could change their model at any time. They are not fully open source. | |||||||
5 | ||||||||||||
6 | Project browsing | |||||||||||
7 | Centralized listing of modules/themes/distros | Critical | Yes | No, by design. | Partial; hosted by d.o | Leave centralized list of projects on Drupal.org. Site builders need something accessible they can interact with. See http://plugins.jquery.com/ for an example. | ||||||
8 | An easy way to tell the "canonical" version of a given project (modules/views vs. modules/views_by_some_other_person) | Critical | Yes | No (while forks are clearly marked as such, that doesn't help) | Partial; Forks are possible, but we'll have control over the structure | This actually is one of the chief problems with github: just because something is a fork it doesn't mean it didn't become the official once the original got abandoned. | Yes, assuming these were all placed as reposotories under the "drupal" user. This is what other projects (e.g. Symfony (Symfony just places their core components/bundles, which is the same as everything in core/modules). Their contrib is all over the place) do. | Needs research to see if Github can handle placing 30,000+ repos under one user. | If a Github project isn't active, there is a serious problem, even for me as a developer when it comes to 'well, what do I use now?' For our non-technical end users, we would be leading them off a plank. | Having every project on drupal.org under one organization owner would probably not be the best use of GitHub. People would not be able to create their own projects. | We will still have a "project catalog" on Drupal.org, right? That one would have a field that points to the "official" project. What am I missing? | |
9 | Easy to edit project pages | Major | Yes, optionally control who has access to edit this. | Yes, optionally automatically display README files from codebase | Yes | Each have additional nuance that's useful. | ||||||
10 | Easy to discover projects | Major | Partial. Project taxonomy pages w filters, but needs improvement. | Project discovery happens on Drupal per the plugins.jquery.com model. See 'Centralized listing ...' | ||||||||
11 | Easy to find specific projects | Major | Partial. Projects have a consistent URL format that makes them easy to find that way, but e.g. searching for "views" doesn't always show Views module (although this could probably be fixed with a tweak) | Partial; Much better filters and search algorithm, instant search, search *inside the code*, rank by popularity, etc, but hard to identify canonical project version. | Partial; Much better filters and search algorithm, instant search, search *inside the code*, rank by popularity, etc, but hard to identify canonical project version. | |||||||
12 | Management of Obsolete/Abandoned projects | Nice to have | Yes, handled by taxonomy | No | No | |||||||
13 | ||||||||||||
14 | Development | |||||||||||
15 | Pull Requests | Major | No | Yes | Yes | Pull requests integrate into the development process. Write code, run command, poof, record is there. Push more commits, poof, available to all. No need to code, diff, figure out interdiff, post, upload. | Issues can be merged without needing to fetch and apply a patch | It requires a lot more work to create a PR than a diff: Must create a branch, commit to it (with a useful commit message as it will be kept for eternity), push it back, create PR. Of course, very simple patches can be done in-browser. What about the rest? | If you edit the file in the UI, it makes one for you | A noob can create a pull request by clicking around. You must use the command line to create a diff and apply a patch. This still has the problem of a commit message (which is unnecessary for the noob). | Do we think that PR from noobs are usefull? (yes, it promotes openness) | |
16 | Code review tools | Major | Partial; Nothing in Project* module itself, but the Dreditor browser script provides line-by-line review capabilities | Yes, fully-featured ability to comment on specific lines of code, and collapse no longer relevant comments as new versions of the code are posted. | Yes, fully-featured ability to comment on specific lines of code, and collapse no longer relevant comments as new versions of the code are posted. | Git* blows Drupal.org out of the water. | Integration of tools like http://pareview.sh/ would go a long way towards improving d.o usability | |||||
17 | Drupal-centric code testing automatically available for projects | Major | Yes | Yes. Travis CI might work. If not, we can integrate current testbots with GitHub very tightly (see issue status API) | Partial? Travis CI might work. If not, we can integrate current testbots with GitHub very tightly (see issue status API), there's also a GitLab-CI project, but it will not work for d.o. | Tie. (jthorson) I disagree ... until the test suite is 100% PHPUnit, Travis is out. And integration of GitHub with current testbots introduces added workload and instability. Sad as it is, Drupal.org wins as long as we're on hour+ long simpletest runs. | https://circleci.com/ plays very nicely with github | Note that we need to support D6, D7 and D8. | ||||
18 | Testing results clearly visible | Major | Partial; Yes, but need to scroll down to the most recent patch. It's only whether it passed at the time of the patch being uploaded. | Yes; actually better here because there's only one status, at the top of the issue, clearly visible. Not sure how often it updates. | Yes; actually better here because there's only one status, at the top of the issue, clearly visible. Not sure how often it updates. | Github wins. (jthorson) Visibility is a Tie, post d.o D7. | Github doesn't win; what if there are multiple approaches to an issue? | Then there's multiple pull requests, because they're different code. Both could reference a single issue, however. (PRs are a subcase of issue on GitHub.) | GitHub isn't nearly as easy to understand as d.o | |||
19 | Ability to suggest changes to files completely from the browser. | Major | No, and unlikely to ever be done due to ROI of the feature. | Yes. And this could be a game-changer for non-/less-technical contributions: documentation, UX, markup, etc. | Yes. | Github wins. | Moral of the story: what we want is in-browser changes. | Only very simple changes really work this way and they can't be done on a new branch so it's not good for much more than fixing typos. GitHub is better IF you are a committer on the project. But you still can't do things like create an issue branch without a local repo. | ||||
20 | First-class commit credit for all involved in creating a change | Major | No, and this is a huge source of consternation, particularly in the core queue. | Yes. Because all interim progress on a PR happens as commits, full credit is given to all developers. Reviewers, less so, but they are spottily credited in the current system anyway. And with in-browser editing, there's a good chance they would/could just make the changes directly, rather than marking issues "needs work" for minutia, which would be a *huge* improvement over the status quo. | ? | Github wins, unless we do what Sam suggests in the comment, which would be a Drupalism. | Huh? Why this is not solved by a PR? If everyone commits... I also don't understand. — webchick. | |||||
21 | Delegated access controls (note delegated) | Major | Yes, Drupal.org allows user x to add user y as committer to project z | Partial; GitHub allows this too, but fewer options or roles (can't assign someone only a maintainer of issues or wiki and not code). Project settings > Collaborators | Yes, Gitlab allows user x to add user y as committer to project z | Drupal.org wins. | If all projects are owned by Drupal, how is access delegated? | You can't have all projects owned by a user, won't work. | Not one user; one Organization. | |||
22 | Familiarity to new contributors who are used to github | Nice to have | No | Yes | Yes, very similar to github | One of the key arguments for Git was "everyone already knows it, the community is voting with their feet". The same logic applies here. Drupal.org is proprietary, GitHub is the de facto standard. | If someone comes from Jira, Bugzilla, Trac then github issues are not the normal and the Drupal issue queue would feel more normal. This feels like testing the systems to see which one is most-like-github and...duh...github wins. | I think that is the point. GitHub is the most GitHub like. A lot of folks like GitHub a lot and even more use it. Experience with other systems really doesn't influence your GitHub preference or lack thereof. | ||||
23 | Open collaboration in creating code | Critical | Yes | Partial; GitHub allows 'repo collab', but it does not allow anyone to push to a branch attached to a PR | Drupal.org. | |||||||
24 | ||||||||||||
25 | Issue queue management | |||||||||||
26 | Statuses | Critical | Yes | Partial | Partial | Github status API is oriented to testing and doesn't support every status drupal.org does; Missing "needs work" and "postponed, maintainer needs more info" greatly in Drush GitHub queue. (edit - can be handled with tags, but maintainer's responsibility) | Also issue titles can't be changed by anyone in GH, which helps a lot when issues evolve. | Importantly, Gitlab can be configured to use an external issue tracker. See https://groups.drupal.org/node/313158#comment-954473 | ||||
27 | Components | Critical | Yes | No (tags might replace it but clumsy) | No (tags might replace it but clumsy) | Github loses. | Symfony uses title prefixes for components. Definitely weaker than project_issue. | Projects (i.e. drush) separate "bugs" from support requests in moving to GitHub but the real world delination isn't always so clear. | ||||
28 | "Community tagging" to allow ad-hoc initiatives | Critical | Yes, and this is a killer feature we use all the time. | No; in fact, you need to be an admin of a project in order to set any of these properties. | Drupal.org wins. | |||||||
29 | "Crowd source" issue status/metadata management (e.g. random passers-by being able to mark issues as duplicate, "needs work", clarify title, etc.) | Critical | Yes, and this is used extensively to onboard new contributors and take work off of busy maintainers' plates. | No; you need to be an admin of a project in order to set any of these properties. | Drupal.org wins. | |||||||
30 | Ability to move issues between projects easily | Major | Yes, and this is used extensively to foster cross-project collaboration without losing the history of an issue. | No | No | In theory possible on GH by closing and creating a new one, but that doesn't solve the use case thus D.O still wins. | ||||||
31 | Notifications | Major | Partial -- Users can (un)subscribe to an issue. Cannot manage spcifics (ex. "Dont notify me if I made the comment") Cannot "CC" someone using @username (this would be very hard to implment because d.o. allows spaces in usernames) Offers e-mail based notifications but it's very, very hard to find where to turn this on. | Yes Users can (un)watch; Users have fine-grained control over notifications Users can "CC" with @username Offers both e-mail and web-based subscriptions (a little red indicator in the top right corner). | Yes Users can (un)watch; Users have fine-grained control over notifications Users can "CC" with @username | Github wins; D.O. could be greatly improved to meet many of these needs | ||||||
32 | Milestones — ability to apply issues to a specific goal and show progress on overall initiative | Major | No. We currently abuse "meta" issues for this, though. D7 drupal.org upgrade improves this with both parent and related issue functionality. | Yes | ||||||||
33 | Ability to add consistent, community-specific elements to all Drupal-related projects, including e.g. 'Report a security issue' | Major | Yes | Partial -- CONTRIBUTING.md is automatically linked on the form, but users still need to click a link; the text does not appear directly. | Partial -- CONTRIBUTING.md is automatically linked on the form, but users still need to click a link; the text does not appear directly. | Even though security.drupal.org is separate, Drupal.org still wins by having easy links for 'Report security issue' | No private issues on Github, not easy to hook into the page to help direct people on where they should report security issues privately. | Given one of Drupal's defining strengths is its security track record, this needs to be a critical requirement. | ||||
34 | Automatic association of referenced issues in commits | Major | No | Yes | Yes | This is related to the git web viewer, which is totally integrated in Github | ||||||
35 | Images in-lined automatically | Nice to have | No; Dreditor makes it slightly less tedious with an "Embed" button. | Yes | Yes | Github wins | Common Drupal issue solvable with contrib modules depending on the desired workflow. | |||||
36 | User avatars | Nice to have | Not yet (bluecheese already supports them b/c of g.d.o) | Yes, but requires you to sign up for a Wordpress account to get a 'Gravatar' | Yes, but requires you to sign up for a Wordpress account to get a 'Gravatar' | This assumes having these is a good thing | Avatars are opt-in. They're a good thing. | D.o could easily add these; we have them on most every *.d.o site except the main one. No idea why we haven't. But this is a minor feature compared to everything else. | Right. It's rated "nice to have" :) | |||
37 | Non-Image Attachments: the ability to upload other types of files than images directly to issue comments. | Nice to have | Yes | No | Yes | Minor inconvenience at most; we can link to Dropbox or some other upload site, though this will likely lead to more linkrot. | The gist integeration on github also helps. | |||||
38 | ||||||||||||
39 | Release management | |||||||||||
40 | Consistent, predictable release names | Critical | Yes | No; Social convention to use proper semver; not software-enforced. | Tons of functionality builds off of this, from localize.drupal.org, Update Status module, Drush PM* tools, etc. | |||||||
41 | Ability to mark releases as security releases | Critical | Yes | No? | Even if we adopt github for repos, issues, wikis, the update xml and security tag etc. would need to stay on d.o and would continue to work (and require maintenance) there. | There's no "release" concept in GH at all. All you can do is give proper names to your tags. | ||||||
42 | Semantic versioning | Major | No; currently stalled on community bike-shed-o-rama (jthorson) Policy, not technical limitation. Project* supports custom version string formats. | Yes, by default. | D.o's version number system needs 1,000 words to explain how it works: https://drupal.org/node/467026. Semantic versioning is the standard. | |||||||
43 | Md5 checksums for each release/package | Major | Yes | No? | The md5s (along with packaging) would need to stay on d.o | Shouldn't we be using something stronger than MD5 at this point anyway? Not really. This is just making sure what you got is what was served and that what was served matches what was packaged. | ||||||
44 | Reliability of packaging system | Major | Yes, but release builder breaks occasionally and needs percussive maintenance | Yes, you download via git or zip/tgz, automatically generated | Yes, can automatically generate tgz (not sure about zip) | Github adds extra or extraneously-named folders during packaging that are unwanted in Drupal downloads. Gitlab seems to not do this. | Consider the D.org packaging system has to add metadata to .info files. Can that be replicated with Github? | |||||
45 | Download data as a quality metric | Major | Yes. | No. | No | |||||||
46 | ||||||||||||
47 | Other | |||||||||||
48 | Network stability & availability | Critical | Yes; goes down from time to time, too but has low downtime (greggles has teh graphs). | Partial (grep their twitter feed for "DDOS" or "outage") Note that they do have https://status.github.com/ which is great, and seems to indicate their uptime is right around 98-99%. | Yes | Github is a very common target for DDOS attacks - how many times in the last year has it had major outages? (FYI they had one this evening right after Webchick posted this spreadsheet.) | ||||||
49 | Security track record | Critical | Partial; for the most part excellent, though there have been two times in the past 10 years that all user passwords needed to be reset. | Partial; they've had a few major blunders over the years | ||||||||
50 | Mobile-friendly browsing interface | Major | No | Yes | Yes | We need to solve this regardless. Doesn't seem germane. | ||||||
51 | Robust Web API for 3rd party integration | Nice to have | Partial; Some RSS for queues, and JSON for issue nodes, but not much else D7 upgrade plans to use restws. | Yes | Yes | Github wins. We'd need to wait until Drupal.org is running D8 or we get restws enabled on D7 for us to have better API support. | Naturally we could build one, but GitHub's already exists and there are already PHP wrapper libs for it. | |||||
52 | Desktop applications | Nice to have | No | Yes; see http://mac.github.com/ and http://windows.github.com/. | No | This is really nice, and offers an extremely easy to use interface for non-technical and technical collaborators alike. | Not sure what the No means for d.o. There is no custom-built app for d.o but many great git gui apps (Tower.app on Mac is one) that work well. | |||||
53 | Ability to bookmark or "star" projects. | Major | Partial -- No native support, but there is always ⌘D. | Yes | Very useful feature. | |||||||
54 | Usable Git web viewer | Major | Partial | Yes | Yes | |||||||
55 | Anonymous visitors can view projects and issues | Critical | Yes | Yes | No; but will soon be fixed | |||||||
56 | Open source | Major | Yes | No | Yes | |||||||
57 | Tight integration with main website | Major | Yes; issue queues have same "look and feel" as forums, which have same "look and feel" as rest of website. Makes issue queues feel more approachable. Only one login. | No; issue queues look like Github, and are highly geared towards developers. There are concerns that this could make them less easy to approach for non-developers. Need both a Drupal.org and Github login. | ? | |||||||
58 | ||||||||||||
59 | ||||||||||||
60 | ||||||||||||
61 | ||||||||||||
62 | ||||||||||||
63 | ||||||||||||
64 | 13 | 10 | 10 | Total | ||||||||
65 | 1 | 1 | 1 | |||||||||
66 | 5 | 4 | 4 | |||||||||
67 | ||||||||||||
68 | ||||||||||||
69 | 0 | Critical | Does this mean that github loses out on criticals and majors and it's all nice to have? I said above: what we want is in browser edit. | |||||||||
70 | 1 | 3 | ||||||||||
71 | 0 | 6 | ||||||||||
72 | 7 | 8 | Major | |||||||||
73 | 2 | 0 | ||||||||||
74 | 1 | 5 | ||||||||||
75 | 1 | 5 | Nice to have | |||||||||
76 | 1 | 0 | ||||||||||
77 | 4 | 1 | ||||||||||
78 | ||||||||||||
79 | ||||||||||||
80 | ||||||||||||
81 | ||||||||||||
82 | ||||||||||||
83 | ||||||||||||
84 | ||||||||||||
85 | ||||||||||||
86 | ||||||||||||
87 | ||||||||||||
88 | ||||||||||||
89 | ||||||||||||
90 | ||||||||||||
91 | ||||||||||||
92 | ||||||||||||
93 | ||||||||||||
94 | ||||||||||||
95 | ||||||||||||
96 | ||||||||||||
97 | ||||||||||||
98 |