Developing MIT App Inventor using git and github.com
1.1.1 The main or “upstream” repository
1.1.3 Your Local Repository and Working directory
2.1 Fork the “upstream” repository
2.3 App Inventor 1 or App Inventor 2
3.1 Getting ready to make a change
3.2 Making changes. staging, and committing
3.4 Cleaning up after a change and otherwise updating your fork and local repositories
3.4.1 Getting rid of your development branch
The purpose of this document is to describe the intended code “workflow” that Open Source contributors will follow on github.com when they intend to contribute their code back to the main MIT repository.
All code that becomes part of the MIT App Inventor System must be tested by the developer working on the code and all code must be reviewed and tested by an experienced MIT developer prior to being accepted into the official MIT App Inventor source code repository and placed into production by MIT.
The “git” software itself does not include a code review support component. However the Github website where we are hosting our code offers an integrated code review system. We will discuss how we use this system in this document.
git is a distributed source code repository systems. Each contributor will usually have a copy of the entire code repository, including all changes and history. A developer will start with an up-to-date copy of the code repository and “commit” changes to this local copy. Those changes are then either “pushed” up to the parent repository or “pulled” from the parent.
A good introduction can be had on common “workflows” by doing “man gitworkflows” on a unix system with git installed. General information on git may be found at http://git-scm.com/book. See also Understanding Git Conceptually by Charles Duan.
For a primer on git, please go through CodeSchool Try Git.
Once you get started you will be working with two copies of the App Inventor source. We’ll define names for them here and use those names in this document.
This is the official source code repository for MIT App Inventor. It resides in GitHub at:
https://github.com/mit-cml/appinventor-sources.git
Only reviewed changes are entered into this repository.
Each developer has their own copy of the repository located at GitHub. You create it by selecting the “Fork” button on the main repository. You will use this repository to stage changes that are destined for the main repository, or simply to keep your own code that you do not plan to contribute back. You can create branches and generally do what you want with this repository. It effectively belongs to you, the developer. You can also destroy and recreate this repository if you wish. It is located at:
git@github:<your-user-name-at-github>/appinventor-sources.git
(once you create it).
Your local repository resides on your own computer. You create it by cloning your FORK repository. You will stage changes to your FORK repository here in your local repository.
As part of your local repository, when you clone a git repository, not only is the repository itself copied, but a copy of the most recent version of the “master” branch is placed in the working directory. After a clone, you will likely see the working directory. The local repository is located in a subdirectory of the working directory named “.git”. The contents of .git are manipulated by the git program and are not generally readable by humans.
Git repositories may have multiple “branches.” The primary branch is named “master” (as opposed to “trunk” in other source control solutions). You create a working directory by checking out a branch. Initially after your repository is cloned, an implicit “git checkout master” was done.
Note that there are two main branches in the App Inventor repository, master contains the code for App Inventor 2, and ai1 contains the code for App Inventor 1.
The first step to accessing and using the MIT App Inventor source code is to get an account at github.com. Click on the “Plans/Pricing and Signup” button. Signup for a free (open source) account.
You should also upload an “SSH” public key to your account (see “Account Settings”). If you do not have an SSH key, you will need to create one. Follow this link for details on ssh.
GitHub itself contains a lot of documentation. A good place to start is http://help.github.com/mac-set-up-git/ (this page is oriented to Mac users, but there are prominent links right at the beginning pointing to Linux and Windows versions).
Navigate to: https://github.com/mit-cml/appinventor-sources
Once selected you will see a row of buttons near the top. Select the “Fork” button.
This will create a private copy of the repository just for you. This is called a “fork” of the main repository.
You now need to create the local repository on your own computer. You do this with (swap USER with your user name):
git clone git@github.com:USER/appinventor-sources.git
Next, do:
cd appinventor-sources
git remote -v
You should see:
origin git@github.com:USER/appinventor-sources.git (fetch)
origin git@github.com:USER/appinventor-sources.git (push)
USER is your Github username. Your fork repository is now the “origin” of your local clone. However your fork is not automatically updated as the main (upstream) repository is updated with other people’s work. However you will be expected to periodically merge their work in with your own. When you submit your work for review, you will be required to have tested your changes with the latest App Inventor official sources from the upstream repository.
In order to update your local repository with changes from the main (upstream) repository you need to add it as an additional “remote” repository. Do this from your appinventor-sources directory:
git remote add upstream git@github.com:mit-cml/appinventor-sources.git
If you do a “git remote -v” You should now see something like:
origin git@github.com:USER/appinventor-sources (fetch)
origin git@github.com:USER/appinventor-sources (push)
upstream git@github.com:mit-cml/appinventor-sources (fetch)
upstream git@github.com:mit-cml/appinventor-sources (push)
(again, USER will be your Github username).
After cloning, you should find a file in the top folder called ‘sample-.gitignore’. This file should be renamed to simply ‘.gitignore’. Note that after renaming it, the file will probably be hidden from view in Unix type systems.
As it can be read in its man page, this file specifies files that are not intended to be tracked. For example, if you use an editor that creates certain types of temporary files (for instance file*.swp using vim), you can add the extension of the file to .gitignore. Any additional folders that should not be tracked, should also be added there.
The MIT repository has two main branches, master and ai1:
We recommend you work with App Inventor 2, and the master branch will be automatically checked out when you clone your repository.
If you want to work on AI1, you will need to checkout the ai1 branch:
git checkout -b ai1 origin/ai1
The previous command will checkout the ai1 branch for you, and automatically change into it.
The local repository on your own computer has a working directory associated with it. During normal development you will create a new topic-based branch, checkout that branch and do your work on that branch.
We’ll presume that you have a change to make to App Inventor 2. Start by ensuring that your fork repository’s master branch and the local clone on your computer are both up-to-date with the upstream repository.
Do the following:
git checkout master ← Work on the master branch
git pull --ff-only origin master ← Update local clone from your fork’s master branch
git pull --ff-only upstream master ← Update local clone from the upstream repository
git push origin master ← Update your fork repository’s master branch
You should also get the tags in the main repository, as they are used to keep in sync with the build server:
git fetch --tags upstream master
Now pick a name for a “topic” of “feature” branch which you will create for our change. Let’s say you are adding the “foo” component. You might then name your branch “foocomp” (it doesn’t really matter, it is for your convenience, but using names that make sense to you is the best bet). You do this via:
git checkout -b foocomp
This will create a new branch named “foocomp” off of the master branch. Your working directory will now be associated with the foocomp branch. You can verify this by using
git branch
to view your branches, and you should see something like:
* foocomp
master
(Note that ‘git checkout -b foocomp’ would be the equivalent to doing ‘git branch foocomp; git checkout foocomp’)
In general you can have lots of branches in git that relate in a tree structure. That’s a great way to get confused, so try to stay with just the master branch and a branch for the change you’re working on. If you need to work on several changes at once, you can create a separate branch for each change but it is very easy to get confused if you’re switching between those branches in the same local clone of the repository. It is less error-prone to work on the different branches in different local clones of your fork.
You can now make your changes to the working directory. As you work, it’s a good idea to periodically commit your changes. A commit creates a checkpoint in your local repository. That’s useful, because you can back up to a previous commit point if you get fouled up as you are working. You can also ask git to show you the differences between your current state and a given commit point, and other useful and confusing things. See http://learn.github.com/p/normal.html for this, and more.
You do a commit with the “git commit” command and a brief message describing the commit, like
git commit -m “I finished the first method for this awesome component”
Warning: Commits occur purely on your local machine, not on your fork on GitHub. We’ll deal with pushing things to GitHub below.
One tricky property of git is that changes must be “staged” before they can be committed. If you modify a bunch of files and then commit, nothing will be committed until you explicitly “git add” them to the list of files to be committed. You can keep track of all this with the aid of the
git status
command. It will show you a list of “changes to be committed”, i.e., changes that will become part of the next commit. It will also show you a list of things that are “changed but not updated” - things that must be staged in order to be added to “changes to be committed”. You can stage these by using “git add” on them one at a time, or you can use
git add -u
to stage all the file modifications and deletions. You can also cause the staging to happen automatically, by using git commit with the -a flag:
git commit -a -m ‘Here is the next method for the awesome component’
Note: if you intend to use ‘-u’, make sure your .gitignore file is set properly (as explained in section 2.2), otherwise some unwanted files might end up in your commit.
Note: If you create a new file as part of your coding, you need to explicitly “git add” it in order to get it “tracked” by git. The git status command will show you such files as “changes to be committed”. You’ll also see a bunch of files listed that you did not create, including the build files that are generated as part of the ant build process. Do not add these to git to be staged. They are not part of the App Inventor source.
OK, so now you’ve done some work and want to get it reviewed and submitted so it can be added to the main repository of App Inventor sources.
NOTE: Don’t even think about doing this unless you have thoroughly tested your work, and also run the tests with “ant tests”.
NOTE2: There are a number of JavaScript tests that rely on having phantomJS in your PATH. More information about these tests can be found in this README file.
And of course, as you’ve been developing, you’ve been periodically doing
git pull upstream master
to make sure you are incorporating other people’s submitted changes into your code.
When you’re finally ready, do another pull from the upstream(main) repository and run the tests again, just to make sure. If everything still seems OK, do another commit and then push your branch to your fork on GitHub with:
git push origin foocomp
where “foocomp” is replaced by whatever you called your branch. This will create the foocomp branch on your personal fork repository in github.
Finally, go to GitHub on the web and look at your local fork. Select the foocomp branch using the branch pulldown menu at the upper left. Then press the button at the top of the screen marked “Pull Request”. This will start a formal review, complete with incorporated patch sets.
GitHub does not automatically assign a reviewer, but after a reviewer has been assigned to your request, you’ll get emails when the reviewer sends comments. The process will probably take a number of comments back and forth, so do not despair!
After your changes are accepted into the upstream(main) repository, you will want to download them to your fork and local repository. You have them on the foocomp branch, but they now reside in the main repository on the master branch. Your foocomp branch is effectively a vestige that exists now only in your fork and local repositories (the actual code changes you made are on the master branch).
This next part is a little counter intuitive, as there is no way to “pull” from the main repository directly to your fork. Instead you pull to your local repository and then push to your fork. This is why we setup the upstream(main) remote in section 2.1.
First, checkout the latest version of the master branch (do this even if you are using the same local directory where you made your changes that will be pulled in):
git checkout master
Now do a:
git pull upstream master
This should pull your changes (and any other changes) from the main repository to your local repository and place them on the master branch. Now you can:
git push origin master
This will place those changes onto the master branch of your fork repository.
Now if you are done with foocomp, and all its valid changes are on the master branch, you can get rid of it. Do this:
git branch -d foocomp
This will remove it from your local repository. Now do:
git push origin :foocomp
Note the “:”, this tells git you are removing the foocomp branch from “origin” (which is your fork repository).
Git will not (easily) let you remove a branch that hasn’t been merged into another branch (master in our case). If you get an error during the “git branch -d foocomp” make sure you have done the “git pull upstream master” after your changes are accepted and applied to the upstream repository.
More information and details can be found in the GitHub documentation at http://help.github.com/send-pull-requests/.
NOTE: this sections has focused on the dev workflow for App Inventor 2. If you want to make changes in App Inventor 1, you can follow exactly the same workflow but you will want to use the ai1 branch instead of master.
The following slide decks can be used as quick guides for Contributing to App Inventor 2 and App Inventor 2 development workflow.
Unless you are familiar with github, you won’t be able to simply follow the slide decks without reading this document, so please do read this one first, and then use the quick guides as refreshers.
When you perform git operations on github.com, github recognizes and authenticates your access via SSH public/private keys. If you already use SSH you should have a directory named “.ssh” under your home directory. This directory will contain your keys and other related information. In particular the file “id_rsa.pub” contains your RSA public SSH key. This is the key that you upload to your github account (you will actually open the file in a text editor and paste the contents into a web form on github). DO NOT UPLOAD id_rsa (the version without the .pub) as that is your private key. This key should not be shared with anyone!
If you do not yet have SSH keys, the command to create them is “ssh-keygen”. If invoked without arguments it should create an RSA and DSA key for you. Once you have run ssh-keygen, you should have the id_rsa.pub file needed in the above paragraph.