Amazon EC2 Mininet Setup
Amazon has generously offered to provide $100 Amazon Web Services (AWS) account credits to each student in Stanford CS244, Advanced Topics in Networking, for Spring 2012. This credit will provide plenty of time to run each assignment, and as a side benefit, you’ll get a taste of what computing in the cloud is like. If you’re familiar with SSHing into a remote machine using public keys, creating and logging into an instance will be no problem. If not, it’s time to learn, and it’s a useful skill.
The TAs would prefer students to spend their time doing assignments and learning, rather than setting up infrastructure - so if there’s some way to improve these instructions, please let the TAs know. Thanks!
Request an Account Credit
- Form a group of exactly 2. This will be your group for all assignments and projects.
- Email the staff to get your account credit code:
- List of Group members
- List of each group member’s OS
- You should receive the response within a day, ONLY if you are in Stanford CS244, Advanced Topics in Networking. In the meantime, you should set up the AWS account.
Setup AWS Account
Prerequisites: credit card, telephone number.
- Register an account at aws.amazon.com. You can use a single account for your group or keep individual accounts.
- You will need a credit card to create an AWS account, but it will only be charged if you exceed the $100 credit. Most students will use less than $20 of their credit.
- You may also need a telephone for account verification.
- EC2 stands for Elastic Compute Cloud, and it is the Amazon service for virtual machines on demand. You may need to wait a little bit until Amazon verifies your account.
- Apply the $100 credit to your account:
- Go to console.aws.amazon.com
- At top right, click on your account name, then the Account Activity drop-down.
- Click Payment Method on the left side once re-logged in.
- In Redeem an AWS Credit Coupon, click Redeem AWS Credits.
- Enter your code and click Redeem.
Setup Public Key Authentication
Prerequisite: SSH client. Pre-installed on Mac and most Linux distros. On Windows, install Git Bash shell as described in the GitHub Windows tutorial.
Setting up public key authentication is probably the scariest part of the whole process, if you haven’t done it before. Don’t worry. It’s really not that bad.
Please check every possibility on this page, ask a friend for help, AND Google your error message before posting on Piazza. GitHub has a nice page for SSH key debugging. SSH key debugging can be slow to debug remotely.
If you’re familiar with the process and already have a key, skip to the “Only if you generated... “ bullet point below.
- Generate a public/private keypair ONLY if you don’t have one already.
- Generate a key pair locally.
- Check for key files first in ~/.ssh/. If don’t see an {id_rsa, id_rsa.pub} pair, continue.
- ssh-keygen -t rsa -b 1024
- Give the key a passphrase. You’ll need to remember this to activate the key.
- Make sure you generate a 1024-bit key (-b 1024)! Amazon only supports 1024-bit keys as of March 2012.
- Give the key a name if you’d like, e.g., amazon_key_rsa
- By default, the keys will go in ~/.ssh/; id_rsa is the private key, while id_rsa.pub is the public key. When later uploading this key to Amazon, make sure to upload id_rsa.pub. id_rsa should never need to leave your local computer, though it may be convenient if you want to share access to a single VM between group members.
- Windows: follow the GitHub tutorial. This will add the Git bash shell, which includes an SSH client with key generation. You don’t actually need to know any git commands. If you can successfully connect to GitHub using the key, you should be able to use it with Amazon just fine.
- Make sure your private key is usable by the OS (probably unnecessary)
- Linux or OS X: the permissions for a locally-generated key will probably be correct, but you may still need to add the key using ssh-add. If that command fails, you may need to start an SSH agent with ‘exec ssh-agent bash’, then run ssh-add. Alternately, pass the key in when starting your SSH client using the -i option and provide the path to your renamed key.
- Windows: consult the documentation for your SSH client.
- Add the Key pair to your Amazon account:
- Go to console.aws.amazon.com and select the EC2 tab
- In the left pane, click on NETWORK & SECURITY -> Key Pairs
- Add your local key by clicking on the Import Key Pair button at the top, and copy in the public key. This is the one with a .pub extension.
Launch an EC2 Instance
For CS244, the TAs provide a pre-built VM for you to use. This will have everything you need to run Mininet pre-installed; you’ll only need to pull in code for specific assignments on top. To create your EC2 instance:
- Go to the AWS Console and select the EC2 tab.
- Click on INSTANCES -> Instances in the left Navigation pane.
- Under My Instances in the right pane, click Launch Instance.
- In the Create a New Instance box that pops up:
- Stick with the Quick Launch Wizard.
- Give your instance a name, any name.
- Choose a key pair. You added one to the account previously, right?
- In Choose a Launch Configuration, click on More Amazon Machine Images.
- Click Continue
- Search for the CS244 Amazon Machine Image (AMI): cs244-mininet or ami-cb8851a2. An AMI is a just a disk image stored on Amazon servers. You must use this one.
- Click Continue
- Click Edit details
- Type: For Assignment 1, we’ve found that a c1.medium (High CPU, medium memory) instance works well. Select instance type: c1.medium. Do NOT use t1.micro!
- Security Settings: Security Group: use the default, quicklaunch-1, which will get created and ensure SSH access. Later, as described below, you may want to enable other ports for webserver access. Note that pings will be blocked. If you get the security group wrong, don’t worry. You can always modify the settings of that security group.
- Click save details
- Wait for your instance to boot. It may take a minute or two. In the meantime, learn the difference between Stop and Terminate actions, then modify the security group settings below.
- The difference between Stop and Terminate actions:
- Stop means shut down your instance but leave its disk image on Amazon’s server, ready to start again within a minute or so.
- Terminate means kill the instance permanently and delete its backing disk image. Terminated instances cannot be restored.
- If you get a “Your account is currently being verified..” error, modify the security group settings below, then come back in a bit.
- If your instance launches successfully, go to the next section.
Modify Security Group Settings
- The quicklaunch-1 setting provides SSH access on port 22 but rejects everything else. To get any other ports past the firewall, you’ll need to explicitly tell Amazon about those ports.
- (Optional, but Highly Recommended) The Python SimpleHTTPServer module is an easy way to access files created on your instance over the web. By default it starts on port 8000. To add a rule to allow requests to this port:
- click on NETWORK & SECURITY -> Security Groups on the left-side Navigation Pane.
- click on quicklaunch-1 on the top-right pane.
- On the bottom right pane, click on the Inbound tab. Create a new Custom TCP rule for port 8000, then click the + button to add the rule.
- Click Apply Rule Changes.
- (Optional) If you’d like to be able to ping your instance, add an inbound rule for All ICMP traffic.
- Click Apply Rule Changes.
Logging in to your Instance
High-level idea: use public-key encryption to connect to your instance via an SSH terminal.
- Remember: the default user in the CS244 VM is ubuntu!
- Make sure to have your private key loaded on the machine from which you’re SSHing in.
- Find the public DNS address using the AWS console. Go to INSTANCES -> Instances and click on the just-launched. instance. In the bottom right pane, look for Public DNS and copy this: e.g., ubuntu@ec2-23-20-163-166.compute-1.amazonaws.com. You may need to expand or scroll the pane to see this field.
- In an SSH terminal, connect to the Public DNS address.
- Within a few seconds, you should be connected and logged in.
- Troubleshooting:
- Try using the -i option for SSH to use a specific private key.
- Try the -v option for SSH to use verbose logging.
- Make sure you’re not using sudo.
- Check the GitHub SSH issues page.
Testing Mininet
As a quick sanity check, make sure that you can run Mininet on your instance:
sudo mn --link tc,bw=10 --test iperf
This command will start a single-switch topology with 10Mbps links and run iperf between them. The output should look something like this:
*** Creating network
*** Adding controller
*** Adding hosts:
h1 h2
*** Adding switches:
s1
*** Adding links:
(10.00Mbit) (10.00Mbit) (h1, s1) (10.00Mbit) (10.00Mbit) (h2, s1)
*** Configuring hosts
h1 h2
*** Starting controller
*** Starting 1 switches
s1 (10.00Mbit) (10.00Mbit)
*** Iperf: testing TCP bandwidth between h1 and h2
*** Results: ['9.48 Mbits/sec', '10.2 Mbits/sec']
*** Stopping 2 hosts
h1 h2
*** Stopping 1 switches
s1 ...
*** Stopping 1 controllers
c0
*** Done
completed in 9.457 seconds
Logging out of the Instance
- Stop your instance, but do not Terminate it. You don’t want to burn through your $100 credit before the end of the class and have to pay for EC2 time out of your own pocket.
Logging back in to your Instance
- Go to the AWS console, click on EC2, click on your instance, and go to Instance Actions -> Start. Take note of the public DNS entry, which will change every time you stop and start the instance (but not when you restart the instance).
Managing your AWS Credit
This section has no action items, but it does have some useful tips.
- Amazon has a pricing list here for all services:
- Go to console.aws.amazon.com.
- Click on your username in the top right, then click on the Account Activity drop-down.
- Click on Details
- Be aware that your credit may get used for data transfer in and out of Amazon data centers.
At this point, you should be ready to begin Assignment 1. You can do some optional setup below, or skip to the assignment.
[Optional] Easier Login
Linux or OS X: For easier login you may want to add an entry to ~/.ssh/config with the name of the VM and a few parameters. Then you can log in to the VM with `ssh vmname’, with no password needed
Add the following to ~/.ssh/config, with the correct hostname from Public DNS:
Host cs244
User ubuntu
Hostname ec2-XX-YY-ZZ...compute-1.amazonaws.com
If using a non-default key, you may want to specify the key location below Hostname and replace yourname:
IdentityFile /Users/yourname/yourkey.pem
[Optional] Changing the Username
Perhaps you like to use your SUNET id when logging into other services, like GitHub, or other remote services using SSH. The benefit of this approach is that it sets the default username.
Adding your own user name to sudo requires a few steps. Since authentication is keypair based, it helps to copy the key over to the new username first.
On remote EC2 instance, replacing yourid with the correct username:
sudo adduser yourid
sudo mkdir /home/yourid/.ssh/
sudo cp /home/ubuntu/.ssh/authorized_keys /home/yourid/.ssh/authorized_keys
sudo chown -R yourid.yourid /home/yourid/.ssh
Also modify /etc/sudoers with a line that gives your username root privileges.
sudo bash -c "echo \"yourid ALL=NOPASSWD: ALL\" >> /etc/sudoers"
Then you can login with your normal username. You may want to change the User line in ~/.ssh/config on your local machine, too.
[Optional] Agent Forwarding
Many students will want to use a source code revision control system. The TAs are fans of BitBucket, which gives free unlimited private accounts to those with .edu addresses. Services like BitBucket and GitHub use SSH public key encryption, and it’s good practice to only keep your private key on your local machine. When logging in, add the -A option to OpenSSH, and it will make your private key available to the remote EC2 instance, without storing it there.
(Optional) Other Resources
EC2 command-line tools
http://aws.amazon.com/developertools/351
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/setting-up-your-tools.html
SSH tutorials:
Linux tutorial
Deprecated Content
- Option 2 (Deprecated): Generate the key pair using the Amazon console.
- Go to console.aws.amazon.com and select the EC2 tab
- In the left pane, click on NETWORK & SECURITY -> Key Pairs
- Generate a new key by clicking on the Create Key Pair button.
- Save the downloaded key to a secure place on your local machine.
- If the key was generated through Amazon, you will definitely need to add the key to the SSH agent and will probably need to set the permissions for it.