Introduction: The Basics of the Web

This is a class about creating interactions between web servers and clients, whether those clients are web pages, Processing scripts, networked Arduinos, or any other piece of internet-connected software or hardware.

The class describes the interaction among the three basic units you will learn to program:

You will learn the basic set of tools you'll need to create these three elements, and to design how they interact. As with anything involving computers, you will have to master the technical details involved; this class is here to teach you those details. However, the model of the web is so simple that it can be described even before you learn those details.

The web isn't a thing. It's a set of rules for how computers interact. In fact, "the web", considered as a whole, is just the sum of all the transactions between computers that abide by those rules.

The simplest form of web interaction is clicking a link. When you click a link on a web page, you are asking the browser to send a request to the server. The request is "Please send me the contents of the page available at this link." This is the most basic form of interaction on the web, carried out literally billions of times a day, and every one of those interactions relies on the browser sending a request for new content to a server, and the server replying with the new content.

Now imagine a slightly more complicated version of interaction: filling out a form on a web page, submitting the form, and getting a response. Imagine, for example, a simple form that does nothing more than ask you your name:

[__________] What is your name?

This is a single-field form, and the interaction can be sketched out very simply: When you fill out the contents of a form on a web page and hit Enter, the browser sends a request to the server’s web address (its URL) to take and process the results of that form.

Now imagine the particular web address (or URL) that the form sends the data to contains instructions to take the name you submit and to send back a web page that says “Hello” plus that name. The server would receive the name you submitted via the form, act on its instructions for processing the data, and send the results back as a reply to the browser. (Here’s a working example of this sort of interaction.)

The technical part of this interaction is split between two computers: the computer where the browser is running, and the computer where the server is running. The browser shows you the web page with the form in it. It also accepts your input into this form, and sends this as a request to the server. The server accepts the request, processes it, and returns the results. The browser then displays those results.

User                           Client (The Browser)            Server

Type your name

Hit ‘Enter’

        

Send name to the server.

Receive request from client.

Read name.

Return page saying “Hello ”+name

Receive data from server.

Render it as a web page.

You see the resulting page.

There are lots of little details to making something as simple as this form work, but the core idea is this: the web is a set of rules for interactions between software running on two or more computers (or phones, or devices, etc). From here, we’ll move on to describing the technologies you’ll use to build those interactions.

WHAT IS SINATRA?

Sinatra is a domain-specific programming language (DSL) that is designed to make it easy to build webapps and websites. Sinatra is itself written in the programming language Ruby, so all Sinatra code is not Ruby code (but not all Ruby code is Sinatra code.) In this class, you will first learn to write simple scripts and then expand to create more complex interactions. (Sinatra programs can be called scripts or applications interchangeably.) As you learn to write Ruby scripts, you will also learn the basics of Ruby syntax.

Throughout, this documentation assumes that you're working on the ITP server, in your /sinatra folder.

THE BASIC WEB INTERACTION: HTTP REQUESTS

Open up a browser. Type in www.google.com. Press enter.

Congratulations! You've just made an HTTP request!

To be more specific, your browser just made an HTTP request. HTTP -- hypertext transfer protocol -- is a standard for requesting and delivering data across the internet, whether web pages, images, videos or other kinds of data.

Your browser is an HTTP client. It formats your input, in this case www.google.com, into an HTTP request and sends it to an HTTP server, in this case located somewhere in one of Google's gigantic server farms. The server at Google then sends a response back to the client, in this case, the content of www.google.com.

Browsers are just one kind of HTTP client, but anything that can initiate an HTTP request to a server and accept the reply, such as a Processing sketch, is, by definition, an HTTP client.

HTTP requests and responses form the basis for communication between a client and a server. The Sinatra applications you write are server-side applications that take in HTTP requests and respond to them based on a block of code you've written. When a user (or client) requests a URL that has been defined in your Sinatra app, your code defines what gets included (text? images? sounds?) in the response.

There are several kinds of HTTP requests, but the two main ones we'll be dealing with are:

GET Requests

When you enter a url into the browser or click a link, you are triggering a GET request.

Get requests ask the server for a particular file, image, webpage, etc located at a specific URL. "Hey browser -- GET me whatever is located at the server located at www.google.com"

POST requests

POST requests are used to send data to a server to be stored or processed. Think entering information into a form, or uploading a profile picture to a web service. "Hey server -- I'm POST-ing some information to you. Do something with it!"

Other HTTP requests include HEAD, PUT, and DELETE. We will not concentrate on these types of requests in this class, but for more info on what these do, click here: http://docs.oracle.com/javaee/1.4/tutorial/doc/HTTP2.html.

YOUR FIRST SINATRA APP

Open up an FTP client (we recommend Cyberduck; class software described here), set up the login credentials for stu.itp.nyu.edu, and navigate to your folder on the ITP server. Inside, you should see a folder called Sinatra. Open it up. Then navigate to the example_sinatra_app folder.

As its name suggests, example_sinatra_app is a ready-to-go Sinatra application we've created for you for the purposes of getting started in this class. Inside, there are several folders and files:

app.rb : This is where your Sinatra code lives. (The .rb file extension indicates that the file is to be interpreted by Ruby.) This file defines the ways that the server responds to HTTP requests from clients.

/public : The /public directory is where you should put any static files that your web application uses. These can include images, HTML, CSS, and Javascript files.

/tmp : A folder with one file, always_restart.txt, inside. This folder has been created specifically for the ITP installation of Ruby -- you can effectively ignore it.

Once you have familiarized yourself with Cyberduck and your /sinatra folder, open up a code editor. We recommend Komodo Edit, because it will let you work on and save remote files without requiring a separate ‘Upload’ step. When you have Komodo open, click on "File → Open → Remote File…"

Now click on the electric plug+page icon to the right of the top line in the dialogue box. You'll be prompted to set up a new connection. Enter in your ITP server credentials (make sure to use SFTP!) and  the app.rb file in your ITP sinatra folder. When you're done, hit "OK". Now select your server from the dropdown list under "server." Navigate to your sinatra folder, and then to your example_sinatra_app folder. Open the app.rb file.

This file is the first Sinatra application you will work on.

Hello World

When you are done editing it, you should have a very simple working webserver. When you send a request to this webserver, Ruby will interpret the instructions you have put in the file and execute them.

The first thing you need to know about Ruby is which bits of the file are there for humans to read, and which ones are there for Ruby to interpret. Human-readable lines are called comments; in Ruby, everything following # on a line is a comment -- the hash character (#) says to Ruby “This is a note for humans -- please ignore everything (or everything else) on this line.”

# Comments look like this...

### ...or this

“This is not a comment” # but after the hash sign is a comment

You should turn all the the existing code in app.rb (This is often called ‘commenting out’ code -- it’s a useful technique for leaving code in a program but momentarily keeping Ruby from interpreting it.) After you have commented out the existing code, type in the following:

require 'sinatra'

# When a GET request comes to my webserver at /helloworld

# send back the text “Hello!”

get '/helloworld' do          

    'Hello!'

end

This is the simplest possible Sinatra webserver, with only four lines of working code (and two lines of comments, which are optional, and can be varied as you like.)

Code Line 1: require 'sinatra'

You'll need to include this once at the top of every Sinatra application file. This instructs ruby to look for the Sinatra gem. (In Ruby, optional bits of functionality you can include in your program are called gems.) Since Sinatra is an optional bit of functionality, you have to tell Ruby explicitly that you want to use Sinatra to run your application.

Code Line 2: get '/helloworld' do

The get...do tells your server to look for HTTP GET requests and do something, and '/helloworld' says the request will come to a URL ending in /helloworld

In Sinatra, this is called a route. Routes are a core concept in Sinatra; a route is composed of an action (in this case get)and a path (in this case /helloworld). Most of your work will be deciding what routes you want your application to have, and how you want them to work.

For now, you want one route, get /helloworld, and you want it to reply Hello!

Code Line 3:     'Hello!'

This single line says “When a cleint sends a GET request to this URL, send back a line of text reading Hello!” (The indentation is for human readability; it helps us keep track of which code is conceptually inside which route.)

The word is in single-quotes, as a way of telling Ruby “Treat this as a string of text, rather than as an instruction in Sinatra.’ Double-quotes would work equally well. A mix of single and double-quotes, though, or unbalanced quotes, or no quotes -- “Hello!’ or “Hello! or Hello! -- will all cause the server to crash. (Computers are dumb, so so dumb. And picky. They never do what you want, they only do what you tell them, and if you tell them things in a way they don’t understand, they crash.)

Code Line 4: end

end does what it says, telling Sinatra “This is the end of this route.” Later, you will have more than one route in an application, and you routes will be more complex, but this simple get...do...end version demonstrates the basics of what a route does.

Now it’s time to test your server.

Save the file. Now, open up your browser and go to http://itp.nyu.edu/~yourNetId/sinatra/example_sinatra_app/helloworld (replace ~yourNetID with your actual NYU net id in all these examples.)

You should see "Hello!" If you do, congratualtions! You've just made your first Sinatra app.

If it doesn’t work, check that you have saved and uploaded the file, that you have put the correct URL in the browser, and that your network connection to itp.nyu.edu is working. Then give it a minute and try again -- sometimes the ITP server takes a moment to load or transfer files.

If it still doesn’t work, cut and paste the whole script, from require ‘sinatra’ through end, and replace everything in app.rb with that copied version, save it, and test it again.

HELLO WORLDS: WHAT IS A ROUTE?

As you saw with the simplest version of app.rb, a route in Sinatra consists of an action and a path, followed by some instructions to the server. In the abstract, the simplest Sinatra route looks like this:

action 'path' do

    instructions to the server

end

The instructions to the server define how the server responds to the request, including what the user sees when he or she navigates to the path. In the Hello World example, we defined the server's response as the word "Hello!" When we executed a get request with our browser browser, the server returned "Hello!" and the browser displayed it.

The action get is a reference to an HTTP GET request. This route will be executed by the server only when the server receives a GET request. Were the browser to send a POST request, or PUT, or DELETE, the server would not say “Hello!” Instead, it would return an error.

The path, in this case /helloworld, tells Sinatra which URL to respond to. When you typed in the URL in your browser, you can see that after the domain name, itp.nyu.edu, and then your ~netID, the URL points to your Sinatra folder, your example folder, and then to /helloworld.

You'll notice, though, that you didn't type in a full on URL into the 'path' section of our Hello World route. Instead, you typed in a relative URL -- just the /helloworld bit. This relative URL is all you needed because app.rb knows where it is installed, so it doesn't care about the full URL; it only cares about that path relative to where the app.rb file is located.

As a result, you can assume that any 'path' you define in your sinatra application takes into account a root URL -- the path to the folder where your app.rb file lives. In our Hello World example, this root URL is:

http://itp.nyu.edu/~yourNetID/sinatra/example_sinatra_app/

Later, when you learn how to create additional sinatra applications on the ITP server, you'll have a folder and an app.rb file (or another_app.rb, or kittens_and_unicorns.rb) for each of these different applications. Each app's root URL will still be the path to the folder where that application's app.rb file lives.

http://itp.nyu.edu/~yourNetID/sinatra/example_sinatra_app/

http://itp.nyu.edu/~yourNetID/sinatra/another_app/

You can define any number of paths you want off of your root URL. In your app.rb file, you should now add more routes, like this:

require 'sinatra'

# When a GET request comes to my webserver at /helloworld

# send back the text “Hello!”

get '/helloworld' do          

    'Hello!'

end

### Now add three new routes

# GET request to the base URL plus /goodbyeworld

get '/goodbyeworld' do          

    'Goodbye!'

end

# GET request to the base URL plus /joke

get '/joke' do          

    'What did the Buddha say to the hot dog guy? Make me one with everything!'

end

# GET request to the base URL ONLY, a path of ‘/’

get '/' do

    'Wll your base URLs are belong to us"

end

### Now add a route of your own here, in the same form

After you finish this excercise and save the file, you should now be able to visit 5 different URLs from your browser, the original

itp.nyu.edu/~yourNetId/sinatra/example_sinatra_app/helloworld 

plus

itp.nyu.edu/~yourNetId/sinatra/example_sinatra_app/goodbyeworld 

itp.nyu.edu/~yourNetId/sinatra/example_sinatra_app/joke 

itp.nyu.edu/~yourNetId/sinatra/example_sinatra_app/

plus one of your own making. Now we’re getting enough different routes to start to think about managing them.

Route Matching: Order

In order for a Sinatra route to execute, both the action and the path must match the request. The first route that satisfies the request is executed. Sinatra then stops looking for matches and resets, waiting for the next request to come in.

Look at these two routes:

get '/helloworld' do

    "i'm the first match!"

end

get '/helloworld' do

    "i match too"

end

If you direct your browser to the URL ending in /helloworld, you'll only ever see "i'm the first match!". The route containing “i match too” will never execute. The first route that satisfies the request gets executed.

If you change the action from get to post on the first route and visit /helloworld,your script will skip the first route (because browser requests are get by default; post requests are generated by forms). The second route will still match, though, returning “i match too”.

If you change the action from get to post in both of the above routes, and then try to access them, you'll get a "404/Sinatra doesn't know this ditty" error.

Returning to the Client

In all of the examples above, the instructions to the server have been composed of a single line of code. What happens if you add another?

get '/helloworld' do

    "hello"

    "goodbye"

end

In this case, if you direct your browser to the URL ending in /helloworld, you'll only ever see "goodbye”. Even though there are two strings in the route above, the last line (and only the last line) is always returned to the client.

Review of Basic Routes

Routes are the basic building blocks of Sinatra applications. When you direct your browser to a base URL in your Sinatra folder:

1) Your browser sends a GET request to that URL

2) The Sinatra script at the root URL receives the request, then looks in app.rb to see if there is a route that matches your request

3) The route has to match both action (like get) and path (like /helloworld).

3a) If there is not, it returns an error page to your browser (the “404/Ditty” page.)

4) The first route that matches is the one that gets returned to the browser

5) If there is a matching route, the script executes whatever code is inside the route.

6) The script treats the last line of the code as the response, and returns it to your browser.

7) The data returned by the script shows up in your browser.