Jem Young

@jemyoung

Senior Software Engineer

Slides

jemyoung.com/fsfe

Part 1

getting started

• “Full Stack”
• Command line
• Shells
• How the internet works
• VIM

What is “Full Stack?”

Frontend

Backend

magic?

Frontend

browsers

desktop apps

mobile devices

cars

televisions

appliances

security

reliability

databases

APIs

data analytics

platform

Backend

What is “Full Stack?”

Full Stack Engineer

Someone who can build an application from start to finish

Why Full Stack?

Command Line

Command Line

• Fast
• Consistent
• Easier to automate

​

Why the command line?

Command Line

Exercise

Command Line

• cd - change directory
• ls - list directory contents
• pwd - print working directory
• mkdir - make directory
• rmdir - remove directory

• cat - show file contents
• man - command manual
• less - show file contents by page
• rm - remove file
• echo - repeat input

Shells

Shells

Shells

Shell

Command interpreter to interface with system

Terminal

Runs shell applications

$ echo $0

Shells

Show current shell

shell

application

kernel

How does the internet work?

How does the internet work?

Internet

A system of globally interconnected devices

How does the internet work?

Intranet

Private internet

How does the internet work?

IP

Internet Protocol

IP Address

A label assigned to an internet connected device

How does the internet work?

IPv4

8.8.8.8

IPv6

2001:4860:4860:8888

How does the internet work?

TCP

Transmission Control Protocol

UDP

User Datagram Protocol

Ping

Exercise

$ ping twitter.com

How does the internet work?

DNS

Domain Name System

How does the internet work?

subdomain

domain

tld

How does the internet work?

jemyoung.com

====>

23.23.185.61

How does the internet work?

Nameservers

Map domains to IP addresses

Trace Route

Exercise

$ traceroute google.com

$ man traceroute

Traceroute manual

Run traceroute

$ traceroute frontendmasters.com

How does the internet work?

ICMP

Internet Control Message Protocol

ping

traceroute

How does the internet work?

Packet

(Again) How does the internet work?

domain

IP

TCP

?

VIM

VIM

VIM

Vi Improved

VIM

command mode

insert mode

last line mode

Primary mode

Text editing

Searching, saving, exiting

VIM

command mode

insert mode

last line mode

ESC

i

:

VIM

ESC

:q!

HOW TO QUIT VIM

VIM

https://linuxmoz.com/vi-commands-cheat-sheet/

​

http://vim.wikia.com/wiki/Copy,_cut_and_paste

VIM

Exercise

$ vi

$ man vi

Creating and editing in VIM

Exercise

$ vi temp

Part 2

servers ftw

• Servers
• The cloud
• VPS
• SSH
• Buying a domain

Servers

Servers

What does a server do?

Servers

Let’s make a server!

Exercise

$ vi simpleServer.js

$ node simpleServer.js

Run NodeJS script

Servers

Servers

Servers

Datacenter

The cloud

The cloud

The cloud

The cloud

VPS

Virtual Private Server

Buying a VPS

Exercise

VPS

www.digitalocean.com

Operating Systems

Operating Systems

Operating Systems

Two main types of server operating systems

windows

unix

BSD

solaris

linux*

freeBSD

OSX/MacOS

ubuntu

red hat

*there are a lot more than 3

debian

Operating Systems

Kernel

Utilities

Operating Systems

Operating Systems

LTS

Long Term Support

SSH

SSH

username/password

Authentication

ssh key

SSH

SSH

Secure Socket Shell

SSH

public key

private key

SSH key pair

SSH

your computer

server

(secret)

encrypted messages

public key

private key

SSH Key

Exercise

$ cd ~/.ssh

​

$ ls

​

$ ssh-keygen

Move into .ssh directory

List files

Generate ssh key

public key

private key

Logging in

Exercise

$ ssh root@YOUR_IP

​

​

​

SSH into server

$ ssh -i ~/.ssh/fsfe root@YOUR_IP

​

​

​

SSH into server with key file

$ ssh-add -K ~/.ssh/fsfe

​

​

​

$ vi ~/.ssh/config

​

​

​

Add private key to keychain

Make sure keychain is active

$ ssh root@YOUR_IP

​

​

​

SSH into server

Buying a domain

Exercise

Buying a domain

www.namecheap.com

DNS Records

DNS Records

A record

Maps name to IP address

CNAME

Maps name to name

DNS Records

www.jemyoug.com

23.23.185.61

DNS Records

blog.jemyoug.com

23.23.185.61

$ dig blog.jemyoung.com

​

​

​

DNS Records

Domain Setup

Exercise

DNS Records

Add 2 A Records with your IP address

• @
• www

• Use digital ocean to add domain
• Update nameserver on namecheap

Part 3

Satisfying Server Setup

• Server setup
• Nginx
• NodeJS

​

Server Setup

Server Setup

1. Update software
2. Create a new user
3. Make that user a superuser
4. Enable login for new user
5. Disable root login

​

Server Setup

# apt update

​

# adduser $USERNAME

# apt upgrade

​

Update software

Upgrade software

Add new user

Server Setup

# usermod -aG sudo $YOUR_USERNAME

Add user to “sudo” group

# su $YOUR_USERNAME

Switch user

$ sudo cat /var/log/auth.log

Check sudo access

Server Setup

$ cd ~

Change to home directory

$ mkdir -p ~/.ssh

Create a new directory if it doesn’t exist

$ vi ~/.ssh/authorized_keys

​

Create authorized_keys file and paste PUBLIC key

Server Setup

$ exit

# exit

Exit

$ ssh $USERNAME@IP_ADDRESS

Login with new user

Server Setup

$ chmod 644 ~/.ssh/authorized_keys

Change file permissions

$ sudo vi /etc/ssh/sshd_config

Disable root login

Server Setup

$ sudo service sshd restart

Restart ssh daemon

Nginx

Nginx

Nginx

(engine-x)

• Reverse proxy
• Web server
• Proxy server

Nginx

App

Database

request

Nginx

App

Database

request

Nginx

Nginx

Exercise

$ sudo apt install nginx

$ sudo service nginx start

Install nginx

Start nginx

Nginx

domain

IP

Nginx

?

Nginx

$ sudo less /etc/nginx/sites-available/default

Show nginx configuration

Nginx

base directory for requests

Nginx

location block

Nginx

directive

Nginx

html defaults

Default page

Exercise

$ sudo vi /var/www/html/index.html

Create and edit index.html

NodeJS

NodeJS

$ sudo apt install nodejs npm

$ sudo apt install git

NodeJS

Install NodeJS and npm

Install git

Part 4

Master of the stack

• Application Server
• Process Manager
• Git

​

Application Architecture

Application Architecture

Application Setup

Exercise

$ sudo chown -R $USER:$USER /var/www

$ mkdir /var/www/app

$ cd /var/www/app && git init

Change ownership of the www directory to the current user

Create application directory

Move into app directory and initialize empty git repo

$ mkdir -p ui/js ui/html ui/css

$ touch app.js

$ npm init

Create directories

Create empty app file

Initialize project

$ npm i express --save

$ vi app.js

install express

edit app

https://www.dropbox.com/s/8wlwz5wdp7ohe16/Screenshot%202019-09-18%2022.34.07.png?dl=0

https://expressjs.com/en/starter/hello-world.html

)

$ node app.js

Run application

Nginx

domain

IP

Nginx

Express

?

Nginx

location / {

proxy_pass URL_TO_PROXY_TO;

}

Proxy pass

Exercise

$ sudo vi /etc/nginx/sites-available/default

​

Edit nginx config

location / {

proxy_pass http://127.0.0.1:3000/;

}

What about server restarts?

Process Manager

Process Manager

• Keeps your application running
• Handles errors and restarts
• Can handle logging and clustering

Process Manager

Exercise

$ sudo npm i -g pm2

​

$ pm2 start app.js

​

$ pm2 save

$ pm2 startup

​

Install PM2

Start PM2

Setup auto restart

Version Control

Version Control

• Git
• Subversion
• Mercurial

​

Version Control

Record changes to a file system to preserve the history.

Static Files

Static Files

app.use(express.static(‘ui’))

Git

Exercise

1. Create git repository
2. Create SSH key
3. Add SSH key to Github
4. Add remote repo
5. Push local repository to Github*

*Don’t forget to add a .gitignore file so you don’t commit node_modules/

Congratulations!

Further Exploration

Install Fail2ban

• https://www.techrepublic.com/article/how-to-install-fail2ban-on-ubuntu-server-18-04/

ExpressJS performance tips

• http://expressjs.com/en/advanced/best-practice-performance.html

​

​

Recap

Frontend

Backend

magic?

Recap

domain

IP

Web server

Application server

Server

Recap

• Command line basics
• How the internet works
• How to create and manage a web server
• Create a Node application

Part 1

bit of this, bit of that

• Standard streams
• Redirection
• Finding things
• Nginx

​

​

Standard Streams

Standard Streams

• standard output
• stdout
• standard input
• stdin
• standard error
• stderr

Redirection

Redirection

• |
• read from stdout
• >
• write stdout to file
• >>
• append stdout to file
• <
• read from stdin
• 2>
• read from stderr

Redirection

foo > bar

What does this do?

Finding Things

Finding things

grep

find

search file names

search file contents

Finding things

find

find

directory

option

file/folder

Finding things

find

useful options

• -name
• -type
• -empty
• -executable
• -writable

Find

Exercise

$ find /var/log/nginx -type f -name “*.log”

​

Find all log files in /var/log

$ find / -type d -name log

​

Find all directories with the name ‘log’

Finding things

grep

$ grep -i ‘jem’ /var/www

grep

options

search

expression

directory

$ zgrep FILE

search inside gzip file

Grep

Exercise

$ ps aux | grep node

Find running node processes

Nginx

Nginx - Redirect

location /help {

return 301 https://developer.mozilla.org/en-US/;

}

Nginx - Subdomain

server {

listen 80;

listen [::]:80; # IPV6 notation

​

server_name test.jemisthe.best;

​

location / {

proxy_pass http://localhost:3000;

}

}

Nginx - Gzip

/etc/nginx/nginx.conf

Part 2

Security

• Controlling access
• Securing applications
• Firewalls
• Permissions
• Upgrade NodeJS

​

​

​

Security

$ sudo cat /var/log/auth.log

Read auth.log

Security

What could someone do if they gained root access to your server?

Security

• SSH
• Firewalls
• Updates
• Two factor authentication
• VPN

Updates

$ sudo apt install unattended-upgrades

$ cat /etc/apt/apt.conf.d/50unattended-upgrades

Application Updates

Install unattended upgrades

View conf file

Firewalls

Firewall

Firewall

“A network security device that monitors incoming and outgoing network traffic and decides whether to allow or block specific traffic based on a defined set of security rules.”

https://www.cisco.com/c/en/us/products/security/firewalls/what-is-a-firewall.html

Firewall

nmap

Exercise

$ man nmap

$ nmap YOUR_SERVER_IP_ADDRESS

$ sudo apt install nmap

$ nmap -sV YOUR_SERVER_IP_ADDRESS

Install nmap

Run nmap

Run nmap with more service/version info

Firewall

port

communication endpoint that maps to a specific process or network service

$ less /etc/services

Firewall

iptables -p tcp --dport 80 -j REJECT

Firewall

Firewall

ufw

uncomplicated firewall

$ ufw allow ssh

Firewall

deny

reject

http

https

ufw

Exercise

$ sudo ufw allow ssh

$ sudo ufw status

$ sudo ufw enable

Check firewall status

Enable ssh

Enable firewall

How would you create a ufw rule to block all HTTP connections?

sudo ufw reject http

Permissions

Permissions

https://isabelcastillo.com/linux-chmod-permissions-cheat-sheet

Upgrade node

Exercise

$ curl -sL https://deb.nodesource.com/setup_10.x -o nodesource_setup.sh

$ sudo bash nodesource_setup.sh

$ sudo apt install nodejs

Download setup script from nodesource

Run script

Install nodejs

$ sudo npm update -g

Application Updates

Update outdated packages

Part 3

HTTP/*

• HTTP
• Headers
• Status codes
• HTTPS
• Certbot
• HTTP/2
• HTTP/3

​

​

HTTP

HTTP

HTTP

hypertext transport protocol

HTTP

request

response

HTTP

HTTP

request

GET / HTTP/1.1

Host: jemisthe.best

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36

Accept: text/html

Accept-Encoding: gzip, br

Accept-Language: en,en-US

HTTP Headers

HTTP

request

GET / HTTP/1.1

Host: jemisthe.best

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36

Accept: text/html

Accept-Encoding: gzip, br

Accept-Language: en,en-US

HTTP

common headers

HTTP

response

HTTP/1.1 200 OK

Server: nginx/1.14.0 (Ubuntu)

Date: Wed, 25 Sep 2019 02:13:13 GMT

Content-Type: text/html; charset=utf-8

Content-Length: 12

Status Codes

Status Codes

HTTP Status Code

Indicates the status of an HTTP request

Status Codes

Status Codes

What is the proper status code for a successful POST request?

Headers and Status Codes

Exercise

In your app, create a path that returns 418 and a custom header.

hint: res.status(), res.set()

HTTPS

HTTPS

frontendmasters.com

Visa 4235529596901600

HTTPS

frontendmasters.com

Visa 4235529596901600

Evil Server

HTTPS

*Ifkfi24fh243&&o9#$kD#!8b

https://frontendmasters.com

HTTPS

Certbot

Exercise

https://certbot.eff.org/

HTTP/2

HTTP/2

TCP

TCP

TCP

http/1.1

HTTP/2

TCP

http/2

http/2

Exercise

$ sudo vi /etc/nginx/sites-available/default

Edit nginx config

listen 443 http2 ssl;

HTTP/3

HTTP/3

QUIC

Quick UDP Internet Connections

Part 4

Containers

• Microservices
• Containers
• Orchestration
• Load balancers
• Deployments

​

​

Microservices

Microservices

Microservices

Architecture of loosely connected services

Microservices

Containers

The cloud

Containers

VPS

Containers

Ubuntu

NodeJS

MySQL

Python

Nginx

Containers

Ubuntu

NodeJS

MySQL

Nginx

Python

Containers

OSX

NodeJS

MySQL

Nginx

Python

Fedora

NodeJS

MySQL

Nginx

Python

Containers

• Lightweight
• Portable
• Easier for development
• Easier to manage
• Faster startup
• Decouple application from infrastructure

Containers

• Amazon ECS
• Apache Mesos
• CoreOS rkt

Orchestration

Orchestration

Orchestration

• Docker Swarm
• Amazon EKS
• Apache Mesos
• AKS

Kubernetes

“K8s”

Load Balancers

Load Balancers

90%

15%

?

request

Load Balancers

90%

15%

request

Load Balancer

Load Balancers

Scheduling Algorithms

• Round Robin*
• IP Hashing
• Random Choice
• Least Connections
• Least Load

top

Exercise

$ top

$ sudo apt install htop

$ htop

Display running processes

Install htop

Display running processes

Load Balancers

Load Balancers

https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/

Deployments

Deployments

Part 5

Saving Data

• Files
• Databases

​

​

​

Files

Files

Why not save everything to a file?

Databases

Databases

• SQL
• Tables
• Related data
• Strict structure

relational

Databases

• NoSQL
• Data agnostic
• Loose structure

non-relational

Redis

$ sudo apt install redis-server

$ sudo vi /etc/redis/redis.conf

$ sudo systemctl restart redis.service

Redis

Install redis server

Edit config to start with system

Restart redis server

Redis

https://github.com/NodeRedis/node_redis

MySQL

MySQL

Install mysql server

Run setup

$ sudo apt install mysql-server

$ mysql_secure_installation

MySQL

https://github.com/mysqljs/mysql

Part 6

Final Project

• Websockets

​

​

Websockets

Websockets

Websocket

Persistent bidirectional connection between client and server

Websockets

location / {

proxy_set_header Upgrade $http_upgrade;

proxy_set_header Connection "upgrade";

​

proxy_pass http://127.0.0.1:3000;

}

Final Project

Final Project

1. Create a chat bot using websockets

https://github.com/young/fsfev2

Congratulations!