Unit 1

Docker is a very powerful containerization platform – not to mention very popular. It is used extensively and provides a large number of integrations, including GitHub, Jenkins, Kubernetes and Terraform. We will be exploring the concepts and how best we can leverage Docker.

Presented by – Seshagiri Sriram

Getting Started Docker

www.guvi.in

2

Why do we need Docker?

Shipping Transportation Challenges

www.guvi.in

3

​

​

Problem

• When goods are transported, they have to pass through a variety of different means Ex:- trucks, forklifts, cranes, trains, and ships.
• These means have to be able to handle a wide variety of goods of different sizes and with different requirements e.g. sacks of coffee, drums of hazardous chemicals, boxes of electronic goods, fleets of luxury cars, and racks of refrigerated lamb).
• This was a cumbersome and costly process, requiring manual labor, such as dock workers, to load and unload items by hand at each transit point

www.guvi.in

4

​

​

Solution

• The transport industry was revolutionized by the introduction of the intermodal container.
• These containers come in standard sizes and are designed to be moved between modes of transport with a minimum of manual labor.

Advantages :

• All transport machinery is designed to handle these containers, from the forklifts and cranes to the trucks, trains, and ships.
• For Ex :- Refrigerated and insulated containers are available for transporting temperature sensitive goods, such as food and pharmaceuticals.
• The benefits of standardization also extend to other supporting systems, such as the labeling and sealing of containers.
• This means the transport industry can let the producers of goods worry about the contents of the containers so that it can focus on the movement and storage of the containers themselves.

Shipping Transportation Challenges

Relevance in IT Industry

www.guvi.in

5

​

​

Current IT Industry Problem :

• We have a similar issue to the one seen by the transport industry—we have to continually invest substantial manual effort to move code between environments.
• A typical modern system may include Javascript frameworks, NoSQL databases, message queues, REST APIs, and backends all written in a variety of programming languages
• This stack has to run partly or completely on top of a variety of hardware—from the developer’s laptop and the in-house testing cluster to the production cloud provider.

​

www.guvi.in

6

​

​

Solution

Much as the intermodal containers simplified the transportation of goods, Docker containers simplify the transportation of software applications.

Developers can concentrate on building the application and shipping it through testing and production without worrying about differences in environment and dependencies.

Operations can focus on the core issues of running containers, such as allocating resources, starting and stopping containers, and migrating them between servers.

Relevance in IT Industry

How Does Docker Fit Into Devops Eco-System

www.guvi.in

7

Docker forms a lethal combination with Jenkins:

The lethal combination of Jenkins and Docker is proving to be very valuable for DevOps teams.

By leveraging the tight integration with source code control mechanisms such as Git, Jenkins can initiate a build process each time a developer commits his code. This process results in a new Docker image which is instantly available across environments.

Organizations are deploying private Docker registries to publish and maintain their internal Docker images

​

Docker makes it easier to test exactly what you deploy:

Docker containers encourage a central tenet of continuous delivery – reuse the same binaries at each step of the pipeline to ensure no errors are introduced in the build process itself. The complexities of the configuration Management tools are considerable and complicated to manage. Most of the problems stem from complex integrations.

Example: You have to make sure that you are using the same libraries and other important elements every time.

Making sure that all those environments are perfectly aligned can be a lot of work.

Docker solves that because you are starting with a given base image, and it’s guaranteed to be the same image everywhere.

www.guvi.in

8

Docker has revolutionized software packaging and deployment:

Instead of deploying the final set of artifacts such as EXE and JAR files to the target environment, ops teams are now packaging the entire application as a Docker Image.

This image shares the same build version before getting published to a central registry.

It is then picked up by various environments – development, testing, staging, and production – for final deployment

​

Docker containers provide the basis for immutable infrastructure:

Applications can be added, removed, cloned and/or its constituencies can change, without leaving any residues behind. Whatever mess a failed deployment can cause is constrained within the container.

Deleting and adding become so much easier that you stop thinking about how to update the state of a running application.

How Does Docker Fit Into Devops Eco-System

www.guvi.in

9

What is Docker ?

Docker

www.guvi.in

10

Docker is a Containerization platform which packages your application and all its dependencies together in the form of Containers so as to ensure that your application works seamlessly in any environment be it Development or Test or Production.

Host OS

Docker Engine

App 1

BINS / LIBS

Container 1

Container 2

App 2

BINS / LIBS

www.guvi.in

11

Containers v/s Virtual Machines

Virtualization; Adopted By VMs

www.guvi.in

12

Host OS

Hypervisor

Guest OS

Guest OS

Guest OS

BINS�&�LIBS

BINS� &� LIBS

BINS �&�LIBS

App 1

App 2

App 3

Virtualization Technique

VMs

• Multiple OS In Same Machine
• Easy Maintenance & Recovery
• Lower Total Cost Of Ownership

Advantages

Disadvantages

• Multiple VMs Lead To Unstable Performance
• Hypervisors Are Not As Efficient As Host OS
• Long Boot-Up Process ( Approx. 1 Minute )

Containerization

www.guvi.in

13

Host OS

App 3

Container Engine

App 1

App 2

Containerization Technique

BINS & LIBS

BINS�&�LIBS

Containers

• Containers On Same OS Kernel Are Lighter & Smaller
• Better Resource Utilization Compared To VMs
• Short Boot-Up Process ( 1/20^th of a second )

BINS & LIBS

Advantages Over Virtualization

Note: Containerization Is Just Virtualization At The OS Level

www.guvi.in

14

Benefits of Docker over VM’s

VM vs. Docker

www.guvi.in

15

Resource/Memory Utilization

www.guvi.in

16

Total Memory:

16 GB

4 GB

2 GB

🡪

Memory Used: 9 GB

Memory wasted: 7 GB

VM 1

3 GB

1 GB

2 GB

4 GB

VM 2

VM 3

In case of Virtual Machines

🡪

6 GB

4 GB

6 GB

7 Gb of Memory is blocked and cannot be allotted to a new VM

www.guvi.in

17

Total Memory:

16 GB

4 GB

2 GB

🡪

Memory Used: 9 GB

Memory wasted: 7 GB

VM 1

3 GB

1 GB

2 GB

4 GB

VM 2

VM 3

Memory Allotted: 4 GB

App 1

App 2

App 3

Memory Used: 9 GB

In case of Virtual Machines

In case of Docker

🡪

🡪

6 GB

4 GB

6 GB

Total Memory:

16 GB

Memory Allotted: 35 GB

Memory Allotted: 10 GB

7 Gb of Memory is blocked and cannot be allotted to a new VM

Only 9 GB memory utilized;�7 GB can be allotted to a new Container

4 GB

3 GB

2 GB

Resource/Memory Utilization

www.guvi.in

18

Host OS Kernel

Guest OS

Guest OS

BINS & LIBS

BINS & LIBS

Ruby v1

Ruby v2

Host OS Kernel

Ruby v1

Ruby v2

BINS & LIBS

New Builds 🡪 Multiple OS 🡪 Separate Libraries 🡪 Heavy 🡪 More Time

New Builds 🡪 Same OS 🡪 Separate Libraries 🡪 Lightweight 🡪 Less Time

Build 1

Build 2

Build 1

Build 2

In case of Virtual Machines

In case of Docker

BINS & LIBS

Container

Virtual Machines

Building and Deployment

Integration in VMs

www.guvi.in

19

Integration In Virtual Machines Is Possible, But:

• Costly Due To Infrastructure Requirements
• Not Easily Scalable

Jenkins

When to use what?

• Containers allow you to run more applications on a physical machine than VMs.
• If resources are a constraint, containers may be a better choice.
• With containers you can create a portable, consistent operating environment for development, testing, and deployment.
• From a security viewpoint, important subsystems like SELINUX are outside the scope of containers
• therefore a person with super user privileges in a container can in theory undermine underlying operating system.
• There is no easy way to make a packaged box handle all upstream dependencies.
• Wrong packaging is harder to debug and resolve, thus increasing time spent in QA activities.

​

www.guvi.in

20

When to use what?

• What is the scope of your work? In our example, we have only Jenkins.
• If we needed to run other applications, it makes more sense to use a VM over a container.
• Do you plan to use this instance of Jenkins only on one OS? If yes, proceed with a container over a VM.
• The thumb rules are
• If planning to use multiple instances of an application over minimum number of servers, containers are the best options.
• If planning to run multiple applications with greater requirement for security, then use VM.
• In all cases, a proper Cost Benefit Analysis process is required to decide which option is best for your organization.

​

www.guvi.in

21

Docker – A DevOps Ally

www.guvi.in

22

Jenkins

Integration in Docker is Faster, Cheap & Easily Scalable

www.guvi.in

23

Architecture

Docker Architecture

www.guvi.in

24

docker pull

docker build

docker run

Docker Daemon

Client

Docker_Host

Registry

Containers

Images

Docker Daemon

• The Docker daemon runs on a host machine.
• The user uses the Docker client to interact with the daemon.

​

www.guvi.in

25

• Responsible for creating, running, and monitoring containers
• Building and storing images

Why Use Docker Daemon?

Docker Client

• The Docker client is the primary user interface to Docker.
• It accepts commands and configuration flags and communicates with a Docker daemon via HTTP.
• One client can even communicate with multiple unrelated daemons.

www.guvi.in

26

• Since all communication has to be done over HTTP - it is easy connect to remote Docker
• The API used for communication with daemon allows developers to write programs that interface directly with the daemon, without using Docker

Why Use Docker Daemon?

Docker Registry

• Docker Registry is a storage component for Docker Images
• We can store the Images in either Public / Private repositories
• Docker Hub is Docker’s very own cloud repository

www.guvi.in

27

• Control where your images are being stored
• Integrate image storage with your in-house development workflow

Why Use Docker Registries?

Private or Public Registry?

• When choosing between these, some points to consider are:
• Performance, depending mainly on roll-out frequency and cluster size.
• Security issues such as access control and digitally signing Docker image.
• With a private registry, you
• Are in full control.
• No external dependencies in the CD pipeline, so your build really do build faster (or appear to be faster).
• Have to manage storage yourself – and this can increase drastically as the adoption of Devops and number of build increase.
• Before doing any decisions,
• Evaluate how many builds/images are being pushed and the average increase in size.
• Also factor in growth rates for your applications/builds
• These will give you metrices for determining network bandwidth and storage requirements.

www.guvi.in

28

Which Public Registry?

• Factors for making a selection include:
• Costs for storage and support for number of images
• Workflow support including integration with your version control system
• Some of them like Coreos Quay.IO fully support integration with GIT.
• You should not have to drastically change your workflow to support external registries.

​

www.guvi.in

29

Docker Images & Containers

www.guvi.in

30

• Read Only Template Used To Create Containers
• Built By Docker Users
• Stored In Docker Hub Or Your Local Registry

Docker Images

Docker Containers

• Isolated Application Platform
• Contains Everything Needed To Run The Application
• Built From One Or More Images

run

Docker Architecture In Action

www.guvi.in

31

docker pull

docker build

docker run

Docker Daemon

Client

Docker_Host

Registry

Containers

Images

build

Docker Architecture In Action

www.guvi.in

32

docker pull

docker build

docker run

Docker Daemon

Client

Docker_Host

Registry

Containers

Images

build

pull

Docker Architecture In Action

www.guvi.in

33

docker pull

docker build

docker run

Docker Daemon

Client

Docker_Host

Registry

Containers

Images

build

pull

run

www.guvi.in

34

Docker’s Plugins And Plumbing

​

Docker - Plugin’s And Plumbing

www.guvi.in

35

Docker - Plugin’s And Plumbing

www.guvi.in

36

• Docker’s on premise solution for storing and managing Docker images
• A local version of Docker Hub that can integrate with an existing security infrastructure and helps organizations comply with regulations regarding the storage and security of data
• Only non–open source product from Docker Inc.

Docker Trusted Registry

• Used to group several Docker hosts, allowing the user to treat them as a unified resource

Docker Swarm – Docker’s Clustering Solution

Orchestration and cluster management

• In large container deployments, tooling is essential in order to monitor and manage the system
• Each new container needs to be placed on a host, monitored, and updated
• The system needs to respond to failures or changes in load by moving, starting, or stopping containers appropriately.
• Several competing solutions in the area, including Kubernetes from Google

www.guvi.in

37

Docker Topics

Docker Topics

www.guvi.in

38

www.guvi.in

39

Working with Docker Images

What is an Image?

• An image is basically a text file with a set of pre-written commands and saved as a file usually called as a Docker file
• Docker images are made up of multiple layers which are read-only filesystem
• A layer is created for each instruction in a Docker file and sits on top of the previous layers
• When an image is turned into a container the Docker engine takes the image and adds a read-write filesystem on top (as well as initializing various settings such as the IP address, name, ID, and resource limits)

www.guvi.in

40

Few Basic Commands

www.guvi.in

41

• Few Basic Commands:
• $ docker help :- Displays all the useful commands for Docker and other general help commands
• $ docker images :- Displays a list of existing images in Docker system. It also displays the following details:
• REPOSITORY: Name of the repository
• TAG: Every image has an attached tag.
• IMAGE ID: Each image is assigned a unique ID
• CREATED: The date when the image was created
• SIZE: The size of the image

​

Few Basic Commands

www.guvi.in

42

• $ docker ps :- Displays the list of active containers. It also displays the following details:
• CONTAINER ID: Each container is assigned a unique ID
• IMAGE: Every image has an attached tag.
• COMMAND: Each image is assigned a unique ID
• CREATED : The date when the image was created
• STATUS: This shows whether the container is active or not
• PORTS: The number of exposed ports (needed for networking)
• NAMES: Name of container which is automatically assigned by Docker. It contains first name, “_” and last name
• $ docker ps – a :- Displays the list of all the container processes which are running or have run in the past

Create Our First Image: Hello-World

www.guvi.in

43

• The following actions will be performed to pull the image:
• Search for images which start with “hello” word from Docker hub
• Select the image which we are going to pull
• Pull the selected image from Docker Hub
• Search for the copy of the image in our local repository
• Initiate a container based on “Hello-world” image
• Examine container Details

​

Search – Hello World

www.guvi.in

44

• Command: $ docker search hello

• Command: $ docker pull hello-world

Image Pull – Hello World

www.guvi.in

45

• Command: $ docker images

Result Details:

• REPOSITORY: This is our local repository
• TAG: Since we did not specify the Tag in our command, we got the latest version
• IMAGE ID: This is the ID of the image
• CREATED: This is the time when the image was created
• SIZE: This is the size of the image

Image History

www.guvi.in

46

• Image History:
• Can get the history of the base image to understand how the base image was built
• Can see the different layers that were used during the build of image
• Gives ImageID, command that created particular layer and the size of the layer

​

• Command: $ docker history ImageName

​

Image History – Why

www.guvi.in

47

• Psychologically users tend to use the latest images. But sometimes we may need to use older images for following reasons.
• Supporting clients who have not yet been migrated to newer technologies. Assume Client A is requiring Library Version 1.0 which was released around January 2013. An image built around this time is more likely to be supporting Library Version 1.0
• History also allows to see if a particular images (esp. from a 3^rd party) is even under active development/support.
• As an example, if an image was last built in January 2012, it is unlikely we will consider this as a base image for our development.

​

Tagging

www.guvi.in

48

• Tagging:
• Each image has a default tag associated with it
• Default tag is set by the image maintainer
• Command: $docker tag centos (Image ID or Image Name) Tag (Name of the desired tag)

Tagging – Types and best Practices

www.guvi.in

49

• Docker uses the term tagging to refer to a label applied to an image (e.g. –t) or refer to string applied to end of image name (e.g. jenkins:latest). The latter is usually referred to as a version tag.
• For version tags, there are really no clear cut best practices
• Usually practices for versioning in GIT are applied as is to docker tagging also.
• Docker’s automated builds lets a user link a “version tag” to either to a branch or a tag in the git history.
• A “branch” in this case can refer either to a different git branch or merely a different sub-directory.
• Matching to a Git tag provides the most clear-cut use of the docker version-tag; providing a relatively static version stable link. (Food for thought: Is this a good practice or not?)
• Using the version tag to indicate any other difference is a widespread practice but with no clear use case except for supporting multiple dockerfiles in same repository.

​

Image Distribution - Repositories

www.guvi.in

50

• What is a Repository?
• A collection of images.
• There can be three kinds of repositories
• Local: Can be saved on the system. All the images which are pulled from Public or Private repository gets saved on the local repository
• Private:
• Can get one free from Docker
• If you need other private repositories then you need to pay
• Requires Username and Password
• Public (Docker Hub):
• Need to sign up. We will be discussing this in detail

Hierarchy For Image Storing

www.guvi.in

51

• There is a hierarchical system for storing images, where following terminology is used:

Best practices for Image Storing

www.guvi.in

52

• As far as possible, use namespaces (We will be dealing with namespaces later on)
• Version tags as far as possible should be mapped to GIT branching tags and not used for indicating other differences.
• Clearly indicate for personal images, the use case for the image e.g. dev, qa, production.
• Assume you are the person responsible for pushing images to the repository. Automate the process of pushing images to the repository, which involves:
• Push images
• Remove images locally
• Repull images (Sounds close enough to GIT Best Practices?)

Pushing Images to Docker Hub

• Step 1: To push the images to Docker Hub, first login to Docker Hub
• Step 2: Create a new public repository with your name
• Step 3: Make sure your local image’s name is the same as Docker repo’s name
• Step 4: If it is not the same, tag your local image to give it the same repo name as the repository you created on Docker Hub using the command: $ docker tag <Local image name> <Docker Hub repo name>
• Step 4: Now, to push the Image to Docker Hub, use the command: $ docker push <Docker Hub repo name>

www.guvi.in

53

Pushing Images to Docker Hub(Contd..)

www.guvi.in

54

Pushing Images to Docker Hub(Contd..)

www.guvi.in

55

Pushing Images to Docker Hub(Contd..)

www.guvi.in

56

• Image is pushed to Docker hub using command: $ docker push seshagirisriram(username)/sriram_hello_world(Name of image)

Image Namespaces

www.guvi.in

57

• Namespaces:
• Namespacing ensures users cannot be confused about where images have come from
• Example: If using the Centos image, it is the official image from Docker Hub and not some other registry’s version of Centos image
• Following are three namespaces pushed Docker images, which can be identified from the image name:
• Names Prefixed With A String:
• Names prefixed with a string and /, such as /nginx belong to the “user” namespace
• These are images on Docker Hub that have been uploaded by a given user
• Example: docker/nginx is the nginx image uploaded by the user docker

​

​

Image Namespaces

www.guvi.in

58

• Simple Names:
• Names such as Debian and Ubuntu, with no prefixes or /s, belong to “root” namespace
• There are official images for most common software packages, which should be your first port of call when looking for an image to use

​

• Names Prefixed With Hostname or IP:
• Names prefixed with a hostname or IP are images hosted on third-party registries (not the Docker Hub)
• These include self - hosted registries for organizations, as well as competitors to the Hub, such as quay.io
• Example: localhost:5000/wordpress refers to an WordPress image hosted on a local registry

​

​

​

Image Selection – Base Image

www.guvi.in

59

• Base Images
• When creating your own images, you will need to decide which base image to start from
• The best-case scenario is that you don’t need to create an image at all you can just use an existing one and mount your configuration files and/or data into it
• This is likely to be the case for common application software, such as databases and web servers, where there are official images available
• In general, you are better off using an official image than rolling your own

• Image Selection – Base Image(Contd..)

www.guvi.in

60

• Benefits of Using Base Images
• You get the benefit of other people’s work and experience in figuring out how best to run the software inside a container

​

• If Base Image Is Not Available
• If there is a particular reason an official image doesn’t work for you, consider opening an issue on the parent project, as it is likely others are facing similar problems or know of workarounds.

• When to select base image and when to create

www.guvi.in

61

• Benefits of Using Base Images
• You get the benefit of other people’s work and experience in figuring out how best to run the software inside a container

​

• If Base Image Is Not Available
• If there is a particular reason an official image doesn’t work for you, consider opening an issue on the parent project, as it is likely others are facing similar problems or know of workarounds.

www.guvi.in

62

Containers

What we will cover

www.guvi.in

63

• Attaching to a container
• Container Life Cycle:
• Initiating a container
• Examining existing containers
• Naming a container
• Attaching to a container
• Stopping a container
• Restarting a container
• Removing a container
• Inspect our existing containers and look for important information

Connection Modes

www.guvi.in

64

• Container can be connected in the following two modes:

Detached Mode Vs Root Mode

www.guvi.in

65

Detached Mode

• Command: $ docker run –itd ubuntu:xenial
• I – Interactive
• T – Connected to terminal
• D – Detached mode

Root User Mode

• Command: $ docker run –it ubuntu:xenial
• I – interactive
• T – connected to terminal

​

Detached Mode Vs Root Mode(Contd..)

www.guvi.in

66

Detached Mode

• User manages from Daemon
• Container does not exit after the process within the container is over.
• Container could be stopped at a later stage though.
• Using the $docker attach command user can attach as a root user
• Allows control over other containers

Root User Mode

• User manages from the Root
• Container exits after the process within the container is over
• Container can be restarted at a later stage though
• Using the $docker exit command user can attach to the daemon
• Allows control over the container to which user is attached

Outputs in connected modes

www.guvi.in

67

• Command: $ docker ps -a

Selection between Connection Modes

www.guvi.in

68

• Containers started in detached mode exit when the root process used to run the container exits.
• A container in detached mode cannot be automatically removed when it stops.
• If this is not your use case and you do wish to automatically remove them, then you would not use the detached mode option.
• 2 examples that do not use detached mode is the use of Dockers plugins in Jenkins where we want to remove the containers after a Jenkins job is executed. This runs in a root user mode with attaching to the foreground and pretending to be a pseudo terminal and on completion close the container.
• Another example will be the starting of a service – where we need to start the service. However, using “service nginx start” with the –d option starts the nginx server but this cannot be used as is since container stops after the command executed.
• The detached mode is used when you want to run one off commands and (possibly) have the output of the commands send it outputs to some shared data volume for processing later.

Docker File System - Initiate A Container

​

www.guvi.in

69

• Command: docker run hello-world:latest

Container can be initiated using both the “Image name” as well as “Image ID” along with the required tag

Examine Existing Containers

www.guvi.in

70

• Command: docker ps

• Question: What happened to the container we initiated from our image “hello-world?

Examine Existing Containers

www.guvi.in

71

• Command: docker ps

Question: What happened to the container we initiated from our image “hello-world?

• Command: docker ps -a

Display inference:

This container with the name “pedantic_jepsen” , with CONTAINER ID (7a45c308c3fa) ,got created 3 minutes ago from the and exited 3 minutes ago as soon as its processes were executed. e

Naming Containers

www.guvi.in

72

• Docker assigns default names to the container. The usual format would be firstname_secondname.
• However Docker gives us the privilege to name our containers. This can be done with the use of the –name command

Getting Attached To a Container

www.guvi.in

73

• When we are running containers in Detached mode (Daemonised Mode) we can still attach to the container if required.
• Use the docker attach command to do so.

​

• Stopping a container: Default Mode

www.guvi.in

74

• When container is initiated in detached mode. It keeps running. It can be stopped in following two ways:
• Get attached to the root and exit
• Stop the container using the “stop” command

By default a container exists as soon as its processes are executed

Restarting A Container

www.guvi.in

75

• Start a stopped container by using the “start” command: $ docker start

Inspect Containers

www.guvi.in

76

• Inspect containers and to view important information regarding config, IP address etc. where both the running and stopped containers can be inspected by the command: $ docker inspect

Why would we want to inspect containers?

www.guvi.in

77

• The basic usage of inspecting containers is
• to verify that everything is fine
• debug issues (if any)
• The inspect command is used to inspect
• Containers
• Networks
• Nodes
• ……..
• The classical usage of these commands are in the Docker Pipeline plugin of Jenkins. This is used to work with Docker Swarm to inspect all nodes in a Swarm that can be used to run the jobs
• As a very advanced use case, the output of the docker inspect is used as an input to data visualizers to present your organization with real time data on containers, networks and nodes used in your docker infrastructure.

Removing Images: Use Cases

www.guvi.in

78

With Associated Containers

Without Associated Containers

Removing by force

Removing Images: With Associated Containers

www.guvi.in

79

• Removing Images with attached container:
• Docker will not allow you to remove an image if there are containers attached with the image
• The running containers will have to be removed first and then the image can be removed
• Using –f command, the images can be removed forcefully making the associated containers orphan which is not advisable though.

With Associated Containers

Without Associated Containers

Removing by force

Removing Images: Without Associated Containers

www.guvi.in

80

• Can use the command: $ docker rmi (Image Name/ID)

​

With Associated Containers

Without Associated Containers

Removing by force

Removing Images: Removing by force

​

www.guvi.in

81

• Docker allows to remove images with associated containers forcefully. This is not recommended though.
• Can use the –f command to perform this action.

With Associated Containers

Without Associated Containers

Removing by force

When to use remove with –f option?

• Use this very rarely and only for images that have containers.
• As best practices, containers should be disposed off when they are done.
• If this is not possible, you will need to script to dispose of the containers before removing the image.
• REITERATED WARNING – use of the –f option is never a good option, unless you know that the image itself warranted removal because of
• Poor performance
• Security issues

www.guvi.in

82

www.guvi.in

83

Docker Networking

Default Networks

www.guvi.in

84

• When Docker is installed, it creates three networks automatically, which can be listed using the docker network Command : @docker ls

Default Networks - Types

www.guvi.in

85

• Bridge Network:
• The bridge network represents the docker0 network present in all Docker installations
• Docker daemon connects containers to this network by default.

​

• Host Network:
• The host network adds a container on the hosts network stack.
• You’ll find the network configuration inside the container is identical to the host.

​

• None Network:
• The none network adds a container to a container-specific network stack. That container lacks a network interface.

Default Bridge Network Details

www.guvi.in

86

• With the exception of the bridge network, you really don’t need to interact with these default networks.
• While you can list and inspect them, you cannot remove them. They are required by your Docker installation.
• Command : $ docker network inspect bridge
• The Engine automatically creates a Subnet and Gateway to the network.
• Any new containers get added to this network

Default Bridge Network Details

www.guvi.in

87

When to use the bridge network?

• If you have a set of containers,
• each providing micro-services to the other (and)
• Should not be exposed to the external world
• The bridge network is the best choice.
• This is typically used with layered architectures.
• Be Aware that services in these containers are not exposed to outside networks.

www.guvi.in

88

• Taking the same example from above, if you have an application that needs to expose part of the network and not all, then we use custom bridge networks that expose and publish custom ports.

​

• Again a tiered architecture, where you want to connect to a web application, which in turn connects to backend application or database services is an example of same – which is standard web application deployment practice

​

• A bridge network is useful in cases where you want to run a relatively small network on a single host.

​

• You can, however, create significantly larger networks by creating an overlay network.

When to use the user defined bridge network?

www.guvi.in

89

When to use the user defined bridge network?

www.guvi.in

90

Exposing Ports

www.guvi.in

91

• Ports are exposed in the container so that the container can be using the container IP. We can connect to the http address of that port.
• We are going to direct the port that is listening to http on a container to an underlying port on our host
• We can redirect the exposed ports to the host ports.
• There are two ways of exposing the ports – by commands: P or p
• P: Any ports that are exposed by the container, any random port between 32768 and 65000 will be available on the host machine. This is the ports available for Docker to pick randomly
• The available ports can be seen in different ways
• Command to connect to a container: $ docker run – d – name = nginx – P nginx:latest

Exposing Ports(Contd..)

www.guvi.in

92

• Two ways to connect to a container:
• Connect if IP Address is known and I am on the host machine
• I am on the network of my host machine and the IP address and port of the container.
• It can also be obtainer through Docker port command : $ docker port nginx-demo $CONTAINERPORT

Exposing Ports: Use Case

www.guvi.in

93

Connection Through Host Port

www.guvi.in

94

Getting Ports – Binding Ports

www.guvi.in

95

• What if I want to bind it to a specific port on my server rather than any value which is picked in a range $ docker run –d –p 8080:80

Free or Binding ports?

• It is generally preferred to use binding ports esp. for services that are external facing and do not need to change e.g. a web server needs to be exposed on port 80.

​

• It is also used (indirectly) as a proxy so that end users do no need to remember non-standard ports like 8081 (Technically firewall rules do not need to be tweaked much).

​

www.guvi.in

96

www.guvi.in

97

Docker Volumes

• Docker - Volumes

www.guvi.in

98

• Volumes are files or directories that are directly mounted on the host and not part of the normal union file system
• Docker filesystems are temporary by default
• You can create, modify, and delete files as you wish
• If container is stopped and restarted, all the changes will be lost: any files you previously deleted will now be back, and any new files or edits you made won't be present
• Docker Data Volumes allow you to store data in separate place which can then be used as a normal folder - all changes are persistent
• Command: docker run –ti –v /hostLog:/log ubuntu
• Run second container, where volume can be shared: docker run –ti --volumes-from firstContainerName ubuntu

Volumes – Use Cases

www.guvi.in

99

To Keep Data Persistent

Share Data Between Host and Container

Share Data With Other Containers

Volumes – Use Cases

www.guvi.in

100

To Keep Data Persistent

Share Data Between Host and Container

Share Data With Other Containers

• There's no way to directly create a "data volume" in Docker - Instead create a data volume container with a volume attached to it. For any containers to connect to the data volume container, need to use the Docker's –volumes from option to grab the volume from this container and apply them to the current container
• Commands:
• # Create the data volume Container

docker create -v /tmp --name datacontainer Ubuntu

• # The above creates a container called data container in the directory /tmp.
• # use it..

docker run -t -i --volumes-from datacontainer ubuntu /bin/bash

• # from now on any data written to /tmp is persisted

Volumes – Use Cases

www.guvi.in

101

To Keep Data Persistent

Share Data Between Host and Container

Share Data With Other Containers

• Example:

docker run -d -v ~/nginxlogs:/var/log/nginx -p 5000:80 -i nginx

• In above example, folder ~/nginxlogs on host machine is mapped to /var/log/nginx
• Writes to /var/log/nginx in container will be reflected to ~/nginxlogs on host
• Similarly writes to ~/nginxlogs on host will be reflected back to /var/log/nginx on container

​

Volumes – Use Cases

www.guvi.in

102

To Keep Data Persistent

Share Data Between Host and Container

Share Data With Other Containers

• In addition to mounting a host directory in your container, some Docker volume plugins allow you to provision and mount shared storage, such as iSCSI, NFS, or FC.
• A benefit of using shared volumes is that they are host-independent.
• A volume can be made available on any host that a container is started on as long as it has access to the shared storage backend, and has the plugin installed.
• The –volumes-from option is used to share information between services.

​

Volumes – Use Cases

www.guvi.in

103

To Keep Data Persistent

Share Data Between Host and Container

Share Data With Other Containers

• Other Commands:

# Example of using flocker

docker run -d -P --volume-driver=flocker -v my-named-volume:/webapp --name web training/webapp python app.py

# (or) Create a volume

docker volume create -d flocker -o size=20GB my-named-volume �# use it..

docker run -d -P -v my-named-volume:/webapp --name web training/webapp python app.py

Which is the best way?

• Mounting host directories is usually used.
• Any write inconsistencies have to be managed by applications themselves – not DOCKER.
• For larger data volumes, docker has plugins to manage NFS or other shared storage volumes.
• The factors determining which ones to choose will depend on
• Your volume requirements
• Cost of solution

www.guvi.in

104

www.guvi.in

105

Docker Files

Docker File

www.guvi.in

106

• Docker file is the basic building block of Docker containers
• Docker file is a file with a set of instructions written in it. It forms the basis for any image in Docker.
• Almost every time any base image is going to based upon another image. You are going to pick up a base image and build up on that image.

Docker File – Creation Steps

www.guvi.in

107

• Step 1: Create Directory
• Step 2: Create a named file subdirectory directory
• Step 3: Input the set of instructions
• Step 4: Save the file
• Step 5: Build the image

Docker File Main Sections

www.guvi.in

108

• Following are the main sections of the Docker file:
• FROM and MANINTAINER
• How to RUN commands
• How to set up ENV (environments)
• Difference between CMD vs RUN
• How to EXPOSE ports

Docker File – FROM and MAINTAINER

www.guvi.in

109

FROM:

• Every Docker file starts with this command
• It shows where is the base image coming from
• Will pick up an image from Docker hub or some other repository and make some changes for ex: environmental changes, or expose your ports etc. and then save the file.
• Example: FROM debian:stable

​

Docker File – FROM and MAINTAINER

www.guvi.in

110

MAINTAINER:

• The section of the Docker file shows the maintainer or the owner of the Docker file
• It requires certain format – It requires the name and the email id
• Following is the format
• MAINTAINER name <emailid >
• FROM debian:stable
• MAINTAINER docker<sriram@gmail.com >

​

Docker File – RUN

www.guvi.in

111

RUN:

• Set of actions you want to perform on the base image, where the modification of the base image starts.
• These actions have to be performed with root images
• Commands will be executed in the exact way you write it

​

​

​

​

​

​

​

Example:

​

FROM debian:stable

MAINTAINER docker<sriram@gmail.com >

RUN apt –get update

RUN apt –get upgrade

Docker File – RUN(Contd..)

www.guvi.in

112

RUN:

• Following steps will happen when you save this file and build an image
• Step 1: It is going to pull the Debian base image
• Step 2: It will set up the MAINTAINER in the container 
• Step 3: IT will update the packages
• Step 4: IT will upgrade the packages

Docker File – ENV

www.guvi.in

113

ENV:

• Can set up the environment variable
• By setting this up we can pass a variable that we need to pass inside the container that runs on base image
• Following format is required to set up this directive: ENV MYVALUE -test
• When you run the container this value will have to be passed using “echo $MYVALUE”

​

Example:

​

FROM debian:stable

MAINTAINER docker<sriram@gmail.com >

RUN apt –get update

RUN apt –get upgrade

ENV MYVALUE -test

Docker File – EXPOSE

www.guvi.in

114

EXPOSE:

• Command to expose any ports to expose through your container to the underlying host operating system with mapping redirect. We can get to the containers through containers IP’s.
• Ports are set up in the Docker file to be exposed.
• Following format is required to set up this directive: EXPOSE 80 (Port number which you want to expose)
• When you run docker ps for this container you will see the information for the ports which are exposed

Example:

FROM debian:stable

MAINTAINER docker<sriram@gmail.com >

RUN apt –get update

RUN apt –get upgrade

ENV MYVALUE -test

EXPOSE 80

EXPOSE 24

Docker File – CMD

www.guvi.in

115

CMD:

• Command for starting up of a service of some kind
• Anything that is after a command is a list of things to run within any container that is initiated on a base image
• All the actions to run when the containers are initiated is described in this section
• Following format is required to set up this directive: CMD [“usr/sbin/apache2ctl”]

​

Example:

FROM debian:stable

MAINTAINER docker<sriram@gmail.com >

RUN apt –get update && apt –get upgrade

ENV MYVALUE -test

EXPOSE 80

EXPOSE 24

CMD [“usr/sbin/apache2ctl”, “-D”, “FOREGROUND”]

Create Docker File - Demo

www.guvi.in

116

• Step 1: Create a directory called 'custom' and change it. In this directory, create an empty file called ”Dockerfile"
• Step 2: Edit ‘Dockerfile’ created in Step #1. This configuration file should be written to perform following actions:
• Use base Centos 6 latest version image from public repository
• Identify your email address as the author and maintainer of this image
• Update the base OS after initial import of the image
• Install the Open-SSH Server
• Install Apache Web Server
• Expose ports 22 and 80 to support the services installed
• Step 3: Build custom image from ’Dockerfile' as created above. Name/tag this new image as ”customimg/test:v1". Once the image is built, verify the image appears in your list.

Questions

www.guvi.in

117

Should I include my code with COPY/ADD or a volume?

• You can add your code to the image using COPY or ADD directive in a Docker file.
• This is useful if you need to relocate your code along with the Docker image, for example when you’re sending code to another environment (production, CI, etc.).
• Prefer COPY over ADD as image size is reduced with COPY over ADD.  
• You should use a volume if you want to make changes to your code and see them reflected immediately, for example when you’re developing code and your server supports hot code reloading or live-reload.  
• There may be cases where you’ll want to use both. You can have the image include the code using a COPY, and use a volume in your Compose file to include the code from the host during development. The volume overrides the directory contents of the image.

​

​

​

www.guvi.in

118

Docker Compose

www.guvi.in

119

Docker Compose: Multi Container Applications

www.guvi.in

120

Without using Compose

• Build and run one container at a time
• Manually connect containers
• Manual Dependency Management

​

​

​

With Compose

• Define multi-container apps in a single file
• Single command to deploy entire apps
• Handles Dependencies
• Works with Networking, Volumes, Swarm

​

​

​

Docker Compose: Multi Container Applications

www.guvi.in

121

Without using Compose

​

​

​

With Compose

​

​

​

docker run -e MYSQL_ROOT_PASSWORD=<pass> -e MYSQL_DATABASE=wordpress --name wordpressdb -v "$PWD/database":/var/lib/mysql -d mysql:latest

docker run -e WORDPRESS_DB_PASSWORD=<pass> --name wordpress --link wordpressdb:mysql -p 80:80 -v "$PWD/html":/var/www/html

-d wordpress

services:

db:

image: mysql:5.7 volumes:

- db_data:/var/lib/mysql environment: MYSQL_ROOT_PASSWORD:

MYSQL_DATABASE: wordpress wordpress:

depends_on:

- db

image: wordpress:latest ports:

- "80:80"

environment: WORDPRESS_DB_PASSWORD=<pass>

Docker Compose

www.guvi.in

122

Compose works in all environments: production, staging, development, testing, as well as CI workflows.

A tool for defining and running multi- container Docker applications

With Compose, you use a YAML file to configure your application’s services.

With a single command, you create and start all the services from your configuration

Docker Compose

www.guvi.in

123

Define your app’s environment with a Dockerfile

Define the services that make up your app in Docker Compose file

Run the CLI:

​

docker-compose up

Docker Compose: Multi Container Applications

www.guvi.in

124

C

Services

Volumes

Networking

B

A

Docker Compose: Multi Container Applications

www.guvi.in

125

containers:

web:

build: .

command: python app.py ports:

- "5000:5000"

volumes:

• .:/code environment:
• PYTHONUNBUFFERED=1

redis:

image: redis:latest

command: redis-server --appendonly yes

Docker Compose: Multi Container Applications

www.guvi.in

126

Docker Compose: Multi Container Applications

www.guvi.in

127

Using .env file

Docker Compose: Multi Container Applications

www.guvi.in

128

Commands:

​

docker compose up

docker compose down

docker compose run –e DEBUG=1 <services>

Docker Compose: Multi Container Applications

www.guvi.in

129

• A network called myapp_default is created.
• Name is based on directory
• A container is created using web’s configuration.

It joins the network myapp_default under the

name web.

• A container is created using db’s configuration.
• Joins the network myapp_default under the name db.

Docker Compose: Multi Container Applications

www.guvi.in

130

www.guvi.in

131

Q &A

www.guvi.in

132

We're done!

Thank you for your time and participation.

​

​