The concept of image

The image can be understood as the container of the application, and the docker is used to load and unload the container.

The docker image contains the file system and its contents required to start the container, so it is used to create and start the container.

The docker image adopts a layered construction mechanism. The bottom layer is bootfs and the top layer is rootfs

• bootfs: the file system used for system boot, including bootloader and kernel. After the container is started, it will be unloaded to save memory resources
• rootfs: located above bootfs and represented as the root file system of the docker container
  • In the traditional mode, when the system starts, the kernel will mount rootfs to the "read-only" mode first, and then re mount it to the read-write mode after the integrity self-test is completed
  • In docker, rootfs is mounted in the "read-only" mode by the kernel, and then an additional "writable" layer is mounted through the "joint mount" technology

Note: when a container is deleted, its own "writable" layers will be deleted together

docker mirror layer

The image at the lower level is called a parent image, and the image at the lowest level is called a base image;
The top layer is the "read-write" layer, and the lower layer is the "read-only" layer.

docker storage driver

docker provides a variety of storage drivers to store images in different ways. The following are common storage drivers:

• AUFS
• OverlayFS
• Devicemapper
• Btrfs
• VFS

AUFS

AUFS (another Union FS) is a Union FS, which is a file level storage driver. AUFS is a layered file system that can transparently cover one or more existing file systems, merging multiple layers into a single-layer representation of the file system. Simply put, it supports mounting different directories to the file system under the same virtual file system. This file system can overlay and modify files layer by layer. No matter how many layers below are read-only, only the top file system is writable. When a file needs to be modified, AUFS creates a copy of the file, uses CoW to copy the file from the read-only layer to the writable layer for modification, and the results are also saved in the writable layer. In Docker, the lower read-only layer is image, and the writable layer is Container.

It is said that the AUFS file system has 3W lines of code, while the ext4 file system has only about 4000-5000 lines of code. These codes are to be integrated into the kernel. Later, when the AUFS application is to be incorporated into the kernel code, linuz thought the code is too bloated, so he refused. Therefore, the AUFS file system has not been the file system of the linux kernel for a long time. If you want to use the AUFS file system, you must patch the kernel and compile and use it. However, the redhat series operating system has always been known for its stability and will not do such extraordinary things. Therefore, it is impossible to use AUFS in the redhat series operating system. The docker on ubuntu uses AUFS by default.

OverlayFS

Overlay is supported by Linux kernel after 3.18. It is also a kind of Union FS. Unlike AUFS, overlay has only two layers: an upper file system and a lower file system, representing the image layer and Container layer of Docker respectively. When a file needs to be modified, use CoW to copy the file from the read-only lower to the writable upper for modification, and the results are also saved in the upper layer. In Docker, the lower read-only layer is image, and the writable layer is Container. At present, the latest overlay FS is overlay 2.

Both AUFS and Overlay are federated file systems, but AUFS has multiple layers, while Overlay has only two layers. Therefore, when copying on write, if the file is large and there are lower layers, AUSF will be slower. Moreover, Overlay is incorporated into the linux kernel mainline, but AUFS does not. At present, AUFS has been basically eliminated.

DeviceMapper

Device mapper is supported by Linux kernel after 2.6.9. It provides a mapping framework mechanism from logical devices to physical devices. Under this mechanism, users can easily formulate and implement storage resource management strategies according to their own needs. Both AUFS and OverlayFS are file level storage, while device mapper is block level storage. All operations are direct operations on blocks, not files. The device mapper driver will first create a resource pool on the block device, and then create a basic device with a file system on the resource pool. All images are snapshots of the basic device, and the container is a snapshot of the image. Therefore, the file system in the container is a snapshot of the file system of the basic device in the resource pool, and no space is allocated for the container. When a new file is to be written, a new block is allocated in the container's image and data is written. This is called time allocation. When you want to modify an existing file, use CoW to allocate block space for the container snapshot, copy the data to be modified to a new block in the container snapshot, and then modify it.

OverlayFS is a file level storage and device mapper is a block level storage. When a file is very large and the modified content is very small, Overlay will copy the whole file regardless of the modified content size. Obviously, modifying a large file takes more time than a small file, while block level only copies the blocks that need to be modified, not the whole file, In this scenario, device mapper is obviously faster. Because the block level directly accesses the logical disk, it is suitable for IO intensive scenarios. The performance of Overlay is relatively stronger in the scenario of complex internal program, large concurrency but less io.

docker registry

When the container is started, the docker daemon will attempt to obtain the relevant image locally. When the local image does not exist, it will download the image from the Registry and save it locally.

Registry is used to save the docker image, including the hierarchy and metadata of the image. Users can create their own registry or use the official Docker Hub.

Classification of docker registry:

• Sponsor Registry: a third-party Registry for customers and Docker communities
• Mirror Registry: a third-party Registry for customers only
• Vendor Registry: the registry provided by the vendor that publishes the docker image
• Private Registry: a registry provided by private entities with firewalls and additional security layers

Composition of docker registry:

• Repository
  • An image repository consisting of all iterative versions of a particular docker image
  • Multiple repositories can exist in a Registry
    • Repository can be divided into "top-level warehouse" and "user warehouse"
    • The format of user warehouse name is "user name / warehouse name"
  • Each warehouse can contain multiple tags, and each tag corresponds to a mirror image
• Index
  • Maintain user accounts, verification of images, and public namespace information
  • It is equivalent to providing a retrieval interface for Registry to complete user authentication and other functions

Images in Docker Registry are usually created by developers, and then pushed to "public" or "private" Registry for saving for other people's use, such as "deployment" to production environment.

Making docker image

In most cases, we implement mirroring based on an existing basic image of others, which we call base image. For example, a pure version of minimized centos, ubuntu, or debian.

So where does this minimized centos image come from? In fact, this basic image is usually manually created by the relevant maintenance personnel of Docker Hub, that is, the official Docker. The production of this basic image is very easy for the official professionals of Docker, but it is not so easy for end users.

Docker Hub

Docker Hub is a cloud based registration service that allows you to link to the code base, build images and test them, store manually pushed images, and link to the Docker cloud so that you can deploy images to the host.

It provides centralized resources for container image discovery, distribution and change management, user and team collaboration, and workflow automation throughout the development pipeline.

Docker Hub provides the following main functions:

• Image Repositories

  • Find and extract images from community and official image repositories, and manage, push, and extract images from private image repositories that you can access.

• Automated Builds

  • Automatically creates a new image when changes are made to the source code repository

• web hooks

  • Webhooks is an automated build feature that allows you to trigger actions after a successful push to the warehouse.

• Organizations

  • Create a workgroup to manage access to the mirrored warehouse

• GitHub and Bitbucket Integration

  • Add the Hub and your Docker Images to your current workflows.

    Add hub and Docker images to the current workflow.

Acquisition of docker image

To obtain a Docker image from a remote registry (such as your own Docker registry) and add it to the local system, use the Docker pull command:

1. # docker pull <registry>[:<port>]/[<namespace>/]<name>:<tag>

It is the host that provides docker distribution service over TCP (default: 5000)

And identify the specific image controlled by in the registry

• Some registries also support raw; For these, is optional
• However, when it is included, the additional hierarchy level provided can be used to distinguish mirrors with the same attributes

Additional levels of hierarchy

Namespace	Examples(/)
organization	redhat/kubernetes, google/kubernetes
login(username)	Alice/application, bob/application
role	devel/database, test/database, prod/database

Image generation

Image generation method:

• Dockerfile
• Container based fabrication
• Docker Hub automated builds

Container based mirroring

Create a new image from container's changes

Usage:

docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]

Options	Default	Description
—author, -a		Author (e.g., "John Hannibal Smith hannibal@a-team.com")
-c, --change list		Apply Dockerfile instruction to the created image
-m, --message string		Commit message
-p, --pause	true	Pause container during commit

[root@localhost ~]# docker pull centos
Using default tag: latest
latest: Pulling from library/centos
7a0437f04f83: Pull complete 
Digest: sha256:5528e8b1b1719d34604c87e11dcd1c0a20bedf46e83b5632cdeac91b8c04efc1
Status: Downloaded newer image for centos:latest
docker.io/library/centos:latest

[root@localhost ~]# docker images
REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
nginx        latest    08b152afcfae   6 days ago     133MB
httpd        latest    73b8cfec1155   6 days ago     138MB
centos       latest    300e315adb2f   7 months ago   209MB

[root@localhost ~]# docker run --name a1 -it 300e315adb2f /bin/bash
[root@3e337d66b2eb /]# ls
bin  dev  etc  home  lib  lib64  lost+found  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
#Configure yum source
[root@3e337d66b2eb ~]# cd /etc/
[root@3e337d66b2eb etc]# mv yum.repos.d yum.repos
[root@3e337d66b2eb etc]# mkdir yum.repos.d
[root@3e337d66b2eb etc]# cd yum.repos.d/
[root@3e337d66b2eb yum.repos.d]# vi aa.repo
[aa1]
name=aa1
baseurl=https://mirrors.aliyun.com/centos/8/AppStream/x86_64/os/
gpgcheck=0
enabled=1

[aa2]
name=aa2
baseurl=https://mirrors.aliyun.com/centos/8/BaseOS/x86_64/os/
gpgcheck=0
enabled=1
#Installing nginx
[root@3e337d66b2eb ~]# yum -y install nginx
[root@3e337d66b2eb ~]# vi /etc/nginx/nginx.conf
daemon off;    #Add this to make him run at the front desk

When creating an image, we cannot close the container. We must make it running, so we must start another terminal and execute it

[root@localhost ~]# docker commit -p a1
sha256:4d0b1c77dda1eee1b2fc9f138f1b9a202731437afe846e32268580c882b45ea6
[root@localhost ~]# docker images
REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
<none>       <none>    4d0b1c77dda1   11 seconds ago   318MB
nginx        latest    08b152afcfae   6 days ago       133MB
httpd        latest    73b8cfec1155   6 days ago       138MB
centos       latest    300e315adb2f   7 months ago     209MB
[root@localhost ~]# docker tag 4d0b1c77dda1 phsama/a1:v1.0
[root@localhost ~]# docker images
REPOSITORY   TAG       IMAGE ID       CREATED         SIZE
phsama/a1    v1.0      4d0b1c77dda1   2 minutes ago   318MB
nginx        latest    08b152afcfae   6 days ago      133MB
httpd        latest    73b8cfec1155   6 days ago      138MB
centos       latest    300e315adb2f   7 months ago    209MB

Log in to the image warehouse and upload the image

[root@localhost ~]# docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: phsama
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[root@localhost ~]# docker push phsama/a1:v1.0
The push refers to repository [docker.io/phsama/a1]
86fb8858fa19: Pushed 
2653d992f4ef: Layer already exists 
v1.0: digest: sha256:e696ffeaec4bea04957360b727c225c5532612a8ff8f28f76fa2cc6d158ba0fb size: 741

Create a container using the newly generated image

[root@localhost ~]# docker pull phsama/a1:v1.0
v1.0: Pulling from phsama/a1
7a0437f04f83: Already exists 
18589bfbba1d: Pull complete 
Digest: sha256:e696ffeaec4bea04957360b727c225c5532612a8ff8f28f76fa2cc6d158ba0fb
Status: Downloaded newer image for phsama/a1:v1.0
docker.io/phsama/a1:v1.0
[root@localhost ~]# docker images
REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
phsama/a1    v1.0      4d0b1c77dda1   2 hours ago    318MB
nginx        latest    08b152afcfae   6 days ago     133MB
httpd        latest    73b8cfec1155   6 days ago     138MB
centos       latest    300e315adb2f   7 months ago   209MB
[root@localhost ~]# docker run -it --name t1 phsama/a1:v1.0
[root@912a7b85b582 /]# ls etc/nginx/
conf.d	      fastcgi.conf.default    koi-utf	  mime.types.default  scgi_params	   uwsgi_params.default
default.d     fastcgi_params	      koi-win	  nginx.conf	      scgi_params.default  win-utf
fastcgi.conf  fastcgi_params.default  mime.types  nginx.conf.default  uwsgi_params

It can be seen that the newly generated image contains the newly added nginx, but there is a problem at this time, that is, what is the default process to be started by the container? Here, the / bin/bash process is started by default, but we want to start an nginx site, so we need to set the default process started by the container to nginx when creating the image, so that we can quickly build a simple nginx site through the newly generated image

[root@localhost /]# docker inspect t1
....
            "Cmd": [
                "/bin/bash"
            ],
....

Regenerate image and upload

[root@localhost etc]# docker commit -c 'CMD ["/usr/sbin/nginx"]' -p t1 phsama/a1:v1.1
[root@localhost etc]# docker push phsama/a1:v1.1

Create a container using the newly generated image

[root@localhost ~]# docker rmi phsama/a1:v1.1
Untagged: phsama/a1:v1.1
Untagged: phsama/a1@sha256:24ba42ec2e5eadff08574973ced7dd3afa3cf893f84926fe0b88c12b82b487f3
Deleted: sha256:609c64640085d88e037493552b65c25af566830117d00c26a4a2ed86874063cd
Deleted: sha256:6464ed0b296d495b0785197dbecc917bab09c06271e79e1e8546f64273be2b0d
[root@localhost ~]# docker pull phsama/a1:v1.1
v1.1: Pulling from phsama/a1
7a0437f04f83: Already exists 
18589bfbba1d: Already exists 
87b507679834: Pull complete 
Digest: sha256:24ba42ec2e5eadff08574973ced7dd3afa3cf893f84926fe0b88c12b82b487f3
Status: Downloaded newer image for phsama/a1:v1.1
docker.io/phsama/a1:v1.1
[root@localhost ~]# docker images
REPOSITORY   TAG       IMAGE ID       CREATED         SIZE
phsama/a1    v1.1      609c64640085   4 minutes ago   318MB
phsama/a1    v1.0      4d0b1c77dda1   2 hours ago     318MB
nginx        latest    08b152afcfae   6 days ago      133MB
httpd        latest    73b8cfec1155   6 days ago      138MB
centos       latest    300e315adb2f   7 months ago    209MB
[root@localhost ~]# docker run --name t2 -d phsama/a1:v1.1
[root@localhost ~]# docker container ls
CONTAINER ID   IMAGE            COMMAND             CREATED          STATUS          PORTS     NAMES
7514d8e283ca   phsama/a1:v1.1   "/usr/sbin/nginx"   20 seconds ago   Up 19 seconds             t2
[root@localhost ~]# curl 172.17.0.2
....
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
    <head>
        <title>Test Page for the Nginx HTTP Server on Red Hat Enterprise Linux</title>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

Import and export of images

If there are two hosts, we make an image on host 1. What if host 2 wants to use this image?

We can push the image to the image warehouse on host 1, and then pull the image down on host 2. This method is more troublesome. If I just test it, I can run on another host after doing the image on one host. There is no need to push it to the warehouse and then pull it locally.

At this time, we can package the image into a compressed file based on the existing image, and then copy it to another host to import it. This is the import and export function of the image.

In docker, we use docker save for export and docker load for import.

Execute docker save to export the image on the host on which the image has been generated

[root@localhost ~]# docker save -o nginx.tar.gz phsama/a1:v1.1
[root@localhost ~]# ls
anaconda-ks.cfg  nginx.tar.gz

Use scp to transfer to another host, and then execute docker load to import the image on another host without an image

[root@localhost ~]# docker load -i nginx.tar.gz
Loaded image: phsama/a1:v1.1
[root@localhost ~]# docker images
REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
phsama/a1    v1.1      609c64640085   12 minutes ago   318MB