The concept of image
The image can be understood as the container of the application, and the docker is used to load and unload the container.
The docker image contains the file system and its contents required to start the container, so it is used to create and start the container.
The docker image adopts a layered construction mechanism. The bottom layer is bootfs and the top layer is rootfs
- bootfs: the file system used for system boot, including bootloader and kernel. After the container is started, it will be unloaded to save memory resources
- rootfs: located above bootfs and represented as the root file system of the docker container
- In the traditional mode, when the system starts, the kernel will mount rootfs to the "read-only" mode first, and then re mount it to the read-write mode after the integrity self-test is completed
- In docker, rootfs is mounted in the "read-only" mode by the kernel, and then an additional "writable" layer is mounted through the "joint mount" technology
Note: when a container is deleted, its own "writable" layers will be deleted together
docker mirror layer
The image at the lower level is called a parent image, and the image at the lowest level is called a base image;
The top layer is the "read-write" layer, and the lower layer is the "read-only" layer.
docker storage driver
docker provides a variety of storage drivers to store images in different ways. The following are common storage drivers:
AUFS (another Union FS) is a Union FS, which is a file level storage driver. AUFS is a layered file system that can transparently cover one or more existing file systems, merging multiple layers into a single-layer representation of the file system. Simply put, it supports mounting different directories to the file system under the same virtual file system. This file system can overlay and modify files layer by layer. No matter how many layers below are read-only, only the top file system is writable. When a file needs to be modified, AUFS creates a copy of the file, uses CoW to copy the file from the read-only layer to the writable layer for modification, and the results are also saved in the writable layer. In Docker, the lower read-only layer is image, and the writable layer is Container.
It is said that the AUFS file system has 3W lines of code, while the ext4 file system has only about 4000-5000 lines of code. These codes are to be integrated into the kernel. Later, when the AUFS application is to be incorporated into the kernel code, linuz thought the code is too bloated, so he refused. Therefore, the AUFS file system has not been the file system of the linux kernel for a long time. If you want to use the AUFS file system, you must patch the kernel and compile and use it. However, the redhat series operating system has always been known for its stability and will not do such extraordinary things. Therefore, it is impossible to use AUFS in the redhat series operating system. The docker on ubuntu uses AUFS by default.
Overlay is supported by Linux kernel after 3.18. It is also a kind of Union FS. Unlike AUFS, overlay has only two layers: an upper file system and a lower file system, representing the image layer and Container layer of Docker respectively. When a file needs to be modified, use CoW to copy the file from the read-only lower to the writable upper for modification, and the results are also saved in the upper layer. In Docker, the lower read-only layer is image, and the writable layer is Container. At present, the latest overlay FS is overlay 2.
Both AUFS and Overlay are federated file systems, but AUFS has multiple layers, while Overlay has only two layers. Therefore, when copying on write, if the file is large and there are lower layers, AUSF will be slower. Moreover, Overlay is incorporated into the linux kernel mainline, but AUFS does not. At present, AUFS has been basically eliminated.
Device mapper is supported by Linux kernel after 2.6.9. It provides a mapping framework mechanism from logical devices to physical devices. Under this mechanism, users can easily formulate and implement storage resource management strategies according to their own needs. Both AUFS and OverlayFS are file level storage, while device mapper is block level storage. All operations are direct operations on blocks, not files. The device mapper driver will first create a resource pool on the block device, and then create a basic device with a file system on the resource pool. All images are snapshots of the basic device, and the container is a snapshot of the image. Therefore, the file system in the container is a snapshot of the file system of the basic device in the resource pool, and no space is allocated for the container. When a new file is to be written, a new block is allocated in the container's image and data is written. This is called time allocation. When you want to modify an existing file, use CoW to allocate block space for the container snapshot, copy the data to be modified to a new block in the container snapshot, and then modify it.
OverlayFS is a file level storage and device mapper is a block level storage. When a file is very large and the modified content is very small, Overlay will copy the whole file regardless of the modified content size. Obviously, modifying a large file takes more time than a small file, while block level only copies the blocks that need to be modified, not the whole file, In this scenario, device mapper is obviously faster. Because the block level directly accesses the logical disk, it is suitable for IO intensive scenarios. The performance of Overlay is relatively stronger in the scenario of complex internal program, large concurrency but less io.
When the container is started, the docker daemon will attempt to obtain the relevant image locally. When the local image does not exist, it will download the image from the Registry and save it locally.
Registry is used to save the docker image, including the hierarchy and metadata of the image. Users can create their own registry or use the official Docker Hub.
Classification of docker registry:
- Sponsor Registry: a third-party Registry for customers and Docker communities
- Mirror Registry: a third-party Registry for customers only
- Vendor Registry: the registry provided by the vendor that publishes the docker image
- Private Registry: a registry provided by private entities with firewalls and additional security layers
Composition of docker registry:
- An image repository consisting of all iterative versions of a particular docker image
- Multiple repositories can exist in a Registry
- Repository can be divided into "top-level warehouse" and "user warehouse"
- The format of user warehouse name is "user name / warehouse name"
- Each warehouse can contain multiple tags, and each tag corresponds to a mirror image
- Maintain user accounts, verification of images, and public namespace information
- It is equivalent to providing a retrieval interface for Registry to complete user authentication and other functions
Images in Docker Registry are usually created by developers, and then pushed to "public" or "private" Registry for saving for other people's use, such as "deployment" to production environment.
Making docker image
In most cases, we implement mirroring based on an existing basic image of others, which we call base image. For example, a pure version of minimized centos, ubuntu, or debian.
So where does this minimized centos image come from? In fact, this basic image is usually manually created by the relevant maintenance personnel of Docker Hub, that is, the official Docker. The production of this basic image is very easy for the official professionals of Docker, but it is not so easy for end users.
Docker Hub is a cloud based registration service that allows you to link to the code base, build images and test them, store manually pushed images, and link to the Docker cloud so that you can deploy images to the host.
It provides centralized resources for container image discovery, distribution and change management, user and team collaboration, and workflow automation throughout the development pipeline.
Docker Hub provides the following main functions:
- Find and extract images from community and official image repositories, and manage, push, and extract images from private image repositories that you can access.
- Automatically creates a new image when changes are made to the source code repository
- Webhooks is an automated build feature that allows you to trigger actions after a successful push to the warehouse.
- Create a workgroup to manage access to the mirrored warehouse
GitHub and Bitbucket Integration
Add the Hub and your Docker Images to your current workflows.
Add hub and Docker images to the current workflow.
Acquisition of docker image
To obtain a Docker image from a remote registry (such as your own Docker registry) and add it to the local system, use the Docker pull command:
- # docker pull <registry>[:<port>]/[<namespace>/]<name>:<tag>
It is the host that provides docker distribution service over TCP (default: 5000)
And identify the specific image controlled by in the registry
- Some registries also support raw; For these, is optional
- However, when it is included, the additional hierarchy level provided can be used to distinguish mirrors with the same attributes
Additional levels of hierarchy
|role||devel/database, test/database, prod/database|
Image generation method:
- Container based fabrication
- Docker Hub automated builds
Container based mirroring
Create a new image from container's changes
docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]
|—author, -a||Author (e.g., "John Hannibal Smith firstname.lastname@example.org")|
|-c, --change list||Apply Dockerfile instruction to the created image|
|-m, --message string||Commit message|
|-p, --pause||true||Pause container during commit|
[root@localhost ~]# docker pull centos Using default tag: latest latest: Pulling from library/centos 7a0437f04f83: Pull complete Digest: sha256:5528e8b1b1719d34604c87e11dcd1c0a20bedf46e83b5632cdeac91b8c04efc1 Status: Downloaded newer image for centos:latest docker.io/library/centos:latest [root@localhost ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE nginx latest 08b152afcfae 6 days ago 133MB httpd latest 73b8cfec1155 6 days ago 138MB centos latest 300e315adb2f 7 months ago 209MB [root@localhost ~]# docker run --name a1 -it 300e315adb2f /bin/bash [root@3e337d66b2eb /]# ls bin dev etc home lib lib64 lost+found media mnt opt proc root run sbin srv sys tmp usr var #Configure yum source [root@3e337d66b2eb ~]# cd /etc/ [root@3e337d66b2eb etc]# mv yum.repos.d yum.repos [root@3e337d66b2eb etc]# mkdir yum.repos.d [root@3e337d66b2eb etc]# cd yum.repos.d/ [root@3e337d66b2eb yum.repos.d]# vi aa.repo [aa1] name=aa1 baseurl=https://mirrors.aliyun.com/centos/8/AppStream/x86_64/os/ gpgcheck=0 enabled=1 [aa2] name=aa2 baseurl=https://mirrors.aliyun.com/centos/8/BaseOS/x86_64/os/ gpgcheck=0 enabled=1 #Installing nginx [root@3e337d66b2eb ~]# yum -y install nginx [root@3e337d66b2eb ~]# vi /etc/nginx/nginx.conf daemon off; #Add this to make him run at the front desk
When creating an image, we cannot close the container. We must make it running, so we must start another terminal and execute it
[root@localhost ~]# docker commit -p a1 sha256:4d0b1c77dda1eee1b2fc9f138f1b9a202731437afe846e32268580c882b45ea6 [root@localhost ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE <none> <none> 4d0b1c77dda1 11 seconds ago 318MB nginx latest 08b152afcfae 6 days ago 133MB httpd latest 73b8cfec1155 6 days ago 138MB centos latest 300e315adb2f 7 months ago 209MB [root@localhost ~]# docker tag 4d0b1c77dda1 phsama/a1:v1.0 [root@localhost ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE phsama/a1 v1.0 4d0b1c77dda1 2 minutes ago 318MB nginx latest 08b152afcfae 6 days ago 133MB httpd latest 73b8cfec1155 6 days ago 138MB centos latest 300e315adb2f 7 months ago 209MB
Log in to the image warehouse and upload the image
[root@localhost ~]# docker login Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username: phsama Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@localhost ~]# docker push phsama/a1:v1.0 The push refers to repository [docker.io/phsama/a1] 86fb8858fa19: Pushed 2653d992f4ef: Layer already exists v1.0: digest: sha256:e696ffeaec4bea04957360b727c225c5532612a8ff8f28f76fa2cc6d158ba0fb size: 741
Create a container using the newly generated image
[root@localhost ~]# docker pull phsama/a1:v1.0 v1.0: Pulling from phsama/a1 7a0437f04f83: Already exists 18589bfbba1d: Pull complete Digest: sha256:e696ffeaec4bea04957360b727c225c5532612a8ff8f28f76fa2cc6d158ba0fb Status: Downloaded newer image for phsama/a1:v1.0 docker.io/phsama/a1:v1.0 [root@localhost ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE phsama/a1 v1.0 4d0b1c77dda1 2 hours ago 318MB nginx latest 08b152afcfae 6 days ago 133MB httpd latest 73b8cfec1155 6 days ago 138MB centos latest 300e315adb2f 7 months ago 209MB [root@localhost ~]# docker run -it --name t1 phsama/a1:v1.0 [root@912a7b85b582 /]# ls etc/nginx/ conf.d fastcgi.conf.default koi-utf mime.types.default scgi_params uwsgi_params.default default.d fastcgi_params koi-win nginx.conf scgi_params.default win-utf fastcgi.conf fastcgi_params.default mime.types nginx.conf.default uwsgi_params
It can be seen that the newly generated image contains the newly added nginx, but there is a problem at this time, that is, what is the default process to be started by the container? Here, the / bin/bash process is started by default, but we want to start an nginx site, so we need to set the default process started by the container to nginx when creating the image, so that we can quickly build a simple nginx site through the newly generated image
[root@localhost /]# docker inspect t1 .... "Cmd": [ "/bin/bash" ], ....
Regenerate image and upload
[root@localhost etc]# docker commit -c 'CMD ["/usr/sbin/nginx"]' -p t1 phsama/a1:v1.1 [root@localhost etc]# docker push phsama/a1:v1.1
Create a container using the newly generated image
[root@localhost ~]# docker rmi phsama/a1:v1.1 Untagged: phsama/a1:v1.1 Untagged: phsama/a1@sha256:24ba42ec2e5eadff08574973ced7dd3afa3cf893f84926fe0b88c12b82b487f3 Deleted: sha256:609c64640085d88e037493552b65c25af566830117d00c26a4a2ed86874063cd Deleted: sha256:6464ed0b296d495b0785197dbecc917bab09c06271e79e1e8546f64273be2b0d [root@localhost ~]# docker pull phsama/a1:v1.1 v1.1: Pulling from phsama/a1 7a0437f04f83: Already exists 18589bfbba1d: Already exists 87b507679834: Pull complete Digest: sha256:24ba42ec2e5eadff08574973ced7dd3afa3cf893f84926fe0b88c12b82b487f3 Status: Downloaded newer image for phsama/a1:v1.1 docker.io/phsama/a1:v1.1 [root@localhost ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE phsama/a1 v1.1 609c64640085 4 minutes ago 318MB phsama/a1 v1.0 4d0b1c77dda1 2 hours ago 318MB nginx latest 08b152afcfae 6 days ago 133MB httpd latest 73b8cfec1155 6 days ago 138MB centos latest 300e315adb2f 7 months ago 209MB [root@localhost ~]# docker run --name t2 -d phsama/a1:v1.1 [root@localhost ~]# docker container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 7514d8e283ca phsama/a1:v1.1 "/usr/sbin/nginx" 20 seconds ago Up 19 seconds t2 [root@localhost ~]# curl 172.17.0.2 .... <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"> <head> <title>Test Page for the Nginx HTTP Server on Red Hat Enterprise Linux</title> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
Import and export of images
If there are two hosts, we make an image on host 1. What if host 2 wants to use this image?
We can push the image to the image warehouse on host 1, and then pull the image down on host 2. This method is more troublesome. If I just test it, I can run on another host after doing the image on one host. There is no need to push it to the warehouse and then pull it locally.
At this time, we can package the image into a compressed file based on the existing image, and then copy it to another host to import it. This is the import and export function of the image.
In docker, we use docker save for export and docker load for import.
Execute docker save to export the image on the host on which the image has been generated
[root@localhost ~]# docker save -o nginx.tar.gz phsama/a1:v1.1 [root@localhost ~]# ls anaconda-ks.cfg nginx.tar.gz
Use scp to transfer to another host, and then execute docker load to import the image on another host without an image
[root@localhost ~]# docker load -i nginx.tar.gz Loaded image: phsama/a1:v1.1 [root@localhost ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE phsama/a1 v1.1 609c64640085 12 minutes ago 318MB