How to realize image multi-level construction under Docker?

In the early versions of Docker, for the image construction of compiled languages (such as C, Java and Go), we can only put the compilation of applications and the preparation of running environment in one Dockerfile, which leads to a large volume of images, thus increasing the storage and distribution cost of images.

1. Build with additional scripts

In order to reduce the size of the image, we need an additional script to separate the compiling process and running process of the image.

Compilation stage: responsible for compiling our code into executable binary files.
Runtime build node: prepare the dependent environment for the application to run, and then copy the compiled executable object to the image.

Take an HTTP service developed by Go as an example, and the code is as follows:

package main

import (
	"fmt"
	"io"
	"net/http"
)

func getInfoHandler(w http.ResponseWriter, r *http.Request){
	user := r.URL.Query().Get("user")
	if user != "" {
		io.WriteString(w,fmt.Sprintf("hello [%s]\n",user))
	}else{
		io.WriteString(w,"hello [Stranger]\n")
	}

	for k, v := range r.Header{
		io.WriteString(w,fmt.Sprintf("%s=%s",k,v))
	}
}

func health(w http.ResponseWriter, r *http.Request){
	fmt.Fprintln(w,http.StatusOK)
}

func main() {
	http.HandleFunc("/",getInfoHandler)
	http.HandleFunc("/health",health)
	http.ListenAndServe(":8080",nil)
}

Building the Go service as an image is divided into two stages: code compilation stage and image construction stage.

When we build the image, the image needs to include the Go language compilation environment. In the compilation stage of the application, we can use the Dockerfile.build file to build the image. The contents of Dockerfile.build are as follows:

FROM golang:1.15
WORKDIR /go/src/httpServer/
COPY main.go .
RUN CGO_ENABLED=0 GOOS=linux go build -o http-server .

Dockefile.Build can help us compile code into executable binary files. We use the following Dockerfile to build a running environment:

FROM alpine:latest  
WORKDIR /root/
COPY http-server .
CMD ["./http-server"]

We put the compilation of the application and the preparation steps of the running environment into a build.sh script file, as follows:

#!/bin/sh   
echo Building http-server:build    # Declare the shell file, and then output the start build information

# Use the Dockerfile.build file to build a temporary image HTTP server: build
docker build -t http-server:build . -f Dockerfile.build

# Create a container named builder with the HTTP server: build image
# This container contains the compiled HTTP server binaries.
docker create --name builder http-server:build  

# Use the docker cp command to copy the HTTP server file from the builder container to the current build directory
# And delete the temporary container named builder.
docker cp builder:/go/src/httpServer/http-server ./http-server  
docker rm -f builder

# Output start building image information.
echo Building http-server:latest

# Build the runtime image and delete the temporary file HTTP server
docker build -t http-server:latest .
rm ./http-server

Above, we use the Dockerfile.build file to compile the application, and use the Dockerfile file to build the running environment of the application. Then we create a temporary container, copy the compiled HTTP server file to the current build directory, then copy the file to the image of the running environment, and finally specify the container startup command as HTTP server.

Although this method can realize the compilation and running environment of separate images, we need to introduce an additional build.sh script file, and in the construction process, we also need to create a temporary container builder to copy the compiled HTTP server file, which makes the whole construction process more complex and the whole construction process is not transparent enough.

In order to solve this problem, Docker launched the solution of multistage build in 17.05.

2. Use multistage build

Docker allows us to use multiple FROM statements in Dockerfile, and each FROM statement can use different basic images. The final generated image is based on the last FROM, so we can declare multiple FROM in a Dockerfile, and then selectively copy the files generated in one stage to another stage, so as to realize the final image and only retain the environment and files we need. The main use scenario of multi-stage construction is to separate the compilation environment and the running environment.

Next, we use multi-stage build to streamline the above process without multi-stage build:

# Compile and generate HTTP server
FROM golang:1.15
WORKDIR /go/src/httpServer/
COPY main.go .
RUN CGO_ENABLED=0 GOOS=linux go build -o http-server .

# Build runtime mirror
FROM alpine:latest  
WORKDIR /root/
# --from=0 means to copy the file from the first stage construction result to the current construction stage
COPY --from=0 /go/src/httpServer/http-server .  
CMD ["./http-server"]

To build a mirror:

docker build -t http-server:latest .

3. Other uses of multi-stage construction

3.1. Name the construction phase

By default, each construction phase is not named. These construction phases can be referenced by the order in which the FROM instruction appears. The sequence number of the construction phase starts FROM 0. However, in order to improve the readability of Dockerfile, we need to give a name to some construction stages, so that even if we reorder the content processes in Dockerfile or add new construction stages later, the COPY instructions in other construction processes do not need to be modified.

Optimize the Dockerfile above:

# Compile and generate HTTP server
FROM golang:1.15 AS builder
WORKDIR /go/src/httpServer/
COPY main.go .
RUN CGO_ENABLED=0 GOOS=linux go build -o http-server .

# Build runtime mirror
FROM alpine:latest  
WORKDIR /root/
COPY --from=builder /go/src/httpServer/http-server .  
CMD ["./http-server"]

In the first build phase, we use the AS instruction to name this phase builder. Then use the -- from=builder instruction in the second build phase to copy the file from the first build phase, making the Dockerfile more readable.

3.2. Stop at a specific construction stage

Sometimes, our construction phase is very complex. We want to debug in the code compilation phase, but multi-stage construction defaults to all phases of building Dockerfile. In order to reduce the construction time of each debugging, we can use the target parameter to specify the phase in which the construction stops.

For example, I just want to debug the Dockerfile file at the compilation stage. You can use the following command:

# docker build -t http-server:latest . --target builder

Sending build context to Docker daemon  6.656kB
Step 1/4 : FROM golang:1.15 AS builder
 ---> 40349a2425ef
Step 2/4 : WORKDIR /go/src/httpServer/
 ---> Using cache
 ---> 90a929dd37de
Step 3/4 : COPY main.go .
 ---> Using cache
 ---> fee18351e532
Step 4/4 : RUN CGO_ENABLED=0 GOOS=linux go build -o http-server .
 ---> Using cache
 ---> e7356e46c9c4
Successfully built e7356e46c9c4
Successfully tagged http-server:latest

3.3. Use the existing image as the construction phase

When using multi-stage construction, you can not only copy files from the defined stages in the Dockerfile, but also copy files from a specified image using the COPY --from command. The specified image can be an existing local image or an image on a remote image warehouse.

For example, we copy the compiled binaries from the above compilation environment image

# Build runtime mirror
FROM alpine:latest  
WORKDIR /root/
COPY --from=http-server:latest /go/src/httpServer/http-server .  
CMD ["./http-server"]

Build using the following command:

# docker build -t http-server:v2.0 . 

Sending build context to Docker daemon   7.68kB
Step 1/4 : FROM alpine:latest
 ---> c059bfaa849c
Step 2/4 : WORKDIR /root/
 ---> Running in bc1e94677d94
Removing intermediate container bc1e94677d94
 ---> ab8143431ed0
Step 3/4 : COPY --from=http-server:latest /go/src/httpServer/http-server .
 ---> 133f25c0d3f0
Step 4/4 : CMD ["./http-server"]
 ---> Running in d625e202db78
Removing intermediate container d625e202db78
 ---> 00a4c8370bc9
Successfully built 00a4c8370bc9
Successfully tagged http-server:v2.0

For another example, when we want to copy the configuration file of nginx official image to our own image, we can use the following instructions in Dockerfile:

COPY --from=nginx:latest /etc/nginx/nginx.conf /etc/local/nginx.conf

Keywords: Docker

Added by aa-true-vicious on Tue, 07 Dec 2021 02:05:57 +0200

Programming VIP

How to realize image multi-level construction under Docker?