Deploy elk log analysis system to consume more computer hardware. If you use virtual machine for test deployment, it is recommended to allocate more hardware resources. Otherwise, when elk container is running, it will not work normally. I will allocate 5G of memory to the docker host, four CPU s.

I. environmental preparation

I use a docker host here (for deployment of docker service, please refer to the blog: Detailed configuration of Docker installation ), whose IP address is 192.168.20.6, on which the elk container runs.

2. Configure the elk container of docker host

[root@docker01 ~]# echo "vm.max_map_count = 655360" >> /etc/sysctl.conf     
#Change its virtual memory
[root@docker01 ~]# sysctl -p     #Refresh kernel parameters
vm.max_map_count = 655360        #If the container fails to operate normally, the parameter value can be increased appropriately
[root@docker01 ~]# docker pull sebp/elk            #elk image size is more than 2G, so it is recommended to download it to the local area before running the container
[root@docker01 ~]# docker run -itd -p 5601:5601 -p 9200:9200 -p 5044:5044 -e ES_HEAP_SIZE="3g" -e LS_HEAP_SIZE="1g" --name elk sebp/elk
#Running elk container based on sebp/elk
# "- e es? Heap? Size =" 3G ": is to limit the memory size used by elasticsearch
# -E LS? Heap? Size = "1g": limit the memory size used by logstash

At this point, you can access the following interface by accessing the 5601 port of the docker host through the browser (all the following operations can be carried out by looking at the figure, which has been marked on the figure):

When you see the following page (pay attention to select RPM tab), follow the prompt command below to execute on our docker host.

Execute the prompt command operation on the above page, as follows:

[root@docker01 ~]# curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.4.0-x86_64.rpm
#Download rpm package
[root@docker01 ~]# rpm -ivh filebeat-7.4.0-x86_64.rpm    #Install the downloaded rpm package
[root@docker01 ~]# vim /etc/filebeat/filebeat.yml
======== Filebeat inputs ==========
filebeat.inputs:         #Modify the contents below filebeat.inputs
  enabled: true           #Change to true to enable filebeat
  paths:         #Modify the path section to add the log path to collect
    - /var/log/messages         #Specify the file path of the system log
    - /var/lib/docker/containers/*/*.log     #This path is the log path of all containers
========== Kibana =============
host: "192.168.20.6:5601"      #Remove the comment symbol in this line, and fill in kibana's listening port and address
------------ Elasticsearch output ------------ 
  hosts: ["192.168.20.6:9200"]    #Modify to the listening address and port of Elasticsearc
#After modifying the above configuration, save and exit
[root@docker01 ~]# filebeat modules enable elasticsearch     #Enable elasticsearch module
[root@docker01 ~]# filebeat setup   #Initialization of filebeat, waiting time is a little longer
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
Loaded machine learning job configurations
Loaded Ingest pipelines
#If the above information appears, the initialization is successful
[root@docker01 ~]# service filebeat start     #Start filebeat

After performing the above operations, click "Dicover" below to view the logs, as follows:

If there are new logs in the last 15 minutes, and the time of the docker host is also in a synchronous state, you can consider executing the following command to restart the elk container and view it again.

[root@docker01 ~]# systemctl daemon-reload       #Reload profile
[root@docker01 ~]# docker restart elk          #Restart the elk container

When the above page can be accessed normally, we run a container to output a character every 10 seconds, and then check whether kibana can collect the related log information of the container, as follows:

[root@docker01 ~]# docker run busybox sh -c 'while true;do echo "this is a log message from container busybox!";sleep 10;done'
#Run the container and output a character every ten seconds

Then look at the following figure, and operate in sequence, as follows:

If you can see the corresponding log information, the elk container is running normally.

————————Thank you for reading————————