HAProxy https configuration, four-tier load balancing, and access control for load balancing services

In the previous section, we talked about the configuration of haproxy's access control ACL. For a review, please refer to https://www.cnblogs.com/qiuhom-1874/p/12817773.html ; Today let's talk about the configuration of haproxy's https, tcp-based four-tier load balancing, and access control.

First let's look at the configuration of haproxy's https; what HTTPS are I won't go into much detail here. Please refer to the instructions for Certificate Application https://www.cnblogs.com/qiuhom-1874/p/12237944.html ; in the haproxy configuration file, we explicitly state that we listen on a port that requires ssl protocol access, similar to listen 443 in nginxSSL configuration; in addition to crt, we need to specify the certificate; unlike nginx, haproxy's certificate content contains private key information; so after we apply for the certificate, we need to merge the certificate file content with the private key file;

Example: Configure haproxy to listen on port 443 and support https

Tip: The contents in the red box indicate listening on port 443 and explicitly specify access using SSL protocol; the certificate file is/etc/haproxy/ssl/haproxy.pem

Create ssl directory and certificate file

[root@docker_node1 ~]# mkdir -p /etc/haproxy/ssl
[root@docker_node1 ~]# cd /etc/haproxy/ssl
[root@docker_node1 ssl]# ls
[root@docker_node1 ssl]# (umask 066;openssl genrsa -out haproxy.key 2048)
Generating RSA private key, 2048 bit long modulus
e is 65537 (0x10001)
[root@docker_node1 ssl]# ls
[root@docker_node1 ssl]# openssl req -new -x509 -key haproxy.key -out haproxy.crt -subj "/CN=www.test.com"
[root@docker_node1 ssl]# ls
haproxy.crt  haproxy.key
[root@docker_node1 ssl]# cat haproxy.crt haproxy.key > haproxy.pem
[root@docker_node1 ssl]# ls
haproxy.crt  haproxy.key  haproxy.pem
[root@docker_node1 ssl]# openssl  x509 -in haproxy.pem -noout -text
        Version: 3 (0x2)
        Serial Number:
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=www.test.com
            Not Before: May  2 14:41:50 2020 GMT
            Not After : Jun  1 14:41:50 2020 GMT
        Subject: CN=www.test.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
            X509v3 Authority Key Identifier: 

            X509v3 Basic Constraints: 
    Signature Algorithm: sha256WithRSAEncryption
[root@docker_node1 ssl]# 

Tip: I will use -subj directly to specify certificate/CN to generate self-signed certificate; haproxy's certificate requires that certificate information and private key information be combined together, so we need to merge certificate file information with new private key information by redirection;

Testing: Restart haproxy for browser access

Tip: You can see that our access with https is normal, but the certificate browser does not trust it, so we will be prompted that it is not safe;

* Redirect request for port 80 to fixed 443

Tip: The configuration in the red box indicates that the http request is redirected to https, which is similar to the rewritten url in nginx;

Testing: Visit in your browser to see if you want to rewrite HTTP as https?

Tip: You can see that our access is accessed using http, and in the response message the browser is told to access using location; this enables the full site of https;

Pass user-requested protocols and ports (frontend or backend) to back-end

Tip: The content in the red box means setting the value of the dst_port variable in the X-Forwarded-port header in the request message to the back end; adding the X-Forwarded-Proto header with the value of HTTPS when the request protocol is https; these two settings are usually required for the front end HTTPS requests to be passed to the back end through X-Forwarded-Proto when the back end application server requires HTTPS access;

Testing: Configure the log format on the back-end server, log these two headers separately, and then access to see if the values recorded in the log for these two headers are the 443 port and https protocol we visited?

Tip: Define the httpd log format above to record the values at the beginning of {X-Forwarded-Port and X-Forwared-Proto, respectively

Use a browser to access the test and see the information recorded in the log

Tip: You can see that the corresponding access port and protocol are passed to the back-end server in the log.

haproxy does tcp load balancing based on 4-tier proxy

Example: Load balancing mysql based on tcp

Tip: The above configuration indicates that tcp protocol is used to proxy back-end server;

Backend server environment preparation

[root@docker_node1 ~]# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
wordpress           latest              c3fa1c8546fb        44 hours ago        540MB
mysql               5.7                 f965319e89de        4 days ago          448MB
httpd               2.4.37-alpine       dfd436f9a5d8        16 months ago       91.8MB
[root@docker_node1 ~]# docker run --name db1 -d --net bridge -e MYSQL_ROOT_PASSWORD=admin mysql:5.7
[root@docker_node1 ~]# docker run --name db2 -d --net bridge -e MYSQL_ROOT_PASSWORD=admin mysql:5.7 
[root@docker_node1 ~]# docker ps 
CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                 NAMES
1bff841c226a        mysql:5.7             "docker-entrypoint.s..."   7 seconds ago       Up 6 seconds        3306/tcp, 33060/tcp   db2
8cade03e4f28        mysql:5.7             "docker-entrypoint.s..."   12 seconds ago      Up 12 seconds       3306/tcp, 33060/tcp   db1
3572c621f827        wordpress             "docker-entrypoint.s..."   9 hours ago         Up 9 hours          80/tcp                wordpress
f07288d607e5        httpd:2.4.37-alpine   "httpd-foreground"       7 days ago          Up 13 hours         80/tcp                web3
971595b7f409        httpd:2.4.37-alpine   "httpd-foreground"       7 days ago          Up 13 hours         80/tcp                web2
5c74f3be1868        httpd:2.4.37-alpine   "httpd-foreground"       7 days ago          Up 13 hours         80/tcp                web1
[root@docker_node1 ~]# docker exec -it db1 /bin/sh
# mysql -padmin
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.30 MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> grant all on *.* to "myuser"@'172.17.0.%' identified by 'admin';
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> \q
# exit
[root@docker_node1 ~]# docker exec -it db2 /bin/sh
# mysql -padmin
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.30 MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> grant all on *.* to "myuser"@'172.17.0.%' identified by 'admin';
Query OK, 0 rows affected, 1 warning (0.01 sec)

mysql> \q
# exit
[root@docker_node1 ~]# 

Tip: When starting the mysql container, we need to set a password for the root; when creating a new user, the host IP address we specify should be the address of the docker0 bridge, although it is based on a 4-tier tcp proxy, it still needs to change the source ip;

Testing: Restart haproxy and use the mysql client to access port 3306 that haproxy listens on to see if it can properly connect to the back-end mysql container?

Tip: You can see that mysql clients on other hosts can normally access port 3306 that haproxy listens on;

haproxy does access control based on 4-tier tcp

Tip: tcp-request connection indicates that incoming connections are operated upon according to Layer 4 conditions; the above configuration denies connections with source address

Testing: Restart haproxy and connect using the mysql client on to see if it works?

Tip: You can see that mysql on cannot be connected with the mysql tool on;

Example: only host access with open source address, deny access to other hosts

Tip: tcp-request connection allows access to be identified by accept and rejects by reject; if it is used in combination, all entries except ACL entries matched above can be represented without writing conditions; unlike http-request, HTTP-request allows access to be identified by allow and deny is rejected;

Testing: Connect mysql client on host with source address to mysql client on and mysql client on source address other than to see if they can connect?

Tip: You can see that using the mysql client at the source address other than cannot connect to the backend mysql container; this is because it is not matched by the ACL we defined, so it will be matched by tcp-request connect reject; thus, the connection will be rejected;

Last topic, we talked about the configuration of the status page of haproxy in the previous blog; in which we demonstrated how to manage the back-end server by clicking on the mouse through the page; this is a challenge for us to monitor haproxy itself and the back-end server; smart you must have noticed that there is an s in the global configuration section of haproxyConfigurations such as Tat socket/var/lib/haproxy/stats; let's move on to what this configuration is for;

Tip: The above configuration means binding monitoring page information to a socket file; we can achieve the purpose of haproxy by sending specific instructions to the socket file

Example: Pass help information to/var/lib/haproxy/stats via socat and have them print help pages

[root@docker_node1 ~]# echo "help"|socat stdio /var/lib/haproxy/stats 
Unknown command. Please enter one of the following commands only :
  clear counters : clear max statistics counters (add 'all' for all counters)
  clear table    : remove an entry from a table
  help           : this message
  prompt         : toggle interactive mode with prompt
  quit           : disconnect
  show info      : report information about the running process
  show pools     : report information about the memory pools usage
  show stat      : report counters for each proxy and server
  show errors    : report last request and response errors for each proxy
  show sess [id] : report the list of current sessions or dump this session
  show table [id]: report table usage stats or dump this table's contents
  get weight     : report a server's current weight
  set weight     : change a server's weight
  set server     : change a server's state or weight
  set table [id] : update or create a table entry's data
  set timeout    : change a timeout setting
  set maxconn    : change a maxconn setting
  set rate-limit : change a rate limiting value
  disable        : put a server or frontend in maintenance mode
  enable         : re-enable a server or frontend which is in maintenance mode
  shutdown       : kill a session or a frontend (eg:to release listening ports)
  show acl [id]  : report available acls or dump an acl's contents
  get acl        : reports the patterns matching a sample for an ACL
  add acl        : add acl entry
  del acl        : delete acl entry
  clear acl <id> : clear the content of this acl
  show map [id]  : report available maps or dump a map's contents
  get map        : reports the keys and values matching a sample for a map
  set map        : modify map entry
  add map        : add map entry
  del map        : delete map entry
  clear map <id> : clear the content of this map
  set ssl <stmt> : set statement for ssl

[root@docker_node1 ~]# 

Tip: From the help information above, we can see that we can manage the backend server by piping the corresponding instructions to the / var/lib/haproxy/stats file.

Example: Mark web1 disable d with the socat command

Tip: We executed marking web1 disable, which prompted us that we do not have permissions; let's configure the permissions next

Tip: The above configuration indicates that the socket file has admin privileges. Next, mark web1 disable (modifying the haproxy configuration file requires restarting haproxy)

Tip: After we modified the permissions, there are no permissions reported and other errors when we execute the above command. Next, we open the status page to see what the status of web1 is.

Tip: You can see that web1 has become a maintenance mode;

Example: List information for monitoring page indicator data

[root@docker_node1 ~]# echo "show info " |socat stdio /var/lib/haproxy/stats
Name: HAProxy
Version: 1.5.18
Release_date: 2016/05/10
Nbproc: 4
Process_num: 3
Pid: 6374
Uptime: 0d 0h05m04s
Uptime_sec: 304
Memmax_MB: 0
Ulimit-n: 8039
Maxsock: 8039
Maxconn: 4000
Hard_maxconn: 4000
CurrConns: 0
CumConns: 3
CumReq: 10
MaxSslConns: 0
CurrSslConns: 0
CumSslConns: 0
Maxpipes: 0
PipesUsed: 0
PipesFree: 0
ConnRate: 0
ConnRateLimit: 0
MaxConnRate: 1
SessRate: 0
SessRateLimit: 0
MaxSessRate: 1
SslRate: 0
SslRateLimit: 0
MaxSslRate: 0
SslFrontendKeyRate: 0
SslFrontendMaxKeyRate: 0
SslFrontendSessionReuse_pct: 0
SslBackendKeyRate: 0
SslBackendMaxKeyRate: 0
SslCacheLookups: 0
SslCacheMisses: 0
CompressBpsIn: 0
CompressBpsOut: 0
CompressBpsRateLim: 0
ZlibMemUsage: 0
MaxZlibMemUsage: 0
Tasks: 14
Run_queue: 1
Idle_pct: 100
node: docker_node1

[root@docker_node1 ~]# 

Tip: The above information is the indicator data in the monitoring page;

Example: online web1

View the status of web1 on the Monitor page

Tip: You can see that web1 is online normally; smart enough, you must have thought of dynamically managing the back-end server in this way; so you can achieve many functions, such as monitoring status indicators in haproxy through zabbix; dynamically uploading and downloading servers through scripts; dynamically modifying the weight of back-end servers (this needs to be done according to the algorithm, ifScheduling algorithms are supported dynamically but not statically;

Keywords: Linux MySQL SSL Docker Oracle

Added by amandas on Sun, 03 May 2020 09:12:33 +0300