Accelerating the web with Varnish

Question:

By configuring the Varnish cache server, you can achieve the following goals

• Accelerating back-end web services with varnish
• The proxy server can cache remote web server pages locally
• The remote web server is transparent to the client
• Using cache mechanism to improve the response speed of website
• Use the varnishadm command to manage cached pages
• Use the varnishstat command to view the varni status

Scheme:

Install the varnish cache server through source code compilation

• Compile and install the varnish software

Modify the configuration file, cache the agent web server, and realize the web acceleration function

Three virtual machines are used, one as a web server, one as a varnish proxy server, and the other as a Linux client for testing

For the deployment of web server, in this experiment, you only need to install nginx or httpd software, start the service, and generate the test home page file. By default, the root path of httpd website is / var/www/html /, and the home page file is index Htnml default

As a proxy server, the biggest difference between varnish and nginx proxy is that varnish has a caching function. When a client accesses varnish for the first time, varnish forwards it like nginx, and then returns it to the client. However, at this time, a cache will be reserved on the varnish server. If a client accesses this content next time, It will be returned directly from the varnish proxy server to the client without forwarding again

If this content can be cached to servers across the country, it can also be called CDN (content distribution network)

Steps:

Implement the above example

Step 1: build a web server

1) In order to facilitate the experiment, we use httpd

yum -y install httpd

2) Start the service (if nginx is installed, please close it, otherwise the port conflicts)

The httpd service listens to client requests through the tcp80 port by default

systemctl start httpd
ss -nultp |grep 80

3) Creating test files for web access

In the root directory of the website, / var/www/html, create a file named index HTML homepage file

echo "this is web" > /var/www/html/index.html

Step 2: deploy the varnish cache server

1) Compile and install software

 yum -y install readline-devel gcc #Install software dependent packages
 yum -y  install pcre-devel
 yum -y  install python3-docutils.noarch
 pip3 install Sphinx
 useradd -s /sbin/nologin varnish  #Create user
 wget https://varnish-cache.org/_downloads/varnish-6.0.8.tgz
 tar -xf varnish-6.0.8.tgz
 cd varnish-6.0.8/
 ./configure
 make && make install

2) Copy the startup script and configuration file (there is no configuration file for this software by default)

cp varnish-6.0.8/etc/example.vcl /usr/local/etc/default.vcl

3) Modify profile

vim /usr/local/etc/default.vcl
#About 18 lines
...
backend default {
    .host = "192.168.20.10";
    .port = "80";
}
...

4) Start service

varnishd -f /usr/local/etc/default.vcl
 ss -nultp | grep 80
#Possible problems in starting the service, port conflict. Check whether any software occupies port 80, such as httpd or nginx
#Other common options for the varnishd command
#varnishd -f plus the location of the configuration file is used to start the service
#varnishd -s malloc,128M defines that varnish uses memory as a cache
#varnishd -s file,/var/lib/varnish_storage.bin,1G defines that varnish uses file cache
#Using memory as a cache (malloc) has the advantages of fast speed and the disadvantages of restarting the cache

Step 3: client test

Testing with the client

Access the varnish proxy server first

It can be seen from the figure that the access is successful. After refreshing the web page again, we can see from the status that the cache is used

Step 4: other operations

1) View the varish log

varnishlog   #Detailed log
varnishncsa

2) Update the cached data. After the background web server updates the page content, the user accesses the proxy server to see the previous data, indicating that the data in the cache has expired and needs to be updated (it will also be updated automatically by default (usually 2 minutes), but not in real time)

We quickly modify the content of the web, and then use the browser to view it again

As can be seen from the figure, we are still accessing the cache

Wait two minutes and refresh again

You can see that the web is refreshed normally

manual update
varnishadm
varnishban req.url ~.*    #Empty cached data, support regular expressions