Squid traditional, transparent proxy service configuration, log analysis, reverse proxy, ACL access control

Squid proxy server

Squid mainly provides cache acceleration and application layer filtering control functions

1. Working mechanism of agency

  • Instead of the client requesting data from the website, you can hide the user's real IP address
  • The obtained Web page data (static Web elements) is saved in the cache and sent to the client, so that it can respond quickly the next time the same data is requested

2. Type of squid agent

  • Traditional proxy: it is applicable to the Internet. You need to specify the address and port of the proxy server on the client
  • Transparent proxy: the client does not need to specify the address and port of the proxy server, but redirects the Web access to the proxy server through the default route and firewall policy
  • Reverse proxy: if the requested resources are cached in the Squid reverse proxy server, the requested resources will be returned directly to the client; Otherwise, the reverse proxy server will request resources from the background WEB server, and then return the requested response to the client. At the same time, it will also cache the response locally for use by the next requester

3. Benefits of using agents

  • Improve Web access speed
  • Hide the real IP address of the client

Install Squid service

The server	            address
 Client( windows10)

1. Close the firewall and security mechanism

systemctl stop firewalld
systemctl disable firewalld
setenforce 0

2. Compile and install Squid

mount /dev/sr0 /mnt

#Install environment dependent packages
yum -y install gcc gcc-c++ make

tar zxvf squid-3.5.28.tar.gz -C /opt/
cd /opt/squid-3.5.28

./configure --prefix=/usr/local/squid \    #Point to the installation directory path 
--sysconfdir=/etc \                        #Specify the configuration file path	
--enable-arp-acl \						   #MAC address control to prevent clients from using ip spoofing 
--enable-linux-netfilter \				   #Using kernel filtering
--enable-linux-tproxy \					   #Support transparent mode
--enable-async-io=100 \					   #Asynchronous IO to improve storage performance
--enable-err-language="Simplify_Chinese" \  #Display language of error message
--enable-underscore \                       #Allow underscores in URLs
--enable-poll \								#Turn off the default use poll mode
--enable-gnuregex							#Using gnu regular expressions

Compile and install added modules
cd /opt/squid-3.5.28

./configure --prefix=/usr/local/squid \
--sysconfdir=/etc \
--enable-arp-acl \
--enable-linux-netfilter \
--enable-linux-tproxy \
--enable-async-io=100 \
--enable-err-language="Simplify_Chinese" \
--enable-underscore \
--enable-poll \
make && make install

ln -s /usr/local/squid/sbin/* /usr/local/sbin/ #Optimization path

useradd -M -s /sbin/nologin squid   #Create program user

chown -R squid:squid /usr/local/squid/var/ 

3. Modify Squid's configuration file

vim /etc/squid.conf					#The operation mechanism of this file is to match from top to bottom, and the matching is not completed from bottom to top
#Line 56 insert
http_access allow all				#Put on HTTP_ Before access deny all, allow any client to use the proxy service
http_access deny all
http_port 3128						#Used to specify the address and port that the proxy service listens to (the default port number is 3128)

#Line 61 inserts cache management users and groups
cache_effective_user squid			#Add, specify the program user, which is used to set the account of initialization and runtime cache. Otherwise, the startup will not succeed
cache_effective_group squid			#Add, specify account basic group

coredump_dir /usr/local/squid/var/cache/squid		#Specify cache file directory

4.Squid operation control

#Check whether the configuration file syntax is correct
squid -k parse
#Start Squid

squid -z 					#-The z option is used to initialize the cache directory
squid						#Start squid service

netstat -anpt | grep "squid"

5. Write Squid startup control service script

vim /etc/init.d/squid               
#chkconfig: 2345 90 25          #2345 is the default startup level, 90: startup level, 90th startup (0-100) 25: stop level       

case "$1" in
     netstat -natp | grep squid &> /dev/null
     if [ $? -eq 0 ]
       echo "squid is running"
       echo "Starting squid..."
     $CMD -k kill &> /dev/null
     rm -rf $PID &> /dev/null
     [ -f $PID ] &> /dev/null
        if [ $? -eq 0 ]
            netstat -natp | grep squid
            echo "squid is not running"
      $0 stop &> /dev/null
      echo "Closing squid..."
         $0 start &> /dev/null
      echo "Starting squid..."
      $CMD -k reconfigure
      $CMD -k parse
      echo "Usage: $0{start|stop|status|reload|check|restart}"

chmod +x /etc/init.d/squid   
chkconfig --add squid    #Added to the system to identify
chkconfig --level 35 squid on   #3.5 automatically start the service at the operation level

Building traditional proxy server

1. squid server (

vim /etc/squid.conf
http_access allow all
http_access deny all
http_port 3128
cache_effective_user squid
cache_effective_group squid
63 Row insertion

cache_mem 64 MB				#Specify the size of the memory space used by the cache function to maintain the frequently accessed WEB objects. The capacity is preferably a multiple of 4, and the unit is MB. It is recommended to set it to 1 / 4 of the physical memory
reply_body_max_size 10 MB			#The maximum file size that users are allowed to download, in bytes. When downloading a Web object that exceeds the specified size, a prompt of "request or access too large" will appear on the error page of the browser. The default setting is 0, which means no restriction
maximum_object_size 4096 KB			#The maximum object size allowed to be saved to the cache space, in KB. Files exceeding the minimum limit will not be cached, but will be forwarded directly to the user

service squid restart
systemctl restart squid

2. Modify firewall rules

iptables -F
iptables -I INPUT -p tcp --dport 3128 -j ACCEPT
iptables -L INPUT

3.apache1 installation of http service

systemctl stop firewalld.service
setenforce 0
yum -y install httpd
systemctl start httpd
netstat -natp | grep 80

4. Configure the proxy service of the client browser (ip: gw:

4. Apache 1 viewing logs

tail -f /var/log/httpd/access_log

Build transparent proxy server

host	            IP address	                                     Services, installation packages, tools
Squid-Server	ens33:  ens36: squid-3.5.28.tar.gz	
Web1	                              httpd
 client Windows                            -

① Install the http service and restart the service

yum -y install httpd
systemctl restart httpd.service 

2.Squid server configuration (ens33, ens36)
① Power off, add a network card, change to host only mode, power on, ifconfig, view the name of the new network card, and edit the network card configuration file

cd /etc/sysconfig/network-scripts/
cp ifcfg-ens33 ifcfg-ens36
vim ifcfg-ens36
systemctl restart network

② Change profile support transparent mode

vim /etc/squid.conf
http_access allow all
http_access deny all

http_port transparent

service squid reload #If it has been started before and overloaded, it can also be restarted. If it has not been started, start

③ Turn on routing forwarding

echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
sysctl -p

Note that the virtual machine can ping the web server of the network segment in the host mode only. Even if route forwarding is not enabled, the client cannot Ping the same server. It is necessary to add a static route

Add static route on web server

route add -net gw

④ Add firewall rules (client access does not need to specify an intermediate server, but is forwarded through the firewall)

Modify firewall rules (from 136 network segments in the future): 80/443 Port redirected to port 3128, 443: https)
iptables -F
iptables -t nat -F
iptables -t nat -I PREROUTING -i ens36 -s -p tcp --dport 80 -j REDIRECT --to 3128	#This is the Internet interface ens36
iptables -t nat -I PREROUTING -i ens36 -s -p tcp --dport 443 -j REDIRECT --to 3128	
iptables -I INPUT -p tcp --dport 3128 -j ACCEPT     #If you restart, you need to reconfigure this rule

3. Client test (

View new records of Squid access log

tail -f /usr/local/squid/var/logs/access.log

Squid proxy server detects that the client accesses the target website

Check the new records in the Web access log, which shows that the external network port of the proxy server replaces the access of the client

tail -f /var/log/httpd/access_log

ACL access control

1. In the configuration file squid In conf, ACL access control is realized through the following two steps
(1) Use acl configuration items to define the conditions to be controlled;
(2) Via http_ The access configuration item controls "allow" or "deny" access to the defined list.

1. Define access control list
acl list name list type list content
List name: the name is user-defined, which is equivalent to giving ACL a name (a bit similar to shell script variable)
List type: the predefined value of Squid must be used to correspond to different types of control conditions
List content: refers to the specific object to be controlled. The corresponding contents of different types of lists are also different. There can be multiple values (separated by spaces and the relationship of "or")

Mode 1
vim /etc/squid.conf
acl host src 				        #The source address is, which is added separately

acl MYLAN src 						#Segment based
acl destionhost dst				#The destination address is
acl MC20 maxconn 20									#Maximum concurrent connections 20
acl PORT port 21									#Target port 21
acl DMBLOCK dstdomain .qq.com						#Target domain, matching all sites in the domain
acl BURL url_regex -i ^rtsp:// ^emule:// 			# URLs starting with rtsp: / /, eMule: /, - i means case is ignored The structure of url regular expression, which controls the filtering ^ rtsp (Protocol) based on the specified protocol
acl PURL urlpath_regex -i \.mp3$ \.mp4$ \.rmvb$		#Based on access to the end of the file mp3,. mp4,. URL path at the end of rmvb
acl WORKTIME time MTWHF 08:30-17:30					#Based on the access time control, it is 8:30 ~ 17:30 from Monday to Friday

Insert in the first clause: http_access deny host

Mode 2:
mkdir /etc/squid				#Enable object list management
vim dest.list			
vim /etc/squid.conf				#Call the contents of the specified file list
#The called list is the content in the form of a file. You need to define the file to the list first
acl List name of the file definition dst (Destination address, file path)
For example: acl destionhost dst "/etc/squid/dest.list"
http_access deny destionhost				#Note that if it is a rejection list, it needs to be placed in http_access allow all

http_port 3128

systemctl restart squid

Squid log analysis

1. Install image processing software package

#You need to use the online source. If an error occurs, you can modify the dns and gateway in the network card configuration file back to the original

yum install -y pcre-devel gd gd-devel

mkdir /usr/local/sarg
tar zxvf sarg-2.3.7.tar.gz -C /opt/

cd /opt/sarg-2.3.7

./configure --prefix=/usr/local/sarg --sysconfdir=/etc/sarg --enable-extraprotection
make && make install

2. Modify the configuration file

vim /etc/sarg/sarg.conf

7 Line uncomment
access_log /usr/local/squid/var/logs/access.log		#Specify access log file

25 Line uncomment
title "Squid User Access Reports"					#Page title

120 Line uncomment
output_dir /var/www/html/squid-reports						#Report output directory

178 Line uncomment
user_ip no											#Display with user name

184 Line uncomment, modify
topuser_sort_field connect reverse					#In top sorting, the specified connection times are arranged in descending order, and the ascending order is normal

190 Line uncomment, modify
user_sort_field connect reverse						#For user access records, the number of connections is sorted in descending order

206 Line uncomment, modify
exclude_hosts /usr/local/sarg/noreport				#Specifies files that are not included in the sorted site list

257 Line uncomment
overwrite_report no									#Overwrite logs with the same name and date

289 Line uncomment, modify
mail_utility mailq.postfix							#Send mail report command

434 Line uncomment, modify
charset UTF-8										#Specifies the character set UTF-8

518 Line uncomment
weekdays 0-6										#Week cycle of top ranking

525 Line uncomment
hours 0-23											#Time period of top ranking

633 Line uncomment
www_document_root /var/www/html						#Specify page root

Add is not included in the site file, and the added domain name will not be displayed in the sorting

touch /usr/local/sarg/noreport

ln -s /usr/local/sarg/bin/sarg /usr/local/bin/
sarg --help


sarg				#Start a record


yum install httpd -y
systemctl start httpd

3. Add scheduled tasks, execute and generate reports every day

vim /usr/local/sarg/report.sh

#Get current date
TODAY=$(date +%d/%m/%Y)
#Get one week ago today
YESTERDAY=$(date -d "1 day ago" +%d/%m/%Y)
/usr/local/sarg/bin/sarg -l /usr/local/squid/var/logs/access.log -o /var/www/html/squid-reports -z -d $YESTERDAY-$TODAY &> /dev/null
exit 0
chmod +x /usr/local/sarg/report.sh

crontab -e
0 0 * * * /usr/local/sarg/report.sh

Squid reverse proxy

If the requested resource is cached in the Squid reverse proxy server, the requested resource is directly returned to the client; Otherwise, the reverse proxy server will request resources from the background Web server, and then return the requested response to the client. At the same time, it will also cache the response locally for the next requester to use.
(it is recommended not to continue based on the above experiment, which may cause the squid server to fail to start at port 80)
Working mechanism:
● cache web page objects to reduce duplicate requests
● assign Internet request rotation training or weight to intranet Web server
● acting as the user's request, avoiding the user's direct access to the Web server and improving security

1. Squid server ( network card nat mode
Because httpd in the experiment will occupy port 80 of the server, it must be closed
Add a web2 server
Turn on the firewall
systemctl start firewalld
Local httpd shutdown
systemctl stop httpd

iptables -F				#Clear firewall rules
iptables -t nat -F		#The emptying rule can't do internal and external forwarding, so it didn't pass 3 a.m. and then the nat mode succeeded
iptables -I INPUT -p tcp --dport 3128 -j ACCEPT

vim /etc/squid.conf

60 Row modification, inserting
http_port accel vhost vport
cache_peer parent 80 0 no-query originserver round-robin max_conn=30 weight=1 name=web1
cache_peer parent 80 0 no-query originserver round-robin max_conn=30 weight=1 name=web2
cache_peer_domain web1 web2 www.gg.com

systemctl restart squid
netstat -natp | grep 80

http_ Port 80 access Vhost vport #squid changes from a cache to a web server using the acceleration mode. At this time, squid listens to requests at port 80 and binds to the request port (vhost vport) of the web server. At this time, squid does not need to forward requests. Instead, squid directly takes data from the cache or directly requests data from the bound port.

parent	:Represents the parent node
no-query:	Do not query, directly obtain data
originserver:	Specify source server
round-robin:	appoint squid The request is distributed to one of the parent nodes by polling
max_conn:	Specify the maximum number of connections
weight:	Specify weights
name:	Set alias

2.web1 and web2 (install httpd service and configure web page content)

systemctl stop firewalld.service 
setenforce 0
yum install -y httpd
systemctl start httpd

echo "this is web1" >> /var/www/html/index.html

echo "this is web2" >> /var/www/html/index.html

3. Domain name mapping of client (

modify C:\Windows\System32\drivers\etc\hosts file www.gg.com

4.squid server
Delete the previous acl reject source address

Added by jasonc310771 on Thu, 17 Feb 2022 18:31:23 +0200