In the previous article, I used openresty to build https proxy and log printing. With the passage of time, the openresty I built has been on the company's production server. A new problem appeared on Friday, so I began to explore! So what's the problem? How to solve it? Let's look down with questions!

After this problem occurred, my colleagues in the company came to me and said, "I checked on the Internet that the client_max_body_size parameter is 20m. Why is the request_body printed in the log empty?". Then I was confused at that time, so I restored the modified configuration to its original appearance. I saw the following big error message. I thought: the error message is very obvious. Full of confidence, I asked him for the requested message and interface address. I found that the request message was so large that the original Notepad opened and collapsed directly. Finally, I started troubleshooting the problem!

The openresty request body is too large, resulting in 413 error reports

First of all, I think that since nginx proxy is used to forward the interface address, the IP and domain name of the server where nginx is located must be interconnected with the forwarding address. You can use ping or telnet to check. I think this is necessary, because the problems you check must be based on the premise of network interoperability. Otherwise, it will waste your time and patience during the investigation, which will also affect your investigation ideas and mood! So after checking, my network is interconnected, so continue to check!

Secondly, nginx is a lightweight service. If the request is too large, you need to change the size limit of the request body, and your request is sent from the browser. The browser has a content length of the request header, which is 1m by default. Therefore, we must first change the size limit of nginx to accept the request to solve the 413 error problem.

client_max_body_size 50m;

After configuration, I found that 413 reported no error. When I initiated a request, I found that the postman console reported an error of 500, which proves that the request body is indeed empty, because the request body you should transmit was not transmitted, resulting in the back-end reported 500 error. And I can't see the request when the log is printed_ The content of body is simply request_body is empty, so why? I have configured to resolve the request_ The configuration item whose body is empty, why is it still empty?

Then I thought of client_body_buffer_size to cache requests. The Buffer size allocated by Nginx to the request body. If the requested data is less than the client_body_buffer_size directly stores the data in memory first. If the requested value is greater than client_body_buffer_size less than client_max_body_size, the data will be stored in the temporary file first. In which temporary file? The answer is in nginx/client_body_temp. So we also need to ensure that the client_body_temp has read-write permission. Otherwise, when the transmitted data is greater than the client_body_buffer_size, an error will be reported if it fails to write into the temporary file. You can find it in error Log to view the log written into the temporary file. After my investigation, the result is that I really don't have read-write permission, or even the permission to see the folder!

Before I ask the company's system administrator for permission, I can summarize that the transmitted data is greater than that of the client_max_body_size, it must be unsuccessful. Less than client_body_buffer_size is stored directly in memory and efficiently. If greater than client_body_buffer_size less than client_max_body_size will store temporary files. Temporary files must have permissions. If you want efficiency, set the client_max_body_size client_ body_ buffer_ The size value will not be stored directly in the temporary memory.

I climbed down the pole, found the administrator and opened the client_body_temp has read and write permissions, but it is found that the problem is not solved. In fact, I probably came here when I browsed other people's solutions online, but I still didn't solve it. I thought: is it because my configuration has not been loaded? This time, I didn't reload as usual. Instead, I chose another way to kill all the processes of nginx and restart nginx. When I found that dependence didn't work, there was still a problem. At this time, I gradually lost my patience!

Suddenly, the brain flashed an idea, is this request timeout, because there was no configuration item to configure timeout in my configuration before! So I configured the timeout configuration item, reload reloaded the configuration file, and suddenly found that the problem was solved!

proxy_connect_timeout 300;
proxy_read_timeout 300;
proxy_send_timeout 300;  

Knowledge sharing

The problem has been solved, and some knowledge points have been found in the process, that is, the use of http1 in nginx forwarding 0 and http1 Problem with version 1. Because at first I suspected the problem of keep alive, which is obviously not the problem!

nginx uses http1 by default when reverse proxy HTTP protocol 0 And http1 0 and http1 The difference between http1 and http1 0 does not support HTTP keep alive. Partners in need can also be found and configured in the configuration file given below!

HTTP type		describe
http1.0	Keep alive is not supported	Connection Close notifies the back-end server to actively close the connection. The request of any client generates a TIME-WAIT connection on the back-end server
http1.1	Support keep alive	Enable http1.0 on Nginx 1 sends a request to the back end and supports keep alive

#user  nobody; # User group
worker_processes  1;

# Open error log and pid comment
error_log  logs/error.log;
error_log  logs/error.log  notice;
error_log  logs/error.log  info;

pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    
    # In order to print the content of the log, which is essential for better troubleshooting, I only print the content of the log
    log_format main  escape=json '{ "@timestamp": "$time_local", '
                         '"upstream_addr": "$upstream_addr",'
                         '"request_time": "$request_time", '
                         '"status": "$status", '
                         '"request": "$request", '
                         '"host":""$host",'
                         '"http_uri": "$uri",'
                         '"Request message":"$request_body",'
                         '"response message":"$resp_body" }'

    # Enable log cache to avoid excessive memory use
    open_log_file_cache max=1000 inactive=20s valid=1m min_use=2;
    sendfile        on;
    keepalive_timeout  65;
   
    # The following solves the problem that the request body is too large
    client_max_body_size 50m;
   
    # Solve the request below_ Problem with empty body 
    # Specify how many and how large buffers are needed locally to buffer FastCGI responses
    fastcgi_buffers 32 16k;
    # The maximum number of bytes that the buffer agent can buffer client requests
    client_body_buffer_size 1024k;

    # HTTPS server
    #
    server {
        listen       443 ssl; # https default port
        server_name  xxx.com; # domain name
		
		# The following configurations are certificate configurations
        ssl_certificate      /data/openresty-1.15/nginx/conf/server.pem;
        ssl_certificate_key  /data/openresty-1.15/nginx/conf/server.key;

    	# The following is the configuration of certificate suite
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;
        
        # Add the following configuration
        charset utf-8;
        set $resp_body "";
        access_log  /data/openresty-1.15/nginx/logs/nginx.log  main; # Configure log path

        location / {
            #root   html;
            #index  index.html index.htm;
            
            # Enable forced acquisition request message log
            lua_need_request_body on;
            log_escape_non_ascii off;
            # lua
            body_filter_by_lua '
                local resp_body = string.sub(ngx.arg[1], 1, 1000)
                ngx.ctx.buffered = (ngx.ctx.buffered or "") .. resp_body
                if ngx.arg[2] then
                    ngx.var.resp_body = ngx.ctx.buffered
                end
            ';
            
            # The following configurations are request timeout configurations
            proxy_connect_timeout 300;
            proxy_read_timeout 300;
            proxy_send_timeout 300;       
            
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            
            # Specify http1 Version 1
            # proxy_http_version 1.1;
            # Set the Connection to an empty string to prevent the delivery of the header to the back end
            # proxy_set_header Connection "";
			# proxy_set_header X-Forwarded-Proto $scheme;
            # proxy_set_header X-Forwarded-For $remote_addr;
            # Specifies that the forwarding request header carries the host
            # proxy_set_header Host $host;
            # proxy_set_header X-Real-IP $remote_addr;

            proxy_pass https://IP: port / index jsp; #  Address or domain name forwarded by proxy
        }
    }
}