Welcome to my GitHub

Here we classify and summarize all the original works of Xinchen (including supporting source code): https://github.com/zq2599/blog_demos

Overview of this article

• This article is the sixth in the "Java extends Nginx" series, Above The five handlers form the basic framework of nginx clojure development. The preliminary evaluation can support simple demand development, but nginx clojure does not stop at handler and provides rich expansion capabilities. The two filter s in this article are the commonly used capabilities
• There are two types of filters: header filter and body filter. Nginx clojure's positioning of them is the processing of header and body respectively, which will be described in detail below

Nginx Header Filter

• As the name suggests, header filter is a filter used for header processing. It has the following characteristics:

1. Header filter is a location level configuration. You can develop a header filter and configure it to be used in different locations
2. The header filter must implement the NginxJavaHeaderFilter interface, and the function code is written in the doFilter method
3. If the doFilter method returns PHASE_DONE, the nginx clojure framework will continue to execute other filters and handlers. If the returned is not PHASE_DONE, the nginx clojure framework will treat the current filter as an ordinary content handler and immediately return the return value of doFilter to the client
4. It is officially suggested to use header filter to dynamically process the header of response (add, delete and modify header items)

• Next, try developing a header filter, remember One of the Java extensions to Nginx: Hello, Nginx clojure The / java interface in the article is the simplest helloworld level location, and the content handler is hellohandler Java, which will be used later when verifying the header filter function

• First use the postman request / java interface to see the response header before using the header filter, as shown in the following figure:
  

• Next, add a location with the following configuration: content handler or hellohandler Java, added header_filter_type and header_filter_name:

location /headerfilterdemo {
	content_handler_type 'java';
    content_handler_name 'com.bolingcavalry.simplehello.HelloHandler';

    # The type of header filter is java
    header_filter_type 'java';
    # header
    header_filter_name 'com.bolingcavalry.filterdemo.RemoveAndAddMoreHeaders';
}

• The class that performs the header filter function is removeandaddmoreheaders Java, as shown below, the content type is modified, and two header items xfeep header and Server are added:

package com.bolingcavalry.filterdemo;

import nginx.clojure.java.Constants;
import nginx.clojure.java.NginxJavaHeaderFilter;
import java.util.Map;

public class RemoveAndAddMoreHeaders implements NginxJavaHeaderFilter {
    @Override
    public Object[] doFilter(int status, Map<String, Object> request, Map<String, Object> responseHeaders) {
        // Deleting before adding is equivalent to modifying the content type value
        responseHeaders.remove("Content-Type");
        responseHeaders.put("Content-Type", "text/html");

        // Add two header s
        responseHeaders.put("Xfeep-Header", "Hello2!");
        responseHeaders.put("Server", "My-Test-Server");

        // Return to PHASE_DONE tells the nginx clojure framework that the current filter is normal and can continue to execute other filters and handler s
        return Constants.PHASE_DONE;
    }
}

• Compile and build both simple hello and filter demo maven projects to get simple-hello-1.0-snapshot jar and filter-demo-1.0-snapshot jar these two jars, put them into the nginx/jars directory, and then restart nginx
• Use postman to request / headerfilterdemo, and compare the response header with / java, as shown in the figure below. It can be seen that deletion, addition and addition are normal. In addition, since the Server configuration item already exists, the result of the put operation in the filter is to modify the value of the configuration item:
  
• Here is the introduction of the header filter. Next, we will look at the body filter. As the name suggests, this is a filter used to process the response body. Unlike the header filter, the response body has different types, so the body filter can not be generalized, and needs to be developed and used in different scenes

The first scene of Nginx Body Filter: string faced Java body filter

• The function of Body Filter is very clear: modify the value of the original response body and return it to the client
• If the body of the response is a string, the body filter is relatively simple. Note the following rules:

1. Inherit the abstract class StringFacedJavaBodyFilter,
2. When processing a web request, the doFilter method may be invoked many times. A isLast entry is used to mark whether the current call is the last time (true represents the last time).
3. The return value of the doFilter method is the same as the previous nginxjavaringhandler Like the invoke method, it is a one-dimensional array with only three elements: status, headers and filtered_ Chunk, once the status value is not empty, the nginx clojure framework will use the return value of this doFilter as the last call and return it to the client
4. Combining the characteristics of 2 and 3, we should pay attention to when coding: assuming that doFilter will be called 10 times for a web request (the value of each body input parameter is a part of the whole response body), then the isLast of the first 9 times is equal to false, and the isLast of the 10th time is equal to true. Assuming that the status returned when calling doFilter method for the first time is not empty, It will cause the doFilter for the next 9 times to no longer be called!

• The following actual combat uses the previous hellohandler again Java acts as a content handler because the body it returns is a string
• First add a location configuration, body_filter_type and body_filter_name is the configuration item of body filter:

# demo of body filter, response body is of string type
location /stringbodyfilterdemo {
	content_handler_type 'java';
	content_handler_name 'com.bolingcavalry.simplehello.HelloHandler';

	# The type of body filter is java
	body_filter_type 'java';
    # Class of body filter
    body_filter_name 'com.bolingcavalry.filterdemo.StringFacedUppercaseBodyFilter';
}

• StringFacedUppercaseBodyFilter. The Java source code is as follows (please focus on the notes). It can be seen that the function of the filter is to change the original body to uppercase, and the value of isLast is checked in the code. When isLast is equal to false, the value of status remains null, so as to ensure that the call of doFilter will not end in advance, so as to return the complete body:

package com.bolingcavalry.filterdemo;

import nginx.clojure.java.StringFacedJavaBodyFilter;
import java.io.IOException;
import java.util.Map;

public class StringFacedUppercaseBodyFilter extends StringFacedJavaBodyFilter {
    @Override
    protected Object[] doFilter(Map<String, Object> request, String body, boolean isLast) throws IOException {
        if (isLast) {
            // isLast equals true, indicating that the doFilter method is called for the last time during the current web request,
            // Body is the last part of the complete response body,
            // At this time, the returned status should not be empty, so the nginx clojure framework will complete the execution process of the body filter and return the status and aggregated body to the client
            return new Object[] {200, null, body.toUpperCase()};
        }else {
            // isLast equals false, which means that the doFilter method will continue to be invoked in the current web request process. The current call is only one of many times.
            // Body is a part of the complete response body,
            // At this time, the returned status should be empty, so that the nginx clojure framework will continue the execution process of body filter and continue to call doFilter
            return new Object[] {null, null, body.toUpperCase()};
        }
    }
}

• After compilation, construction and deployment, access / stringbodyfilterdemo with postman, and the response is as follows. It can be seen that the content of the body has been capitalized, which is in line with expectations:
  
• The next step is to learn the body filter, but this time the body type is binary stream (stream faced Java body filter)

The second scenario of Nginx Body Filter: binary stream body (stream faced Java body filter)

• When the response body is a binary stream, if you want to read and write the response body, nginx clojure's suggestion is to execute it in the body filter, which is specially used in the scenario of binary stream body. It has the following characteristics:

1. Implement the interface NginxJavaBodyFilter (note the difference: the filter of string body inherits the abstract class StringFacedJavaBodyFilter),
2. When processing a web request, the doFilter method may be invoked many times. A isLast entry is used to mark whether the current call is the last time (true represents the last time).
3. The return value of the doFilter method is the same as the previous nginxjavaringhandler Like the invoke method, it is a one-dimensional array with only three elements: status, headers and filtered_ Chunk, once the status value is not empty, the nginx clojure framework will use the return value of this doFilter as the last call and return it to the client
4. Combining the characteristics of 2 and 3, we should pay attention to when coding: assuming that doFilter will be called 10 times for a web request (the value of each body input parameter is a part of the whole response body), then the isLast of the first 9 times is equal to false, and the isLast of the 10th time is equal to true. Assuming that the status returned when calling doFilter method for the first time is not empty, It will cause the doFilter for the next 9 times to no longer be called!
5. The doFilter method has an input parameter named bodyChunk, which represents a part of the real response body (assuming that there are ten doFilter calls in a web request, the bodyChunk of each doFilter can be regarded as one tenth of the complete response body). Here is a key point: bodyChunk is only valid during the current doFilter execution, Do not save bodyChunk for use elsewhere (for example, in the member variable of body filter)
6. Continue to look at the return value of the doFilter method. Just mentioned that the return value is a one-dimensional array with only three elements: status, headers and filtered_ Chunk. If the status and headers have been set before (for example, in content handler or header filter), the returned status and headers will be ignored (that is, in fact, nginx clojure framework only judges whether the status is empty and is used to end the processing flow of body filter. It doesn't care about the specific value of status)
7. Look at the third element of the return value of the doFilter method, filtered_chunk, which can be one of the following four types:

• File, viz. java.io.File

• String

• InputStream

• Array/Iterable, e.g. Array/List/Set of above types

• Next, enter the actual combat. The detailed steps are as follows:
  

• The first is to develop a web interface that returns binary streams. In order to simplify and save trouble, another capability of nginx clojure is directly used to implement: clojure type services in nginx The following contents can be added to conf. although the code is not java, it can be barely understood (it's OK to understand it, after all, it's not the key point), that is, 1024 lines of strings are continuously written, and the content of each line is' 123456789 ':

location /largebody {
	content_handler_type 'clojure';
    content_handler_code '
    	(do
        	(use \'[nginx.clojure.core])
            (fn[req]
            	{:status 200
                 :headers {}
                 :body (for [i (range 1024)] "123456789\n")})
        )';
}

• The next step is to focus on the binary stream oriented body filter, streamfacedbody filter Java, the body filter used to process binary streams. It can be seen that this is a very simple logic. You can use this InputStream according to your actual needs:

package com.bolingcavalry.filterdemo;

import nginx.clojure.NginxChainWrappedInputStream;
import nginx.clojure.NginxClojureRT;
import nginx.clojure.java.NginxJavaBodyFilter;
import java.io.IOException;
import java.io.InputStream;
import java.util.Map;

public class StreamFacedBodyFilter implements NginxJavaBodyFilter {

    @Override
    public Object[] doFilter(Map<String, Object> request, InputStream bodyChunk, boolean isLast) throws IOException {
        // Here, only the binary file length is printed to the log, and you can modify it according to the actual business situation
        NginxClojureRT.log.info("isLast [%s], total [%s]", String.valueOf(isLast), String.valueOf(bodyChunk.available()));

        // The reading position of the index record of the member variable of NginxChainWrappedInputStream should be reset after using up this time, because bodyChunk may also be read in codes other than doFilter
        ((NginxChainWrappedInputStream)bodyChunk).rewind();

        if (isLast) {
            // isLast equals true, indicating that the doFilter method is called for the last time during the current web request,
            // Body is the last part of the complete response body,
            // At this time, the returned status should not be empty, so the nginx clojure framework will complete the execution process of the body filter and return the status and aggregated body to the client
            return new Object[] {200, null, bodyChunk};
        }else {
            // isLast equals false, which means that the doFilter method will continue to be invoked in the current web request process. The current call is only one of many times.
            // Body is a part of the complete response body,
            // At this time, the returned status should be empty, so that the nginx clojure framework will continue the execution process of body filter and continue to call doFilter
            return new Object[] {null, null, bodyChunk};
        }
    }
}

• And in nginx Conf and let StreamFacedBodyFilter process the body returned by / largebody. As shown below, add an interface / streambodyfilterdemo, which will be directly transmitted to / largebody, and use StreamFacedBodyFilter to process the response body:

        location /streambodyfilterdemo {
            # The type of body filter is java
            body_filter_type java;
            body_filter_name 'com.bolingcavalry.filterdemo.StreamFacedBodyFilter';
            proxy_http_version 1.1;
            proxy_buffering off;
            proxy_pass http://localhost:8080/largebody;
        }

• After writing, compile the jar file, copy it to the jars directory, and restart nginx
• Access / streambodyfilterdemo on postman, and the response is as follows, which meets the expectation:
  
• Recheck the file nginx-clojure-0.5.2/logs/error Log, you can see the log of StreamFacedBodyFilter, which proves that the body filter has indeed taken effect. In addition, you can also see that the doFilter method of StreamFacedBodyFilter object will be called many times by neginx clojure in one request:

2022-02-15 21:34:38[info][23765][main]isLast [false], total [3929]
2022-02-15 21:34:38[info][23765][main]isLast [false], total [4096]
2022-02-15 21:34:38[info][23765][main]isLast [false], total [2215]
2022-02-15 21:34:38[info][23765][main]isLast [true], total [0]

• So far, we have completed the filter and learning practice of header and body together. We have almost understood the general functions of Nginx clojure, but the series of Java extended Nginx is not over yet. There are still wonderful contents that will appear one after another. Please pay attention. Xinchen's original will live up to your expectations~

Source download

• The complete source code of Java extension Nginx can be downloaded from GitHub. The address and link information are shown in the table below( https://github.com/zq2599/blog_demos):

name	link	remarks
Project Home	https://github.com/zq2599/blog_demos	The project is on the home page of GitHub
git warehouse address (https)	https://github.com/zq2599/blog_demos.git	The warehouse address of the source code of the project, https protocol
git warehouse address (ssh)	git@github.com:zq2599/blog_demos.git	The warehouse address of the source code of the project, ssh protocol

• There are multiple folders in this git project. The source code of this article is in the filter demo sub project under the nginx clojure tutorials folder, as shown in the red box below:
  

• This article involves nginx For the modification of conf, the complete reference is here: https://raw.githubusercontent.com/zq2599/blog_demos/master/nginx-clojure-tutorials/files/nginx.conf

You're not alone. Xinchen's original accompanies you all the way

1. Java series
2. Spring collection
3. Docker series
4. kubernetes series
5. Database + middleware series
6. DevOps series