Two encoding formats for Post requests: application/x-www-form-urlencoded and multipart/form-data

In common business development, POST requests are often used in these places: when front-end forms are submitted, when interface code is invoked, and when the Postman test interface is used.Let's take a look:

1. When a front-end form is submitted

application/x-www-form-urlencoded

Form code:

<form action="http://localhost:8888/task/" method="POST">
First name: <input type="text" name="firstName" value="Mickey&"><br>
Last name: <input type="text" name="lastName" value="Mouse "><br>
<input type="submit" value="Submit">
</form>

The test found that the interface was accessible properly. As you can see from the Chrome developer tools, the form upload encoding format is application/x-www-form-urlencoded (in Request Headers), and the parameter format is key=value&key=value.

We can see that the server knows that the parameters are symbolized &spaced and that if &is required in the parameter values, it must be encoded.The encoding format is application/x-www-form-urlencoded (links the parameters of key-value pairs with & and converts the spaces to + plus if there are any;Has special symbols, converts special symbols to ASCII HEX values).

application/x-www-form-urlencoded is the default encoding format for browsers.For Get requests, is the parameter converted?Key=value&key=value format, after connecting to url

ps: You can test the form at this web address:http://www.runoob.com/try/try.php?filename=tryhtml_form_submit

multipart/form-data

So how does the server know where each parameter begins and ends when it receives POST requests using multipart/form-data?

<form action="http://localhost:8888/task/" method="POST" enctype="multipart/form-data">
First name: <input type="text" name="firstName" value="Mickey&"><br>
Last name: <input type="text" name="lastName" value="Mouse "><br>
<input type="submit" value="Submit">
</form>

We can see from the developer's tools that multipart/form-data does not encode parameters and uses a boundary (splitter line), which is equivalent to the value of & boundary - --- Web**AJv3.

File Upload

The uploaded file also specifies the encoding format as multipart/form-data.

<form action="http://localhost:8888/testFile" enctype="multipart/form-data" method="POST">
<input type="file" name="file">
<input type="submit" value="Submit">
</form>

If it is a SpringMVC project, the SpringBoot project does not need to be able to accept multipart/form-data type parameters on the server and configure the following in the spring context.

<bean id="multipartResolver" class="org.springframework.web.multipart.commons.CommonsMultipartResolver">
    <property name="defaultEncoding" value="utf-8"></property>
</bean>

We can simulate form submission through the FormData object, send data using the original XMLHttpRequest, and let us see the format in the Chrome development tool:

<form id="form">
    First name: <input type="text" name="firstName" value="Mickey"><br>
    Last name: <input type="text" name="lastName" value="Mouse"><br>
    <input type="file" name="file"><br>
</form>

<button onclick="submitForm()">Submit</button>

<script>
    function submitForm() {
        var formElement = document.getElementById("form");

        var xhr = new XMLHttpRequest();
        xhr.open("POST", "/task/testFile");
        xhr.send(new FormData(formElement));
    }
</script>

The format is as follows:

2. When calling interface code

1. Use the application/x-www-form-urlencoded encoding format to set the Request property in the code to invoke the interface, which can be implemented as follows:

private static String doPost(String strUrl, String content) {
        String result = "";

        try {
            URL url = new URL(strUrl);
            //A new URLConnection object is obtained by calling url.openConnection(), and the result is cast to HttpURLConnection.
            HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
            urlConnection.setRequestMethod("POST");
            //Setting the connection's timeout value to 30,000 milliseconds throws a SocketTimeoutException exception
            urlConnection.setConnectTimeout(30000);
            //Set the read timeout to 30,000 milliseconds, which throws the SocketTimeoutException exception
            urlConnection.setReadTimeout(30000);
            //Use url connections for output so that getOutputStream () can be used.The output stream returned by getOutputStream() is used to transfer data
            urlConnection.setDoOutput(true);
            //Set the Common Request property to the default browser encoding type
            urlConnection.setRequestProperty("content-type", "application/x-www-form-urlencoded");
            //The output stream returned by getOutputStream() to write parameter data.
            OutputStream outputStream = urlConnection.getOutputStream();
            outputStream.write(content.getBytes());
            outputStream.flush();
            outputStream.close();
            //The interface method is invoked.The input stream returned by getInputStream() reads the returned data.
            InputStream inputStream = urlConnection.getInputStream();
            byte[] data = new byte[1024];
            StringBuilder sb = new StringBuilder();
            //InputStream reads 1024 byte s into the data each time. When there is no data in inputSteam, the inputStream.read(data) value is -1
            while (inputStream.read(data) != -1) {
                String s = new String(data, Charset.forName("utf-8"));
                sb.append(s);
            }
            result = sb.toString();
            inputStream.close();
        } catch (IOException e) {
            e.printStackTrace();
        }

        return result;
    }

    public static void main(String[] args) {
        String str = doPost("http://localhost:8888/task/", "firstName=Mickey%26&lastName=Mouse ");
        System.out.println(str);
    }

2. When calling an interface using the multipart/form-data encoding format to set the Request property in code, the boundary value can be specified when setting the Content-Type to let the server know how to split the parameters it accepts.Call interface through the following code:

private static String doPost(String strUrl, Map<String, String> params, String boundary) {
    String result = "";

    try {
        URL url = new URL(strUrl);
        HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
        urlConnection.setRequestMethod("POST");
        urlConnection.setConnectTimeout(30000);
        urlConnection.setReadTimeout(30000);
        urlConnection.setDoOutput(true);
        //Set the universal request property to multipart/form-data
        urlConnection.setRequestProperty("content-type", "multipart/form-data;boundary=" + boundary);
        DataOutputStream dataOutputStream = new DataOutputStream(urlConnection.getOutputStream());

        for (String key : params.keySet()) {
            String value = params.get(key);
            //Be careful!Here are \r (Enter: Move the current position to the beginning of the line), \n (Break: Move the current position to the beginning of the line) to be used together
            dataOutputStream.writeBytes("--" + boundary + "\r\n");
            dataOutputStream.writeBytes("Content-Disposition: form-data; name=\"" + encode(key) + "\"\r\n");
            dataOutputStream.writeBytes("\r\n");
            dataOutputStream.writeBytes(encode(value) + "\r\n");
        }
        //The last separator ends with'--'
        dataOutputStream.writeBytes("--" + boundary + "--");
        dataOutputStream.flush();
        dataOutputStream.close();
        InputStream inputStream = urlConnection.getInputStream();
        byte[] data = new byte[1024];
        StringBuilder sb = new StringBuilder();
        while (inputStream.read(data) != -1) {
            String s = new String(data, Charset.forName("utf-8"));
            sb.append(s);
        }
        result = sb.toString();
        inputStream.close();
    } catch (Exception e) {
        e.printStackTrace();
    }

    return result;
}

private static String encode(String value) throws UnsupportedEncodingException {
    return URLEncoder.encode(value, "UTF-8");
}

public static void main(String[] args) {
    Map<String, String> params = new HashMap<>();
    params.put("firstName", "Mickey");
    params.put("lastName", "Mouse");

    //Custom boundaries have two requirements: use values that do not appear in the HTTP data sent to the server;And use the same value for the split position in the request message
    String boundary = "abcdefg";
    String str = doPost("http://localhost:8888/testFile", params, boundary);
    System.out.println(str);
}

From debug, you can see the following values for dataOutputStream:

3. When using Postman test interface

1. POST Request-> Body-> x-www-form-urlencoded


When you switch to x-www-form-urlencoded, Headers automatically add Content-Type:application/x-www-form-urlencoded

When Send is requested, at this point Code can view the same data as Content-Type and Form Data in the Chrome development tools (Request Headers)

2. POST Request-> Body-> form-data

Equivalent to an html form request, the value can be a Text or a file.

You can either specify the encoding format without manually specifying it, or you can specify the encoding as multipart/form-data

The dividing line at the line should be omitted.

You can change the type in the upper left corner to see the corresponding Headers code, which is common in the following three types:

Java OK HTTP:

JavaScript Jquery AJAX:
JavaScript XHR:

summary

Two encoding formats for POST requests: application/x-www-urlencoded, which is the default encoding format for browsers, is used for key-value pair parameters with an interval between them;Multpart/form-data is commonly used for binary files, such as files, or for key-value pair parameters that are connected to a string of character transfers (refer to Java OK HTTP).In addition to these two encoding formats, application/json is also frequently used.

Interface Code

@RequestMapping("/task")
public class TaskController {

    @RequestMapping("/")
    public String index(String firstName, String lastName) {
        return firstName + lastName;
    }

    @RequestMapping("/testFile")
    public String testFile(String firstName, String lastName, MultipartFile file) {
        String result = firstName + lastName;
        if (file != null) {
            result += (file.getOriginalFilename() != null) ? file.getOriginalFilename() : "";
        }
        return result;
    }
}

Four common POST submissions

The HTTP request methods specified in the HTTP/1.1 protocol are OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, CONNECT.POST is generally used to submit data to the server. This paper mainly discusses several ways POST submits data.

We know that the HTTP protocol is an application layer specification based on the TCP/IP protocol and is transmitted in ASCII code.The specification divides HTTP requests into three parts: the status line request line, the request header, and the message body.Similar to the following:

<method> <request-URL> <version>
<headers>

<entity-body>

The protocol specifies that data submitted by POST must be placed in the entity-body, but it does not specify how the data must be encoded.In fact, the format of the message body is entirely up to the developer as long as the last HTTP request sent satisfies the above format.

However, it makes sense for the server to parse the data successfully before it is sent out.Common server-side languages such as php, python, and their framework all have built-in capabilities to automatically parse common data formats.The service side usually knows how the message body in the request is encoded based on the Content-Type field in the headers, and then parses the body.So when it comes to POST submission data scheme, there are two parts, Content-Type and encoding method of message body.They are now officially introduced.

application/x-www-form-urlencoded

This should be the most common way for POST to submit data.The browser's native <form>form, if the enctype property is not set, will eventually submit the data as application/x-www-form-urlencoded.Requests are similar to the following (irrelevant request headers are omitted in this article):

POST http://www.example.com HTTP/1.1
Content-Type: application/x-www-form-urlencoded;charset=utf-8

title=test&sub%5B%5D=1&sub%5B%5D=2&sub%5B%5D=3

First, Content-Type is specified as application/x-www-form-urlencoded;Second, the submitted data is encoded as key1=val1&key2=val2, and both keys and Vals are URL transcoded.Most server-side languages support this approach very well.For example, $_in PHP POST['title'] can get the value of title, $_POST['sub'] can get a subarray.

This is also the way we submit data with Ajax many times.For example, Ajax for JQuery and QWrap, the default value for Content-Type is "application/x-www-form-urlencoded";Charset=utf-8".

multipart/form-data

This is another common way to submit POST data.When we use forms to upload files, we must make the enctype of the <form>form equal to multipart/form-data.Look directly at a request example:

POST http://www.example.com HTTP/1.1
Content-Type:multipart/form-data; boundary=----WebKitFormBoundaryrGKCBY7qhFd3TrwA

------WebKitFormBoundaryrGKCBY7qhFd3TrwA
Content-Disposition: form-data; name="text"

title
------WebKitFormBoundaryrGKCBY7qhFd3TrwA
Content-Disposition: form-data; name="file"; filename="chrome.png"
Content-Type: image/png

PNG ... content of chrome.png ...
------WebKitFormBoundaryrGKCBY7qhFd3TrwA--

This example is slightly more complex.First, a boundary is generated to split different fields, which is very long and complex to avoid duplication with the body content.The Content-Type then indicates that the data is encoded as multipart/form-data, and what the boundary of this request is.The message body is divided into several parts with similar structure according to the number of fields, each starting with--boundary, followed by content description information, then carriage return, and finally the specific content of the field (text or binary).If you are transferring files, include file name and file type information.The message body ends with the--boundary--flag.For a detailed definition of multipart/form-data, go to rfc1867 to see it.

This method is generally used to upload files and is well supported by various server languages.

Both of the above POST data methods are natively supported by the browser, and only native <form>forms in the current standard support these two methods (specified by the enctype attribute of the <form>element, defaulted to application/x-www-form-urlencoded).Entype actually supports text/plain, but it is rarely used.

With more and more Web sites, especially WebApp, fully using Ajax for data interaction, we can completely define new ways of data submission to facilitate development.

application/json

application/json This Content-Type is certainly familiar as a response header.In fact, more and more people now use it as a request header to tell the server that the message body is a serialized JSON string.Due to the popularity of the JSON specification, all browsers except the lower version of IE support JSON.stringify natively, and server-side languages also have functions to handle JSON, so there is no trouble using JSON.

It is also useful that the JSON format supports structured data that is much more complex than key-value pairs.Remember when I was working on a project a few years ago, the level of data I had to submit was very deep, and I submitted it after serializing the data JSON.At that time, however, I was using the JSON string as val, still in key-value pairs and submitting it as x-www-form-urlencoded.

The Ajax feature in Google's Angular JS submits JSON strings by default.For example, the following code:

var data = {'title':'test', 'sub' : [1,2,3]};
$http.post(url, data).success(function(result) {
    ...
});

The final request sent is:

POST http://www.example.com HTTP/1.1 
Content-Type: application/json;charset=utf-8

{"title":"test","sub":[1,2,3]}

This scheme makes it easy to submit complex structured data, especially for RESTful interfaces.Major package capture tools, such as Chrome's own developer tools, Firebug, Fiddler, display JSON data in a tree structure, which is very friendly.But there are also some server-side languages that do not support this approach, such as php that cannot pass through $_The POST object gets its content from the above request.At this point, you need to do it yourself: when Content-Type is application/json in the request header, from php://input Get the original input stream in and json_decode as an object.Some php frameworks are already doing this.

Of course, Angular JS can also be configured to submit data using x-www-form-urlencoded.Refer to this article if necessary.

text/xml

XML-RPC (XML Remote Procedure Call) was mentioned earlier in my blog.It is a remote call specification that uses HTTP as the transport protocol and XML as the encoding method.A typical XML-RPC request is as follows:

POST http://www.example.com HTTP/1.1 
Content-Type: text/xml

<?xml version="1.0"?>
<methodCall>
    <methodName>examples.getStateName</methodName>
    <params>
        <param>
            <value><i4>41</i4></value>
        </param>
    </params>
</methodCall>

The XML-RPC protocol is simple, functional and available in all languages.It is also widely used, such as the XML-RPC Api of WordPress, the ping Service of search engines, and so on.There are also libraries in JavaScript that support data interaction in this way, which is a good way to support existing XML-RPC services.However, I personally think the XML structure is too bulky, and JSON is more flexible and convenient for general scenarios.

Keywords: Java Spring RESTful

Added by brij_theinvader on Thu, 02 Sep 2021 19:28:07 +0300