Go Language Programming Notes 17: Web Service
Source: wallpapercave.com
Through a series of articles, I introduced how to build a Web application in Go language, specifically a website. In fact, not all Web applications exist in the form of websites, and a considerable part of them are Web services. Compared with the former, the latter has a wider range of applications. Its front end may be a website front end written by pure Js, a mobile APP, or even another Web application.
So this article will introduce how to build a Web Service.
The concept of Web Service here is different from that of Apache. It refers to Web applications that provide services through API.
Before describing web services, we should first explain two popular text transfer formats: XML and JSON. In fact, most web services use one of them as the carrier of API.
XML
Take a look at a typical XML text:
<?xml version="1.0" encoding="UTF-8"?> <article id="1" uid="1"> <Content>this is a art's content.</Content> <comments> <comment id="1" uid="1">first comment content.</comment> <comment id="2" uid="1">second comment content.</comment> <comment id="3" uid="2">third comment content.</comment> </comments> </article>
The garbled code in the Content tag is the escape of the 'symbol.
analysis
XML itself is not complex. Like HTML, it is composed of a series of tags. The relevant definitions of XML are not explained here. Let's look directly at how to parse in Go language:
package main
import (
"encoding/xml"
"fmt"
"io"
"io/ioutil"
"os"
"strings"
)
type Article struct {
XMLName xml.Name `xml:"article"`
Id int `xml:"id,attr"`
Content string `xml:content`
Comments []Comment `xml:"comments>comment"`
Uid int `xml:"uid,attr"`
}
func (a *Article) String() string {
var comments []string
for _, c := range a.Comments {
comments = append(comments, c.String())
}
scs := strings.Join(comments, ",")
return fmt.Sprintf("Article(Id:%d,Content:'%s',Comments:[%s],Uid:%d)", a.Id, a.Content, scs, a.Uid)
}
type Comment struct {
XMLName xml.Name `xml:"comment"`
Id int `xml:"id,attr"`
Content string `xml:",chardata"`
Uid int `xml:"uid,attr"`
}
func (c *Comment) String() string {
return fmt.Sprintf("Comment(Id:%d,Content:'%s',Uid:%d)", c.Id, c.Content, c.Uid)
}
func main() {
fopen, err := os.Open("art.xml")
if err != nil {
panic(err)
}
defer fopen.Close()
content, err := ioutil.ReadAll(fopen)
if err != nil && err != io.EOF {
panic(err)
}
art := Article{}
err = xml.Unmarshal(content, &art)
if err != nil {
panic(err)
}
fmt.Println(art.String())
}
The package used by Go to process XML format is encoding/xml.
And in Go Language Programming Notes 16: storing data - konjac black tea's blog (icexmoon.xyz) Similar to the ORM introduced in, to parse an XML text into a Go structure, you need to establish a mapping relationship from XML to structure, which is also reflected in the field label of the structure.
The so-called "field label" is actually the part marked with special single quotation marks after the structure field.
There are several types of field labels:
- XML: "< tag_name >", which means that the name in the current tag is tag_ The value of the child tag of name.
- XML: "< attr_name >, attr" means that the name of the current tag is attr_ Name attribute.
- XML: "innerxml" refers to the XML text contained in the current tag.
- XML: "chardata" refers to the value of the current tag.
- XML: "a > b > c" refers to the a tag contained in the current tag and the c tag contained in the B tag. Using this annotation, you can directly make the field span several levels and correspond to one or several word tags.
In particular, generally, the parser will use the name of the structure to match the label. For example, use the structure Article to match the < Article > label. However, if the structure name and label are inconsistent, you need to specify the corresponding label by adding an additional field XMLName to the structure:
type Article struct {
XMLName xml.Name `xml:"article"`
...
}
The type of XMLName is XML Name, the field label is XML: "< tag_name >".
After building a mapping relationship through field labels, it is easy to read data from a file or data stream to a string or byte sequence, and then call xml.. Unmarshal function to decode.
In most cases, there is no problem parsing XML text in this way, but sometimes it is inappropriate for some XML files or byte streams with huge content. It may be a difficult task to read the XML completely into memory.
Therefore, the XML package also provides the option of "parsing sentence by sentence", which can parse large-volume XML text on the premise of saving memory:
...
func main() {
fopen, err := os.Open("art.xml")
if err != nil {
panic(err)
}
defer fopen.Close()
d := xml.NewDecoder(fopen)
var comments []Comment
for {
token, err := d.Token()
if err == io.EOF {
//xml parsing completed
break
}
if err != nil {
//Parsing error
panic(err)
}
switch node := token.(type) {
case xml.StartElement:
if node.Name.Local == "comment" {
cmmt := Comment{}
d.DecodeElement(&cmmt, &node)
comments = append(comments, cmmt)
}
}
}
art := Article{}
art.Comments = comments
fmt.Println(art.String())
}
It is mainly divided into these steps:
- Using XML Newdecoder creates a parser from a file or data stream.
- Use the for loop and d.Token() to get XML nodes one by one from the parser.
- Using switch The case statement transforms the XML node of the interface type downward to determine whether it is a start tag. If yes, and the tag needs to be processed, call d.DecodeElement to process the specific tag.
code
Encoding variables in Go language into XML text is equivalent to the "inverse process" of parsing. The main workload is still to explain the mapping relationship through the field labels of the structure. The subsequent encoding process is easy:
func main() {
art := Article{
Id: 1,
Content: "this is a art's content.",
Uid: 1,
Comments: []Comment{
{
Id: 1,
Content: "first comment content.",
Uid: 1,
},
{
Id: 2,
Content: "second comment content.",
Uid: 1,
},
{
Id: 3,
Content: "third comment content.",
Uid: 2,
},
},
}
rest, _ := xml.Marshal(art)
fopen, err := os.OpenFile("art.xml", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
panic(err)
}
defer fopen.Close()
fopen.Write(rest)
}
If you observe the generated file art XML will be found to be a long single line string with poor readability. Of course, this does not affect the normal parsing of XML related programs, but it will cause some difficulties if you need to manually troubleshoot XML problems. In addition to directly formatting XML files with the help of IDE formatting tools, you can also directly use Go to generate XML text with good readability:
...
func main() {
...
rest, _ := xml.MarshalIndent(art, "", "\t")
fopen, err := os.OpenFile("art.xml", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
panic(err)
}
defer fopen.Close()
fopen.Write(rest)
}
Use the MarshalIndent function to specify prefix characters and indent symbols to be added when outputting XML text. In the example, \ t is specified as the indent. The number of indentation symbols used here will be adjusted according to the level of XML nodes, so the final output result is a multi-level style with strong readability:
<article id="1" uid="1"> <Content>this is a art's content.</Content> <comments> <comment id="1" uid="1">first comment content.</comment> <comment id="2" uid="1">second comment content.</comment> <comment id="3" uid="2">third comment content.</comment> </comments> </article>
But now there is another problem. The generated XML text is not complete, and there is no first line of XML declaration (the line containing XML version information).
It is also easy to add the first line of XML:
...
func main() {
...
fopen.Write([]byte(xml.Header))
fopen.Write(rest)
}
Constant XML The header contains the first line of XML.
Similar to XML parsing, XML text with huge content can also be "encoded step by step" to save memory overhead:
...
func main() {
...
fopen, err := os.OpenFile("art.xml", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
panic(err)
}
defer fopen.Close()
fopen.Write([]byte(xml.Header))
encoder := xml.NewEncoder(fopen)
encoder.Indent("", "\t")
encoder.Encode(art)
}
For coding, you don't need to implement logic like parsing, just use XML Newencoder creates an encoder from a file or data stream, and then encodes it using the encoder.
JSON
Although the function of XML is very powerful, it can even make XML have the function of "self verification" by writing DTD text for XML text, and even automatically generate XML structure description file.
However, XML has a disadvantage. Even if it carries a small part of content, it also needs to use a lot of additional content to fill various label structures, which makes the effective information content in the whole text low, which means that the transmission efficiency is very low.
Therefore, for some small applications, using XML as API carrier is not a good idea.
In contrast, JSON is much more friendly and simple in structure, which means that it is easy to build, has high effective information content, and has high transmission efficiency.
Look at a typical JSON text:
{
"id": 1,
"content": "this is a art's content.",
"contents": [
{
"id": 1,
"content": "first comment content.",
"uid": 1
},
{
"id": 2,
"content": "second comment content.",
"uid": 1
},
{
"id": 3,
"content": "third comment content.",
"uid": 2
}
],
"uid": 1
}
JSON is much simpler than XML. It has no concepts such as attributes. It is composed of key value pairs.
analysis
The method of parsing JSON is almost the same as that of XML, but the field label of JSON is much simpler than that of XML:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
"strings"
)
type Article struct {
Id int `json:"id"`
Content string `json:"content"`
Comments []Comment `json:"contents"`
Uid int `json:"uid"`
}
func (a *Article) String() string {
var comments []string
for _, c := range a.Comments {
comments = append(comments, c.String())
}
scs := strings.Join(comments, ",")
return fmt.Sprintf("Article(Id:%d,Content:'%s',Comments:[%s],Uid:%d)", a.Id, a.Content, scs, a.Uid)
}
type Comment struct {
Id int `json:"id"`
Content string `json:"content"`
Uid int `json:"uid"`
}
func (c *Comment) String() string {
return fmt.Sprintf("Comment(Id:%d,Content:'%s',Uid:%d)", c.Id, c.Content, c.Uid)
}
func main() {
fopen, err := os.Open("art.json")
if err != nil {
panic(err)
}
defer fopen.Close()
content, err := ioutil.ReadAll(fopen)
if err != nil {
panic(err)
}
art := Article{}
err = json.Unmarshal(content, &art)
if err != nil {
panic(err)
}
fmt.Println(art.String())
}
Similarly, if you are reading JSON text with large content, you need to use additional skills:
...
func main() {
fopen, err := os.Open("art.json")
if err != nil {
panic(err)
}
defer fopen.Close()
decoder := json.NewDecoder(fopen)
art := Article{}
err = decoder.Decode(&art)
if err != nil {
panic(err)
}
fmt.Println(art.String())
}
code
The way JSON text is encoded is almost the same as XML:
...
func main() {
art := Article{
Id: 1,
Content: "this is a art's content.",
Uid: 1,
Comments: []Comment{
{
Id: 1,
Content: "first comment content.",
Uid: 1,
},
{
Id: 2,
Content: "second comment content.",
Uid: 1,
},
{
Id: 3,
Content: "third comment content.",
Uid: 2,
},
},
}
rest, _ := json.Marshal(art)
fopen, err := os.OpenFile("art.json", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
panic(err)
}
defer fopen.Close()
fopen.Write(rest)
}
Similarly, the Marshal l function is used by default, and the output is in the form of reading unfriendly single line string. To output reading friendly JSON text:
...
func main() {
...
rest, _ := json.MarshalIndent(art, "", "\t")
fopen, err := os.OpenFile("art.json", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
panic(err)
}
defer fopen.Close()
fopen.Write(rest)
}
If you want to output large JSON text:
...
func main() {
...
fopen, err := os.OpenFile("art.json", os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
panic(err)
}
defer fopen.Close()
encoder := json.NewEncoder(fopen)
encoder.SetIndent("", "\t")
encoder.Encode(art)
}
Web Service
Now we can learn how to create a Web Service. For convenience, here is Go Language Programming Notes 16: storing data - konjac black tea's blog (icexmoon.xyz) Build on the simple bbs built in, and use JSON as the carrier of API.
The API is built in the way of REST (representative state transfer).
REST
REST is an idea that treats URL requests as access to resources.
For traditional URLs, the URL to get the details of a post may be as follows:
http://127.0.0.1:8080/article?id=2
The URL for deleting a post may look like this:
http://127.0.0.1:8080/del_art?art_id=6
The URL for adding a post might look like this:
http://127.0.0.1:8080/add_article
Traditional URLs use verbs + nouns in URLs to show intention. A URL represents a deterministic behavior. The HTTP method has almost no meaning, and is only related to the coding method of the specific report style when the form is submitted.
If you use REST to build the URL, the URL to get the post details may be:
GET http://127.0.0.1:8080/api/article/2
URL to delete post:
DELETE http://127.0.0.1:8080/api/article/1
URL to add post:
POST http://127.0.0.1:8080/api/article
The key to REST is that the URL itself represents a resource that can be accessed through HTTP. Combined with the HTTP method, you can know whether to add, delete or obtain the corresponding resources. Such a URL is undoubtedly much simpler.
Because the HTML specification stipulates that only GET and POST are HTTP method s that must be implemented, and other methods are not, HTML itself cannot directly use REST to build URL s, and can only be implemented with the help of other front-end programming languages such as Js.
Because REST is used here to build the API of Web Service, the original website functions should be retained, and the parameters should be parsed from the URL, so it is used in Go Language Programming Notes 13: processor - konjac black tea's blog (icexmoon.xyz) The third-party multiplexer httprouter mentioned in to add routing support to the API.
...
func main() {
...
apiRouter := httprouter.New()
http.Handle("/api/", apiRouter)
apiRouter.POST("/api/login", api.ApiLogin)
apiRouter.GET("/api/articles", api.ApiLoginCheck(api.ApiAllArticles))
apiRouter.GET("/api/article/:id", api.ApiLoginCheck(api.ApiArticleDetail))
apiRouter.DELETE("/api/article/:id", api.ApiLoginCheck(api.ApiDelArticle))
apiRouter.POST("/api/article", api.ApiLoginCheck(api.ApiAddArticle))
http.ListenAndServe(":8080", nil)
}
Access token
Websites usually use Cookie/Session mechanism as authentication method, but this often doesn't work on Web services. Because Web Service clients are diverse, there may not be a complete cookie mechanism (such as command line tool curl).
Therefore, it is more common for web services to use "access token" as the authentication mechanism.
Generally speaking, the MD5 access token itself is a non valid user access token or a non valid user access token in the form of a simple access token. Each subsequent request from the client will be accompanied by the access token on the information, which is used by the server to verify whether the user is valid.
There are many ways to generate access tokens, and many mature third-party packages can be used. Here I build the simplest access token:
package api
import (
"crypto/md5"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"strconv"
"time"
"github.com/icexmoon/go-notebook/ch17/rest-xml/model"
)
const SECURITY_TOKEN = "stn"
type Token struct {
Id int `json:"id"`
Expire time.Time `json:"expire"`
SecCode string `json:"scode"`
}
//Generate security code
func (t *Token) GetSecurityCode() (sc string, err error) {
user := model.User{Id: t.Id}
user.Get()
sc = strconv.Itoa(t.Id) + user.Password + t.Expire.Format("2006-01-02 15:04:05") + SECURITY_TOKEN
hash := md5.New()
hash.Write([]byte(sc))
sc = hex.EncodeToString(hash.Sum(nil))
return
}
//Generate access token string token
func (t *Token) String() (st string, err error) {
t.SecCode, err = t.GetSecurityCode()
if err != nil {
return
}
jsonBytes, err := json.Marshal(t)
if err != nil {
return
}
st = base64.StdEncoding.EncodeToString(jsonBytes)
return
}
//Generate Token from string token parsing
func (t *Token) Parse(token string) (err error) {
bBytes, err := base64.StdEncoding.DecodeString(token)
if err != nil {
return
}
err = json.Unmarshal(bBytes, t)
return
}
//Judge whether it is a valid token
func (t *Token) Validate() error {
//Expiration check
if t.Expire.Before(time.Now()) {
return errors.New("token is out of date, please login again")
}
//Security code check
sc, err := t.GetSecurityCode()
if err != nil {
fmt.Println(err)
return err
}
if t.SecCode == sc {
return nil
}
return errors.New("token is invalid, please login again")
}
//Get a new Token
func NewToken(id int) *Token {
token := Token{Id: id}
//The validity period is set to 1 day
token.Expire = time.Now().Add(24 * time.Hour)
return &token
}
The access Token itself contains three attributes:
- ID, representing the currently logged in user ID.
- Expire, representing the expiration time of the token.
- SecCode, security code, to ensure that the information in the access token will not be tampered with by the client.
The security code is generated by the MD5 value of user ID + Token expiration time + user password + security code Token.
The advantage of this is that the security code itself contains the user password, but does not expose the user password, and can be used to verify whether the user ID and Token expiration time have been tampered with. And you can change a security code Token after the server security code Token is leaked.
With such a security code, you can compare the security code transmitted from the client with the security code generated by the server. If it is consistent, it means that the access token is correct and the login identity is valid. Because the token contains expiration time, the token can be invalidated after a certain time, and the user can log in again to obtain the token.
In fact, such tokens can be directly passed between the client and the server in the form of JSON, but considering that most tokens exist in some encoded string, they are also passed in JSON and base64.
Sign in
...
func getParam(r *http.Request, param interface{}) error {
len := r.ContentLength
bodyBytes := make([]byte, len)
_, err := r.Body.Read(bodyBytes)
// fmt.Println(string(bodyBytes))
if err != nil && err != io.EOF {
return err
}
err = json.Unmarshal(bodyBytes, param)
if err != nil {
return err
}
return nil
}
func ApiLogin(rw http.ResponseWriter, r *http.Request, p httprouter.Params) {
param := struct {
Data struct {
Name string `json:"name"`
Password string `json:"password"`
} `json:"data"`
}{}
err := getParam(r, ¶m)
if err != nil {
fmt.Println(err)
http.Error(rw, "login error", http.StatusInternalServerError)
return
}
n := param.Data.Name
pwd := param.Data.Password
user, ok := model.CheckLogin(n, pwd)
if !ok {
http.Error(rw, "login error", http.StatusInternalServerError)
return
}
t := NewToken(user.Id)
data := struct {
Success bool `json:"success"`
Data struct {
Token string `json:"token"`
} `json:"data"`
}{}
data.Success = true
data.Data.Token, err = t.String()
if err != nil {
fmt.Println(err)
http.Error(rw, "login error", http.StatusInternalServerError)
return
}
jBytes, err := json.Marshal(data)
if err != nil {
fmt.Println(err)
http.Error(rw, "login error", http.StatusInternalServerError)
return
}
rw.Header().Set("Content-Type", "application/json")
rw.Write(jBytes)
}
...
The login logic is mainly to obtain the user name and password from the JSON message format, generate an access token and return it after verifying the password.
In order to obtain parameters from the report style conveniently, a tool function getParam is created here.
ApiPost
There are many ways to test the interface, but when I use curl, I find that when I use cmd or PowerShell to send a request with JSON through curl on the WIndows platform, it will fail to parse on the server, showing that the JSON content is in the wrong format, with an escape character at the beginning.
Guess it may be a Windows platform coding problem.
So here I use ApiPost for testing. This is an interface debugging tool. Official website:
To be honest, the UI design of this tool is quite bad. Many functions need to be searched for a long time, which is quite inhuman. But how to say, there is always better than no, just use it
The software is divided into two parts as a whole. On the left navigation bar, you can create a folder as a project and add multiple interfaces to the project.
Click on the top left to create a new interface.
Modify the interface name, http method and url here.
On the right side, modify the http encoding method and select application/json.
Fill in the requested JSON string here, and click beautify to format it (of course, it does not affect the result).
Then click send, and the results will be displayed below.
The request and return value of my test are attached here:
{
"data": {
"name": "111",
"password": "111"
}
}
{
"success": true,
"data": {
"token": "eyJpZCI6MSwiZXhwaXJlIjoiMjAyMi0wMS0wMlQxMTo0Mzo0NS40OTgzOTYzKzA4OjAwIiwic2NvZGUiOiJlMjBkZDViYWEyMzVhNWIyNTNjNmQ4NzRmNzk2ZmEzNCJ9"
}
}
In addition to the login interface, I also implemented interfaces such as adding posts, viewing posts, deleting posts and so on. It will not be explained one by one here. For the complete code, see:
Don't care about the directory name. In fact, I'm going to make two versions, one with XML and the other with JSON. Later
For my API post interface documents, see:
- https://docs.apipost.cn/preview/3bafb43d8b2a7438/70727999212468d5
Of course, there are some defects. For example, most errors can be returned in the form of http status 200 and user-defined error code and error information in the return information. In addition, some of the returned information is actually in the format of the Model layer, which is not conducive to direct use by the client. The interface for viewing post details also does not contain reply content, and there is no interface for reply, etc. Those who are interested can improve themselves.
Thank you for reading.