Go a library of gabs a day

brief introduction

JSON is a very popular data exchange format. Each programming language has many libraries for operating JSON, including standard libraries and third-party libraries. The standard library in Go language has built-in JSON operation library encoding/json. We have also introduced it before, which is specifically used for query JSON string Library gjson And specifically for modification JSON string Library sjson , there is also a very convenient command-line tool for manipulating JSON data jj . Today we will introduce a JSON tool library—— gabs . gabs is a library for querying and modifying JSON strings. It uses encoding/json to convert a general JSON string into map[string]interface {}, and provides a convenient method to operate map[string]struct {}.

Quick use

The code in this article uses Go Modules.

Create directory and initialize:

$ mkdir gabs && cd gabs
$ go mod init github.com/darjun/go-daily-lib/gabs

Install gabs. The latest version is v2. V2 is recommended:

$ go get -u github.com/Jeffail/gabs/v2

use:

package main

import (
  "github.com/Jeffail/gabs/v2"
  "fmt"
)

func main() {
  jObj, _ := gabs.ParseJSON([]byte(`{
    "info": {
      "name": {
        "first": "lee",
        "last": "darjun"
      },
      "age": 18,
      "hobbies": [
        "game",
        "programming"
      ]
    }
    }`))

  fmt.Println("first name: ", jObj.Search("info", "name", "first").Data().(string))
  fmt.Println("second name: ", jObj.Path("info.name.last").Data().(string))
  gObj, _ := jObj.JSONPointer("/info/age")
  fmt.Println("age: ", gObj.Data().(float64))
  fmt.Println("one hobby: ", jObj.Path("info.hobbies.1").Data().(string))
}

First, we call the gabs.ParseJSON() method to parse the incoming JSON string and get a gabs.Container object. Subsequently, query and modify the parsed data through the gabs.Container object.

gabs provides three query methods:

Call the Path() method with paths separated by;
Pass each part of the path into the Search() method as a variable parameter;
Call the JSONPointer() method with / delimited paths.

The internal implementation of the above methods finally calls the same method, but there are some differences in use. be careful:

The three methods eventually return a gabs.Container object. We need to call its Data() to get the internal data, and then do a type conversion to get the actual data;
If the incoming path is incorrect or there is no data under the path, the internal data of the gabs.Container object returned by the Search/Path method is nil, that is, the Data() method returns nil and the JSONPointer method returns err. Pay attention to null pointer and error judgment in actual use;
If the data type corresponding to a part of the path is array, you can add an index to read the data under the corresponding index, such as info.hobbies.1;
The JSONPointer() parameter must start with /.

Operation results:

$ go run main.go
first name:  lee
second name:  darjun
age:  18
one hobby:  programming

Query JSON string

In the previous section, we introduced that there are three ways to represent paths in gabs. These three methods correspond to three basic query methods:

Search (hierarchy... String): also has a short form S;
Path(path string): paths are separated by;
JSONPointer(path string): paths are separated by /.

Their basic usage has been described above. For arrays, we can also do recursive queries on each array element. In the following example, we return the name, age and relation ship fields of each element in the array members in turn:

func main() {
  jObj, _ := gabs.ParseJSON([]byte(`{
    "user": {
      "name": "dj",
      "age": 18,
      "members": [
        {
          "name": "hjw",
          "age": 20,
          "relation": "spouse"
        },
        {
          "name": "lizi",
          "age": 3,
          "relation": "son"
        }
      ]
    }
  }`))

  fmt.Println("member names: ", jObj.S("user", "members", "*", "name").Data())
  fmt.Println("member ages: ", jObj.S("user", "members", "*", "age").Data())
  fmt.Println("member relations: ", jObj.S("user", "members", "*", "relation").Data())

  fmt.Println("spouse name: ", jObj.S("user", "members", "0", "name").Data().(string))
}

Run the program, output:

$ go run main.go
member names:  [hjw lizi]
member ages:  [20 3]
member relations:  [spouse son]
spouse name:  hjw

It is easy to see that arrays encountered in the path can be handled in the following two cases:

If the next partial path is *, the remaining path query is applied to all array elements, and the results are returned in an array;
Otherwise, the next path part must be an array index, and the remaining path queries are applied to the elements where the index is located.

Looking at the source code, we can see that in fact, Path/JSONPointer first parses the path into the form of hierarchy... String, and finally calls the searchStrict method:

func (g *Container) Search(hierarchy ...string) *Container {
  c, _ := g.searchStrict(true, hierarchy...)
  return c
}

func (g *Container) Path(path string) *Container {
  return g.Search(DotPathToSlice(path)...)
}

func (g *Container) JSONPointer(path string) (*Container, error) {
  hierarchy, err := JSONPointerToSlice(path)
  if err != nil {
    return nil, err
  }
  return g.searchStrict(false, hierarchy...)
}

func (g *Container) S(hierarchy ...string) *Container {
  return g.Search(hierarchy...)
}

The searchStrict method is not complicated. Let's take a brief look:

func (g *Container) searchStrict(allowWildcard bool, hierarchy ...string) (*Container, error) {
  object := g.Data()
  for target := 0; target < len(hierarchy); target++ {
    pathSeg := hierarchy[target]
    if mmap, ok := object.(map[string]interface{}); ok {
      object, ok = mmap[pathSeg]
      if !ok {
        return nil, fmt.Errorf("failed to resolve path segment '%v': key '%v' was not found", target, pathSeg)
      }
    } else if marray, ok := object.([]interface{}); ok {
      if allowWildcard && pathSeg == "*" {
        tmpArray := []interface{}{}
        for _, val := range marray {
          if (target + 1) >= len(hierarchy) {
            tmpArray = append(tmpArray, val)
          } else if res := Wrap(val).Search(hierarchy[target+1:]...); res != nil {
            tmpArray = append(tmpArray, res.Data())
          }
        }
        if len(tmpArray) == 0 {
          return nil, nil
        }
        return &Container{tmpArray}, nil
      }
      index, err := strconv.Atoi(pathSeg)
      if err != nil {
        return nil, fmt.Errorf("failed to resolve path segment '%v': found array but segment value '%v' could not be parsed into array index: %v", target, pathSeg, err)
      }
      if index < 0 {
        return nil, fmt.Errorf("failed to resolve path segment '%v': found array but index '%v' is invalid", target, pathSeg)
      }
      if len(marray) <= index {
        return nil, fmt.Errorf("failed to resolve path segment '%v': found array but index '%v' exceeded target array size of '%v'", target, pathSeg, len(marray))
      }
      object = marray[index]
    } else {
      return nil, fmt.Errorf("failed to resolve path segment '%v': field '%v' was not found", target, pathSeg)
    }
  }
  return &Container{object}, nil
}

In fact, you go down the path layer by layer and encounter an array. If the next part is a wildcard *, here is the processing code:

tmpArray := []interface{}{}
for _, val := range marray {
  if (target + 1) >= len(hierarchy) {
    tmpArray = append(tmpArray, val)
  } else if res := Wrap(val).Search(hierarchy[target+1:]...); res != nil {
    tmpArray = append(tmpArray, res.Data())
  }
}
if len(tmpArray) == 0 {
  return nil, nil
}
return &Container{tmpArray}, nil

If * is the last part of the path, all array elements are returned:

if (target + 1) >= len(hierarchy) {
  tmpArray = append(tmpArray, val)
}

Otherwise, apply the remaining path to query each element, and the query result is append ed to the slice to be returned:

else if res := Wrap(val).Search(hierarchy[target+1:]...); res != nil {
  tmpArray = append(tmpArray, res.Data())
}

On the other hand, if it is not a wildcard, the next path must be an index. Take the element of this index and continue to query:

index, err := strconv.Atoi(pathSeg)

ergodic

gabs provides two methods to easily traverse arrays and objects:

Children(): returns the slices of all array elements. If this method is called on the object, Children() will return the slices of all values of the object in an uncertain order;
ChildrenMap(): returns the key and value of the object.

See example:

func main() {
  jObj, _ := gabs.ParseJSON([]byte(`{
    "user": {
      "name": "dj",
      "age": 18,
      "members": [
        {
          "name": "hjw",
          "age": 20,
          "relation": "spouse"
        },
        {
          "name": "lizi",
          "age": 3,
          "relation": "son"
        }
      ]
    }
  }`))

  for k, v := range jObj.S("user").ChildrenMap() {
    fmt.Printf("key: %v, value: %v\n", k, v)
  }

  fmt.Println()

  for i, v := range jObj.S("user", "members", "*").Children() {
    fmt.Printf("member %d: %v\n", i+1, v)
  }
}

Operation results:

$ go run main.go
key: name, value: "dj"
key: age, value: 18
key: members, value: [{"age":20,"name":"hjw","relation":"spouse"},{"age":3,"name":"lizi","relation":"son"}]

member 1: {"age":20,"name":"hjw","relation":"spouse"}
member 2: {"age":3,"name":"lizi","relation":"son"}

The source code of these two methods is very simple. I suggest you take a look~

Existence judgment

gabs provides two methods to check whether there is data on the corresponding path:

Exists(hierarchy ...string)；
ExistsP(path string): the method name ends with P, indicating that paths separated by. Are accepted.

See example:

func main() {
  jObj, _ := gabs.ParseJSON([]byte(`{"user":{"name": "dj","age": 18}}`))
  fmt.Printf("has name? %t\n", jObj.Exists("user", "name"))
  fmt.Printf("has age? %t\n", jObj.ExistsP("user.age"))
  fmt.Printf("has job? %t\n", jObj.Exists("user", "job"))
}

function:

$ go run main.go
has name? true
has age? true
has job? false

Get array information

gabs provides several convenient query methods for values whose type is array.

Get array size: arraycount / arraycount P. methods without suffix accept variable parameters as paths. Methods with P as suffix need to pass in. Separated paths;
Gets the element of an index of the array: ArrayElement/ArrayElementP.

Example:

func main() {
  jObj, _ := gabs.ParseJSON([]byte(`{
    "user": {
      "name": "dj",
      "age": 18,
      "members": [
        {
          "name": "hjw",
          "age": 20,
          "relation": "spouse"
        },
        {
          "name": "lizi",
          "age": 3,
          "relation": "son"
        }
      ],
      "hobbies": ["game", "programming"]
    }
  }`))

  cnt, _ := jObj.ArrayCount("user", "members")
  fmt.Println("member count:", cnt)
  cnt, _ = jObj.ArrayCount("user", "hobbies")
  fmt.Println("hobby count:", cnt)

  ele, _ := jObj.ArrayElement(0, "user", "members")
  fmt.Println("first member:", ele)
  ele, _ = jObj.ArrayElement(1, "user", "hobbies")
  fmt.Println("second hobby:", ele)
}

Output:

member count: 2
hobby count: 2
first member: {"age":20,"name":"hjw","relation":"spouse"}
second hobby: "programming"

Modification and deletion

We can use gabs to construct a JSON string. According to the type of value to be set, gabs divides the modified methods into two categories: original value, array and object. The basic operation process is the same:

Call gabs.New() to create a gabs.Container object, or ParseJSON() to parse the gabs.Container object from the existing JSON string;
Call the method to set or modify the key value, or delete some keys;
Generate the final JSON string.

Original value

As we said earlier, gabs uses three ways to express paths. When setting, you can also specify where to set the value in these three ways. The corresponding method is:

Set (value interface {}, hierarchy... String): pass in all parts of the path as variable parameters;
SetP(value interface{}, path string): each part of the path is separated by a dot;
SetJSONPointer(value interface{}, path string): each part of the path is separated by / and must start with /.

Example:

func main() {
  gObj := gabs.New()

  gObj.Set("lee", "info", "name", "first")
  gObj.SetP("darjun", "info.name.last")
  gObj.SetJSONPointer(18, "/info/age")

  fmt.Println(gObj.String())
}

Finally generated JSON string:

$ go run main.go
{"info":{"age":18,"name":{"first":"lee","last":"darjun"}}}

We can also call the StringIndent method of gabs.Container to add prefix and indent to make the output more beautiful:

fmt.Println(gObj.StringIndent("", "  "))

Observe output changes:

$ go run main.go
{
  "info": {
    "age": 18,
    "name": {
      "first": "lee",
      "last": "darjun"
    }
  }
}

array

Compared with the original value, the operation of array is much more complex. We can create a new array, or add or delete elements from the original array.

func main() {
  gObj := gabs.New()

  arrObj1, _ := gObj.Array("user", "hobbies")
  fmt.Println(arrObj1.String())

  arrObj2, _ := gObj.ArrayP("user.bugs")
  fmt.Println(arrObj2.String())

  gObj.ArrayAppend("game", "user", "hobbies")
  gObj.ArrayAppend("programming", "user", "hobbies")

  gObj.ArrayAppendP("crash", "user.bugs")
  gObj.ArrayAppendP("panic", "user.bugs")
  fmt.Println(gObj.String())
}

We first create arrays by Array/ArrayP under path user.hobbies and user.bugs, and then call ArrayAppend/ArrayAppendP to add elements to these two arrays. Now we should be able to distinguish the path of what format it accepts according to whether the method has a suffix and what the suffix is!

Operation results:

{"user":{"bugs":["crash","panic"],"hobbies":["game","programming"]}}

In fact, we can even omit the above array creation process, because ArrayAppend/ArrayAppendP will automatically create an object if it detects that there is no value on the intermediate path.

Of course, we can also delete the array elements of an index and use the ArrayRemove/ArrayRemoveP method:

func main() {
  jObj, _ := gabs.ParseJSON([]byte(`{"user":{"bugs":["crash","panic"],"hobbies":["game","programming"]}}`))

  jObj.ArrayRemove(0, "user", "bugs")
  jObj.ArrayRemoveP(1, "user.hobbies")
  fmt.Println(jObj.String())
}

After the deletion is completed, there are still:

{"user":{"bugs":["panic"],"hobbies":["game"]}}

object

Object/ObjectI/ObjectP is used to create objects in the specified path, where ObjectI refers to the creation under the specific index of the array. Generally, it is enough for us to use the Set class method. If the intermediate path does not exist, it will be created automatically.

Object deletion uses the Delete/DeleteP group of methods:

func main() {
  jObj, _ := gabs.ParseJSON([]byte(`{"info":{"age":18,"name":{"first":"lee","last":"darjun"}}}`))

  jObj.Delete("info", "name")
  fmt.Println(jObj.String())

  jObj.Delete("info")
  fmt.Println(jObj.String())
}

Output:

{"info":{"age":18}}
{}

Flatten

The flatten operation refers the deeply nested fields to the outermost layer. gabs.Flatten returns a new map[string]interface {}, the interface {} is the value of the leaf node in JSON, and the key is the path of the leaf. For example: {foo ": [{bar": "1"}, {bar ":" 2 "}]} returns {foo.0.bar":"1","foo.1.bar":"2"} after executing the flatten operation.

func main() {
  jObj, _ := gabs.ParseJSON([]byte(`{
    "user": {
      "name": "dj",
      "age": 18,
      "members": [
        {
          "name": "hjw",
          "age": 20,
          "relation": "spouse"
        },
        {
          "name": "lizi",
          "age": 3,
          "relation": "son"
        }
      ],
      "hobbies": ["game", "programming"]
    }
  }`))

  obj, _ := jObj.Flatten()
  fmt.Println(obj)
}

Output:

map[user.age:18 user.hobbies.0:game user.hobbies.1:programming user.members.0.age:20 user.members.0.name:hjw user.members.0.relation:spouse user.members.1.age:3 user.members.1.name:lizi user.members.1.relation:son user.name:dj]

merge

We can combine two Gabs. Containers into one. If there are the same keys under the same path:

If both are object types, merge them;
If both are array types, append all elements in the latter to the previous array;
One of them is an array. After merging, the value of the other key with the same name will be added to the array as an element.

For example:

func main() {
  obj1, _ := gabs.ParseJSON([]byte(`{"user":{"name":"dj"}}`))
  obj2, _ := gabs.ParseJSON([]byte(`{"user":{"age":18}}`))
  obj1.Merge(obj2)
  fmt.Println(obj1)

  arr1, _ := gabs.ParseJSON([]byte(`{"user": {"hobbies": ["game"]}}`))
  arr2, _ := gabs.ParseJSON([]byte(`{"user": {"hobbies": ["programming"]}}`))
  arr1.Merge(arr2)
  fmt.Println(arr1)

  obj3, _ := gabs.ParseJSON([]byte(`{"user":{"name":"dj", "hobbies": "game"}}`))
  arr3, _ := gabs.ParseJSON([]byte(`{"user": {"hobbies": ["programming"]}}`))
  obj3.Merge(arr3)
  fmt.Println(obj3)

  obj4, _ := gabs.ParseJSON([]byte(`{"user":{"name":"dj", "hobbies": "game"}}`))
  arr4, _ := gabs.ParseJSON([]byte(`{"user": {"hobbies": ["programming"]}}`))
  arr4.Merge(obj4)
  fmt.Println(arr4)

  obj5, _ := gabs.ParseJSON([]byte(`{"user":{"name":"dj", "hobbies": {"first": "game"}}}`))
  arr5, _ := gabs.ParseJSON([]byte(`{"user": {"hobbies": ["programming"]}}`))
  obj5.Merge(arr5)
  fmt.Println(obj5)
}

Look at the results:

{"user":{"age":18,"name":"dj"}}
{"user":{"hobbies":["game","programming"]}}
{"user":{"hobbies":["game","programming"],"name":"dj"}}
{"user":{"hobbies":["programming","game"],"name":"dj"}}
{"user":{"hobbies":[{"first":"game"},"programming"],"name":"dj"}}

summary

gabs is a very convenient JSON library, which is very easy to use, and the code implementation is relatively simple, which is worth seeing.

If you find a fun and easy-to-use Go language library, you are welcome to submit an issue on GitHub, the daily library of Go

reference resources

gabs GitHub: https://github.com/Jeffail/gabs
Go one library a day GitHub: https://github.com/darjun/go-daily-lib

Keywords: Java Go http

Added by NICKKKKK on Tue, 12 Oct 2021 10:37:42 +0300

Programming VIP

Go a library of gabs a day

brief introduction

Quick use

Query JSON string

ergodic

Existence judgment

Get array information

Modification and deletion

Original value

array

object

Flatten

merge

summary

reference resources

Popular Keywords