Spring Boot Integration - MongoDB Integration 4 (MongoDB grouping de-duplication and MongoDB table query)

Front of Article

About Versions

rely on Edition
springboot 2.0.8.RELEASE
mongodb 4.0.14

This content is just to introduce the most basic use and configuration of mongodb. As a well-known database, it has quite a lot of advanced usage. It will expand to introduce a lot of content. Of course, I am not the God of related fields. The following content is just to sort out some accumulation of my daily use.Is the accumulation of experience, but also hope to help later students

About Projects

This is also how I've tried to organize my work by touching on the various tools used in springboot.All the methods described below already provide test cases.Because each example involves more code, only part of the code is posted.The whole code is here: https://gitee.com/daifyutils/springboot-samples.

Joint Table Query

Simple Joint Table Query

mongodb multitable queries operate using the Aggregation.lookup API.The four parameters of its method lookup(String from, String localField, String foreignField, String as) are the tables that need to be joined, the joined fields in the main table, the joined tables'fields, and the joined data mapped to the fields in the result set.

The following is a simple join table query that associates the userId in the Order table with the id field in the UserInfo table and saves the UserInfo information from the query into the userInfo field in the result set.

    @Override
    public List<OrderVo> queryOrderAndUserInfo() {

        Criteria criteria = new Criteria();
        MatchOperation match = Aggregation.match(criteria);
        // Associate with another collection as its property
        LookupOperation lookup = Aggregation.lookup("UserInfo",// Join Table
            "userId",// Query table fields
            "_id",// Join fields in tables
            "userInfo");// The name of the property in which the data is returned
        TypedAggregation<Order> noRepeatAggregation2 =
            Aggregation.newAggregation(Order.class,match,lookup);

        AggregationResults<OrderVo> noRepeatDataInfoVos2 = mongoTemplate.aggregate(noRepeatAggregation2, OrderVo.class);
        List<OrderVo> noRepeatDataList2 = noRepeatDataInfoVos2.getMappedResults();
        return noRepeatDataList2;
    }

Joint Table Query - Operates on joined data

When two table data are joined, the data is mapped under an attribute.Its structure is similar to the following format.

{
    "field":"value",
    "field2":"value2",
    "field3":"value3",
    "Mapped Properties" : {
        "field":"value",
        "field2":"value2"
    }
}

For this format, we can use the object.object method to directly access the nested data, while doing some aggregation or sorting or filtering of its contents.

The following example is when, after associating an Order with a UserInfo table, the data for the UserInfo collection is nested under the userInfo property, then the type property of UserInfo is accessed using userInfo.type and sorted according to its content.

    @Override
    public List<OrderVo> queryOrderAndUserInfoSort(Sort.Direction direction) {
        Criteria criteria = new Criteria();
        MatchOperation match = Aggregation.match(criteria);
        // Associate with another collection as its property
        LookupOperation lookup = Aggregation.lookup("UserInfo",
            "userId",// Query table fields
            "_id",// Join fields in tables
            "userInfo");// The name of the property in which the data is returned
        SortOperation sort =
            Aggregation.sort(new Sort(direction,"userInfo.type"));
        TypedAggregation<Order> noRepeatAggregation2 =
            Aggregation.newAggregation(Order.class,match,lookup,sort);

        AggregationResults<OrderVo> noRepeatDataInfoVos2 = mongoTemplate.aggregate(noRepeatAggregation2, OrderVo.class);
        List<OrderVo> noRepeatDataList2 = noRepeatDataInfoVos2.getMappedResults();
        return noRepeatDataList2;
    }

Expansion of Nested Structure

For joins with a one-to-many relationship, the joined data is placed in a collection, and Aggregation.unwind can be used to expand the data.The example above adds the following statement to explain the UserInfos field content expansion

Aggregation.unwind("UserInfos")
  • Pre-expansion structure
{
    "field":"value",
    "UserInfos" : [{"field":"value1"},{"field":"value2"}]
}
  • Expanded structure
{
    "field":"value",
    "UserInfos" : {"field":"value1"}
}
{
    "field":"value",
    "UserInfos" : {"field":"value2"}
}

ps.Note that I did not implement its test code in the code.If you need to test its expansion, you need to redefine its return structure object.

Joint Table Query - Adjust the name of the joined data

Sometimes when we use multitable associations, we use objects with a single layer of data to receive data. Nested data structures at the code level may not be very convenient for us to use.At this point, we can alter the data structure of Agggregation.project as described in the previous section.In the previous article, I described how it can be used to modify data names. Here's how it can modify data structures.In the following example, userInfo.userName is pointed to userName so that the userName, as a property of nested data, is displayed as a data root node after modification.

    @Override
    public List<OrderAndUserVo> queryOrderUserInfo() {
        Criteria criteria = new Criteria();
        MatchOperation match = Aggregation.match(criteria);
        // Associate with another collection as its property
        LookupOperation lookup = Aggregation.lookup("UserInfo",// Join Table
            "userId",// Query table fields
            "_id",// Join fields in tables
            "userInfo");// The name of the property in which the data is returned
        Field userName = Fields.field("userName", "userInfo.userName");
        ProjectionOperation project = Aggregation.project("id","totalMoney","totalProduct","userId","type")
            .andInclude(Fields.from(userName));
        TypedAggregation<Order> noRepeatAggregation2 =
            Aggregation.newAggregation(Order.class,match,lookup,project);

        AggregationResults<OrderAndUserVo> noRepeatDataInfoVos2 = mongoTemplate.aggregate(noRepeatAggregation2, OrderAndUserVo.class);
        List<OrderAndUserVo> noRepeatDataList2 = noRepeatDataInfoVos2.getMappedResults();
        return noRepeatDataList2;
    }

Group Weighting

We use Aggregation.group to group data, but sometimes we want to know what is duplicated when there is multiple duplicate data and different query conditions.

For example, we classify orders for different customers into different types. Now we want to know how many different types of customers exist. By grouping "userId" and "type", we can only get aggregated results for different types of orders for different users.Of course, we can loop through the code to get the number of different types of users.However, this causes the logic to be split into data queries and program code.

The aggregated pipeline is actually a pipeline, and the data processed by the latter is the result of previous pipeline processing.Whether to solve grouping de-weighting or to use the characteristics of its pipelines.For example, in the description above, we scored by "userId" and "type" and got the following results

userId type count
1 1 This userId This type contains the amount of data
1 2 This userId This type contains the amount of data
2 1 This userId This type contains the amount of data
2 2 This userId This type contains the amount of data
2 3 This userId This type contains the amount of data
... ... ...

At this point, we don't know how many different type s exist for each userId.But our subsequent pipelines can group the data again.

We group the data again, then use userId to group the data, and the last count we get is based on the sum of the type values. The result we get here is the final result as shown in the figure below.

userId count
1 Number of type s included in this userId
2 Number of type s included in this userId
... ... ...

Code

The above set of logical usage codes is the following.

    @Override
    public List<GroupVo> getAggGroupDeDuplication() {
        // Grouping by user id and type
        GroupOperation operation1 = Aggregation.group( "userId","type");
        // Group the results again to get the type data for userId
        GroupOperation operation2 = Aggregation.group("userId").count().as("count");

        TypedAggregation<Order> noRepeatAggregation =
            Aggregation.newAggregation(Order.class,operation1,operation2);
        AggregationResults<GroupVo> noRepeatDataInfoVos = mongoTemplate.aggregate(noRepeatAggregation, GroupVo.class);
        List<GroupVo> noRepeatDataList = noRepeatDataInfoVos.getMappedResults();
        System.out.println(JSON.toJSONString(noRepeatDataList));
        return noRepeatDataList;
    }

Personal level is limited, there may be something not clearly described or wrong in the above content. If the development classmates find it, please inform them in time, I will modify the relevant content at the first time. I also hope you can give me a compliment if you can only stay at home and bury your head in the hard code during this New Year Festival.Your compliment is what drives me forward.Here I also wish you all a Happy New Year.

203 original articles were published. 15 were praised. 10,000 visits+
Private letter follow

Keywords: SpringBoot MongoDB Database Attribute

Added by coder9 on Fri, 24 Jan 2020 06:42:43 +0200