I. Mongo cluster

In the project, most of the content is stored in MongoDB, and this part of the data is undoubtedly massive. So how to ensure the capacity and security of the data? The answer is: cluster.

MongoDB has three clustering modes: master-slave cluster (obsolete), replica cluster (recommended) and fragment cluster.

The master-slave cluster is not recommended by the official. Generally, the replica cluster is used to replace the master-slave cluster.

1 master slave cluster (understand)

The master-slave architecture is generally used for backup or read-write separation. It consists of two roles:

• Master: readable and writable. When the data is modified, the oplog will be synchronized to all connected salve s.
• Slave: it is read-only and not writable. It automatically synchronizes data from the Master.

For Mongodb, the master slave architecture is not recommended, because the master slave cannot recover automatically after the Master goes down. Replica Set is recommended.

2 replica cluster [demo]

introduce

In order to prevent a single point of failure, a replica cluster is required, which includes three types of roles:

• Primary node

  Receive all write requests and synchronize the changes to all Secondary. A Replica Set can only have one Primary node. When the Primary node fails, other Secondary or Arbiter nodes will re elect a Primary node. The default read request is also sent to the Primary node for processing If you need to forward to the Secondary, you need the client to modify the connection configuration.

• Replica node (Secondary)

  Maintain the same dataset as the master node. When the primary node hangs up, participate in selecting the primary node.

• Arbiter

  Do not keep data, do not participate in the election, and only conduct the election voting. Using arbiter can reduce the hardware requirements of data storage. Arbiter has little hardware resource requirements when running, but it is important that it and other data nodes should not be deployed on the same machine in the production environment.

Note that the number of Replica Set nodes of an automatic failover must be an odd number. The purpose is to have a majority to make the decision when voting.

Special instructions

Environment construction

Create container

docker run -d --name=master_mongo -p 27011:27017 mongo:4.0.3  --replSet mongo_clus 
docker run -d --name=backup_mongo -p 27012:27017 mongo:4.0.3  --replSet mongo_clus 
docker run -d --name=arbi_mongo   -p 27013:27017 mongo:4.0.3  --replSet mongo_clus

Set replica set name, that is, set cluster name, which must be set, otherwise the cluster cannot be built

to configure

Enter master_mongo container

docker exec  -it master_mongo /bin/bash

Log in to Mongo

mongo

Create cluster

cfg={
  "_id":"mongo_clus",
  members:[{
      _id:0,
      host:"192.168.136.131:27011",
      priority:2
  },{
      _id:1,
      host:"192.168.136.131:27012",
      priority:1
  },{
      _id:2,
      host:"192.168.136.131:27013",
      arbiterOnly:true
  }]
}



rs.initiate(cfg)

The status of the cluster can be viewed through rs.status()

Client test

Connect to the master node using the client

use demo

db.createCollection("user")

db.user.insert({username:'zhangsan',age:18,address:'Beijing Shunyi'})

java code testing

Create the Mongo cluster module in the previous project (not in the flower exploration project)

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.1.0.RELEASE</version>
</parent>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-mongodb</artifactId>
    </dependency>
    <dependency>
        <groupId>org.mongodb</groupId>
        <artifactId>mongodb-driver-sync</artifactId>
        <version>3.9.1</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
    </dependency>
</dependencies>

Entity class

import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.bson.types.ObjectId;
import org.springframework.data.mongodb.core.mapping.Document;

@Data
@NoArgsConstructor
@AllArgsConstructor
@Document(collection = "account")
public class Account {
    private ObjectId id; //Primary key id
    private String username; //user name
    private Double money; //amount of money
}

Startup class

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class MongoClusterApplication {
    public static void main(String[] args) {
        SpringApplication.run(MongoClusterApplication.class,args);
    }
}

application.yml

server:
  port: 9999
  
spring:
  data:
    mongodb:
      uri: mongodb://192.168.136.131:27011,192.168.136.131:27012/demo?connect=replicaSet&replicaSet=mongo_clus

• connect=replicaSet the connection method is replica set
• replicaSet=myrs sets the name of the replica set to mongo_clus

test

@RunWith(SpringRunner.class)
@SpringBootTest
public class MongoTest {

    @Autowired
    private MongoTemplate mongoTemplate;

    @Test
    public void testSave() {
        Account account = new Account();
        account.setUsername("jack");
        account.setMoney(1000D);
        mongoTemplate.save(account);
        
        account = new Account();
        account.setUsername("tom");
        account.setMoney(1000D);
        mongoTemplate.save(account);
    }

    @Test
    public void testFindAll() {
        List<Account> accountList = mongoTemplate.findAll(Account.class);
        for (Account account : accountList) {
            System.out.println(account);
        }
    }
}

Transaction support

Mongodb supports transactions only when it is above 4.0. Versions 4.0-4.2 only support replica cluster transactions, and versions above 4.2 support all cluster transactions

Configure transaction manager

Transaction manager: it has been integrated with Spring

@SpringBootApplication
public class MongoClusterApplication {
    public static void main(String[] args) {
        SpringApplication.run(MongoClusterApplication.class,args);
    }

    @Bean
    public MongoTransactionManager initMongoTransactionManager(MongoDbFactory factory){
        return new MongoTransactionManager(factory);
    }
}

Transfer case

package com.itheima.service;

import com.itheima.domain.Account;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.data.mongodb.core.query.Update;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class AccountService {

    @Autowired
    private MongoTemplate mongoTemplate;

    @Transactional
    public void transfer(String sourceName, String targetName, Double money) {
        //1. Update the account amount of sourceName
        Query query = new Query(Criteria.where("username").is(sourceName));
        Account source = mongoTemplate.findOne(query, Account.class);

        Query query1 = new Query(Criteria.where("username").is(targetName));
        Account target = mongoTemplate.findOne(query1, Account.class);

        //2. Change amount
        Update update = new Update();
        update.set("money", source.getMoney() - money);
        Update update1 = new Update();
        update1.set("money", target.getMoney() + money);


        //3. Update account amount
        mongoTemplate.updateFirst(query, update, Account.class);
        int i=1/0;
        mongoTemplate.updateFirst(query1, update1, Account.class);
    }
}

test

@Autowired
private AccountService accountService;

@Test
public void testTransfer() {
    accountService.transfer("jack", "tom", 100D);
}

3 slice cluster (understand)

introduce

When the amount of data is large, we need to run the data in different machines to reduce the pressure of CPU, memory and IO. Sharding is such a technology.

sharding is a method used by MongoDB to divide large collections into different servers.

As shown in the figure:

For example, if the database has a data set of 1tb and has four slices, then each slice may hold only 256 GB of data. If there are 40 shards, each shard may have only 10TB of data.

Partitioned cluster architecture

• Data Shards

  Save data to ensure high availability and consistency of data. It can be a single mongod instance or a replica set.

  In the production environment, Shard is a Replica Set to prevent a single point of failure of the data slice. There is a PrimaryShard in all shards, which contains data sets that are not divided:

• Query routes

  An instance of mongos. The client directly connects to mongos, and mongos routes the read and write requests to the specified Shard.

  A Sharding cluster can have one mongos or multiple mongos to reduce the pressure of client requests.

• Configure servers (Config servers)

  Save the metadata of the cluster, including the routing rules of each Shard.