1. Background

1.1 There are several problems with this:

1. Write data in the business logic of the GET request!
2. If the concurrency is high, the database pressure is too high;
3. At the same time, if articles are stored in caches and search engines such as ElasticSearch, synchronous updates cache and ElasticSearch updates are too time consuming to synchronize updates, not updating will result in data inconsistencies.

1.2 Solution

• HyperLogLog

HyperLog is one of the Probabilistic data Structures. The basic idea of such data structures is to use algorithms on statistical probability, sacrificing the accuracy of the data to save memory and improve the performance of related operations.

• Design Ideas

1. In order to ensure true blog browsing, the only check is made according to the ip and article id accessed by the user, that is, the same user visits the same article multiple times and the amount of access to changed articles increases by only 1;
2. Store users'views in Redis using opsForHyperLogLogLog (). add (key, value), and update the views to the database through a timed task at midnight when they have low views.

2. Hand-held implementation

2.1 Project Configuration

• sql

DROP TABLE IF EXISTS `article`;

CREATE TABLE `article` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'Primary key',
  `title` varchar(100) NOT NULL COMMENT 'Title',
  `content` varchar(1024) NOT NULL COMMENT 'content',
  `url` varchar(100) NOT NULL COMMENT 'address',
    `views` bigint(20) NOT NULL COMMENT 'Views',
  `create_time` timestamp NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Creation Time',
  PRIMARY KEY (`id`)
) ENGINE=INNODB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;

INSERT INTO article VALUES(1,'Test Article','content','url',10,NULL);

• Projects depend on pom.xml

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
</dependency>
<!--mysql-->
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
</dependency>
<!-- mybatis -->
<dependency>
    <groupId>org.mybatis.spring.boot</groupId>
    <artifactId>mybatis-spring-boot-starter</artifactId>
    <version>1.3.2</version>
</dependency>
<!-- redis -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-pool2</artifactId>
    <version>2.0</version>
</dependency>
<!-- lombok-->
<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <optional>true</optional>
</dependency>

• application.yml

spring:
  # Database Configuration
  datasource:
    url: jdbc:mysql://47.98.178.84:3306/dev
    username: dev
    password: password
    driver-class-name: com.mysql.cj.jdbc.Driver
  redis:
    host: 47.98.178.84
    port: 6379
    database: 1
    password: password
    timeout: 60s  # Connection timeout, type of parameter in 2.0 is Duration, where you need to specify the unit when configuring
    # Connection pool configuration, using jedis or lettuce directly in 2.0 (using lettuce, dependencies must contain commons-pool2 packages)
    lettuce:
      pool:
        # Maximum number of idle connections
        max-idle: 500
        # Minimum number of idle connections
        min-idle: 50
        # Maximum time to wait for available connections, negative unlimited
        max-wait:  -1s
        # Maximum number of active connections, negative unlimited
        max-active: -1


# mybatis
mybatis:
  mapper-locations: classpath:mapper/*.xml
#  type-aliases-package: cn.van.redis.view.entity

2.2 View Side Design

• Customize a comment to add new article views to Redis

@Target({ElementType.PARAMETER, ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface PageView {
    /**
     * describe
     */
    String description()  default "";
}

• Cutting treatment

 @Aspect
@Configuration
@Slf4j
public class PageViewAspect {

    @Autowired
    private RedisUtils redisUtil;

    /**
     * breakthrough point
     */
    @Pointcut("@annotation(cn.van.redis.view.annotation.PageView)")
    public void PageViewAspect() {

    }

    /**
     * Cut-in processing
     * @param joinPoint
     * @return
     */
    @Around("PageViewAspect()")
    public  Object around(ProceedingJoinPoint joinPoint) {
        Object[] object = joinPoint.getArgs();
        Object articleId = object[0];
        log.info("articleId:{}", articleId);
        Object obj = null;
        try {
            String ipAddr = IpUtils.getIpAddr();
            log.info("ipAddr:{}", ipAddr);
            String key = "articleId_" + articleId;
            // Store views in redis
            Long num = redisUtil.add(key,ipAddr);
            if (num == 0) {
                log.info("this ip:{},Visits have been added", ipAddr);
            }
            obj = joinPoint.proceed();
        } catch (Throwable e) {
            e.printStackTrace();
        }
        return obj;
    }
}

• Tool class RedisUtils.java

 @Component
public  class RedisUtils {

    @Resource
    private RedisTemplate<String, Object> redisTemplate;

    /**
     * Delete Cache
     * @param key One or more values can be passed
     */
    public void del(String... key) {
        redisTemplate.delete(key[0]);
    }

    /**
     * count
     * @param key
     * @param value
     */
    public Long add(String key, Object... value) {
        return redisTemplate.opsForHyperLogLog().add(key,value);
    }
    /**
     * Get Total
     * @param key
     */
    public Long size(String key) {
        return redisTemplate.opsForHyperLogLog().size(key);
    }

}

• Tool class IpUtils.java

 @Slf4j
public class IpUtils {

    public static String getIpAddr() {
        try {
            Enumeration<NetworkInterface> allNetInterfaces = NetworkInterface.getNetworkInterfaces();
            InetAddress ip = null;
            while (allNetInterfaces.hasMoreElements()) {
                NetworkInterface netInterface = (NetworkInterface) allNetInterfaces.nextElement();
                if (netInterface.isLoopback() || netInterface.isVirtual() || !netInterface.isUp()) {
                    continue;
                } else {
                    Enumeration<InetAddress> addresses = netInterface.getInetAddresses();
                    while (addresses.hasMoreElements()) {
                        ip = addresses.nextElement();
                        if (ip != null && ip instanceof Inet4Address) {
                            log.info("Acquired ip Address:{}", ip.getHostAddress());
                            return ip.getHostAddress();
                        }
                    }
                }
            }
        } catch (Exception e) {
            log.error("Obtain ip Address failure,{}",e);
        }
        return null;
    }
}

2.3 Synchronization task ArticleViewTask.java

@Component
@Slf4j
public class ArticleViewTask {

    @Resource
    private RedisUtils redisUtil;
    @Resource
    ArticleService articleService;

    // Execute at 1 a.m. every day
    @Scheduled(cron = "0 0 1 * * ? ")
    @Transactional(rollbackFor=Exception.class)
    public void createHyperLog() {
        log.info("Browse Inbound Start");

        List<Long> list = articleService.getAllArticleId();
        list.forEach(articleId ->{
            // Get the number of views for each article in redis and store them in the database
            String key  = "articleId_"+articleId;
            Long view = redisUtil.size(key);
            if(view>0){
                ArticleDO articleDO = articleService.getById(articleId);
                Long views = view + articleDO.getViews();
                articleDO.setViews(views);
                int num = articleService.updateArticleById(articleDO);
                if (num != 0) {
                    log.info("Updated views of the database are:{}", views);
                    redisUtil.del(key);
                }
            }
        });
        log.info("Browse Inbound End");
    }

}

2.4 Test interface PageController.java

@RestController
@Slf4j
public class PageController {

    @Autowired
    private ArticleService articleService;

    @Autowired
    private RedisUtils redisUtil;

    /**
     * When you visit an article, increase its number of views: focused notes
     * @param articleId: Article id
     * @return
     */
    @PageView
    @RequestMapping("/{articleId}")
    public String getArticle(@PathVariable("articleId") Long articleId) {
        try{
            ArticleDO blog = articleService.getById(articleId);
            log.info("articleId = {}", articleId);
            String key = "articleId_"+articleId;
            Long view = redisUtil.size(key);
            log.info("redis Number of browses in the cache:{}", view);
            //Get directly from the cache and add to the previous number
            Long views = view + blog.getViews();
            log.info("Total article views:{}", views);
        } catch (Throwable e) {
            return  "error";
        }
        return  "success";
    }
}

3. Testing

Start the project, test access, request first http://localhost:8080/1 , the log is printed as follows:

2019-03-2623:50:50.047  INFO 2970 --- [nio-8080-exec-1]  cn.van.redis.view.aspect.PageViewAspect  : articleId:1
2019-03-2623:50:50.047  INFO 2970 --- [nio-8080-exec-1] cn.van.redis.view.utils.IpUtils          : Acquired ip Address: 192.168.1.104
2019-03-2623:50:50.047  INFO 2970 --- [nio-8080-exec-1] cn.van.redis.view.aspect.PageViewAspect  : ipAddr:192.168.1.104
2019-03-2623:50:50.139  INFO 2970 --- [nio-8080-exec-1] io.lettuce.core.EpollProvider            : Starting without optional epoll library
2019-03-2623:50:50.140  INFO 2970 --- [nio-8080-exec-1] io.lettuce.core.KqueueProvider           : Starting without optional kqueue library
2019-03-2623:50:50.349  INFO 2970 --- [nio-8080-exec-1] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2019-03-2623:50:50.833  INFO 2970 --- [nio-8080-exec-1] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2019-03-2623:50:50.872  INFO 2970 --- [nio-8080-exec-1] c.v.r.v.web.controller.PageController    : articleId = 1
2019-03-2623:50:50.899  INFO 2970 --- [nio-8080-exec-1] c.v.r.v.web.controller.PageController    : redis Number of browses in the cache: 1
2019-03-2623:50:50.900  INFO 2970 --- [nio-8080-exec-1] c.v.r.v.web.controller.PageController    : Total articles viewed: 11

Looking at the database, the amount of access did not increase, local access again, found that the log prints as follows:

2019-03-2623:51:14.658  INFO 2970 --- [nio-8080-exec-3] 
cn.van.redis.view.aspect.PageViewAspect  : articleId:1
2019-03-2623:51:14.658  INFO 2970 --- [nio-8080-exec-3] cn.van.redis.view.utils.IpUtils          : Acquired ip Address: 192.168.1.104
2019-03-2623:51:14.658  INFO 2970 --- [nio-8080-exec-3] cn.van.redis.view.aspect.PageViewAspect  : ipAddr:192.168.1.104
2019-03-2623:51:14.692  INFO 2970 --- [nio-8080-exec-3] cn.van.redis.view.aspect.PageViewAspect  : this ip:192.168.1.104,Visits have been added
2019-03-2623:51:14.752  INFO 2970 --- [nio-8080-exec-3] c.v.r.v.web.controller.PageController    : articleId = 1
2019-03-2623:51:14.760  INFO 2970 --- [nio-8080-exec-3] c.v.r.v.web.controller.PageController    : redis Number of browses in the cache: 1
2019-03-2623:51:14.761  INFO 2970 --- [nio-8080-exec-3] c.v.r.v.web.controller.PageController    : Total articles viewed: 11

• Timed task triggered, log printed as follows

2019-03-27 01:00:00.265  INFO 2974 --- [   scheduling-1] cn.van.redis.view.task.ArticleViewTask   : Browse Inbound Start
2019-03-27 01:00:00.448  INFO 2974 --- [   scheduling-1] io.lettuce.core.EpollProvider            : Starting without optional epoll library
2019-03-27 01:00:00.449  INFO 2974 --- [   scheduling-1] io.lettuce.core.KqueueProvider           : Starting without optional kqueue library
2019-03-27 01:00:00.663  INFO 2974 --- [   scheduling-1] cn.van.redis.view.task.ArticleViewTask   : Updated views of the database are as follows:11
2019-03-27 01:00:00.682  INFO 2974 --- [   scheduling-1] cn.van.redis.view.task.ArticleViewTask   : Browse Inbound End

Looking at the database, we find that the number of browses in the database has increased to 11. At the same time, the number of browses in Redis has disappeared, indicating success!

4. Source Code and Description

4.1 Source Address

Github sample code