Spike project 05 - Page Optimization Technology

1. Page cache + URL cache + object cache

1.1 page caching

Page caching is to save the requested pages in redis. This caching technology is generally used for pages that do not change information frequently and are accessed more times, so that they do not need to be loaded dynamically every time
Product list page cache: 1 take cache 2 render manually 3 output results

  1. Modify / goods / to in GoodsController_ List / requests the returned content to return the content of the html page.
	@RequestMapping(value = "/to_list", produces = "text/html")
    @ResponseBody
    public String toList(HttpServletRequest request, HttpServletResponse response, Model model, MiaoshaUser user) {

        //Fetch cache
        String html = redisService.get(GoodsKey.getGoodsList,"", String.class);
        if (!StringUtils.isEmpty(html)) {
            return html;
        }
        //Query product list
        List<GoodsVo> goodsList = goodsService.listGoodsVo();
        model.addAttribute("goodsList", goodsList);
        //return "goods_list";
        WebContext ctx = new WebContext(request, response, request.getServletContext(), request.getLocale(), model.asMap());
        //Manual rendering
        html = thymeleafViewResolver.getTemplateEngine().process("goods_list", ctx);
        if (StringUtils.isNotBlank(html)) {
            redisService.set(GoodsKey.getGoodsList, "", html);
        }
        return html;
    }
  1. Write goodskey (store key and expireTime in redis)
public class GoodsKey extends BasePrefix{

    private GoodsKey(int expireSeconds, String prefix) {
        super(expireSeconds, prefix);
    }

    public static GoodsKey getGoodsList = new GoodsKey(60, "gl");

}

1.2 URL caching

The URL cache here is equivalent to the page cache, only for the details page / goods/to_detail/{goodsId}
Different detail pages display different cache pages + rendering is essentially the same

1.3 object caching (finer grained caching)

Object caching is to put objects in the cache
MiaoshaUserservice.java

/**
     * Get objects from the database and optimize to get object data from the cache
     * First get the user data from the cache, if not, then get it from the database, and put the data into the cache
     * @param id
     * @return
     */
    public MiaoshaUser getById(long id) {
        //Fetch cache
        MiaoshaUser user = redisService.get(MiaoshaUserKey.getById, "" + id, MiaoshaUser.class);
        if (user != null) {
            return user;
        }
        //Fetch database
        user = miaoshaUserMapper.selectById(id);
        if (user != null) {
            redisService.set(MiaoshaUserKey.getById, "" + id, user);
        }
        return user;
    }
/**
     * Update user password: it involves object cache -- if you update the relevant data of object cache, you need to process the cache
     * Synchronize the information cached in the database and redis, otherwise the data will be inconsistent
     */
    public boolean updatePassword(String token, long id, String formPass) {
        //Get user
        MiaoshaUser user = getById(id);
        if (user == null) {
            throw new GlobalException(CodeMsg.MOBILE_NOT_EXIST);
        }
        //Update database
        MiaoshaUser toBeUpdate = new MiaoshaUser();
        toBeUpdate.setId(id);
        toBeUpdate.setPassword(MD5Util.formPassToDBPass(formPass, user.getSalt()));
        miaoshaUserMapper.update(toBeUpdate);
        //Process the cache to prevent data inconsistency between the database and redis cache. All caches involving this object should be processed
        redisService.delete(MiaoshaUserKey.getById, "" + id);
        user.setPassword(toBeUpdate.getPassword());
        redisService.set(MiaoshaUserKey.token, token, user);
        return true;
    }

2. The page is static and the front and rear ends are separated

1. Common technologies AngularJS and Vue.js

2. Advantages: using browser cache

3. Static product details page (separate the page from dynamic content)

How to modify the Controller layer

GoodsController.java

	@RequestMapping(value = "/detail/{goodsId}")
    @ResponseBody
    public Result<GoodsDetailVo> detail(Model model, MiaoshaUser user, @PathVariable("goodsId") long goodsId) {

        GoodsVo goods = goodsService.getGoodsVoByGoodsId(goodsId);
        long startAt = goods.getStartDate().getTime();
        long endAt = goods.getEndDate().getTime();
        long now = System.currentTimeMillis();

        int miaoshaStatus = 0;
        int remainSeconds = 0;

        if (now < startAt) {
            // The countdown to the second kill hasn't started yet
            miaoshaStatus = 0;
            remainSeconds = (int)(startAt - now) / 1000;
        } else if(now > endAt) {
            //The second kill is over
            miaoshaStatus = 2;
            remainSeconds = -1;
        } else {
            //Second kill in progress
            miaoshaStatus = 1;
            remainSeconds = 0;
        }

        GoodsDetailVo vo = new GoodsDetailVo();
        vo.setGoods(goods);
        vo.setUser(user);
        vo.setRemainSeconds(remainSeconds);
        vo.setMiaoshaStatus(miaoshaStatus);
        return Result.success(vo);
    }

Add Vo object

GoodsDetailVo.java

@Data
public class GoodsDetailVo {

    private int miaoshaStatus = 0;
    private int remainSeconds = 0;
    private GoodsVo goods;
    private MiaoshaUser user;
}

Modify common.js and add a time formatting function and a function to get url parameters

//Show loading
function g_showLoading(){
	var idx = layer.msg('Processing...', {icon: 16,shade: [0.5, '#f5f5f5'],scrollbar: false,offset: '0px', time:100000}) ;  
	return idx;
}
//salt
var g_passsword_salt="hmxP@ssw0rd"
// Get url parameters
function g_getQueryString(name) {
	var reg = new RegExp("(^|&)" + name + "=([^&]*)(&|$)");
	var r = window.location.search.substr(1).match(reg);
	if(r != null) return unescape(r[2]);
	return null;
};
//Set the time formatting function and use new Date().format("yyyyMMddhhmmss");
Date.prototype.format = function (format) {
	var args = {
		"M+": this.getMonth() + 1,
		"d+": this.getDate(),
		"h+": this.getHours(),
		"m+": this.getMinutes(),
		"s+": this.getSeconds(),
	};
	if (/(y+)/.test(format))
		format = format.replace(RegExp.$1, (this.getFullYear() + "").substr(4 - RegExp.$1.length));
	for (var i in args) {
		var n = args[i];
		if (new RegExp("(" + i + ")").test(format))
			format = format.replace(RegExp.$1, RegExp.$1.length === 1 ? n : ("00" + n).substr(("" + n).length));
	}
	return format;
};

Put the product details page in the / resources/static directory


And modify its content so that it no longer depends on Thymeleaf template engine
goods_detail.html

<!DOCTYPE HTML>
<html>
<head>
    <title>Product details</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <!-- jquery -->
    <script type="text/javascript" src="/js/jquery.min.js"></script>
    <!-- bootstrap -->
    <link rel="stylesheet" type="text/css" href="/bootstrap/css/bootstrap.min.css" />
    <script type="text/javascript" src="/bootstrap/js/bootstrap.min.js}"></script>
    <!-- jquery-validator -->
    <script type="text/javascript" src="/jquery-validation/jquery.validate.min.js"></script>
    <script type="text/javascript" src="/jquery-validation/localization/messages_zh.min.js"></script>
    <!-- layer -->
    <script type="text/javascript" src="/layer/layer.js"></script>
    <!-- md5.js -->
    <script type="text/javascript" src="/js/md5.min.js"></script>
    <!-- common.js -->
    <script type="text/javascript" src="/js/common.js"></script>
</head>
<body>

<div class="panel panel-default">
  <div class="panel-heading">Second kill product details</div>
  <div class="panel-body">
  	<span id="userTip"> You haven't logged in yet. Please log in and operate again<br/></span>
  	<span>No shipping address prompt...</span>
  </div>
  <table class="table" id="goodslist">
  	<tr>  
        <td>Trade name</td>  
        <td colspan="3" id="goodsName"></td>
     </tr>  
     <tr>  
        <td>Product picture</td>  
        <td colspan="3"><img id="goodsImg" width="200" height="200" /></td>
     </tr>
     <tr>  
        <td>Spike start time</td>  
        <td id="startTime"></td>
        <td>
        	<input type="hidden" id="remainSeconds" />
        	<span id="miaoshaTip"></span>
        </td>
        <td>
        	<form id="miaoshaForm" method="post" action="/miaosha/do_miaosha">
        		<button class="btn btn-primary btn-block" type="submit" id="buyButton">Instant spike</button>
        		<input type="hidden" name="goodsId" id="goodsId"/>
        	</form>
        </td>
     </tr>
     <tr>  
        <td>Original price of goods</td>  
        <td colspan="3" id="goodsPrice"></td>
     </tr>
      <tr>  
        <td>price spike</td>  
        <td colspan="3" id="miaoshaPrice"></td>
     </tr>
     <tr>  
        <td>Inventory quantity</td>  
        <td colspan="3" id="stockCount"></td>
     </tr>
  </table>
</div>
</body>
<script>
$(function(){
	//countDown();
    getDetail();
});

function getDetail() {
    var goodsId = g_getQueryString("goodsId");
    $.ajax({
        url: "/goods/detail/" + goodsId,
        type: "GET",
        success:function(data){
            if (data.code === 0) {
                render(data.data);
            } else {
                layer.msg(data.msg);
            }
        },
        error:function() {
            layer.msg("Client request error");
        }
    })
}

function render(detail) {
    var miaoshaStatus = detail.miaoshaStatus;
    var remainSeconds = detail.remainSeconds;
    var goods = detail.goods;
    var user = detail.user;
    if (user) {
        $("#userTip").hide();
    }
    $("#goodsName").text(goods.goodsName);
    $("#goodsImg").attr("src", goods.goodsImg);
    $("#startTime").text(new Date(goods.startDate).format("yyyy-MM-dd hh:mm:ss"));
    $("#remainSeconds").val(remainSeconds);
    $("#goodsId").val(goods.id);
    $("#goodsPrice").text(goods.goodsPrice);
    $("#miaoshaPrice").text(goods.miaoshaPrice);
    $("#stockCount").text(goods.stockCount);
    countDown();
}

function countDown(){
    //debugger;
	var remainSeconds = $("#remainSeconds").val();
	var timeout;
	//debugger;
	if(remainSeconds > 0){//The second kill hasn't started yet. Countdown
		$("#buyButton").attr("disabled", true);
		$("#miaoshaTip").html(" second kill Countdown: "+ remainSeconds +" seconds ")
		timeout = setTimeout(function(){
			$("#countDown").text(remainSeconds - 1);
			$("#remainSeconds").val(remainSeconds - 1);
			countDown();
		},1000);
	}else if(remainSeconds == 0){//Second kill in progress
		$("#buyButton").attr("disabled", false);
		if(timeout){
			clearTimeout(timeout);
		}
		$("#miaoshaTip").html(" second kill in progress ");
	}else{//The second kill is over
		$("#buyButton").attr("disabled", true);
		$("#miaoshaTip").html(" second kill is over ");
	}
}

</script>
</html>

Modify parts of the goods_list page

4. Static order details page

New CodeMsg

	//Order module 5004XX
    ORDER_NOT_EXIST(500400, "Order does not exist"),

Modify Controller layer

MiaoshaController.java

@RequestMapping("/miaosha")
@Controller
public class MiaoshaController {

    @Autowired
    private GoodsService goodsService;

    @Autowired
    private OrderService orderService;

    @Autowired
    private MiaoshaService miaoshaService;

    /**
     *  QPS:909.3
     *  Abnormal%: 4.38%
     *  5000 * 10
     */
    /**
     * GET POST What's the difference?
     * GET Idempotent < a href = "/ delete? Id = 1212" >
     */
    @RequestMapping(value = "/do_miaosha", method= RequestMethod.POST)
    @ResponseBody
    public Result<OrderInfo> doMiaosha(Model model, MiaoshaUser user, @RequestParam("goodsId")long goodsId) {
        model.addAttribute("user", user);
        if (user == null) {
            return Result.fail(CodeMsg.SESSION_ERROR);
        }
        // Judge inventory
        GoodsVo goods = goodsService.getGoodsVoByGoodsId(goodsId);
        int stock = goods.getStockCount();
        if (stock <= 0) {
            return Result.fail(CodeMsg.MIAO_SHA_OVER);
        }
        // Determine whether the second kill has been reached
        MiaoshaOrder order = orderService.getMiaoshaOrderByUserIdGoodsId(user.getId(), goodsId);
        if (order != null) {
            return Result.fail(CodeMsg.REPEAT_MIAOSHA);
        }
        // Write the order under inventory reduction into the second kill order
        OrderInfo orderInfo = miaoshaService.miaosha(user, goods);
        return Result.success(orderInfo);
    }

}

Add vo object

OrderDetailVo.java

@Data
public class OrderDetailVo {
    private GoodsVo goods;
    private OrderInfo order;
}

Put the order details page in the / resources/static directory


order_detail.html

<!DOCTYPE HTML>
<html>
<head>
    <title>Order details</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <!-- jquery -->
    <script type="text/javascript" src="/js/jquery.min.js"></script>
    <!-- bootstrap -->
    <link rel="stylesheet" type="text/css" href="/bootstrap/css/bootstrap.min.css" />
    <script type="text/javascript" src="/bootstrap/js/bootstrap.min.js"></script>
    <!-- jquery-validator -->
    <script type="text/javascript" src="/jquery-validation/jquery.validate.min.js"></script>
    <script type="text/javascript" src="/jquery-validation/localization/messages_zh.min.js"></script>
    <!-- layer -->
    <script type="text/javascript" src="/layer/layer.js"></script>
    <!-- md5.js -->
    <script type="text/javascript" src="/js/md5.min.js"></script>
    <!-- common.js -->
    <script type="text/javascript" src="/js/common.js"></script>
</head>
<body>
<div class="panel panel-default">
  <div class="panel-heading">Second kill order details</div>
  <table class="table" id="goodslist">
        <tr>  
        <td>Trade name</td>  
        <td id="goodsName" colspan="3"></td>
     </tr>  
     <tr>  
        <td>Product picture</td>  
        <td colspan="2"><img id="goodsImg" wid="200" height="200" /></td>
     </tr>
      <tr>  
        <td>Order price</td>  
        <td colspan="2" id="goodsPrice"></td>
     </tr>
     <tr>
     		<td>Order time </td>  
        	<td id="createDate" colspan="2"></td>
     </tr>
     <tr>
     	<td>Order status</td>  
        <td id="orderStatus">
        </td>  
        <td>
        	<button class="btn btn-primary btn-block" type="submit" id="payButton">Immediate payment</button>
        </td>
     </tr>
      <tr>
     		<td>consignee</td>  
        	<td colspan="2">XXX  18812341234</td>  
     </tr>
     <tr>
     		<td>Receiving address</td>  
        	<td colspan="2">Longbo 1st District, Huilongguan, Changping District, Beijing</td>  
     </tr>
  </table>
</div>

</body>
</html>

<script>
    $(function(){
        getOrderDetail();
    });

    function getOrderDetail() {
        var orderId = g_getQueryString("orderId")
        $.ajax({
            url: "/order/detail/",
            type: "GET",
            data: {
                orderId:orderId
            },
            success:function(data){
                debugger;
                if (data.code === 0) {
                    render(data.data);
                } else {
                    layer.msg(data.msg);
                }
            },
            error:function() {
                layer.msg("Client request error");
            }
        })
    }

    function render(detail) {

        var goods = detail.goods;
        var order = detail.order;
        $("#goodsName").text(goods.goodsName);
        $("#goodsImg").attr("src", goods.goodsImg);
        $("#orderPrice").text(order.goodsPrice);
        $("#createDate").text(new Date(order.createDate).format("yyyy-MM-dd hh:mm:ss"));
        var status = "";
        if(order.status === 0){
            status = "Unpaid"
        }else if(order.status === 1){
            status = "To be shipped";
        }
        $("#orderStatus").text(status);
    }
</script>

Modify part of the product details page


good_detail.html

//seckill
function doMiaosha() {
    $.ajax({
        url: "/miaosha/do_miaosha",
        type: "POST",
        data: {
            goodsId:$("#goodsId").val(),
        },
        success:function(data){
            if (data.code === 0) {
                window.location.href="/order_detail.html?orderId="+data.data.id;
            } else {
                layer.msg(data.msg);
            }
        },
        error:function() {
            layer.msg("Client request error");
        }
    })
}

Add static resource configuration

# static
# SPRING RESOURCES HANDLING (ResourceProperties)
# Enable default resource processing
spring.web.resources.add-mappings=true
# The cache cycle of the resource for the resource handler service. If the duration suffix is not specified, seconds will be used. It can be overridden by the 'spring.web.resources.cache.cachecontrol' property.
spring.web.resources.cache.period=3600
# Whether to enable caching in the resource chain.
spring.web.resources.chain.cache=true
# Whether to enable Spring resource processing chain. By default, it is disabled unless at least one policy is enabled.
spring.web.resources.chain.enabled=true
# Whether to enable parsing of compressed resources (gzip, brotli). Check resource names with ". gz" or ". br" file extensions.
spring.web.resources.chain.compressed=false
# The location of static resources. The default is classpath:/static/
spring.web.resources.static-locations=classpath:/static/

Browser cache

Fix a bug

Modify the order creation method in OrderService

Some problems

Solve the oversold problem

When high concurrency access seckill requests, oversold will occur. We can judge by adding stockcount > 0 to the SQL statement during database inventory reduction

The problem of a person killing many times

  1. First, add a unique index to the userid and goodsid of the miaosha_order table

When the number of people entering the second kill Business > the number of inventory, it is necessary to judge whether the current user succeeds in reducing inventory. If it succeeds, an order will be generated, and an exception will be thrown if it fails

MiaoshaService.java

@Transactional
    public OrderInfo miaosha(MiaoshaUser user, GoodsVo goods) {
        // Write the order under inventory reduction into the second kill order
        int row = goodsService.reduceStock(goods);
        if (row > 0) {
            // order_info miaosha_order
            return orderService.createOrder(user, goods);
        }
        //row == 0 indicates that inventory reduction fails, and the order should not be created. If an error is reported, roll back
        throw new GlobalException(CodeMsg.MIAO_SHA_OVER);
    }

optimization

The miaoshaoOrder information is cached in redis as an object cache

Add OrderKey

OrderKey.java

public class OrderKey extends BasePrefix {

	public OrderKey(String prefix) {
		super(prefix);
	}
	public static OrderKey getMiaoshaOrderByUidGid = new OrderKey("moug");
}

Modify OrderService

OrderService.java

@Service
public class OrderService {

    @Autowired
    private OrderMapper orderMapper;

    @Autowired
    private RedisService redisService;

    public MiaoshaOrder getMiaoshaOrderByUserIdGoodsId(long userId, long goodsId) {
        //return orderMapper.getMiaoshaOrderByUserIdGoodsId(userId, goodsId);
        return redisService.get(OrderKey.getMiaoshaOrderByUidGid, ""+userId+"_"+goodsId, MiaoshaOrder.class);
    }

	@Transactional
    public OrderInfo createOrder(MiaoshaUser user, GoodsVo goods) {
        OrderInfo orderInfo = new OrderInfo();
        orderInfo.setCreateDate(new Date());
        orderInfo.setDeliveryAddrId(0L);
        orderInfo.setGoodsCount(1);
        orderInfo.setGoodsId(goods.getId());
        orderInfo.setGoodsName(goods.getGoodsName());
        orderInfo.setGoodsPrice(goods.getMiaoshaPrice());
        orderInfo.setOrderChannel(1);
        orderInfo.setStatus(1);
        orderInfo.setStatus(0);
        orderInfo.setUserId(user.getId());
        //The insert method will set the self incremented primary key to the object attribute
        orderMapper.insert(orderInfo);
        MiaoshaOrder miaoshaOrder = new MiaoshaOrder();
        miaoshaOrder.setGoodsId(goods.getId());
        miaoshaOrder.setOrderId(orderInfo.getId());
        miaoshaOrder.setUserId(user.getId());
        orderMapper.insertMiaoshaOrder(miaoshaOrder);
        redisService.set(OrderKey.getMiaoshaOrderByUidGid, ""+user.getId()+"_"+goods.getId(), miaoshaOrder);
        return orderInfo;
    }

    public OrderInfo getOrderById(long orderId) {
        return orderMapper.selectById(orderId);
    }
}

Re pressure measurement

First, start the main method of MyUtil class and save the token s of 5000 users in redis

Put the jar package into centos and run it, then start the pressure test and view the generated order information.


3. Static resource optimization

3.1 JS/CSS compression to reduce traffic

3.2 multiple JS/CSS combinations to reduce the number of connections

3.3 CDN nearby access

4. CDN optimization

Keywords: Java Project

Added by xerodefect on Wed, 20 Oct 2021 05:52:37 +0300