- UV(Unique Visitor)
- For independent visitors, you need to remove the weight statistics through the user's IP
- Statistics shall be made for each visit
- Hyperlog has good performance and small storage space
- DAU(Dail Active User)
- For daily active users, it is necessary to remove the weight statistics through the user ID
- Once visited, it is considered active (the definition can be different)
- Bitmap has good performance and can count accurate results
Rediskey
RedisKeyUtil.java
// Single day UV
public static String getUVKey(String date) {
return PREFIX_UV + SPLIT + date;
}
// Interval UV
public static String getUVKey(String startDate, String endDate) {
return PREFIX_UV + SPLIT + startDate + SPLIT + endDate;
}
// Single day active users
public static String getDAUKey(String date) {
return PREFIX_DAU + SPLIT + date;
}
// Interval active users
public static String getDAUKey(String startDate, String endDate) {
return PREFIX_DAU + SPLIT + startDate + SPLIT + endDate;
}
Service
Because Redis is used to store data, you don't need to access the DAO layer, just drop the data in the Service layer.
DataService.java
UV(Unique Visitor)
1. Counts the specified IP into the UV
@Service
public class DataService {
@Autowired
private RedisTemplate redisTemplate;
private SimpleDateFormat df = new SimpleDateFormat("yyyyMMdd");
// Counts the specified IP into the UV
public void recordUV(String ip) {
String redisKey = RedisKeyUtil.getUVKey(df.format(new Date()));
redisTemplate.opsForHyperLogLog().add(redisKey, ip);
}
1) private SimpleDateFormat df = new SimpleDateFormat("yyyyMMdd") specify the date format first
2. Statistics: counts UV s within a specified date range
// Counts UV s within the specified date range
public long calculateUV(Date start, Date end) {
if (start == null || end == null) {
throw new IllegalArgumentException("Parameter cannot be empty!");
}
// Sort out the key s within the date range
List<String> keyList = new ArrayList<>();
Calendar calendar = Calendar.getInstance();
calendar.setTime(start);
while (!calendar.getTime().after(end)) {
String key = RedisKeyUtil.getUVKey(df.format(calendar.getTime())); // Single day UV
keyList.add(key);
calendar.add(Calendar.DATE, 1); // Date + 1
}
// Merge these data
String redisKey = RedisKeyUtil.getUVKey(df.format(start), df.format(end)); // key for generating interval UV s
redisTemplate.opsForHyperLogLog().union(redisKey, keyList.toArray());
// Returns the result of the statistics
return redisTemplate.opsForHyperLogLog().size(redisKey);
}
1) The date parameter passed in is the Data class
2) To count the data within the date range, you need to form a group of Rediskey: List < string > keylist, in which the Calendar class is used to cycle the date
3) Merge data
4) Call redistemplate opsForHyperLogLog(). Size() get statistics
DAU(Dail Active User)
1. Count the specified user into DAU according to userId
public void recordDAU(int userId) {
String redisKey = RedisKeyUtil.getDAUKey(df.format(new Date()));
redisTemplate.opsForValue().setBit(redisKey, userId, true);
}
2. Counts daus within the specified date range
Similar to the calculateUV() method, the difference is that the OR operation is performed on the data within the interval, and the connection Bitop() requires the Byte array of RedisKey to be passed in
public long calculateDAU(Date start, Date end) {
if (start == null || end == null) {
throw new IllegalArgumentException("Parameter cannot be empty!");
}
// Sort out the key s within the date range
List<byte[]> keyList = new ArrayList<>(); // Convert RedisKey to byte array
Calendar calendar = Calendar.getInstance();
calendar.setTime(start);
while (!calendar.getTime().after(end)) {
String key = RedisKeyUtil.getDAUKey(df.format(calendar.getTime()));
keyList.add(key.getBytes());
calendar.add(Calendar.DATE, 1);
}
// Perform OR operation
return (long) redisTemplate.execute(new RedisCallback() {
@Override
public Object doInRedis(RedisConnection connection) throws DataAccessException {
String redisKey = RedisKeyUtil.getDAUKey(df.format(start), df.format(end));
connection.bitOp(RedisStringCommands.BitOperation.OR, // OR operation
redisKey.getBytes(), keyList.toArray(new byte[0][0]));
return connection.bitCount(redisKey.getBytes()); // The number of statistics digits is true
}
});
}
}
Controller
@Controller
public class DataController {
@Autowired
private DataService dataService;
// Statistics page
@RequestMapping(path = "/data", method = {RequestMethod.GET, RequestMethod.POST})
public String getDataPage() {
return "/site/admin/data";
}
// Statistics website UV
@RequestMapping(path = "/data/uv", method = RequestMethod.POST)
public String getUV(@DateTimeFormat(pattern = "yyyy-MM-dd") Date start,
@DateTimeFormat(pattern = "yyyy-MM-dd") Date end, Model model) {
long uv = dataService.calculateUV(start, end);
model.addAttribute("uvResult", uv);
model.addAttribute("uvStartDate", start); // Put the date parameter into the Model so that the page can display the default value
model.addAttribute("uvEndDate", end);
return "forward:/data";
}
// Statistics of active users
@RequestMapping(path = "/data/dau", method = RequestMethod.POST)
public String getDAU(@DateTimeFormat(pattern = "yyyy-MM-dd") Date start,
@DateTimeFormat(pattern = "yyyy-MM-dd") Date end, Model model) {
long dau = dataService.calculateDAU(start, end);
model.addAttribute("dauResult", dau);
model.addAttribute("dauStartDate", start);
model.addAttribute("dauEndDate", end);
return "forward:/data";
}
}
1. The "/ data" path opens the website statistics page
2. path = "/ data/uv" Statistics UV.
1) @ DateTimeFormat(pattern = "yyyy MM DD"): Date start is the processing of date parameters.
The method parameter in the Controller in spring MVC is of type Date. When you want to limit the time format of the incoming request, you can specify it through @ DateTimeFormat. However, if the incoming request parameter does not match the specified format, an error of 400 will be returned.
Time format annotation @ DateTimeFormat in Spring
2) return "forward:/data": forward requests forwarding. It is declared that this method can only handle half of the processing, and another method is needed to handle it. The request is forwarded to the "/ data" path. Since it is still the same request, the "/ data" path should also support requestmethod Post request
https://blog.csdn.net/u011676300/article/details/79657933
http://www.51gjie.com/javaweb/956.html
Interceptor
Data should be recorded for each request, so interceptors are used here
@Component
public class DataInterceptor implements HandlerInterceptor {
@Autowired
private DataService dataService;
@Autowired
private HostHolder hostHolder; // Get current login user
// Execute before Controller
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
// Statistical UV
String ip = request.getRemoteHost(); // Get IP
dataService.recordUV(ip); // Count UV
// Statistical DAU
User user = hostHolder.getUser();
if (user != null) {
dataService.recordDAU(user.getId());
}
return true; // Request to continue downward execution
}
}
Configure DataInterceptor to intercept all requests except static resources
WebMvcConfig.java
@Autowired
private DataInterceptor dataInterceptor;
@Override
public void addInterceptors(InterceptorRegistry registry) {
...
registry.addInterceptor(dataInterceptor)
.excludePathPatterns("/**/*.css", "/**/*.js", "/**/*.png", "/**/*.jpg", "/**/*.jpeg");
}
Template
<!-- content -->
<div class="main">
<!-- website UV -->
<div class="container pl-5 pr-5 pt-3 pb-3 mt-3">
<h6 class="mt-3"><b class="square"></b> website UV</h6>
<form class="form-inline mt-3" method="post" th:action="@{/data/uv}"> // Request path
<input type="date" class="form-control" required name="start" th:value="${#dates. Format (uvStartDate, 'yyyy MM DD')} "/ > / / name =" start "th: value is the formatted date class uvStartDate
<input type="date" class="form-control ml-3" required name="end" th:value="${#dates.format(uvEndDate,'yyyy-MM-dd')}"/>
<button type="submit" class="btn btn-primary ml-3">Start statistics</button>
</form>
<ul class="list-group mt-3 mb-3">
<li class="list-group-item d-flex justify-content-between align-items-center">
Statistical results
<span class="badge badge-primary badge-danger font-size-14" th:text="${uvResult}">0</span>
</li>
</ul>
</div>
< input type = "date" >: input time form default style
Input time form default style modification (input[type = "date"])
Active user templates are similar
<!-- Active user -->
<div class="container pl-5 pr-5 pt-3 pb-3 mt-4">
<h6 class="mt-3"><b class="square"></b> Active user</h6>
<form class="form-inline mt-3" method="post" th:action="@{/data/dau}">
<input type="date" class="form-control" required name="start" th:value="${#dates.format(dauStartDate,'yyyy-MM-dd')}"/>
<input type="date" class="form-control ml-3" required name="end" th:value="${#dates.format(dauEndDate,'yyyy-MM-dd')}"/>
<button type="submit" class="btn btn-primary ml-3">Start statistics</button>
</form>
<ul class="list-group mt-3 mb-3">
<li class="list-group-item d-flex justify-content-between align-items-center">
Statistical results
<span class="badge badge-primary badge-danger font-size-14" th:text="${dauResult}">0</span>
</li>
</ul>
</div>
</div>