preface

What is data desensitization

Data desensitization refers to the deformation of some sensitive information through desensitization rules to realize the reliable protection of sensitive privacy data

Common desensitization rules

Replace, rearrange, encrypt, truncate, mask

Good data desensitization implementation

1. Keep meaningful information before desensitization for the application after desensitization as much as possible
2. Prevent hackers from cracking to the greatest extent

Today we talk about how to customize data desensitization

Overall thinking

This example realizes desensitization by means of substitution, and then quickly realizes desensitization in combination with common framework features, such as the interceptor mechanism of mybatis or the serialization of json

Specific landing

1. Define a desensitization tool class

You can directly reference the hutool toolkit, but it only provides this tool after version 5.6 +
https://www.hutool.cn/docs/#/core/ Tools / information desensitization tools desensitized util

Otherwise, implement one by yourself, as shown below

public class DesensitizedUtils {



    /**
     * Desensitization, using the default desensitization strategy
     * <pre>
     * DesensitizedUtil.desensitized("100", DesensitizedUtils.DesensitizedType.USER_ID)) =  "0"
     * DesensitizedUtil.desensitized("Duan Zhengchun ", desensitizedutils. Desensitizedtype. Machine_name) =" Duan**“
     * DesensitizedUtil.desensitized("51343620000320711X", DesensitizedUtils.DesensitizedType.ID_CARD)) = "5***************1X"
     * DesensitizedUtil.desensitized("09157518479", DesensitizedUtils.DesensitizedType.FIXED_PHONE)) = "0915*****79"
     * DesensitizedUtil.desensitized("18049531999", DesensitizedUtils.DesensitizedType.MOBILE_PHONE)) = "180****1999"
     * DesensitizedUtil.desensitized("No. 289, malianwa street, Haidian District, Beijing, "desensitized utils. Desensitized type. Address") = "malianwa street, Haidian District, Beijing********“
     * DesensitizedUtil.desensitized("duandazhi-jack@gmail.com.cn", DesensitizedUtils.DesensitizedType.EMAIL)) = "d*************@gmail.com.cn"
     * DesensitizedUtil.desensitized("1234567890", DesensitizedUtils.DesensitizedType.PASSWORD)) = "**********"
     * DesensitizedUtil.desensitized("Su d40000 ", desensitizedutils. Desensitizedtype. Car_license) = Su D4***0“
     * DesensitizedUtil.desensitized("11011111222233333256", DesensitizedUtils.DesensitizedType.BANK_CARD)) = "1101 **** **** **** 3256"
     * </pre>
     *
     * @param str              character string
     * @param desensitizedType Desensitization type; Chinese mobile phone can desensitization: user id, Chinese name, id card number, id number, mobile phone number, address, e-mail, password.
     * @return String after desensitization
     * @author dazer and neusoft and qiaomu
     * @since 5.6.2
     */
    public static String desensitized(CharSequence str, DesensitizedType desensitizedType) {
        if (StrUtil.isBlank(str)) {
            return StrUtil.EMPTY;
        }
        String newStr = String.valueOf(str);
        switch (desensitizedType) {
            case USER_ID:
                newStr = String.valueOf(DesensitizedUtils.userId());
                break;
            case CHINESE_NAME:
                newStr = DesensitizedUtils.chineseName(String.valueOf(str));
                break;
            case ID_CARD:
                newStr = DesensitizedUtils.idCardNum(String.valueOf(str), 1, 2);
                break;
            case FIXED_PHONE:
                newStr = DesensitizedUtils.fixedPhone(String.valueOf(str));
                break;
            case MOBILE_PHONE:
                newStr = DesensitizedUtils.mobilePhone(String.valueOf(str));
                break;
            case ADDRESS:
                newStr = DesensitizedUtils.address(String.valueOf(str), 8);
                break;
            case EMAIL:
                newStr = DesensitizedUtils.email(String.valueOf(str));
                break;
            case PASSWORD:
                newStr = DesensitizedUtils.password(String.valueOf(str));
                break;
            case CAR_LICENSE:
                newStr = DesensitizedUtils.carLicense(String.valueOf(str));
                break;
            case BANK_CARD:
                newStr = DesensitizedUtils.bankCard(String.valueOf(str));
                break;
            default:
        }
        return newStr;
    }

    /**
     * [User id] do not provide userId externally
     *
     * @return Primary key after desensitization
     */
    public static Long userId() {
        return 0L;
    }

    /**
     * [Chinese name] only displays the first Chinese character, and others are hidden as two asterisks, such as Li**
     *
     * @param fullName full name
     * @return Name after desensitization
     */
    public static String chineseName(String fullName) {
        if (StrUtil.isBlank(fullName)) {
            return StrUtil.EMPTY;
        }
        return StrUtil.hide(fullName, 1, fullName.length());
    }

    /**
     * [ID number 1 and the last 2
     *
     * @param idCardNum ID
     * @param front     Reserved: front digits in front; Start with 1
     * @param end       Reserved: the following end digits; Start with 1
     * @return ID card after desensitization
     */
    public static String idCardNum(String idCardNum, int front, int end) {
        //ID card cannot be empty
        if (StrUtil.isBlank(idCardNum)) {
            return StrUtil.EMPTY;
        }
        //The length of interception must not exceed the length of the ID number.
        if ((front + end) > idCardNum.length()) {
            return StrUtil.EMPTY;
        }
        //The to be intercepted cannot be less than 0
        if (front < 0 || end < 0) {
            return StrUtil.EMPTY;
        }
        return StrUtil.hide(idCardNum, front, idCardNum.length() - end);
    }

    /**
     * [The first four and the last two of the landlines
     *
     * @param num fixed telephone
     * @return Fixed telephone after desensitization;
     */
    public static String fixedPhone(String num) {
        if (StrUtil.isBlank(num)) {
            return StrUtil.EMPTY;
        }
        return StrUtil.hide(num, 4, num.length() - 2);
    }

    /**
     * [[mobile phone number] the first three digits, the last four digits, and others are hidden, such as 135 * * * * 2210
     *
     * @param num Mobile phone;
     * @return Desensitized mobile phone;
     */
    public static String mobilePhone(String num) {
        if (StrUtil.isBlank(num)) {
            return StrUtil.EMPTY;
        }
        return StrUtil.hide(num, 3, num.length() - 4);
    }

    /**
     * [Address] displays only the region, not the detailed address, for example, Haidian District, Beijing****
     *
     * @param address       Home address
     * @param sensitiveSize Sensitive information length
     * @return Home address after desensitization
     */
    public static String address(String address, int sensitiveSize) {
        if (StrUtil.isBlank(address)) {
            return StrUtil.EMPTY;
        }
        int length = address.length();
        return StrUtil.hide(address, length - sensitiveSize, length);
    }

    /**
     * [E-mail] the e-mail prefix only displays the first letter, other prefixes are hidden, replaced by an asterisk, @ and the following address are displayed, such as D * * @ 126 com
     *
     * @param email mailbox
     * @return Mailbox after desensitization
     */
    public static String email(String email) {
        if (StrUtil.isBlank(email)) {
            return StrUtil.EMPTY;
        }
        int index = StrUtil.indexOf(email, '@');
        if (index <= 1) {
            return email;
        }
        return StrUtil.hide(email, 1, index);
    }

    /**
     * [Password] all characters of the password are replaced by *, for example:******
     *
     * @param password password
     * @return Desensitized password
     */
    public static String password(String password) {
        if (StrUtil.isBlank(password)) {
            return StrUtil.EMPTY;
        }
        return StrUtil.repeat('*', password.length());
    }

    /**
     * [[Chinese license plate] the middle of the license plate is replaced by *
     * eg1: null       -> ""
     * eg1: ""         -> ""
     * eg3: Su D40000 - Su D4***0
     * eg4: Shaanxi A12345D - * * * * D
     * eg5: Jing A123 - if the license plate is wrong, it will not be processed
     *
     * @param carLicense Complete license plate number
     * @return Desensitized license plate
     */
    public static String carLicense(String carLicense) {
        if (StrUtil.isBlank(carLicense)) {
            return StrUtil.EMPTY;
        }
        // Ordinary license plate
        if (carLicense.length() == 7) {
            carLicense = StrUtil.hide(carLicense, 3, 6);
        } else if (carLicense.length() == 8) {
            // New energy license plate
            carLicense = StrUtil.hide(carLicense, 3, 7);
        }
        return carLicense;
    }

    /**
     * Bank card number desensitization
     * eg: 1101 **** **** **** 3256
     *
     * @param bankCardNo Bank card No
     * @return Bank card number after desensitization
     * @since 5.6.3
     */
    public static String bankCard(String bankCardNo) {
        if (StrUtil.isBlank(bankCardNo)) {
            return bankCardNo;
        }
        bankCardNo = StrUtil.trim(bankCardNo);
        if (bankCardNo.length() < 9) {
            return bankCardNo;
        }

        final int length = bankCardNo.length();
        final int midLength = length - 8;
        final StringBuilder buf = new StringBuilder();

        buf.append(bankCardNo, 0, 4);
        for (int i = 0; i < midLength; ++i) {
            if (i % 4 == 0) {
                buf.append(CharUtil.SPACE);
            }
            buf.append('*');
        }
        buf.append(CharUtil.SPACE).append(bankCardNo, length - 4, length);
        return buf.toString();
    }
}

In fact, in this step, desensitization can be completed through replacement. You can call this tool directly in the program. But as a programmer who knows how to be lazy, he is certainly not satisfied with this. So we will further package

2. Custom desensitization annotation

@Target(ElementType.FIELD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface Sensitive {

    DesensitizedType strategy() default DesensitizedType.NONE;

    /**
     * Whether to use dfa algorithm
     * @return
     */
    boolean useDFA() default false;

    /**
     * dfa Replace sensitive characters with "*" by default
     * @return
     */
    String dfaReplaceChar() default "*";


    /**
     * dfa Number of sensitive character replacements
     * @return
     */
    int dfaReplaceCharRepeatCount() default 1;

}

3. Use some framework features to improve efficiency

a. If your project already has mybatis, you can take advantage of the mybatis interceptor feature. The implementation principle is to intercept the results returned from the response, and then desensitize the results

@Intercepts(@Signature(type = ResultSetHandler.class,method = "handleResultSets",args = Statement.class))
public class DesensitizedInterceptor implements Interceptor {
    @Override
    public Object intercept(Invocation invocation) throws Throwable {
        List<Object> list = (List<Object>) invocation.proceed();
        list.forEach(EntityUtils::desensitized);

        return list;
    }

}

b. If the project is a springboot based web project, you can use the jackson custom serialization implementation that comes with springboot. Its implementation is actually desensitization when json is serialized and rendered to the front end.

If this is the case, the user-defined annotation needs to be modified and added

@JacksonAnnotationsInside
@JsonSerialize(using = DesensitizedJsonSerializer.class)

Notes. The shape is as follows

@Target(ElementType.FIELD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@JacksonAnnotationsInside
@JsonSerialize(using = DesensitizedJsonSerializer.class)
public @interface Sensitive {

    DesensitizedType strategy() default DesensitizedType.NONE;

    /**
     * Whether to use dfa algorithm
     * @return
     */
    boolean useDFA() default false;

    /**
     * dfa Replace sensitive characters with "*" by default
     * @return
     */
    String dfaReplaceChar() default "*";


    /**
     * dfa Number of sensitive character replacements
     * @return
     */
    int dfaReplaceCharRepeatCount() default 1;

}

The core code of serialization desensitization logic is as follows

public class DesensitizedJsonSerializer extends JsonSerializer<String> implements ContextualSerializer {

    private Sensitive sensitive;
    @Override
    public void serialize(String s, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
        jsonGenerator.writeString(EntityUtils.getDesensitizedValue(sensitive,s));

    }

    @Override
    public JsonSerializer<?> createContextual(SerializerProvider serializerProvider, BeanProperty beanProperty) throws JsonMappingException {

        sensitive = beanProperty.getAnnotation(Sensitive.class);

        if(!ObjectUtils.isEmpty(sensitive) && String.class.isAssignableFrom(beanProperty.getType().getRawClass())){
            return this;
        }
        return serializerProvider.findValueSerializer(beanProperty.getType(),beanProperty);
    }
}

Example

Take json as an example

1. When defining entity objects, desensitization annotations are added to the desensitized attributes

@Data
@EqualsAndHashCode(callSuper = false)
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class UserDTO {

    private Integer id;

    private String username;

    @Sensitive(strategy = DesensitizedType.PASSWORD)
    private String password;

    @Sensitive(strategy = DesensitizedType.CHINESE_NAME)
    private String fullname;

    @Sensitive(strategy = DesensitizedType.MOBILE_PHONE)
    private String mobile;

    @Sensitive(strategy = DesensitizedType.EMAIL)
    private String email;

    @Sensitive(useDFA = true,dfaReplaceChar = "#",dfaReplaceCharRepeatCount = 3)
    private String remark;
}

2. Write a test controller

@RestController
@RequestMapping("/user")
public class UserController {

    @Autowired
    private UserService userService;


    @GetMapping(value="/list")
    public AjaxResult listUsers(){
       return AjaxResult.success(userService.listUserDTO());
    }

}

test result

Desensitization has been carried out as shown in the figure

Other programmes

1. Data desensitization based on Sharding Sphere

The specific implementation can refer to the following articles

https://jaskey.github.io/blog/2020/03/18/sharding-sphere-data-desensitization/

2. Custom annotation formatting

The main implementation steps are as follows

• 1. Implements the AnnotationFormatterFactory interface
• 2. Create a desensitization formatting class to implement Formatter
• 3. Register the interface implemented by AnnotationFormatterFactory with FormatterRegistry

The specific implementation can refer to the following articles

https://blog.csdn.net/qq_27081015/article/details/103295983

4. Desensitization with fastjson

The main implementation steps are as follows

• 1. Implement the ValueFilter interface and desensitize in process
• 2. Configure fastjson as the default JSON transformation

/**
     * Configure fastjson as the default JSON transformation
     *
     * @return
     */
    @Bean
    public HttpMessageConverters fastJsonHttpMessageConverters() {
        // 1. Define an object that converts messages
        FastJsonHttpMessageConverter fastConverter = new FastJsonHttpMessageConverter();
        // 2. Add the configuration information of fastjson, such as whether to format the returned json data
        FastJsonConfig fastJsonConfig = new FastJsonConfig();
        fastJsonConfig.setSerializerFeatures(SerializerFeature.PrettyFormat);
        fastJsonConfig.setSerializeFilters(new ValueDesensitizeFilter());//Add your own interceptor
        // 3. Add configuration information to the converter
        fastConverter.setFastJsonConfig(fastJsonConfig);
        // 4. Assign converter to HttpMessageConverter
        HttpMessageConverter<?> converter = fastConverter;
        // 5. Return HttpMessageConverters object
        return new HttpMessageConverters(converter);
    }

The specific implementation can refer to the following articles

https://blog.csdn.net/qq_27081015/article/details/103297316

5. Using mybatis mate

For the mybatis plus enterprise (data processing) module, configure the authorization code when using it. It is as follows:

mybatis-mate:
  cert:
    grant: jinTianYiXueKe
    license: GKXP9r4MCJhGID/DTGigcBcLmZjb1YZGjE4GXaAoxbtGsPC20sxpEtiUr2F7Nb1ANTUekvF6Syo6DzraA4M4oacwoLVTglzfvaEyUogW8L7mydqlsZ4+hlm20kK85eLJK1QsskrSJmreMnEaNh9lsV7Lpbxy9JeGCeM0HPEbRvq8Y+8dUt5bQYLklsa3ZIBexir+4XykZY15uqn1pYIp4pEK0+aINTa57xjJNoWuBIqm7BdFIb4l1TAcPYMTsMXhF5hfMmKD2h391HxWTshJ6jbt4YqdKD167AgeoM+B+DE1jxlLjcpskY+kFs9piOS7RCcmKBBUOgX2BD/JxhR2gQ==

Its implementation mechanism is to use json serialization. If you are interested, please refer to the following link

https://gitee.com/baomidou/mybatis-mate-examples

The demo of this article also realizes desensitization based on mybatis mate. The link is as follows
https://github.com/lyb-geek/springboot-learning/tree/master/springboot-desensitization/springboot-desensitzation-mybatis-mate

summary

Sometimes business scenarios can be implemented in a variety of ways. You should know how to judge whether to choose or not. For example, if your project does not use mybatis, but introduces mybatis in order to desensitize, this scheme will add additional complexity, and later maintenance estimation will have to be tossed

demo link

https://github.com/lyb-geek/springboot-learning/tree/master/springboot-desensitization