2021SC@SDUSC

In this analysis, I will briefly introduce and summarize the SimpleRecognizePlate function. Since this function is mainly analyzed by other members of my team, there is no need to analyze it in detail here. The focus is to compare it with the SimpleRecognizePlateByE2E function I analyzed.

The function code is as follows:

def SimpleRecognizePlate(image):
    t0 = time.time()
    images = detect.detectPlateRough(image,image.shape[0],top_bottom_padding_rate=0.1)
    res_set = []
    for j,plate in enumerate(images):
        plate, rect, origin_plate  =plate
        # plate = cv2.cvtColor(plate, cv2.COLOR_RGB2GRAY)
        plate  =cv2.resize(plate,(136,36*2))
        t1 = time.time()

        ptype = td.SimplePredict(plate)
        if ptype>0 and ptype<5:
            plate = cv2.bitwise_not(plate)

        image_rgb = fm.findContoursAndDrawBoundingBox(plate)

        image_rgb = fv.finemappingVertical(image_rgb)
        cache.verticalMappingToFolder(image_rgb)
        print("e2e:", e2e.recognizeOne(image_rgb))
        image_gray = cv2.cvtColor(image_rgb,cv2.COLOR_RGB2GRAY)

        # image_gray = horizontalSegmentation(image_gray)
        cv2.imshow("image_gray",image_gray)
        # cv2.waitKey()

        cv2.imwrite("./"+str(j)+".jpg",image_gray)
        # cv2.imshow("image",image_gray)
        # cv2.waitKey(0)
        print("correcting",time.time() - t1,"s")
        # cv2.imshow("image,",image_gray)
        # cv2.waitKey(0)
        t2 = time.time()
        val = segmentation.slidingWindowsEval(image_gray)
        # print val
        print("Segmentation and recognition",time.time() - t2,"s")
        if len(val)==3:
            blocks, res, confidence = val
            if confidence/7>0.7:
                image =  drawRectBox(image,rect,res)
                res_set.append(res)
                for i,block in enumerate(blocks):

                    block_ = cv2.resize(block,(25,25))
                    block_ = cv2.cvtColor(block_,cv2.COLOR_GRAY2BGR)
                    image[j * 25:(j * 25) + 25, i * 25:(i * 25) + 25] = block_
                    if image[j*25:(j*25)+25,i*25:(i*25)+25].shape == block_.shape:
                        pass


            if confidence>0:
                print("License plate:",res,"Confidence:",confidence/7)
            else:
                pass

                # print "uncertain license plate:", res, "confidence:", confidence

    print(time.time() - t0,"s")
    return image,res_set

1, Overall process

From the sub functions called by this function, we can see that the general process of this function is the same as that of the SimpleRecognizePlateByE2E function:

1. For the rough location of license plate, the detectPlateRough function is also called.

First, print out a shape value of the incoming image, that is, the size of the image, and then use an if judgment to ensure that the frame value of the image is less than 0.2 for subsequent processing. If it is greater than 0.2, an error will be returned and the next process will be stopped.

The next step is to call CV2 Resize() function. The main purpose of this function is to make some zoom settings for the picture.
The most critical step in the recognition process is to call the methods in opencv2, CascadeClassifier() and detectMultiScale(), both of which are applications in face recognition. Of course, a classifier is also loaded in it, that is, for the processed pictures, the specific position and size of the license plate are recognized and stored as an array. Finally, the specific location and size of the recognized license plate are processed on the data and returned to others

2. For fine location of license plate, the finemappingVertical function is called. Firstly, multiple parameters are used to adaptively binarize the region for many times. Then, we analyze the connected domain of each binarized image to find the contour that meets the character length width ratio, and then the contour line is fitted by straight line, so as to find the upper boundary and lower boundary. In this way, we finish the fine positioning of the upper and lower boundaries.

3. Call the slidingWindowsEval function, first use the projection method to segment the characters, obtain the projection on the basis of image filtering, and then use the obtained projection value to find the segmentation points. According to these segmentation points, segment the area of each character on the license plate, and recognize these characters according to the model.

2, Comparative analysis

It can be seen that the ideas of the two functions are roughly the same, which also explains a general process of license plate recognition:

1. Image preprocessing, such as changing into gray image, clipping, rotation and scaling, histogram equalization, etc., make it easy for license plate recognition and processing

2. Locate the approximate position of the license plate in the image. Some processing may be required after positioning

3. Locate the position of the words on the license plate. You can separate the words to determine their position one by one, or you can determine their overall position without segmentation

4. Recognition with machine learning model

The two functions are identical in the first two parts, and the biggest difference is in the third and fourth parts.

SimpleRecognizePlateByE2E adopts OCR optical character recognition technology, which is different from the traditional character recognition method, that is, first segment the characters, and then use the classification algorithm for recognition one by one. OCR optical character recognition technology can recognize seven characters of the license plate without segmenting the characters. It can be seen that there is no text segmentation in this function.

SimpleRecognizePlate uses the template matching method. The template matching method designs a fixed template and an evaluation function to measure the matching degree of the template according to the standard style of the license plate and the structure, size and spacing characteristics of the characters. Then, the template slides from left to right in the normalized image, and the corresponding evaluation value is calculated each time, Finally, the template sliding position with the highest matching degree is selected as the position of character segmentation.