1, Introduction to license plate recognition
License plate recognition technology originated in the early 1980s. Image pattern recognition and computer vision play an important role in it. With the rapid popularization and progress of computer technology, the license plate recognition system was gradually put into the market and used in the middle and late 1980s, but the recognition accuracy and speed are not ideal. At present, license plate recognition systems in developed countries have been widely used in the market, among which the license plate recognition systems of Israeli and Singapore companies are more advanced. Due to different license plate designs, there is no general license plate detection technology. Because Chinese license plates include Chinese characters, foreign license plate recognition systems can not be directly used for domestic license plate recognition, which needs independent research and development in China. The research on license plate recognition technology in China started slightly later than that in western countries. At present, the leading domestic technology is the "Hanwang eye" of the Chinese Academy of Sciences and the visual technology company in Hong Kong. However, the use of these license plate recognition systems put into the market has certain limitations. When the license plate image quality is poor, the license plate tilt, the lighting conditions are not ideal, the recognition rate will be greatly reduced.
In order to overcome the above problems, the license plate images obtained under different lighting conditions are recognized and analyzed in this paper. Because the license plate involves confidential information, there is no public large-scale data set for use and experimental comparison. Firstly, the license plate data is collected and processed uniformly × 240 resolution pictures, establish the test database. The database is divided into two categories: test library 1 is a license plate with good lighting conditions and no shadow; Test library 2 is a shaded license plate with poor lighting conditions. The traditional template matching algorithm is used to establish the license plate recognition system, and the template images are averaged by the median filter algorithm, with a unified resolution of 20 × 40 identification template library. For the convenience of users, a graphical user interaction interface (GUI) is established by using Matlab. Through the license plate experiment in the test library, we can see that the license plate imaging quality affects the recognition rate
Has a great impact. Therefore, when establishing the license plate recognition system, the light supplement at an appropriate angle can effectively improve the recognition accuracy.
1 image preprocessing
Because most license plates are obtained through traffic monitoring and other conditions, due to the surrounding traffic environment, shooting angle, illumination and real-time requirements, the photos obtained are directly used for license plate recognition, and the accuracy is difficult to be guaranteed. Therefore, preprocessing the image can improve the accuracy of recognition.
1.1 image graying
The license plate captured by the camera is usually a color image, that is, RGB image. In order to improve the speed of the license plate recognition system and save memory, the RGB image is first converted into a gray image The three basic colors of red, blue and green can form any color according to different proportions, so each color of the license plate photo can be composed of red, blue and green in different proportions. Grayscale images can be represented by array I, and the data types of array I generally include integer and double precision. Usually 0 represents black and 255 represents white. A conversion method suitable for human visual system is adopted, as shown in formula (1).
Among them, the coefficients a, B and c should be greater than 0, and a+b+c=1, Y represents the gray value, R, G and B represent red, green and blue respectively, a=0.299, b=0.584 and c=0.117 Figure 1 shows the original license plate image with the license plate photo, and Figure 2 shows the gray image of the license plate image after the gray processing of formula (1).
Figure 1 original license plate
Figure 2 gray scale of license plate
1.2 image binarization
Image binarization can not only greatly reduce the amount of data, but also highlight the target contour of the image, which is conducive to the subsequent image positioning and segmentation. In the binarization process of the license plate image, the pixel determined as the target area has a gray value greater than or equal to the threshold, and the calculation is shown in formula (2). After testing, when the threshold th=0.76, the effect is good, and the results are shown in Figure 3.
Fig. 3 binarization diagram of license plate
1.3 image edge detection
In this paper, Roberts operator is used for edge detection. The operator has the advantages of small amount of calculation and fast speed, which is convenient for subsequent real-time processing. Its template is shown in Table 1. If the gradient amplitude G(x, y) is greater than the set threshold, it is judged as an edge. The calculation formula of G(x, y) is shown in formula (3), where the threshold value is 0.15.
Table 1 Roberts operator template
Where f(x, y) is the gray value at the image space (x, y), and G(x, y) is the gradient amplitude of f(x, y). Fig. 4 is a result of edge detection.
Figure 4 edge detection of license plate
2 design of license plate recognition system
2.1 license plate location
The license plate location adopts color feature extraction, and the general license plate area has obvious characteristics. Most of the license plates in China are white characters on a blue background. The traditional method generally divides the reasonable license plate area according to the color characteristics of the license plate and the method of color pixel statistics. It is assumed that the RGB image containing the license plate is collected by the camera, and the horizontal direction is marked as y and the vertical direction is marked as x. Firstly, the color range corresponding to each component of the license plate is determined. Secondly, the number of pixels corresponding to the horizontal direction and the reasonable area of the license plate are calculated. Then, the number of white pixels in the color range in the vertical direction in the segmented horizontal direction area is calculated, and a reasonable threshold is set for positioning. Finally, the license plate area is determined according to the range of the corresponding direction. However, the accuracy of this method is low. In this paper, the color feature extraction and binary classification method proposed in literature [6] is used to improve the algorithm. The calculation method is shown in formula (4). Figure 5 shows the license plate after positioning.
The blue feature can be expressed by equation (4):
Figure 5 license plate location
2.2 license plate character recognition
Before character recognition, the license plate characters are segmented. Character segmentation adopts threshold segmentation, which mainly includes two steps [7].
(1) Determine the threshold for segmentation.
(2) The threshold is compared with the gray value of each point to achieve the purpose of segmentation.
Normalizing the segmented image can effectively scale the size of the character image to obtain a character image with the same size, which is convenient for subsequent character recognition.
Template matching is generally one of the most commonly used recognition methods in digital image processing. Firstly, a template library is established, and then the characters are input into the template to find the best matching template characters.
The template image size is 20 × 40. The template library is composed of numbers 0-9, abbreviations of 32 provinces, Chinese characters and capitalized English letters (O is generally not used as license plate letters). Some images of the template are shown in Figure 6.
Figure 6 some pictures of template library
2, Partial source code
function varargout = gui(varargin)
gui_Singleton = 1;
gui_State = struct('gui_Name', mfilename, ...
'gui_Singleton', gui_Singleton, ...
'gui_OpeningFcn', @gui_OpeningFcn, ...
'gui_OutputFcn', @gui_OutputFcn, ...
'gui_LayoutFcn', [] , ...
'gui_Callback', []);
if nargin && ischar(varargin{1})
gui_State.gui_Callback = str2func(varargin{1});
end
if nargout
[varargout{1:nargout}] = gui_mainfcn(gui_State, varargin{:});
else
gui_mainfcn(gui_State, varargin{:});
end
%End initialization
function gui_OpeningFcn(hObject, ~, handles, varargin)
handles.output = hObject;
guidata(hObject, handles);
function varargout = gui_OutputFcn(~, ~, handles)
varargout{1} = handles.output;
% ======================input image ===============================
function pushbutton1_Callback(hObject, ~, handles)
[filename, pathname]=uigetfile({'*.jpg';'*.bmp'}, 'File Selector');
I=imread([pathname '\' filename]);
handles.I=I;
guidata(hObject, handles);
axes(handles.axes1);
imshow(I);title('Original drawing');
% ======================image processing===============================
function pushbutton2_Callback(~, ~, handles)
I=handles.I;
I1=rgb2gray(I);
I2=edge(I1,'roberts',0.18,'both');
axes(handles.axes2);
imshow(I1);title('Grayscale image');
axes(handles.axes3);
imshow(I2);title('edge detection ');
se=[1;1;1];
I3=imerode(I2,se);%Corrosion operation
se=strel('rectangle',[25,25]);
I4=imclose(I3,se);%Image clustering, filling image
I5=bwareaopen(I4,2000);%Remove the part with cluster gray value less than 2000
[y,x,z]=size(I5);%Returns the dimensions of 15 dimensions, stored in x,y,z in
myI=double(I5);
tic %tic The timing starts, toc end
Blue_y=zeros(y,1);%Generate a y*1 Zero needle
for i=1:y
for j=1:x
if(myI(i,j,1)==1)%If myI The image coordinates are( i，j)The point value is 1, that is, the background color is blue, blue add one-tenth
Blue_y(i,1)=Blue_y(i,1)+1;%Blue pixel statistics
end
end
end
[temp MaxY]=max(Blue_y);
%Y Direction license plate area determination
%temp As vector yellow_y The maximum value in the element of, MaxY Index the value
PY1=MaxY;
while((Blue_y(PY1,1)>=5)&&(PY1>1))
PY1=PY1-1;
end
PY2=MaxY;
while((Blue_y(PY2,1)>=5)&&(PY2<y))
PY2=PY2+1;
end
IY=I(PY1:PY2,:,:);
%X Direction license plate area determination
Blue_x=zeros(1,x);%Further confirmation x License plate area in direction
for j=1:x
for i=PY1:PY2
if(myI(i,j,1)==1)
Blue_x(1,j)=Blue_x(1,j)+1;
end
end
end
PX1=1;
while((Blue_x(1,PX1)<3)&&(PX1<x))
PX1=PX1+1;
end
PX2=x;
while((Blue_x(1,PX2)<3)&&(PX2>PX1))
PX2=PX2-1;
end
PX1=PX1-1;%Correction of license plate area
PX2=PX2+1;
dw=I(PY1:PY2-8,PX1:PX2,:);
t=toc;
axes(handles.axes4);imshow(dw),title('Locate license plate');
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
imwrite(dw,'dw.jpg');%Write color license plate to dw In the file
a=imread('dw.jpg');%Read license plate
b=rgb2gray(a);%Convert license plate image into gray image
imwrite(b,'Gray license plate.jpg');%Write grayscale image to file
g_max=double(max(max(b)));
g_min=double(min(min(b)));
T=round(g_max-(g_max-g_min)/3);%T Is the threshold of binarization
[m,n]=size(b);
d=(double(b)>=T);%d:Binary image
imwrite(d,'Binarization.jpg');
%Before mean filtering
%wave filtering
h=fspecial('average',3);
%Establish a predefined filter operator, average It is mean filtering, and the template size is 3*3
d=im2bw(round(filter2(h,d)));%Use the specified filter h yes h conduct d Mean filtering
imwrite(d,'Mean filtering.jpg');
%Operate on some images
%Expansion or corrosion
se=eye(2);%Identity matrix
[m,n]=size(d);%Return information matrix
if bwarea(d)/m/n>=0.365%Calculate whether the ratio of the total area of the object to the whole area in the binary image is greater than 0.365
d=imerode(d,se);%If greater than 0.365 Then corrosion
elseif bwarea(d)/m/n<=0.235%Calculate whether the ratio of the total area of the object to the whole area in the binary image is less than 0.235
d=imdilate(d,se);%%If less than, the expansion operation is realized
end
imwrite(d,'expand.jpg');
%Find a block with continuous text. If the length is greater than a certain threshold, it is considered that the block is composed of two characters and needs to be segmented
d=qiege(d);
[~,n]=size(d);
k1=1;
k2=1;
s=sum(d);
j=1;
while j~=n
while s(j)==0
j=j+1;
end
k1=j;
while s(j)~=0 && j<=n-1
j=j+1;
end
k2=j-1;
if k2-k1>=round(n/6.5)
[val,num]=min(sum(d(:,[k1+5:k2-5])));
d(:,k1+num+5)=0;%division
end
end
%Re cutting
d=qiege(d);
%Cut out 7 characters
y1=10;
y2=0.25;
flag=0;
word1=[];
while flag==0
[m,n]=size(d);
left=1;
wide=0;
while sum(d(:,wide+1))~=0
wide=wide+1;
end
if wide<y1 %Think it's left interference
d(:,[1:wide])=0;
d=qiege(d);
else
temp=qiege(imcrop(d,[1 1 wide m]));
[m,n]=size(temp);
all=sum(sum(temp));
two_thirds=sum(sum(temp([round(m/3):2*round(m/3)],:)));
if two_thirds/all>y2
flag=1;word1=temp;%word1
end
d(:,[1:wide])=0;d=qiege(d);
end
end
3, Operation results
4, matlab version and references
1 matlab version
2014a
2 references
[1] Cai Limei MATLAB image processing -- theory, algorithm and example analysis [M] Tsinghua University Press, 2020
[2] Yang Dan, Zhao Haibin, long Zhe Detailed explanation of MATLAB image processing example [M] Tsinghua University Press, 2013
[3] Zhou pin MATLAB image processing and graphical user interface design [M] Tsinghua University Press, 2013
[4] Liu Chenglong Proficient in MATLAB image processing [M] Tsinghua University Press, 2015
[5] Yun Runhua, Su Tingting, Ma Xiaowei License plate recognition system based on BP neural network combined template matching [J] Journal of Tsinghua University (NATURAL SCIENCE EDITION), 2013, 53 (9): 1221-1226
[6] Lu Yang Research on license plate recognition algorithm based on BP neural network [D] Daqing: Northeast Petroleum University, 2018
[7] Li Qiang, Zhang Juan An improved recognition method of stained license plate based on template matching [J] Intelligent computers and applications 2019,9(03).
[8] Liang Kai Design and implementation of vehicle license plate recognition system based on MATLAB [D] Harbin: Heilongjiang University, 2018
[9] Liu Xiongfei, Zhu Shengchun Multi feature extraction of license plate characters and recognition algorithm of BP neural network [J] Computer simulation, 2014, 31 (10): 161-164290
[10] Zeng Quan, Tan Beihai License plate recognition system based on SVM and BP neural network [J] Electronic technology, 2016, 29 (1): 98-101