Accuracy improvement of license plate recognition on thai license plate_v2

Prévia do material em texto

Accuracy Improvement of
A Province Name Recognition on
Thai License Plate
Somkheart Kraisin
Information and Communication Technology for Embedded Systems
Sirindhorn International Institute of Technology
Email: m592204040646@studentmail.siit.tu.ac.th
Natsuda Kaothanthong
School of Management Technology
Sirindhorn International Institute of Technology
Email: natsuda@siit.tu.ac.th
Abstract—License plate recognition is mechanism for extract-
ing information on a license plate and converting to an encoded
text. The information on license plate is letters, number, and
province’s name. Although, the license plate recognition for Thai
has been introduced for more than ten years, it has a low accuracy
rate in the province part. This is due to a low resolution of the
province, which has approximately 15-18 pixels of the height.
This research aims to improve the accuracy of the province name
recognition on Thai license plate. The experiments are conducted
on images that were captured from the South of Thailand. A
two step classification is proposed to improve accuracy. Given
an image of a province’s part, it is classified according to the
length of the name. Then, the HOG feature is extracted for
predicting the province’s name using a classifier. The proposed
method achieve 90% accuracy, which is higher than classifying
using only Extreme Learning Machine and Decision Tree.
Index Terms—OCR, Thai License Plate Recognition, HOG,
Extreme Learning Machine
I. INTRODUCTION
Thai License Plate Recognition is a technique that is used
to convert information on image to encoded text [1] [8]
[10] [7] [2] [14] [15]. Information on license plate can be
used to identify car ’s owner. It has been utilized in many
applications such as car park system, vehicle detection system,
car monitoring etc. In the South of Thailand, police have been
used license plate recognition software to preventing the car
bomb.
Many researches were proposed for recognizing the license
plate of Japan [8] [10], Taiwan [7], Brazil [2], China [14],
United Kingdom [15]. However, such methods cannot be
implemented in Thailand because of the different layout and
the language.
Accuracy rate of license plate recognition in Thailand is
limited due to the similarity of Thai characters. Although
many researches were proposed to recognize Thai characters
[1] [12], it cannot uniquely recognized characters as shown
in Fig 1. Moreover, the characters’ height of the province is
approximately 15-18 pixels, which is very small as compared
to the size of the license plate. Therefore, the accuracy of
the province’s name recognition of the current method [1] is
limited.
Fig. 1: Pair of Similar Characters in Thai language
To recognize each characters on province portion in Thai
license plate is practical due to the low resolution of the
province’s part. R. Juntanasub et.al. [11] proposed a method
for recognizing the province’s name using Hausdorff Distance
to compare the similarity of the two names. Although, it
achieved an accuracy of 92%, the recognition time is high. S.
Subhadhira et.al. [1] applied a HOG feature and an Extreme
Learning Machine for a recognition. It has been reported
that the accuracy is 95.05%. However, the accuracy drops to
60.83% when utilizing with the images of a real situation.
In this research, a method to improve an accuracy for
recognizing the province’s name of images captured form the
real situation is proposed. The overview of the method is
depicted in 2. Given a set of images of the province’s part,
it is classified using the length of the image into N groups.
See, S1, S2, and S3 as in Fig.2. For each group, a classifier
is built for recognizing the province’s name.
The paper is organized as follow: the previous work and the
preliminary are presented in Section II and III, respectively.
The proposed method can be found in Section IV. The exper-
imental result and the conclusion are discussed in Section V
and VI, respectively.
II. PREVIOUS WORK
Methods were proposed for recognizing license plates of
different countries: Japan [8] [10], Taiwan [7], Brazil [2],
China [14], and United Kingdom [15]. C. Lin et.al. [7]
proposed a technique that applied a SVM and HOG to detect
the license plate in an image. A Convolution Neural Networks
was used to recognize English characters and Arabic numbers
Images of Province Part
Length Classification
S1 S2 S3
Image
Classification
Image
Classification
Image
Classification
Output
Fig. 2: Overview of the proposed method.
on license plate in Taiwan. There was a misclassification of
the similar characters such as “B” and “8”, “U” and “0”. K.
Miyamoto et.al. proposed a Japan license plate recognition
using an image analysis [10]. Template matching is applied to
recognize the segmented characters on the license plate.
S. Subhadhira et.al. [1] proposed a license plate recognition
for Thai using an Extreme Learning Machine. Given an input
image of a Thai license plate, it is segmented into lower
and upper part. The upper part is divided into two sub-
parts: a series of letters and numbers. The lower part contains
a province name. Each part is binarized using an image
processing tools. Then, the feature of each part of the license
plate is extracted using Histogram of Gradient (HOG) [4] [13].
The classifier called Extreme Learning Machine that is applied
to recognize each part, i.e. a series of letters, numbers, and a
province name. The accuracy rate of recognition on province
name in research is 95.05%. P. Sa-ngamuang [10] proposed
technique to recognize Thai characters on a license plate by
using an Essential-Elements-Based technique. R. Juntanasub
and N. Sureerattanan [11] divided an image into 5 × 5
blocks. The intensity of black color in each block is used
as a representation. Hausdorff Distance Technique to measure
similarity of the two images.
III. PRELIMINARY STUDY
A. Thai License Plate Recognition
Structure of Thai license plate is composed of three parts:
letter series, number, and province name as shown in Fig 3.
The combination of the three parts is used to represent a unique
car in Thailand. In other words, two cars can have the same
combination of the letter series and the number. However, the
two cars can be differentiate using the province’s name. In this
way, the accuracy of the province’s name recognition on the
license plate plays an important role.
B. License Plate Dataset
The dataset of the license plate images was from the car
check point at Songkla province. Only the images that were
taken during the day time were used in this work. Each license
plate is manually cropped from image as shown in Fig 4. The
segmented image is binarized and rotated before segmenting
province part by using the following method.
Fig. 3: Thai License Plate Structure
Fig. 4: Dataset Preparation
To segment upper and lower part from a binarized image
as shown in Fig 5, a vertical and a horizontal projection are
applied as shown in Fig 6 and Fig 7, respectively.
1) Vertical Projection: It is a technique that is used to
separate border frame of license plate in vertical by using
intensity that can be calculated as follow equation.
Pv(j) =
j=H∑
j=0
f(xj , yj) (1)
where f(xj , yj) represents intensity, H represents height of
image.
2) Horizontal Projection: To separate border in horizontal
direction, horizontal projection is applied by using equation
below.
Ph(j) =
j=w∑
j=0
f(xj , yj) (2)
where f(xj , yj) is intensity, w represents width of image.
3) Border position: Let PV be intensity in vertical projec-
tion, border position of image can be calculated by follow
equation.
dv[i] = Pv(i+ 1)− Pv(i) (3)
Maximum and minimum of dv represent border position of
license plate image.
dh[i] = Ph(i+ 1)− Ph(i) (4)
Let ma represents maximum and mi represents minimum.
dv[ma] and dv[mi] are position border in image.
Fig. 5: Binary License Plate Image
Fig. 6: Vertical and Horizontal Projection
Fig. 7: Province Segmentation
C. Histogram of Gradients
Histogram of Gradients isa feature that was proposed
for scene recognition. It is used to represent importance
information of images and was used in character recognition
[12] [16] [17].To compute HOG. It can be divided into 5 steps
as follows.
1) Each image is resized to 160 × 32 pixels and converted
to gray-scale.
2) Images are computed gradients in X and Y direction.
3) Sliding window size 2 × 2 from left to right and top
to bottom of image, to move window it is moved 1
cell/step, it is overlap with last cell 50%, each cell has 8
× 8 blocks.Each cell is computed Histogram of gradients
into 9 bins.
4) Divide each value by k
k =
√√√√i=N∑
i=1
Hv(i) + eps (5)
where eps=1.5× 10−10, Hv represents HOG vector, N
represents length of vector.
5) Create final vector by changing 2-D vector to 1-D vector.
D. Classifier
Extreme Learning Machine (ELM) is a feed forward net-
works. ELM consists of an input layer, a hidden layer, and an
output layers.
Decision tree is a tree-like structure algorithm that is widely
used in Machine Learning. Each node of decision tree is used
to represent attribute, each branch represents outcome of the
nodes and each leaf node is used to represent class label.
IV. PROPOSED METHOD
From our preliminary experiment using HOG feature and
ELM classifer, we found a misclassification of Bangkok
and Pattalung. From our observation, Bangkok and Pattalung
province have different length. In this way, a length is used to
filter the province name before classifying the province name.
For an illustration see Fig. 2.
A. Length Classification
1) Computing Image’s Length: Given an binarized im-
age of the province part on a license plate, which was
segmented using a horizontal and a vertical projection. Let
L = {L1, . . . , Ln} be the length of the n images in the dataset
where Li for i = {1, . . . , n}. The length Li of an image i can
be computed as follow:
Li =
x=end−1∑
x=start
Gi(x, y) (6)
where Gi(x, y) =
{
1 if x < end, (y = 0)
0 otherwise
start represents start position of province’s characters, end
represents last position of province’s characters on province
portion. Gi represents province image in training data.
2) Filtering image’s length: Given the length L of the n
images, they are grouped into b groups using a histogram.
In this work, b = {3, 4} are used to group the images of
the province part. Let I be a length interval, which can be
computed as follow:
I =
Lmax − Lmin
b
, (7)
where Lmax is the maximum length and Lmin is the minimum
length of the images.
Given an interval I and b, the lower value of an interval of
each group is computed using the following equation:
Ijmin = Lmin × j where
{
j = 1..b
}
(8)
The upper value of an interval of each group is computed
using the following equation:
Ijmax = I
j
min + I where
{
j = 1..b
}
(9)
where Ijmin is the lower value and I
j
max is the upper value of
the group j for j = {1..b
}
.
To assign each image into a group Kj for j = {1..b
}
, each
image Gi in the dataset is assigned to the group Kj if the
length is between [Ijmin, I
j
max).
B. Province Name Recognition
To train model, each image in a group Kj for j = {1..b
}
are resize to 160 × 32 pixels. A HOG feature is extracted
to represent an image. Let Hi represents HOG of an image
Gi, Hi is used to train a recognition model. In this work,
an Extreme Learning Machine (ELM) and Decision Tree are
used.
Images of Province Part
Length Classification
S1 S2 S3
Image
Classification
Image
Classification
Image
Classification
Output
Fig. 8: Overview of the province’s name recognition.
C. Recognize Province Name
An overview of the proposed province name recognition is
shown in Fig. 8. Given an testing image of the province part
T , it is classified by its length. If the length of T falls between
the interval [Ijmin, I
j
max), the image T is in the group j. From
the overview in Fig. 8, the length of the image T is in group
2. Then, T is resized to 160 × 32 pixels before extracting a
HOG feature, denoted by HT . Given a feature HT and the
group j, the province name recognition model of the group j
is used to recognize the province’s name.
V. EXPERIMENT
In this experiment, the image dataset is collected from real
situation, which were taken in the South of Thailand. The
dataset composes of 5 provinces (classes). For each class, there
are 69 images where 45 are used for training and the rest is
for testing. Examples of the images in each class is shown in
TableI. From the examples, the length of ‘Bangkok’ class is
the longest while ‘Pattalung’ is the shortest.
TABLE I. Example of images in each class.
Bangkok Narathivat Pattalung Pattani Songkla
A. Experimental Setting
To obtain the number of group for classifying the images’
length, an experiment is conducted in order to show the class
distribution. The distribution of the province images in each
group for b = 3 and b = 4 are shown in Table III and Table
V, respectively. Examples of images in each bin using b = 3
and b = 4 are shown in Table IV and Table VI, respectively.
It can be seen from the result in Table V that b = 4 gives
better distribution since the majority of each group falls in
TABLE II. Example of testing images in each class.
Bangkok Narathivat Pattalung Pattani Songkla
different group. On the other hand, for b = 3 the majority of
the provinces which have similar length are in the first bin.
TABLE III. The distribution of the length of the province
images using 3 groups.
Group Bangkok Narathiwat Pattalung Pattani Songkla
1st 3 36 45 44 45
2nd 41 7 0 1 0
3rd 1 2 0 0 0
TABLE IV. Example of images when using b = 3.
Group Example of Images
1st Bin
2nd Bin
3rd Bin
TABLE V. The distribution of the length of the province
images using 4 groups.
Group Bangkok Narathiwat Pattalung Pattani Songkla
1st 0 0 35 17 36
2nd 0 13 9 22 9
3rd 7 27 1 5 0
4th 38 5 0 1 0
TABLE VI. Example of images in each bin when using b=4.
Group Example of Images
1stBin
2nd Bin
3rd Bin
4th Bin
B. Experimental Result
The experiments are conducted using two classification: 1)
without applying the length classification and 2) the one with
the length classification. For the experiments using the length
classification, they are conducted using two different size of
b, i.e., b = {3, 4}.
The summary of the experimental result is shown in Table
VII. It can be seen from the result that the length classification
can improve the classification accuracy. Comparing the size of
TABLE VII. Accuracy Comparison of Recognition Method
Classification Method Accuracy
(%)
Extreme Learning Machine without length classification 60.83
Decision Tree without length classification 60.83
Extreme Learning Machine and b = 3 histogram 83.33
Decision Tree and b = 3 histogram 66.67
Extreme Learning Machine and b = 4 histogram 90.00
Decision Tree and b = 4 histogram 70.00
TABLE VIII. Confusion Matrix of Decision Tree without
length classification.
Class Bangkok Narathiwat Pattalung Pattani Songkla Accuracy
Bangkok 13 0 1 9 1 54.17%
Narathivat 1 14 2 3 4 58.33%
Pattalung 1 1 18 3 1 75.00%
Pattani 0 5 4 13 2 54.17%
Songkla 1 3 4 2 14 58.33%
Average accuracy 60.83%
the bins, using b = 4 achieves a better result. The detail of the
experimental result can be found in the following subsection.
1) Without Length Classification: In this experiment, the
province’s images are classified using the two methods, i.e.,
Decision Tree and Extreme Learning Machine. Results can be
found in Table VIII and IX, respectively.
From the recognition result, there are misclassified im-
ages. For example, Songkla is recognized as Narathivat and
Bangkok is recognized as Pattani. Regarding to the examples
in Table I, the length of Bangkok is much longer than Pattani.
2) With Length Classification: To improve the classification
as shown in Table VIII and Table IX, the images are classified
according to their length using a histogram.
The classification result using b = 3 histogram with the
Decision Tree and the Extreme Learning Machine are shown in
Table X and TableXI, respectively. It can be seen that accuracy
of the classification using the Extreme Learning Machine is
improved from 60.83% to 83.33%. On the other hand, there
are a number of misclassified of Songkla when using the
Decision Tree.
Table XII and Table XIII shows the classification result
using b = 4 histogram. The accuracy of the Decision Tree
classifier improves from 60.83% to 70% when using b = 4
bins. The misclassification classes from ‘Bangkok’ to ‘Pattani’
is improved from 9 to 1. See, Table X and Table XII for
comparison.
The accuracy using Extreme Learning Machine is 90%,
which is better than the Decision Tree. The classification result
TABLE IX. Confusion Matrix of Extreme Learning Machine
without length classification.
Class Bangkok Narathiwat Pattalung Pattani Songkla Accuracy
Bangkok 12 3 4 4 1 50.00%
Narathiwat 2 18 1 2 1 75.00%
Pattalung 0 2 20 1 1 83.33%
Pattani 2 3 4 12 3 50.00%
Songkla 3 6 3 2 11 45.83%
Average accuracy 60.83%
TABLE X. Confusion Matrix of Decision Tree using b = 3
bins.
Class Bangkok Narathiwat Pattalung Pattani Songkla Accuracy
Bangkok 20 3 0 1 0 83.33%
Narathivat 0 14 3 3 4 58.33%
Pattalung 0 0 19 3 2 79.17%
Pattani 0 0 4 11 9 45.83%
Songkla 0 3 2 3 16 66.67%
Average accuracy 66.67%
TABLE XI. Confusion Matrix of Extreme Learning Machine
using b = 3 bins.
Class Bangkok Narathiwat Pattalung Pattani Songkla Accuracy
Bangkok 24 0 0 0 0 100.00%
Narathivat 0 21 0 1 2 87.50%
Pattalung 0 2 20 2 0 83.33%
Pattani 0 3 4 17 0 70.83%
Songkla 0 4 2 0 18 75.00%
Average accuracy 83.33%
for each class is better that using b = 3 histogram. The
result of ‘Songkla’ province is similar to the one from b = 3
histogram. This is because its length was classified in the same
bins as ‘Pattalung’ and ‘Pattani’. Comparing the result between
b = 3 and b = 4, the one using b = 4 gives a better result.
VI. CONCLUSION
Thai License Plate recognition still has low accuracy to
recognize province name, the main reason is province portion
has low resolution. This research aims to improve accuracy
rate by using two step classification. First step, the image is
classified by filtering image ’s length, second step, image is
classified by using classifier. From the experiment, classifying
image into N bins before recognizing can improve accuracy
rate. In this research, Recognition province name by using
Extreme Learning Machine obtains accuracy at 90%.
TABLE XII. Confusion Matrix of Decision Tree using b = 4
bins.
Class Bangkok Narathiwat Pattalung Pattani Songkla Accuracy
Bangkok 19 4 0 1 0 79.17%
Narathivat 2 20 0 2 0 83.33%
Pattalung 0 2 14 3 5 58.33%
Pattani 0 5 2 14 3 58.33%
Songkla 0 2 2 3 17 70.83%
Average accuracy 70%
TABLE XIII. Confusion Matrix of Extreme Learning Machine
using b = 4 bin.
Class Bangkok Narathiwat Pattalung Pattani Songkla Accuracy
Bangkok 24 0 0 0 0 100.00%
Narathivat 0 22 0 2 0 91.67%
Pattalung 0 2 22 0 0 91.67%
Pattani 0 0 1 22 1 91.67%
Songkla 0 0 3 3 18 75.00%
Average accuracy 90%
ACKNOWLEDGEMENT
This work was supported by the SIIT Young Researcher
Grant, under contract no. SIIT 2017-YRG-NK04. The authors
would like to thank the anonymous reviewers for their valuable
comments and suggestions to improve the quality of the paper.
REFERENCES
[1] S. Subhadhira U. Juithonglang ,“License Plate Recognition Application
Using Extreme Learning Machines ”,2014 Third ICT International
Student Project Conference (ICT-ISPC),2014,p.103-106.
[2] F. Delmar Kurpiel and R. Minetto and B. T. Nassu,”Convolutional neural
networks for license plate detection in images”,2017 IEEE International
Conference on Image Processing (ICIP),2017,pp.3395-3399.
[3] Y. Hou and Z. Ye and W. Xu and L. Ma, “Remote sensing textual
image classification based on extreme learning machine and hybrid
rice optimization algorithm”,2017 9th IEEE International Conference
on Intelligent Data Acquisition and Advanced Computing Systems:
Technology and Applications (IDAACS),2017,p.777-781.
[4] S. Routray and A. K. Ray and C. Mishra, “Analysis of various image
feature extraction methods against noisy image: SIFT, SURF and HOG”,
2017 Second International Conference on Electrical, Computer and
Communication Technologies (ICECCT),2017,p.1-5.
[5] W. Kusakunniran, K. Ngamaschariyakul, C. Chantaraviwat, K. Janvit-
tayanuchit and K. Thongkanchorn, “A Thai license plate localization
using SVM,” 2014 International Computer Science and Engineering
Conference (ICSEC), Khon Kaen, 2014, pp.163-167.
[6] David C. Lambert, Extreme Learning Machine implementation in
Python, (2013), GitHub repository, https://github.com/dclambert/Python-
ELM
[7] C. H. Lin and Y. S. Lin and W. C. Liu , “An efficient license plate
recognition system using convolution neural networks”,018 IEEE Inter-
national Conference on Applied System Invention (ICASI),2018,pp.224-
227.
[8] K. Miyamoto and K. Nagano and M. Tamagawa and I. Fujita and M. Ya-
mamoto, “Vehicle license-plate recognition by image analysis”,Industrial
Electronics, Control and Instrumentation, 1991. Proceedings. IECON
’91, 1991 International Conference,1991,pp.1734-1738 vol.3.
[9] P. Sa-ngamuang and C. Thamnittasana and T. Kondo,“Thai Car License
Plate Recognition Using Essential-Elements-Based Method”,2007 Asia-
Pacific Conference on Communications,2007,pp.41-44.
[10] K. Taniyama and K. Hayashi, “Robust car License Plate Recognition
system verified with 163, 574 images captured in fields”,Proceedings
of the 21st International Conference on Pattern Recognition
(ICPR2012),2012,pp.1273-1276.
[11] R. Juntanasub and N. Sureerattanan, “Car license plate recognition
through Hausdorff distance technique”,17th IEEE International Confer-
ence on Tools with Artificial Intelligence (ICTAI’05),2005,pp.5-9.
[12] T. Siriteerakul,“Mixed Thai-English Character Classification Based on
Histogram of Oriented Gradient Feature”,2013 12th International Con-
ference on Document Analysis and Recognition,2013,pp.847-851.
[13] T. Surasak and I. Takahiro and C. h. Cheng and C. e. Wang and
P. y. Sheng,“Histogram of oriented gradients for human detection in
video”,2018 5th International Conference on Business and Industrial
Research (ICBIR),2018,pp.172-176.
[14] Q. Wang, “License plate recognition via convolutional neural net-
works”,2017 8th IEEE International Conference on Software Engineer-
ing and Service Science (ICSESS),2017,pp.926-929.
[15] X. Zhai and F. Bensaali and S. Ramalingam, “Real-time license plate
localisation on FPGA”,CVPR 2011 WORKSHOPS,2011,pp.14-19.
[16] S. Iamsa-at and P. Horata, “Handwritten Character Recognition Using
Histograms of Oriented Gradient Features in Deep Learning of Artificial
Neural Network”,2013 International Conference on IT Convergence and
Security (ICITCS),2013,pp.1-5.
[17] A. Tikader and N. B. Puhan, “Histogram of oriented gradients for
English-Bengali script recognition”,International Conference for Con-
vergence for Technology-2014,2014,pp.1-5.