Precise and efficient Chinese license plate recognition in the real monitoring scene of intelligent transportation system

doi:10.19682/j.cnki.1005-8885.2022.1014

The Journal of China Universities of Posts and Telecommunications ›› 2022, Vol. 29 ›› Issue (3): 1-14.doi: 10.19682/j.cnki.1005-8885.2022.1014

School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230009, China

Received:2022-02-28 Revised:2022-06-22 Online:2022-06-30 Published:2022-06-30
Contact: Jia Wei E-mail:jiawei@hfut.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (62076086), and the Key Research and
Development Program in Anhui Province (202004d07020008).

Abstract

Abstract: In this paper, the performance of you only look once ( YOLO) series detectors on Chinese license plate recognition (LPR) in the real intelligent transportation system (ITS) monitoring scene is investigated. Specially, a precise and efficient automatic license plate recognition ( ALPR ) system based on the YOLOv4 detector is proposed. The proposed ALPR system contains three stages including vehicle detection, license plate detection (LPD) and LPR. In vehicle detection stage, YOLOv4 detector is directly applied. In LPD stage, YOLOv4-tiny detector is exploited. In the last stage, the YOLOv4-tiny detector with attention mechanism for LPR is proposed to use. In addition, a large Chinese license plate dataset containing 10 500 images collected from all 31 provinces in the Chinese mainland is created. This Chinese license plate dataset is named Hefei University of Technology license plate version 1 (HFUT-LP v1). Particularly, HFUT-LP v1 dataset is collected in the real ITS monitoring scene. In order to compare the performance of different object detection algorithms for ALPR, a variety of object detection algorithms are used to make a comprehensive performance evaluation. Experimental results show that the proposed ALPR system achieves very high accuracy and has very fast processing speed, which is suitable for real-time LPR.

Key words: license plate detection(LPD), license plate recognition(LPR), YOLOv4-tiny detector, attention mechanism, intelligent transportation

CLC Number:

TP391.4

Jia Wei, Gong Chao. Precise and efficient Chinese license plate recognition in the real monitoring scene of intelligent transportation system[J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(3): 1-14.

References

1. SUN D D, LIU L D, ZHENG A H, et al. Visual cognition inspired vehicle re-identification via correlative sparse ranking with multi-view deep features. Advances in Brain Inspired Cognitive Systems: Proceedings of the 9th International Conference on Brain Inspired Cognitive Systems (BICS'18), 2018, Jul 7-8, Xi'an, China. LNAI 10989. Berlin, Germany: Springer, 2018: 54-63

2. LAROCA R, ZANLORENSI L A, GONÇALVES G R, et al. An efficient and layout-independent automatic license plate recognition system based on the YOLO detector. ArXiv Preprint, ArXiv:1909.01754, 2019.

3. Montazzolli S, JUNG C. Real-time brazilian license plate detection and recognition using deep convolutional neural networks. Proceedings of the 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI'17), 2017, Oct 17-20, Niteroi, Brazil. Piscataway, NJ, USA: IEEE, 2017: Aeticle 5-62.

4. SILVA S M, JUNG C. R. License plate detection and recognition in unconstrained scenarios. Computer Vision: Proceedings of the 15th European Conference on Computer Vision (ECCV'18): Part XII, 2018, Sept 8-14, Munich, Germany. LNIP 11216. Berlin, Germany: Springer, 2018: 593-609.

5. LAROCA R, SEVERO E, ZANLORENSI L A, et al. A robust real-time automatic license plate recognition based on the YOLO detector. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN'18), 2018, Jul 8-13, Rio de Janeiro, Brazil. Piscataway, NJ, USA: IEEE, 2018: 10p.

6. HAN J, YAO J, ZHAO J, et al. Multi-oriented and scale-invariant license plate detection based on convolutional neural networks. Sensors, 2019,19(5): Article 1175/1-19.

7. BULAN O, KOZITSKY V, RAMESH P, et al. Segmentation-and annotation-free license plate recognition with deep localization and failure identification. IEEE Transactions on Intelligent Transportation Systems, 2017, 18(9): 2351-2363.

8. GONCALVES G R, DINIZ M A, LAROCA R, et al. Real-time automatic license plate recognition through deep multi-task networks. Proceedings of the 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI'18), 2018, Oct 29-Nov 1, Parana, Brazil. Piscataway, NJ, USA: IEEE, 2018: 110-117.

9. GONCALVES G R, DINIZ M A, LAROCA R, et al. Multi-task learning for low-resolution license plate recognition. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: Proceedings of the 24th Iberoamerican Congress on Pattern Recognition (CIARP'19), 2019, Oct 28-31, Havana, Cuba. LNIP 11896. Berlin, Germany: Springer, 2019: 251-261.

10. LI H, WANG P, YOU M Y, et al. Reading car license plates using deep neural networks. Image and Vision Computing, 2018, 72: 14-23.

11. HSU G S, AMBIKAPATHI A, CHUNG S L, et al. Robust license plate detection in the wild. Proceedings of the 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS'17), 2017, Aug 29-Sept 1, Lecce, Italy. Piscataway, NJ, USA: IEEE, 2017: 6p.

12. XIE L L, AHMAD T, JIN L W, et al. A new CNN-based method for multi-directional car license plate detection. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(2): 507-517.

13. REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Uniﬁed, real-time object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'16), 2016, Jun 27-30, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2016.

14. REDMON J, FARHADI A. YOLO9000: Better, faster, stronger. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'17), 2017, Jul 21-26, Honolulu, HI, USA. Piscataway, NJ, USA: IEEE, 2017.

15. REDMON J, FARHADI A. YOLOv3: An incremental improvement. arXiv e-Prints, arXiv:1804.02767, 2018.

16. BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of object detection. arXiv e-Prints, ArXiv:2004.10934, 2020.

17. ultralytics/yolov5: YOLOv5. https://github.com/ultralytics/yolov5

18. GE Z, LIU S T, WANG F, et al.YOLOX: Exceeding YOLO series in 2021. arXiv e-Prints, arXiv.2107.08430, 2021.

19. DU S, IBRAHIM M, SHEHATA M, et al. Automatic license plate recognition (ALPR): A state-of-the-art review. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 23(2): 311-325.

20. WANG W H, TU J Y. Research on license plate recognition algorithms based on deep learning in complex environment. IEEE Access, 2020, 8: 91661-91675.

21. YAO D H, ZHU W X, CHEN Y J, et al. Chinese license plate character recognition based on convolution neural network. Proceedings of the 2017 Chinese Automation Congress (CAC'17), 2017, Oct 20-22, Jinan, China. Piscataway, NJ, USA: IEEE, 2017: 1547-1552.

22. LIU Y J, HUANG H. Car plate character recognition using a convolutional neural network with shared hidden layers. Proceedings of the 2015 Chinese Automation Congress (CAC'15), 2015, Nov 27-29, Wuhan, China. Piscataway, NJ, USA: IEEE, 2015: 638-643.

23. ABDUSSALAM A, SUN S L, FU M X, et al. Robust model for Chinese license plate character recognition using deep learning techniques. Communications, Signal Processing, and Systems: Proceedings of the 2018 International Conference in Communications, Signal Processing, and Systems (CSPS'18), 2018, Jul 14-16, Dalian, China. LNEE 517. Berlin, Germany: Springer, 2018: 121-127.

24. ZANG D, CHAI Z L, ZHANG J Q, et al. Vehicle license plate recognition using visual attention model and deep learning, Journal of Electronic Imaging, 2015, 24(3): Article 033001/1-10.

25. LIU Y J, HUANG H, CAO J D, et al. Convolutional neural networks-based intelligent recognition of Chinese license plates. Soft Computing, 2018, 22: 2403-2419.

26. DUAN S M, HU W, LI R R, et al. Attention enhanced ConvNet-RNN for Chinese vehicle license plate recognition. Pattern Recognition and Computer Vision: Proceedings of the 1st Chinese Conference on Pattern Recognition and Computer Vision (PRCV'18): Part III, 2018, Nov 23-26, Guangzhou, China. LNIP 11257. Berlin, Germany: Springer, 2018: 417-428.

27. WANG J L, HUANG H, QIAN X H, et al. Sequence recognition of Chinese license plates. Neurocomputing, 2018, 317: 149-158.

28. ZHANG L J, WANG P, LI H, et al. A robust attentional framework for license plate recognition in the wild. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(11): 6967-6976.

29. HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.

30. WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module. Computer Vision: Proceedings of the 15th European Conference on Computer Vision (ECCV'18): Part VII, 2018, Sept 8-14, Munich, Germany. LNIP 11211. Berlin, Germany: Springer, 2018: 3-19.

31. LI X, WANG W H, HU X L, et al. Selective kernel networks. Proceeding of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'19), 2019, Jun 15-20, Long Beach, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 7253-7260

32. CAO Y, XU J R, LIN S, et al. GCNet: Non-local networks meet squeeze-excitation networks and beyond. Proceeding of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW'19), 2019, Oct 27-28, Seoul, Republic of Korea. Piscataway, NJ, USA: IEEE, 2019: 1971-1980.

33. WANG Q L, WU B G, ZHU P F, et al. ECA-Net: Efficient channel attention for deep convolutional neural networks. Proceeding of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020, Jun 13-19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 11531-11539.

34. MISRA D, NALAMADA T, ARASANIPALAI A U, et al. Rotate to attend: Convolutional triplet attention module. Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision (WACV'21), 2021, Jan 3-8, Waikoloa, HI, USA. Piscataway, NJ, USA: IEEE, 2021.

35. ZHOU W G, LI H Q, LU Y J, et al. Principal visual word discovery for automatic license plate detection. IEEE Transactions on Image Processing, 2021, 21(9): 4269-4279.

36. HSU G S, CHEN J C, CHUNG Y Z. Application-oriented license plate recognition. IEEE Transactions on Vehicular Technology, 2021, 62 (2): 552-561.

37. YUAN Y L, ZOU W B, ZHAO Y, et al. A robust and efficient approach to license plate detection. IEEE Transactions on Image Processing, 2017, 26 (3): 1102-1114.

38. XU Z B, YANG W, MENG A J, et al. Towards end-to-end license plate detection and recognition: A large dataset and baseline. Computer Vision: Proceedings of the 15th European Conference on Computer Vision (ECCV'18): Part XII, 2018, Sept 8-14, Munich, Germany. LNIP 11216. Berlin, Germany: Springer, 2018: 261-277.

39. LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector. Computer Vision: Proceedings of the 14th European Conference on Computer Vision (ECCV'16): Part I, 2016, Oct 11-14, Amsterdam, Netherlands. LNIP 9905. Berlin, Germany: Springer, 2016: 21-37.

40. LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV'17), 2017, Oct 22-29, Venice, Italy. Piscataway, NJ, USA: IEEE, 2017.

41. ZHU C C, HE Y H, SAVVIDES M. Feature selective anchor-free module for single-shot object detection. Proceeding of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'19), 2019, Jun 15-20, Long Beach, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 840-849.

42. KIM K, LEE H S. Probabilistic anchor assignment with IoU prediction for object detection. Computer Vision: Proceeding of the 16th European Conference on Computer Vision (ECCV’20): Part XXV, 2020, Aug 23-28, Glasgow, UK. LNIP 12370. Berlin, Germany: Springer, 2020: 355-371

43. REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.

44. LU X, LI B Y, YUE Y X, et al. Grid R-CNN. Proceeding of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'19), 2019, Jun 15-20, Long Beach, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 7253-7260.

45. HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 386-397.

46. CAI Z W, VASCONCELOS N. Cascade R-CNN: High quality object detection and instance segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(5): 1483-1498.

47. PANG J M, CHEN K, SHI J P, et al. Libra R-CNN: Towards balanced learning for object detection. Proceeding of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'19), 2019, Jun 15-20, Long Beach, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 7253-7260.

48. CHEN K, PANG J M, WANG J Q, et al. Hybrid task cascade for instance segmentation. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'19), 2019, Jun 15-20, Long Beach, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 4974-4983.

49. LAW H, DENG J, Cornernet: Detecting objects as paired keypoints. Computer Vision: Proceedings of the 15th European Conference on Computer Vision (ECCV'18): Part XII, 2018, Sept 8-14, Munich, Germany. LNIP 11216. Berlin, Germany: Springer, 2018: 734-750.

50. KONG T, SUN F K, LIU H P, et al. FoveaBox: Beyond anchor-based object detector. IEEE Transactions on Image Processing, 2020, 29: 7389-7398.

51. TIAN Z, SHEN C H, CHEN H, et al. FCOS: Fully convolutional one-stage object detection. Proceeding of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW'19), 2019, Oct 27-28, Seoul, Republic of Korea. Piscataway, NJ, USA: IEEE, 2019: 9626-9635.

52. YANG Z, LIU S H, HU H, et al. RepPoints: Point set representation for object detection. Proceeding of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW'19), 2019, Oct 27-28, Seoul, Republic of Korea. Piscataway, NJ, USA: IEEE, 2019: 9657-9666.

53. DONG Z W, LI G X, LIAO Y, et al. CentripetalNet: Pursuing high-quality keypoint pairs for object detection. Proceeding of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020, Jun 13-19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 10516-10525

54. ZHANG S F, CHI C, YAO Y Q, et al. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. Proceeding of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020, Jun 13-19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 9756–9765.

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

[1]	Wang Xianlun, Wang Guangyu, Cui Yuxia. Facial expression recognition based on improved ResNet [J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(1): 28-38.
[2]	Kong Chao, Ou Weihua, Gong Xiaofeng, Li Weian, Han Jie, Yao Yi, Xiong Jiahao. Face anti-spoofing based on multi-modal and multi-scale features fusion [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(6): 73-82.
[3]	Song Yue, Wu Chengmao, Tian Xiaoping, Song Qiuyu. Enhanced kernel-based fuzzy local information clustering integrating neighborhood membership [J]. The Journal of China Universities of Posts and Telecommunications, 2021, 28(6): 65-81.
[4]	Xue Chenzi, Wei Yifei, Zhang Yong. Performance optimization for smart grid blockchain integrated with fog computing using DDQN [J]. The Journal of China Universities of Posts and Telecommunications, 2021, 28(2): 68-78.
[5]	Wang Zhaoying, Zhou Junhua, Liao Zhonghua, Zhai Xiang, Zhang Lianping. Semantic segmentation of track image based on deep neural network [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(5): 23-33.
[6]	Chen Faquan, Fan Jun. Real-time prediction of the motion tendency of human lower limbs during gait [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(4): 1-7.
[7]	. Recognition of motor imagery tasks for BCI using CSP and chaotic PSO twin SVM [J]. JOURNAL OF CHINA UNIVERSITIES OF POSTS AND TELECOM, 2017, 24(3): 83-90.
[8]	. Smooth support vector machine based on circular tangent function [J]. JOURNAL OF CHINA UNIVERSITIES OF POSTS AND TELECOM, 2016, 23(1): 68-72.
[9]	. Dynamic and combined gestures recognition based on multi-feature fusion in a complex environment [J]. Acta Metallurgica Sinica(English letters), 2015, 22(2): 81-88.
[10]	. Exposing photo manipulation with inconsistent perspective geometry [J]. Acta Metallurgica Sinica(English letters), 2014, 21(4): 83-91.
[11]	LIU Jing , XU Guo-sheng, ZHENG Shi-hui, XIAO Da, GU Li-ze. Data streams classification with ensemble model based on decision-feedback [J]. Acta Metallurgica Sinica(English letters), 2014, 21(1): 79-85.
[12]	Jia-Kuo ZUO. Orthogonal isometric projection for face recognition [J]. Acta Metallurgica Sinica(English letters), 2011, 18(1): 91-97.