Abstract
The recognition of license plates in natural scenes often face challenges such as multi-directional and multi-line variations. Additionally, previous studies have treated license plate detection and recognition as separate tasks, resulting in inefficiencies and error accumulation. To address these challenges, we propose an end-to-end method for license plate detection and recognition using multi-task learning. Firstly, we introduce two parallel branches to detect the horizontal bounding box and the four corners of the license plate, enabling multi-directional license plate detection in a multi-task manner. The outputs from these branches are combined to enhance recognition accuracy. Secondly, we propose to extract global features to perceive character layout and utilize reading order to spatially attend to characters for recognizing multi-line license plates. Finally, we combine detection and recognition using the same backbone, with the detection branch based on multiple deep layers and the recognition branch based on multiple shallow layers, thereby constructing an end-to-end detection and recognition network. Comparative experiments on CCPD and RodoSol datasets validate that our method significantly outperforms state-of-the-art methods, particularly in scenarios involving multi-directional and multi-line license plates.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Atienza, R.: Data augmentation for scene text recognition. In: International Conference on Computer Vision Workshops, Montreal, BC, Canada, pp. 1561–1570. IEEE (2021)
Bochkovskiy, A., Wang, C., Liao, H.M.: Yolov4: optimal speed and accuracy of object detection. CoRR abs/2004.10934 (2020)
Cao, Y., Fu, H., Ma, H.: An end-to-end neural network for multi-line license plate recognition. In: International Conference on Pattern Recognition, Beijing, China, pp. 3698–3703. IEEE (2018)
Chen, S., Liu, Q., Chen, F., Yin, X.: End-to-end multi-line license plate recognition with cascaded perception. In: Fink, G.A., Jain, R., Kise, K., Zanibbi, R. (eds.) ICDAR 2023. LNCS, vol. 14191, pp. 274–289. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-41734-4_17
Chen, S., Tian, S., Liu, Q., Chen, F., Yin, X.: Vertex adjustment loss for multidirectional license plate detection and recognition. In: International Conference on Ubiquitous Intelligence and Computing, Haikou, China, pp. 285–292. IEEE (2022)
Chen, S., et al.: End-to-end trainable network for degraded license plate detection via vehicle-plate relation mining. Neurocomputing 446, 1–10 (2021)
Cheng, Z., Bai, F., Xu, Y., Zheng, G., Pu, S., Zhou, S.: Focusing attention: towards accurate text recognition in natural images. In: International Conference on Computer Vision, Venice, Venice, pp. 5086–5094. IEEE (2017)
Datondji, S.R.E., Dupuis, Y., Subirats, P., Vasseur, P.: A survey of vision-based traffic monitoring of road intersections. IEEE Trans. Intell. Transp. Syst. 17(10), 2681–2698 (2016)
Fan, X., Zhao, W.: Improving robustness of license plates automatic recognition in natural scenes. IEEE Trans. Intell. Transp. Syst. 23(10), 18845–18854 (2022)
Girshick, R.B., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, pp. 580–587. IEEE (2014)
He, K., Gkioxari, G., Dollár, P., Girshick, R.B.: Mask R-CNN. In: International Conference on Computer Vision, Venice, Italy, pp. 2980–2988. IEEE (2017)
Henry, C., Ahn, S.Y., Lee, S.: Multinational license plate recognition using generalized character sequence detection. IEEE Access 8, 35185–35199 (2020)
Huang, Q., Cai, Z., Lan, T.: A single neural network for mixed style license plate detection and recognition. IEEE Access 9, 21777–21785 (2021)
Laroca, R., Cardoso, E.V., Lucio, D.R., Estevam, V., Menotti, D.: On the cross-dataset generalization in license plate recognition. In: International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, pp. 166–178. SCITEPRESS, Online Streaming (2022)
Li, H., Wang, P., Shen, C.: Toward end-to-end car license plate detection and recognition with deep neural networks. IEEE Trans. Intell. Transp. Syst. 20(3), 1126–1136 (2019)
Li, H., Wang, P., Shen, C., Zhang, G.: Show, attend and read: a simple and strong baseline for irregular text recognition. In: AAAI Conference on Artificial Intelligence, Honolulu, Hawaii, USA, pp. 8610–8617. AAAI Press (2019)
Li, Z., Chen, S., Liu, Q., Chen, F., Yin, X.: Anchor-free location refinement network for small license plate detection. In: Yu, S., et al. (eds.) PRCV 2022. LNCS, vol. 13537, pp. 506–519. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-18916-6_41
Lin, T., Dollár, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 936–944. IEEE (2017)
Lin, T., Maji, S.: Visualizing and understanding deep texture representations. In: IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 2791–2799. IEEE (2016)
Liu, Q., Chen, S.-L., Li, Z.-J., Yang, C., Chen, F., Yin, X.-C.: Fast recognition for multidirectional and multi-type license plates with 2D spatial attention. In: Lladós, J., Lopresti, D., Uchida, S. (eds.) ICDAR 2021. LNCS, vol. 12824, pp. 125–139. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86337-1_9
Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 8759–8768. IEEE (2018)
Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Lu, Q., Liu, Y., Huang, J., Yuan, X., Hu, Q.: License plate detection and recognition using hierarchical feature layers from CNN. Multim. Tools Appl. 78(11), 15665–15680 (2019)
Luo, C., Jin, L., Sun, Z.: MORAN: a multi-object rectified attention network for scene text recognition. Pattern Recognit. 90, 109–118 (2019)
Masood, S.Z., Shu, G., Dehghan, A., Ortiz, E.G.: License plate detection and recognition using deeply learned convolutional neural networks. arXiv abs/1703.07330 (2017)
Meng, A., Yang, W., Xu, Z., Huang, H., Huang, L., Ying, C.: A robust and efficient method for license plate recognition. In: International Conference on Pattern Recognition, Beijing, China, pp. 1713–1718. IEEE (2018)
Paidi, V., Fleyeh, H., Håkansson, J., Nyberg, R.G.: Smart parking sensors, technologies and applications for open parking lots: a review. IET Intel. Transport Syst. 12(8), 735–741 (2018)
Qiao, L., et al.: MANGO: a mask attention guided one-stage scene text spotter. In: AAAI Conference on Artificial Intelligence, pp. 2467–2476. AAAI Press, Virtual Event (2021)
Qin, S., Liu, S.: Towards end-to-end car license plate location and recognition in unconstrained scenarios. Neural Comput. Appl. 34(24), 21551–21566 (2022)
Redmon, J., Divvala, S.K., Girshick, R.B., Farhadi, A.: You only look once: unified, real-time object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 779–788. IEEE (2016)
Selmi, Z., Halima, M.B., Alimi, A.M.: Deep learning system for automatic license plate detection and recognition. In: International Conference on Document Analysis and Recognition, Kyoto, Japan, pp. 1132–1138. IEEE (2017)
Shi, B., Wang, X., Lyu, P., Yao, C., Bai, X.: Robust scene text recognition with automatic rectification. In: IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 4168–4176. IEEE Computer Society (2016)
Silva, S.M., Jung, C.R.: License plate detection and recognition in unconstrained scenarios. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11216, pp. 593–609. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01258-8_36
Spanhel, J., Sochor, J., Juránek, R., Herout, A., Marsik, L., Zemcík, P.: Holistic recognition of low quality license plates by CNN using track annotated data. In: International Conference on Advanced Video and Signal Based Surveillance, Lecce, Italy, pp. 1–6. IEEE Computer Society (2017)
Wang, T., et al.: Decoupled attention network for text recognition. In: AAAI Conference on Artificial Intelligence, New York, NY, USA, pp. 12216–12224. AAAI Press (2020)
Wang, W., Yang, J., Chen, M., Wang, P.: A light CNN for end-to-end car license plates detection and recognition. IEEE Access 7, 173875–173883 (2019)
Xu, Z., et al.: Towards end-to-end license plate detection and recognition: a large dataset and baseline. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 261–277. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_16
Yang, Y., Xi, W., Zhu, C., Zhao, Y.: HomoNet: unified license plate detection and recognition in complex scenes. In: Gao, H., Wang, X., Iqbal, M., Yin, Y., Yin, J., Gu, N. (eds.) CollaborateCom 2020. LNICST, vol. 350, pp. 268–282. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67540-0_16
Zhang, L., Wang, P., Li, H., Li, Z., Shen, C., Zhang, Y.: A robust attentional framework for license plate recognition in the wild. IEEE Trans. Intell. Transp. Syst. 22(11), 6967–6976 (2021)
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., Ren, D.: Distance-IoU loss: faster and better learning for bounding box regression. In: AAAI Conference on Artificial Intelligence, New York, NY, USA, pp. 12993–13000. AAAI Press (2020)
Zherzdev, S., Gruzdev, A.: LPRNet: license plate recognition via deep neural networks. arXiv abs/1806.10447 (2018)
Zhou, X., Cheng, Y., Jiang, L., Ning, B., Wang, Y.: Fafenet: a fast and accurate model for automatic license plate detection and recognition. IET Image Process. 17(3), 807–818 (2023)
Zou, Y., et al.: License plate detection and recognition based on yolov3 and ILPRNET. Signal Image Video Process. 16(2), 473–480 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mo, ZL., Chen, SL., Liu, Q., Chen, F., Yin, XC. (2024). Multi-task Learning for License Plate Recognition in Unconstrained Scenarios. In: Barney Smith, E.H., Liwicki, M., Peng, L. (eds) Document Analysis and Recognition - ICDAR 2024. ICDAR 2024. Lecture Notes in Computer Science, vol 14804. Springer, Cham. https://doi.org/10.1007/978-3-031-70533-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-70533-5_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70532-8
Online ISBN: 978-3-031-70533-5
eBook Packages: Computer ScienceComputer Science (R0)
