A new augmentation-based method for text detection in night and day license plate images | Multimedia Tools and Applications Skip to main content
Springer Nature Link
Log in

A new augmentation-based method for text detection in night and day license plate images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Despite a number of methods that have been developed for License Plate Detection (LPD), most of these focus on day images for license plate detection. As a result, license plate detection in night images is still an elusive goal for researchers. This paper presents a new method for LPD based on augmentation and Gradient Vector Flow (GVF) in night and day images. The augmentation involves expanding windows for each pixel in R, G and B color spaces of the input image until the process finds dominant pixels in both night and day license plate images of the respective color spaces. We propose to fuse the dominant pixels in R, G and B color spaces to restore missing pixels. For the results of fusing night and day images, the proposed method explores Gradient Vector Flow (GVF) patterns to eliminate false dominant pixels, which results in candidate pixels. The proposed method explores further GVF arrow patterns to define a unique loop pattern that represents hole in the characters, which gives candidate components. Furthermore, the proposed approach uses a recognition concept to fix the bounding boxes, merging the bounding boxes and eliminating false positives, resulting in text/license plate detection in both night and day images. Experimental results on night images of our dataset and day images of standard license plate datasets, demonstrate that the proposed approach is robust compared to the state-of-the-art methods. To show the effectiveness of the proposed method, we also tested our approach on standard natural scene datasets, namely, ICDAR 2015, MSRA-TD-500, ICDAR 2017-MLT, Total-Text, CTW1500 and MS-COCO datasets, and their results are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16.

Similar content being viewed by others

References

  1. Afifi M, Brown M (2019) What else can fool deep learning? Addressing color constancy errors on deep neural network performance, In Proc ICCV, pp. 243–252

  2. Afifi M, Price B, Cohen S, Brown MS (2019) When color constancy goes wrong: correcting improperly white balanced images, In Proc CVPR, pp. 1535–1544

  3. Ahmed AH, Afifi M, Korashy M, William EK, El-sattar MA, Hafez Z (2015) OCR systems for poor quality images using chain-code representation, In Proc AISI, pp. 151–161

  4. Asif MR, Chun Q, Hussain S, Fareed MS, Khan S (2017) Multinational vehicle license plate detection in complex backgrounds. J Vis Commun Image Represent 46:176–186

    Article  Google Scholar 

  5. Bazazian D, Gomez R, Nicolaou A, Gomez L, Karatazas D, Bagdanov AD (2019) FAST: facilitated and accurate scene text proposals through FCN guided pruning. Pattern Recogn Lett 119:112–120

    Article  Google Scholar 

  6. Boonsim N, Prakoonwit S (2017) Car make and model recognition under limited lighting conditions at night, Pattern Anal Applic, pp. 1195–1207

  7. Ch’ng CK, Chan CS (2017) Total-text: a comprehensive dataset for scene text detection and recognition, In Proc ICDAR, pp. 935–942

  8. Chowdhury PN, Shivakumara P, Raghavendra R, Pal U, Blumenstein M (2019) A new U-net based license plate enhancement model in night and day images, In Proc. ACPR, pp. 749–763

  9. Deng D, Liu H, Li X, Cai D (2018) PixelLink: detecting scene text via instance segmentation, In Proc. AAAI

  10. Deng L, Gong Y, Lin Y, Shuai J, Tu X, Zhang Y, Ma Z, Xie M (2019) Detecting multi-oriented text with corner-based region proposals. Neurocomputing 21:134–142

    Article  Google Scholar 

  11. Du S, Ibrahim M, Shehata M, Badway W (2013) Automatic license plate recognition (ALPR): a state-of-the-art review. IEEE Trans CSVT 23:311–325

    Google Scholar 

  12. Gupta N, Jalal AS (2019) A robust model for salient text detection in natural scene images using MSER feature detector and Grabcut. Multimed Tools Appl 78:10821–10835

    Article  Google Scholar 

  13. He W, Zhang X-Y, Yin F, Liu C-L (2018) Multi-oriented and multi-lingual scene text detection with direct regression. IEEE Trans IP 27:5406–5419

    MathSciNet  Google Scholar 

  14. Kang L, Li Y, Doermann D (2014) Orientation robust text line detection in natural images, In Proc. CVPR, pp. 4034–4041

  15. Karatzas D, Gomez-Bigorda L, Nicolaou A, Ghosh S, Bagdanov A, Iwamura M, Matas J, Neumann L, Chandrasekhar VR, Lu S, Shafait F (2015) ICDAR 2015 competition on robust reading. In Proc. ICDAR, pp. 1156–1160

  16. Khare V, Shivakumara P, Paramesran R, Blumenstein M (2017) Arbitrarily-oriented multi-lingual text detection in video. Multimed Tools Appl 76:16625–16655

    Article  Google Scholar 

  17. Koo H, Kim DH (2013) Scene text detection via connected component clustering and nontext filtering. IEEE Trans. on Image Processing, pp 2296–2305

  18. Li H, Wang P, Shen C (2019) Toward end to end car license plate detection and recognition with deep neural networks. IEEE Trans. ITS 20:1126–1136

    Google Scholar 

  19. Lin CH, Lin YS, Liu WC (2018) An efficient license plate recognition system using convolutional neural networks, In Proc ICASI, pp. 224–227

  20. Liu Y, Jin L (2017) Deep matching prior network: toward tighter multi-oriented text detection, In Proc. CVPR, pp. 3454–3461

  21. Liu Y, Jin L, Zhang S, Luo C, Zhang S (2019) Curved scene text detection via transverse and longitudinal sequence connection. Pattern Recogn 90:337–345

    Article  Google Scholar 

  22. Liu Z, Zhou W, Li H (2019) Scene text detection with fully convolutional neural networks. Multimed Tools Appl 78:18205–18227

    Article  Google Scholar 

  23. Liu Y, Jin L, Zhang S, Zhang S Detecting Curved Text in the Wild: New Dataset and New Solution, arXiv: 1712.02170.

  24. Long S, Ruan J, Zhang W, He X, Wu W, Yao C (2018) Textsnake: a flexible representation for detecting text of arbitrary shapes, In Proc. ECCV, pp , 19–35

  25. Ma J, Shao W, Ye H, Wang L, Wang H, Zheng Y, Xue X (2018) Arbitrary-oriented scene text detection via rotation proposals. IEEE Trans MM 20:3111–3122

    Google Scholar 

  26. Nayef N, Yin F, Bizid I, Choi H, Feng Y, Karatzas D, Luo Z, Pal U, Rigaud C, Chazalon J et al. (2017) ICDAR2017 robust reading challenge on multi-lingual scene text detection and script identification-RRC-MLT, In Proc. ICDAR, pp. 1454–1459

  27. Panahi R, Gholampour I (2017) Accurate detection and recognition of dirty vehicle plate numbers for high speed applications. IEEE Trans ITS 18:767–779

    Google Scholar 

  28. Raghunandan KS, Shivakumara P, Jalab HA, Ibrahim RW, Kumar GH, Pal U, Lu T (2018) Riesz fractional based model for enhancing license plate detection and recognition. IEEE Trans. CSVT 28:2276–2288

    Google Scholar 

  29. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) MobileNetV2: inverted residuals and linear bottleneck, In Proc. CVPR, pp. 4510–4520

  30. Shemarry MSA, Li Y, Abdulla S (2018) Ensemble of adaboost cascades of 3L-LBPs classifiers for license plated detection with low quality images. ESWA 92:216–235

    Google Scholar 

  31. Shi B, Bai X, Belongie S (2017) Detecting oriented text in natural images by linking segments. In Proc. CVPR, pp. 3482–3490

  32. Shivakumara P, Konwer A, Bhowmick A, Khare V, Pal U, Lu T (2017) A new GVF arrow pattern for character segmentation from double line license plate images, In Proc. ACPR, pp. 782–787

  33. Shivakumara P, Roy S, Jalab HA, Ibrahim RW, Pal U, Khare TLV, Wahab AWBA (2019) Fractional means based method for multi-oriented keyword spotting in video/scene/license plate images. Expert Syst Appl 118:1–19

    Article  Google Scholar 

  34. Tian S, Shivakumara P, Phan TQ, Lu T, Tan CL (2015) Character shape restoration system through medial axis points in video. Neurocomputing 161:183–198

    Article  Google Scholar 

  35. Tian Z, Huang W, He T, He P, Qiao Y (2016) Detecting text in natural image with connectionist text proposal network, In Proc ECCV, pp. 56–72

  36. Veit A, Matera T, Neumann L, Matas J, Belongie S (2016) Coco-text: Dataset and benchmark for text detection and recognition in natural images, arXiv preprint arXiv:1601.07140

  37. Xie L, Ahmad W, Jin L, Liu Y, Zhang S (2018) A new CNN based method for multi-directional car license plate detection. IEEE Trans. ITS 19:507–517

    Article  Google Scholar 

  38. Xu Y, Wang Y, Zhou W, Wang Y, Yang Z, Bai X (2019) TextField: learning a deep direction field for irregular scene text detection. IEEE Trans. IP 28:5566–5579

    MathSciNet  MATH  Google Scholar 

  39. Xue C, Lu S, Zhang W (2019) MSR: multi-scale regression for scene text detection, In Proc IJCAI, pp. 989–995

  40. Xue M, Shivakumara P, Zhang C, Lu T, Pal U (2019) Curved text detection in blurred/non-blurred video/scene images, Multimed Tools Appl, pp. 1–25

  41. Yao C, Bai X, Sang N, Zhou X, Zhou S, Cao Z (2016) Scene text detection via holistic, multi-channel prediction, arXiv preprint arXiv:1606.09002

  42. Yin XC, Yin X, Huang K, Hao H (2014) Robust text detection in natural scene images. IEEE Trans. on PAMI, pp 970–983

  43. Yin XC, Pei WY, Zhang J, Hao HW (2015) Multi-orientation scene text detection with adaptive clustering. IEEE Trans PAMI 37:1930–1937

    Article  Google Scholar 

  44. Yuan Y, Zou W, Zhao Y, Wang X, Hu X, Komodakis N (2017) A robust and efficient approach to license plate detection, IEEE Trans., IP, 26, pp 1102–1114

  45. Zamberletti A, Gallo I, Noce L (2015) Augmented text character proposals and convolutional neural networks for text spotting from scene images, In Proc ACPR, pp. 196–200

  46. Zhai X, Bensaai F, Sotudeh R (2012) OCR-based neural network for ALPR, In Proc. ICMST

  47. Zhang Z, Zhang C, Shen W, Yao C, Liu W, Bai X (2016) Multi-oriented text detection with fully convolutional networks, In Proc of CVPR, pp. 4159–4167

  48. Zhang X, Gao X, Tian C (2018) Text detection in natural scene images based on color prior guided MSER. Neurocomputing 307:61–71

    Article  Google Scholar 

  49. Zhang C, Shivakumara P, Xue M, Zhu L, Lu T, Pal U (2018) New fusion based enhancement for text detection in night video footage, In Proc PCM, pp. 46–56

  50. Zhao X, Zhao C, Guo H, Zhu Y, Tang M, Wang J (2019) Elite loss for scene text detection. Neurocomputing 333:284–291

    Article  Google Scholar 

  51. Zhou X, Yao C, Wen H, Wang Y, Zhou S, He W, Liang J (2017) East: an efficient and accurate scene text detector, in Proc. CVPR, pp. 2642–2651

Download references

Acknowledgements

This work is supported by the Natural Science Foundation of China under Grant 61672273 and Grant 61832008, and the Science Foundation for Distinguished Young Scholars of Jiangsu under Grant BK20160021. This work was also partially supported by a Faculty Grant: GPF014D-2019, University of Malaya, Malaysia.

Author information

Authors and Affiliations

  1. Computer Vision and Pattern Recognition Unit, Indian Statistical Institute, Kolkata, India

    Pinaki Nath Chowdhury & Umapada Pal

  2. Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia

    Palaiahnakote Shivakumara

  3. National Key Lab for Novel Software Technology, Nanjing University, Nanjing, China

    Tong Lu

  4. Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo, Sydney, NSW, 2007, Australia

    Michael Blumenstein

Authors
  1. Pinaki Nath Chowdhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Palaiahnakote Shivakumara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Umapada Pal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tong Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Blumenstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tong Lu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chowdhury, P.N., Shivakumara, P., Pal, U. et al. A new augmentation-based method for text detection in night and day license plate images. Multimed Tools Appl 79, 33303–33330 (2020). https://doi.org/10.1007/s11042-020-09681-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09681-0

Keywords

Search

Navigation