Real-Time High Definition License Plate Localization and Recognition Accelerator for IoT Endpoint System on Chip

Shizhen Huang; Mengru Lin; Fan Yu; Ruiqi Chen; Lei Zhang; Yanxiang Zhu

doi:10.6180/jase.202202_25(1).0001

Real-Time High Definition License Plate Localization and Recognition Accelerator for IoT Endpoint System on Chip

Computer Science and Information Engineering

Shizhen Huang^1,2, Mengru Lin^1,2, Fan Yu³, Ruiqi Chen This email address is being protected from spambots. You need JavaScript enabled to view it.^1,2,3, Lei Zhang¹, and Yanxiang Zhu³

¹College of Physics and Information Engineering, Fuzhou University, Fuzhou 350116, China
²Fujian Key Lab of Microelectronics & Integrated Circuits, Fuzhou University, Fuzhou 350002, China
³Verimake Research, Nanjing Qujike Info-tech Co., Ltd., Nanjing 210088, China

Received: December 18, 2020
Accepted: May 24, 2021
Publication Date: July 5, 2021

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202202_25(1).0001

ABSTRACT

Automatic License Plate Recognition (ALPR) systems have become popular application areas of the Internet of Things (IoT). A typical ALPR system always needs powerful processors such as Cortex-A7. However, most known system for Standard Definition (SD) are not suitable for real-time High Definition (HD) image processing and low power consuming requirement in IoT. A HD ALPR accelerator for the IoT endpoint System on Chip (SoC) is proposed in this paper to meet the needs of computations. Based on the programming flexibility of IoT endpoint SoC, it can switch between HD and SD resolutions, which can avoid the specific resolution switching algorithm. A Field Programmable Gate Array (FPGA) chip is transplanted the Cortex-M0 as the IoT endpoint SoC, through the design of ALPR accelerator and Cortex-M0, data communication is achieved by First-In, First-Out (FIFO) with AMBA High-performance Bus (AHB) interface. Heterogeneous implementation of ALPR system has shown that this HD ALPR algorithm can recognize a license plate in 12.5ms, with a success rate of 95.5%. The system utilizes 41,763 Look-Up-Tables (LUTs) without special FPGA IP core. The comparison shows that the system proposed in this paper makes performance of the SoC based on the Cortex-M0 kernel was two times higher than the Cortex-A72 SoC and 39% of the power consumption of Zynq-7000 that is typicalheterogeneous ALPR platform.

Keywords: Automatic License Plate Recognition (ALPR); Internet of Things (IoT); FPGA; SoC; Cortex-M0

REFERENCES

[1] F. Ge, N. Wu, H. Xiao, Y. Zhang, and F. Zhou, (2019) “Compact convolutional neural network accelerator for iot endpoint soc" Electronics 8(5): 497. DOI: 10.3390/ electronics8050497.
[2] A. Al-Zawqari, O. Hommos, A. Al-Qahtani, A. A. H. Farhat, F. Bensaali, X. Zhai, and A. Amira, (2019) “HD number plate localization and character segmentation on the Zynq heterogeneous SoC" Journal of Real-Time Image Processing 16(6): 2351–2365. DOI: 10.1007/ s11554-017-0747-7.
[3] Z. Xu,W. Yang, A. Meng, N. Lu, H. Huang, C. Ying, and L. Huang, (2018) “Towards end-to-end license plate detection and recognition: A large dataset and baseline" Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 11217: 255–271. DOI: 10.1007/978-3-030-01261-8_16.
[4] A. Farhat, O. Hommos, A. Al-Zawqari, A. Al-Qahtani, F. Bensaali, A. Amira, and X. Zhai, (2018) “Optical character recognition on heterogeneous SoC for HD automatic number plate recognition system" Eurasip Journal on Image and Video Processing 2018(1): 1–17. DOI: 10.1186/s13640-018-0298-2.
[5] Y. Jing, (2016) “An Efficient FPGA Implementation of Optical Character Recognition System for License Plate Recognition": DOI: 10.1109/CCECE.2017.7946734.
[6] O. Hommos, A. Al-Qahtani, A. Farhat, A. Al-Zawqari, F. Bensaali, A. Amira, and X. Zhai, (2016) “HD Qatari ANPR system" 2016 International Conference on Industrial Informatics and Computer Systems (CIICS) 2016(2016): 1–5. DOI: 10.1109/ICCSII.2016.7462420.
[7] F. Rosli, A. Elhossini, and B. Juurlink, (2015) “Real-Time Vision System for License Plate Detection and Recognition on FPGA" PARS-Mitteilungen: Vol. 32, Nr. 1:DOI: 10.14279/depositonce-7182.
[8] L. V. Agostini, I. S. Silva, and S. Bampi, (2004) “Parallel color space converters for JPEG image compression" Microelectronics Reliability 44(4): 697–703. DOI: 10.1016/j.microrel.2003.10.016.
[9] C.-C. Cheng, Y.-T. Ching, P.-H. Ko, and Y. Hwu, (2018) “Correction of center of rotation and projection angle in synchrotron X-ray computed tomography" Scientific reports 8(1): 1–9. DOI: 10.1038/s41598-018-28149-8.
[10] Rukundo, O and B. T. Maharaj, (2015) “Optimization of image interpolation based on nearest neighbour algorithm." VISAPP 2014 - Proceedings of the 9th International Conference on Computer Vision Theory and Applications 2014(1): 641–647. DOI: 10 .5220/0004742506410647.
[11] M. J. Atallah, (2001) “Faster image template matching in the sum of the absolute value of differences measure" IEEE Transactions on image processing 10(4): 659–663. DOI: 10.1038/s41598-018-28149-8.
[12] B. Ahirwal, M. Khadtare, and R. Mehta, (2007) “FPGA based system for color space transformation RGB to YIQ and YCbCr" 2007 International Conference on Intelligent and Advanced Systems 2007(2007):1345–1349. DOI: 10.1109/ICIAS.2007.4658603.
[13] ARM, (2009) “ARM Cortex-M0 Processor Technical Reference Manual" ARM Limited Company: Cambridge, UK: 1–8.
[14] A. Paul, B.-W. Chen, K. Bharanitharan, and J.-F. Wang, (2013) “Video search and indexing with reinforcement agent for interactive multimedia services" ACM Transactions on Embedded Computing Systems (TECS) 12(2): 1–16. DOI: 10.1145/2423636.2423643.
[15] S. Di, H. Zhang, X. Mei, D. Prokhorov, and H. Ling, (2016) “A benchmark for cross-weather traffic scene understanding" 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC):2150–2156. DOI: 10.1109/ITSC.2016.7795904.
[16] V. C. Venezia, A. C.-W. Hsiung,W.-Z. Yang, Y. Zhang, C. Zhao, Z. Lin, and L. A. Grant, (2018) “Second generation small pixel technology using hybrid bond stacking" Sensors (Switzerland) 18(2): 667. DOI: 10.3390/ s18020667.
[17] X. Zhai and F. Bensaali, (2015) “Improved number plate character segmentation algorithm and its efficient FPGA implementation" Journal of Real-Time Image Processing 10(1): 91–103. DOI: 10.1007/s11554-012-0258-5.