Abstract
As a consequence of the current edge-processing trend, Convolutional Neural Networks (CNNs) deployment has spread to a rich landscape of devices, highlighting the need to reduce the algorithm’s complexity and exploit hardware-aided computing, as two prospective ways to improve performance on resource-constrained embedded systems. In this work, we refer to a compression method reducing a CNN computational workload based on the complexity of the data to be processed, by pruning unnecessary connections at runtime. To evaluate its efficiency when applied on edge processing platforms, we consider a keyword spotting (KWS) task executing on SensorTile, a low-power microcontroller platform by ST, and an image recognition task running on NEURAghe, an FPGA-based inference accelerator. In the first case, we obtained a 51% average reduction of the computing workload, resulting in up to 44% inference speedup, and 15% energy-saving, while in the latter, a 36% speedup is achieved, thanks to a 44% workload reduction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015)
Jouppi, N.P., et al.: In-datacenter performance analysis of a tensor processing unit. In: ISCA 2017: Proceedings of the 44th Annual International Symposium on Computer Architecture, pp. 1–12, June 2017. https://doi.org/10.1145/3079856.3080246
Azarkhish, E., Rossi, D., Loi, I., Benini, L.: Neurostream: scalable and energy efficient deep learning with smart memory cubes. IEEE Trans. Parallel Distrib. Syst. 22(2), 420–434 (2018)
Desoli, G., et al.: 14.1 a 2.9TOPS/W deep convolutional neural network SoC in FD-SOI 28 nm for intelligent embedded systems. In: 2017 IEEE International Solid-State Circuits Conference (ISSCC), pp. 238–239 (2017)
Movidius: Movidius neural compute stick: accelerate deep learning development at the edge (2020). https://developer.movidius.com/
Chen, Y.-H., Emer, J., Sze, V.: Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks. In: 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), pp. 367–379 (2016)
NVIDIA: Nvidia deep learning accelerator (2020). https://developer.nvidia.com/embedded/buy/tegra-k1-processor
Blott, M., Preusser, T., Fraser, N., Gambardella, G., O’Brien, K., Umuroglu, Y.: FINN-R: an end-to-end deep-learning framework for fast exploration of quantized neural networks. ACM Trans. Reconfigurable Technol. Syst. (TRETS) (2018). https://doi.org/10.1145/3242897
Meloni, P., et al.: NEURAghe: exploiting CPU-FPGA synergies for efficient and flexible CNN inference acceleration on Zynq SoCs. ACM Trans. Reconfigurable Technol. Syst. (TRETS) (2018). https://doi.org/10.1145/3284357
NVIDIA: cuDNN (2020). https://developer.nvidia.com/cudnn
Lai, L., Suda, N., Chandra, V.: CMSIS-NN: efficient neural network kernels for Arm Cortex-M CPUs. CoRR, abs/1801.06601 (2018). http://arxiv.org/abs/1801.06601
ARM-NN (2020). https://www.arm.com/products/silicon-ip-cpu/machine-learning/arm-nn
Han, S., et al.: EIE: efficient inference engine on compressed deep neural network. In: ISCA 2016: Proceedings of the 43rd International Symposium on Computer Architecture, pp. 243–254, June 2016. https://doi.org/10.1109/ISCA.2016.30
Han, S., et al.: ESE: efficient speech recognition engine with sparse LSTM on FPGA. In: FPGA 2017: Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, pp. 75–84, February 2017. https://doi.org/10.1145/3020078.3021745
Theodorakopoulos, I., Pothos, V., Kastaniotis, D., Fragoulis, N.: Parsimonious inference on convolutional neural networks: learning and applying on-line kernel activation rules. CoRR, abs/1701.05221 (2017). https://arxiv.org/abs/1701.05221
Krizhevsky, A.: Learning multiple layers of features from tiny images (2009). https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf
Iandola, F.N., et al.: SqueezeNet: AlexNet-Level accuracy with 50x fewer parameters and \(<\)0.5 mb model size. CoRR, abs/1602.07360 (2016). http://arxiv.org/abs/1602.07360
Zhang, Y., Suda, N., Lai, L., Chandra, V.: Hello edge: keyword spotting on microcontrollers. CoRR, arXiv:1711.07128 (2017)
Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. In: International Conference on Learning Representations 2016, October 2015. https://arxiv.org/abs/1510.00149
Gope, D., Dasika, G., Mattina, M.: Ternary hybrid neural-tree networks for highly constrained IoT applications (2019)
Hua, W., Zhou, Y., De Sa, C., Zhang, Z., Suh, G.E.: Boosting the performance of CNN accelerators with dynamic fine-grained channel gating. In: Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture, ser. MICRO 52, pp. 139–150. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3352460.3358283
Lin, J., Rao, Y., Lu, J., Zhou, J.: Runtime neural pruning. In: Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017). https://proceedings.neurips.cc/paper/2017/file/a51fb975227d6640e4fe47854476d133-Paper.pdf
Tschannen, M., Khanna, A., Anandkumar, A.: StrassenNets: deep learning with a multiplication budget. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 80, pp. 4985–4994. PMLR, 10–15 July 2018. https://proceedings.mlr.press/v80/tschannen18a.html
Kumar, A., Goyal, S., Varma, M.: Resource-efficient machine learning in 2 KB RAM for the internet of things. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 70, pp. 1935–1944. PMLR, 06–11 August 2017. https://proceedings.mlr.press/v70/kumar17a.html
Warden, P.: Speech commands: a dataset for limited-vocabulary speech recognition. CoRR, arXiv:1804.03209 (2018)
Scrugli, M.A., Loi, D., Raffo, L., Meloni, P.: A runtime-adaptive cognitive IoT node for healthcare monitoring. In: Proceedings of the 16th Conference on Computing Frontiers (CF 2019), pp. 350–357, April 2019. https://doi.org/10.1145/3310273.3323160
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Busia, P., Theodorakopoulos, I., Pothos, V., Fragoulis, N., Meloni, P. (2022). Dynamic Pruning for Parsimonious CNN Inference on Embedded Systems. In: Desnos, K., Pertuz, S. (eds) Design and Architecture for Signal and Image Processing. DASIP 2022. Lecture Notes in Computer Science, vol 13425. Springer, Cham. https://doi.org/10.1007/978-3-031-12748-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-12748-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-12747-2
Online ISBN: 978-3-031-12748-9
eBook Packages: Computer ScienceComputer Science (R0)