Abstract
An artificial intelligence (AI) processor is a promising solution for energy-efficient data processing, including health monitoring and image/voice recognition. However, data movements between compute part and memory induce memory wall and power wall challenges to the conventional computing architecture. Recently, the memory-centric architecture has been revised to solve the data movement issue, where the memory is equipped with the compute-capable memory technique, namely, computing-in-memory (CIM). In this paper, we analyze the requirement of AI algorithms on the data movement and low power requirement of AI processors. In addition, we introduce the story of CIM and implementation methodologies of CIM architecture. Furthermore, we present several novel solutions beyond traditional analog-digital mixed static random-access memory (SRAM)-based CIM architecture. Finally, recent CIM tape-out studies are listed and discussed.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Liu L, Qu Z, Deng L, et al. Duet: boosting deep neural network efficiency on dual-module architecture. In: Proceedings of the 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2020. 738–750
Wess M, Manoj P D S, Jantsch A. Neural network based ECG anomaly detection on FPGA and trade-off analysis. In: Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS), 2017. 1–4
Zairi H, Talha M K, Meddah K, et al. FPGA-based system for artificial neural network arrhythmia classification. Neural Comput Appl, 2019, 32: 4105–4120
Chen Y, Luo T, Liu S, et al. Dadiannao: a machine-learning supercomputer. In: Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture, 2014. 609–622
Du Z, Fasthuber R, Chen T, et al. Shidiannao: shifting vision processing closer to the sensor. In: Proceedings of ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA), 2015. 92–104
Pham P, Jelaca D, Farabet C, et al. Neuflow: dataflow vision processing system-on-a-chip. In: Proceedings of IEEE 55th International Midwest Symposium on Circuits and Systems (MWSCAS), 2012. 1044–1047
Chen Y, Krishna T, Emer J, et al. 14.5 eyeriss: an energy-efficient reconfigurable accelerator for deep convolutional neural networks. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), 2016. 262–263
Jouppi N, Young C, Patil N, et al. In-datacenter performance analysis of a tensor processing unit. In: Proceedings of ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA), 2017
Li W, Xu P, Zhao Y, et al. Timely: pushing data movements and interfaces in PIM accelerators towards local and in time domain. In: Proceedings of ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), 2020. 832–845
Chi P, Li S, Xu C, et al. Prime: a novel processing-in-memory architecture for neural network computation in ReRAM-based main memory. In: Proceedings of ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), 2016. 27–39
Zhao Y, Chen X, Wang Y, et al. Smartexchange: trading higher-cost memory storage/access for lower-cost computation. In: Proceedings of ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), 2020. 954–967
Gokhale M, Holmes B, Iobst K. Processing in memory: the Terasys massively parallel PIM array. Computer, 1995, 28: 23–31
Patterson D, Anderson T, Cardwell N, et al. A case for intelligent RAM. IEEE Micro, 1997, 17: 34–44
Hall M, Kogge P, Koller J, et al. Mapping irregular applications to diva, a PIM-based data-intensive architecture. In: Proceedings of the ACM/IEEE Conference on Supercomputing, 1999. 57
Oskin M, Chong F T, Sherwood T. Active pages: a computation model for intelligent memory. In: Proceedings of the 25th Annual International Symposium on Computer Architecture, 1998. 192–203
Kang Y, Huang W, Yoo S M, et al. FlexRAM: toward an advanced intelligent memory system. In: Proceedings of IEEE International Conference on Computer Design, 1999. 192–201
Patterson D, Anderson T, Cardwell N, et al. Intelligent RAM (IRAM): chips that remember and compute. In: Proceedings of IEEE International Solids-State Circuits Conference, 1997. 224–225
Li S, Xu C, Zou Q, et al. Pinatubo: a processing-in-memory architecture for bulk bitwise operations in emerging non-volatile memories. In: Proceedings of the 53rd ACM/EDAC/IEEE Design Automation Conference (DAC), 2016. 1–6
Zhuo Y W, Wang C, Zhang M X, et al. Graphq: scalable PIM-based graph processing. In: Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture. New York: Association for Computing Machinery, 2019
Deng L, Wang G, Li G, et al. Tianjic: a unified and scalable chip bridging spike-based and continuous neural computation. IEEE J Solid-State Circ, 2020, 55: 2228–2246
Li S, Niu D, Malladi K T, et al. Drisa: a DRAM-based reconfigurable in-situ accelerator. In: Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2017. 288–301
Li S, Glova A O, Hu X, et al. Scope: a stochastic computing engine for DRAM-based in-situ accelerator. In: Proceedings of the 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2018. 696–709
Ahn J, Hong S, Yoo S, et al. A scalable processing-in-memory accelerator for parallel graph processing. In: Proceedings of ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA), 2015. 105–117
Chang L, Ma X, Wang Z, et al. CORN: in-buffer computing for binary neural network. In: Proceedings of Design, Automation Test in Europe Conference Exhibition (DATE), 2019. 384–389
Chang L, Ma X, Wang Z, et al. PXNOR-BNN: in/with spin-orbit Torque MRAM preset-XNOR operation-based binary neural networks. IEEE Trans VLSI Syst, 2019, 27: 2668–2679
Gao M, Ayers G, Kozyrakis C. Practical near-data processing for in-memory analytics frameworks. In: Proceedings of International Conference on Parallel Architecture and Compilation (PACT), 2015. 113–124
Peng X, Liu R, Yu S. Optimizing weight mapping and data flow for convolutional neural networks on processing-in-memory architectures. IEEE Trans Circ Syst I, 2020, 67: 1333–1343
Chen Y, Emer J, Sze V. Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks. In: Proceedings of ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), 2016. 367–379
Fleischer B, Shukla S, Ziegler M, et al. A scalable multi-TeraOPS deep learning processor core for AI trainina and inference. In: Proceedings of IEEE Symposium on VLSI Circuits, 2018. 35–36
Samal K, Wolf M, Mukhopadhyay S. Attention-based activation pruning to reduce data movement in real-time AI: a case-study on local motion planning in autonomous vehicles. IEEE J Emerg Sel Top Circ Syst, 2020, 10: 306–319
Yin S, Ouyang P, Liu L, et al. A fast and power-efficient memory-centric architecture for affine computation. IEEE Trans Circ Syst II, 2016, 63: 668–672
JEDEC. High Bandwidth Memory (HBM) DRAM. JESD235A-2015. https://www.jedec.org/standards-documents/docs/jesd235a
Consortium H M C. Hybrid memory cube specification 1.0. 2013. https://yumpu.b4your.com/en/pdf/3015151532/
Koo G, Matam K K, Te I, et al. Summarizer: trading communication with computing near storage. In: Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2017. 219–231
Nair R, Antao S F, Bertolli C, et al. Active memory cube: a processing-in-memory architecture for exascale systems. IBM J Res Dev, 2015, 59: 1–14
Farmahini-Farahani A, Ahn J H, Morrow K, et al. NDA: near-DRAM acceleration architecture leveraging commodity DRAM devices and standard memory modules. In: Proceedings of IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), 2015. 283–295
Si X, Chen J, Tu Y, et al. A Twin-8T SRAM computation-in-memory unit-macro for multibit CNN-based AI edge processors. IEEE J Solid-State Circ, 2020, 55: 189–202
Zhang M, Zhuo Y, Wang C, et al. Graphp: reducing communication for PIM-based graph processing with efficient data partition. In: Proceedings of IEEE International Symposium on High Performance Computer Architecture (HPCA), 2018. 544–557
Dai G, Huang T, Chi Y, et al. GraphH: a processing-in-memory architecture for large-scale graph processing. IEEE Trans Comput-Aided Des Integr Circ Syst, 2019, 38: 640–653
Zhang J, Wang Z, Verma N. In-memory computation of a machine-learning classifier in a standard 6T SRAM array. IEEE J Solid-State Circ, 2017, 52: 915–924
Okumura S, Yabuuchi M, Hijioka K, et al. A ternary based bit scalable, 8.80 TOPS/W CNN accelerator with many-core processing-in-memory architecture with 896K synapses/mm2. In: Proceedings of Symposium on VLSI Technology, 2019
Biswas A, Chandrakasan A P. CONV-RAM: an energy-efficient SRAM with embedded convolution computation for low-power CNN-based machine learning applications. In: Proceedings of IEEE International Solid-State Circuits Conference, 2018. 488–490
Kang M, Gonugondla S K, Shanbhag N R. A 19.4 nJ/decision 364 K decisions/s in-memory random forest classifier in 6T SRAM array. In: Proceedings of the 43rd IEEE European Solid State Circuits Conference, 2017. 263–266
Valavi H, Ramadge P J, Nestler E, et al. A mixed-signal binarized convolutional-neural-network accelerator integrating dense weight storage and multiplication for reduced data movement. In: Proceedings of IEEE Symposium on VLSI Circuits, 2018. 141–142
Kang M, Gonugondla S K, Patil A, et al. A multi-functional in-memory inference processor using a standard 6T SRAM array. IEEE J Solid-State Circ, 2018, 53: 642–655
Gonugondla S K, Kang M, Shanbhag N. A 42 PJ/decision 3.12 TOPS/W robust in-memory machine learning classifier with on-chip training. In: Proceedings of IEEE International Solid-State Circuits Conference, 2018. 490–492
Ramanathan A K, Kalsi G S, Srinivasa S, et al. Look-up table based energy efficient processing in cache support for neural network acceleration. In: Proceedings of the 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2020. 88–101
Eckert C, Wang X, Wang J, et al. Neural cache: bit-serial in-cache acceleration of deep neural networks. In: Proceedings of ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA), 2018. 383–396
Sayal A, Fathima S, Nibhanupudi S S T, et al. 14.4 all-digital time-domain CNN engine using bidirectional memory delay lines for energy-efficient edge computing. In: Proceedings of IEEE International Solid-State Circuits Conference, 2019. 228–230
Sayal A, Nibhanupudi S S T, Fathima S, et al. A 12.08-TOPS/W all-digital time-domain CNN engine using bi-directional memory delay lines for energy efficient edge computing. IEEE J Solid-State Circ, 2020, 55: 60–75
Everson L R, Liu M, Pande N, et al. A 104.8 TOPS/W one-shot time-based neuromorphic chip employing dynamic threshold error correction in 65 nm. In: Proceedings of IEEE Asian Solid-State Circuits Conference (A-SSCC), 2018. 273–276
Everson L R, Liu M, Pande N, et al. An energy-efficient one-shot time-based neural network accelerator employing dynamic threshold error correction in 65 nm. IEEE J Solid-State Circ, 2019, 54: 2777–2785
Amravati A, Nasir S B, Thangadurai S, et al. A 55 nm time-domain mixed-signal neuromorphic accelerator with stochastic synapses and embedded reinforcement learning for autonomous micro-robots. In: Proceedings of IEEE International Solid-State Circuits Conference, 2018. 124–126
Amaravati A, Nasir S B, Ting J, et al. A 55-nm, 1.0–0.4 V, 1.25-pJ/MAC time-domain mixed-signal neuromorphic accelerator with stochastic synapses for reinforcement learning in autonomous mobile robots. IEEE J Solid-State Circ, 2019, 54: 75–87
Chen Z, Gu J. High-throughput dynamic time warping accelerator for time-series classification with pipelined mixed-signal time-domain computing. IEEE J Solid-State Circ, 2021, 56: 624–635
Wan W, Kubendran R, Eryilmaz S B, et al. 33.1 a 74 TMACS/W CMOS-RRAM neurosynaptic core with dynamically reconfigurable dataflow and in-situ transposable weights for probabilistic graphical models. In: Proceedings of IEEE International Solid-State Circuits Conference, 2020. 498–500
Khwa W, Chang M, Wu J, et al. 7.3 a resistance-drift compensation scheme to reduce MLC PCM raw BER by over 100×for storage-class memory applications. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), 2016. 134–135
Wang Z, Zhou H, Wang M, et al. Proposal of toggle spin torques magnetic RAM for ultrafast computing. IEEE Electron Device Lett, 2019, 40: 726–729
Chang L, Ma X, Wang Z, et al. DASM: data-streaming-based computing in nonvolatile memory architecture for embedded system. IEEE Trans VLSI Syst, 2019, 27: 2046–2059
Chang T, Chiu Y, Lee C, et al. 13.4 a 22 nm 1 Mb 1024b-read and near-memory-computing dual-mode STT-MRAM macro with 42.6 GB/s read bandwidth for security-aware mobile devices. In: Proceedings of IEEE International Solid-State Circuits Conference, 2020. 224–226
Zhang S, Huang K, Shen H. A robust 8-bit non-volatile computing-in-memory core for low-power parallel MAC operations. IEEE Trans Circ Syst I, 2020, 67: 1867–1880
Yu Z, Wang Z, Kang J, et al. Early-stage fluctuation in low-power analog resistive memory: impacts on neural network and mitigation approach. IEEE Electron Device Lett, 2020, 41: 940–943
Yang J, Zhu J, Dang B, et al. TaOx synapse array based on ion profile engineering for high accuracy neuromorpic computing. In: Proceedings of China Semiconductor Technology International Conference (CSTIC), 2020. 1–4
Wang Z, Kang J, Bai G, et al. Self-selective resistive device with hybrid switching mode for passive crossbar memory application. IEEE Electron Device Lett, 2020, 41: 1009–1012
Chang L, Wang Z, Zhang Y, et al. Multi-port 1R1W transpose magnetic random access memory by hierarchical bit-line switching. IEEE Access, 2019, 7: 110463
Khwa W, Chen J, Li J, et al. A 65 nm 4 kb algorithm-dependent computing-in-memory SRAM unit-macro with 2.3 ns and 55.8 TOPS/W fully parallel product-sum operation for binary DNN edge processors. In: Proceedings of IEEE International Solid-State Circuits Conference, 2018. 496–498
Su J, Si X, Chou Y, et al. 15.2 a 28 nm 64 kb inference-training two-way transpose multibit 6T SRAM compute-in-memory macro for AI edge chips. In: Proceedings of IEEE International Solid-State Circuits Conference, 2020. 240–242
Dong Q, Sinangil M E, Erbagci B, et al. 15.3 a 351 TOPS/W and 372.4GOPS compute-in-memory SRAM macro in 7 nm FinFet CMOS for machine-learning applications. In: Proceedings of IEEE International Solid-State Circuits Conference, 2020. 242–244
Si X, Tu Y, Huang W, et al. 15.5 a 28 nm 64 kb 6T SRAM computing-in-memory macro with 8b MAC operation for AI edge chips. In: Proceedings of IEEE International Solid-State Circuits Conference, 2020. 246–248
Yue J, Yuan Z, Feng X, et al. 14.3 a 65 nm computing-in-memory-based CNN processor with 2.9-to-35.8 TOPS/W system energy efficiency using dynamic-sparsity performance-scaling architecture and energy-efficient inter/intra-macro data reuse. In: Proceedings of IEEE International Solid-State Circuits Conference, 2020. 234–236
Wang J, Wang X, Eckert C, et al. 14.2 a compute SRAM with bit-serial integer/floating-point operations for programmable in-memory vector acceleration. In: Proceedings of IEEE International Solid-State Circuits Conference, 2019. 224–226
Gonugondla S K, Kang M, Shanbhag N. A 42 PJ/decision 3.12 TOPS/W robust in-memory machine learning classifier with on-chip training. In: Proceedings of IEEE International Solid-State Circuits Conference, 2018. 490–492
Chiu Y C, Zhang Z, Chen J J, et al. A 4-kb 1-to-8-bit configurable 6T SRAM-based computation-in-memory unit-macro for CNN-based AI edge processors. IEEE J Solid-State Circ, 2020, 55: 2790–2801
Wang J, Wang X, Eckert C, et al. A 28-nm compute SRAM with bit-serial logic/arithmetic operations for programmable in-memory vector computing. IEEE J Solid-State Circ, 2020, 55: 76–86
Jia H, Valavi H, Tang Y, et al. A programmable heterogeneous microprocessor based on bit-scalable in-memory computing. IEEE J Solid-State Circ, 2020, 55: 2609–2621
Jiang Z, Yin S, Seo J S, et al. C3SRAM: an in-memory-computing SRAM macro based on robust capacitive coupling computing mechanism. IEEE J Solid-State Circ, 2020, 55: 1888–1897
Yin S, Jiang Z, Seo J, et al. XNOR-SRAM: in-memory computing SRAM macro for binary/ternary deep neural networks. IEEE J Solid-State Circ, 2020, 55: 1733–1743
Biswas A, Chandrakasan A P. CONV-SRAM: an energy-efficient SRAM with in-memory dot-product computation for low-power convolutional neural networks. IEEE J Solid-State Circ, 2019, 54: 217–230
Yang J, Kong Y, Wang Z, et al. 24.4 sandwich-RAM: an energy-efficient in-memory BWN architecture with pulse-width modulation. In: Proceedings of IEEE International Solid-State Circuits Conference, 2019. 394–396
Chih Y D, Lee P H, Fujiwara H, et al. An 89 TOPS/W and 16.3 TOPS/mm2 all-digital SRAM-based full-precision compute-in memory macro in 22 nm for machine-learning edge applications. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), 2021. 252–254
Chen W, Li K, Lin W, et al. A 65 nm 1 Mb nonvolatile computing-in-memory ReRAM macro with sub-16 ns multiply-and-accumulate for binary DNN AI edge processors. In: Proceedings of IEEE International Solid-State Circuits Conference, 2018. 494–496
Xue C, Chen W, Liu J, et al. 24.1 a 1 Mb multibit ReRAM computing-in-memory macro with 14.6 ns parallel MAC computing time for CNN based AI edge processors. In: Proceedings of IEEE International Solid- State Circuits Conference, 2019. 388–390
Yan B, Yang Q, Chen W, et al. RRAM-based spiking nonvolatile computing-in-memory processing engine with precision-configurable in situ nonlinear activation. In: Proceedings of Symposium on VLSI Technology, 2019. 86–87
Su F, Chen W, Xia L, et al. A 462 GOPS/J RRAM-based nonvolatile intelligent processor for energy harvesting IOE system featuring nonvolatile logics and processing-in-memory. In: Proceedings of Symposium on VLSI Technology, 2017. 260–261
Liu Q, Gao B, Yao P, et al. 33.2 a fully integrated analog ReRAM based 78.4 TOPS/W compute-in-memory chip with fully parallel MAC computing. In: Proceedings of IEEE International Solid- State Circuits Conference, 2020. 500–502
Xue C, Chen W, Liu J, et al. Embedded 1-Mb ReRAM-based computing-in- memory macro with multibit input and weight for CNN-based AI edge processors. IEEE J Solid-State Circ, 2020, 55: 203–215
Zha Y, Nowak E, Li J. Liquid silicon: a nonvolatile fully programmable processing-in-memory processor with monolithically integrated ReRAM. IEEE J Solid-State Circ, 2020, 55: 908–919
Wan W, Kubendran R, Gao B, et al. A voltage-mode sensing scheme with differential-row weight mapping for energy-efficient RRAM-based in-memory computing. In: Proceedings of IEEE Symposium on VLSI Technology, 2020. 1–2
Sebastian A, Tuma T, Papandreou N, et al. Temporal correlation detection using computational phase-change memory. Nature Commun, 2017, 8: 1–10
Joshi V, Gallo M L, Haefeli S, et al. Accurate deep neural network inference using computational phase-change memory. Nature Commun, 2020, 11: 1–13
Lee K R, Kim J, Kim C, et al. A 1.02-UW STT-MRAM-based DNN ECG arrhythmia monitoring SOC with leakage-based delay MAC unit. IEEE Solid-State Circ Lett, 2020, 3: 390–393
Jeloka S, Akesh N B, Sylvester D, et al. A 28 nm configurable memory (TCAM/BCAM/SRAM) using push-rule 6T bit cell enabling logic-in-memory. IEEE J Solid-State Circ, 2016, 51: 1009–1021
Ando K, Ueyoshi K, Orimo K, et al. Brein memory: a 13-layer 4.2 k neuron/0.8 m synapse binary/ternary reconfigurable in-memory deep neural network accelerator in 65 nm CMOS. In: Proceedings of Symposium on VLSI Circuits, 2017. 24–25
Slesazeck S, Ravsher T, Havel V, et al. A 2TnC ferroelectric memory gain cell suitable for compute-in-memory and neuromorphic application. In: Proceedings of IEEE International Electron Devices Meeting (IEDM), 2019. 1–4
Yu C, Yoo T, Kim H, et al. A logic-compatible eDRAM compute-in-memory with embedded ADCs for processing neural networks. IEEE Trans Circ Syst I, 2021, 68: 667–679
Acknowledgements
This work was supported by National Key R&D Program of China (Grant No. 2019YFB2204500) and UESTC Research Start-up Funding (Grant No. Y030202059018052).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chang, L., Li, C., Zhang, Z. et al. Energy-efficient computing-in-memory architecture for AI processor: device, circuit, architecture perspective. Sci. China Inf. Sci. 64, 160403 (2021). https://doi.org/10.1007/s11432-021-3234-0
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11432-021-3234-0