Generalised Analog LSTMs Recurrent Modules for Neural Computing

Kazybek Adam, Kamilya Smagulova, Alex James*

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

2 Citations (Scopus)
37 Downloads (Pure)

Abstract

The human brain can be considered as a complex dynamic and recurrent neural network. There are several models for neural networks of the human brain, that cover sensory to cortical information processing. Large majority models include feedback mechanisms that are hard to formalise to realistic applications. Recurrent neural networks and Long short-term memory (LSTM) inspire from the neuronal feedback networks. Long short-term memory (LSTM) prevent vanishing and exploding gradients problems faced by simple recurrent neural networks and has the ability to process order-dependent data. Such recurrent neural units can be replicated in hardware and interfaced with analog sensors for efficient and miniaturised implementation of intelligent processing. Implementation of analog memristive LSTM hardware is an open research problem and can offer the advantages of continuous domain analog computing with relatively low on-chip area compared with a digital-only implementation. Designed for solving time-series prediction problems, overall architectures and circuits were tested with TSMC 0.18 μm CMOS technology and hafnium-oxide (HfO2) based memristor crossbars. Extensive circuit based SPICE simulations with over 3,500 (inference only) and 300 system-level simulations (training and inference) were performed for benchmarking the system performance of the proposed implementations. The analysis includes Monte Carlo simulations for the variability of memristors' conductance, and crossbar parasitic, where non-idealities of hybrid CMOS-memristor circuits are taken into the account.

Original languageEnglish
Article number705050
JournalFRONTIERS IN COMPUTATIONAL NEUROSCIENCE
Volume15
DOIs
Publication statusPublished - 28 Sept 2021
MoE publication typeA1 Journal article-refereed

Keywords

  • analog LSTM
  • crossbar
  • general-purpose LSTM
  • memristors
  • neural networks

Fingerprint

Dive into the research topics of 'Generalised Analog LSTMs Recurrent Modules for Neural Computing'. Together they form a unique fingerprint.

Cite this