Dynamic machine vision with retinomorphic photomemristor-reservoir computing

Hongwei Tan; Sebastiaan van Dijken

doi:10.1038/s41467-023-37886-y

Dynamic machine vision with retinomorphic photomemristor-reservoir computing

Hongwei Tan^*
, Sebastiaan van Dijken^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › Scientific › peer-review

107 Citations (Scopus)

81 Downloads (Pure)

Abstract

Dynamic machine vision requires recognizing the past and predicting the future of a moving object based on present vision. Current machine vision systems accomplish this by processing numerous image frames or using complex algorithms. Here, we report motion recognition and prediction in recurrent photomemristor networks. In our system, a retinomorphic photomemristor array, working as dynamic vision reservoir, embeds past motion frames as hidden states into the present frame through inherent dynamic memory. The informative present frame facilitates accurate recognition of past and prediction of future motions with machine learning algorithms. This in-sensor motion processing capability eliminates redundant data flows and promotes real-time perception of moving objects for dynamic machine vision.

Original language	English
Article number	2169
Pages (from-to)	1-9
Number of pages	9
Journal	Nature Communications
Volume	14
Issue number	1
DOIs	https://doi.org/10.1038/s41467-023-37886-y
Publication status	Published - Dec 2023
MoE publication type	A1 Journal article-refereed

Funding

We gratefully acknowledge E.I. Kauppinen for providing infrastructure support for the electrical measurements. H. Tan. thanks, R. He for inspiration and discussion on the main concept. We acknowledge H. Qin and Y. Zhou for their fruitful discussions and contributions to coding. The project made use of the OtaNano—Micronova Nanofabrication Center and the OtaNano—Nanomicroscopy Center, supported by Aalto University. This work was supported by the Academy of Finland (Grant no. 316973 H. T. and 13293916 S.v.D.).

Access to Document

10.1038/s41467-023-37886-y

Dynamic machine vision with retinomorphic photomemristor-reservoir computingFinal published version, 2.43 MBLicence: CC BY

Cite this

@article{bffcc838539140e381552063b49e1644,

title = "Dynamic machine vision with retinomorphic photomemristor-reservoir computing",

abstract = "Dynamic machine vision requires recognizing the past and predicting the future of a moving object based on present vision. Current machine vision systems accomplish this by processing numerous image frames or using complex algorithms. Here, we report motion recognition and prediction in recurrent photomemristor networks. In our system, a retinomorphic photomemristor array, working as dynamic vision reservoir, embeds past motion frames as hidden states into the present frame through inherent dynamic memory. The informative present frame facilitates accurate recognition of past and prediction of future motions with machine learning algorithms. This in-sensor motion processing capability eliminates redundant data flows and promotes real-time perception of moving objects for dynamic machine vision.",

author = "Hongwei Tan and \{van Dijken\}, Sebastiaan",

note = "Funding Information: We gratefully acknowledge E.I. Kauppinen for providing infrastructure support for the electrical measurements. H. Tan. thanks, R. He for inspiration and discussion on the main concept. We acknowledge H. Qin and Y. Zhou for their fruitful discussions and contributions to coding. The project made use of the OtaNano—Micronova Nanofabrication Center and the OtaNano—Nanomicroscopy Center, supported by Aalto University. This work was supported by the Academy of Finland (Grant no. 316973 H. T. and 13293916 S.v.D.). Publisher Copyright: {\textcopyright} 2023, The Author(s).",

year = "2023",

month = dec,

doi = "10.1038/s41467-023-37886-y",

language = "English",

volume = "14",

pages = "1--9",

journal = "Nature Communications",

issn = "2041-1723",

publisher = "Nature Publishing Group",

number = "1",

}

TY - JOUR

T1 - Dynamic machine vision with retinomorphic photomemristor-reservoir computing

AU - Tan, Hongwei

AU - van Dijken, Sebastiaan

N1 - Funding Information: We gratefully acknowledge E.I. Kauppinen for providing infrastructure support for the electrical measurements. H. Tan. thanks, R. He for inspiration and discussion on the main concept. We acknowledge H. Qin and Y. Zhou for their fruitful discussions and contributions to coding. The project made use of the OtaNano—Micronova Nanofabrication Center and the OtaNano—Nanomicroscopy Center, supported by Aalto University. This work was supported by the Academy of Finland (Grant no. 316973 H. T. and 13293916 S.v.D.). Publisher Copyright: © 2023, The Author(s).

PY - 2023/12

Y1 - 2023/12

N2 - Dynamic machine vision requires recognizing the past and predicting the future of a moving object based on present vision. Current machine vision systems accomplish this by processing numerous image frames or using complex algorithms. Here, we report motion recognition and prediction in recurrent photomemristor networks. In our system, a retinomorphic photomemristor array, working as dynamic vision reservoir, embeds past motion frames as hidden states into the present frame through inherent dynamic memory. The informative present frame facilitates accurate recognition of past and prediction of future motions with machine learning algorithms. This in-sensor motion processing capability eliminates redundant data flows and promotes real-time perception of moving objects for dynamic machine vision.

AB - Dynamic machine vision requires recognizing the past and predicting the future of a moving object based on present vision. Current machine vision systems accomplish this by processing numerous image frames or using complex algorithms. Here, we report motion recognition and prediction in recurrent photomemristor networks. In our system, a retinomorphic photomemristor array, working as dynamic vision reservoir, embeds past motion frames as hidden states into the present frame through inherent dynamic memory. The informative present frame facilitates accurate recognition of past and prediction of future motions with machine learning algorithms. This in-sensor motion processing capability eliminates redundant data flows and promotes real-time perception of moving objects for dynamic machine vision.

UR - https://www.scopus.com/pages/publications/85152555029

U2 - 10.1038/s41467-023-37886-y

DO - 10.1038/s41467-023-37886-y

M3 - Article

C2 - 37061543

AN - SCOPUS:85152555029

SN - 2041-1723

VL - 14

SP - 1

EP - 9

JO - Nature Communications

JF - Nature Communications

IS - 1

M1 - 2169

ER -

Dynamic machine vision with retinomorphic photomemristor-reservoir computing

Abstract

Funding

Access to Document

Other files and links

Fingerprint

Optoelectronic Synapses for Artificial Neuro Networks

COIN: Complex Oxide Interfaces for Nanoelectronics (COIN)

OtaNano

OtaNano - Nanofab

OtaNano - Nanomicroscopy Center

A neuromorphic visual sensor can recognize moving objects and predict their path

Reports on Science Findings from Aalto University School of Science and Technology Provide New Insights (Dynamic machine vision with retinomorphic photomemristor-reservoir computing)

Neuromorphic Sensor Recognizes Motion, Predicts Path

Cite this

Dynamic machine vision with retinomorphic photomemristor-reservoir computing

Abstract

Funding

Access to Document

Other files and links

Fingerprint

Projects

Equipment

Press/Media

Cite this