Computational neuromorphic imaging

A new paradigm to shape conventional optical imaging

Abstract

Computational neuromorphic imaging (CNI), which integrates event cameras, optics, and computational models, represents a promising frontier in optical imaging. The CNI technique makes use of event sensors that encompass time-efficient imaging, high dynamic range reconstruction, and high-sensitivity sensing, and is ideally suitable for detecting ultrafast dynamic information and is robust to challenging environments. CNI encompasses ultrafast dynamic analysis, high-sensitivity sensing, and energy efficiency, offering the transformative potential for academic research and industrial applications from micro to macro settings.

Introduction

A generic computational neuromorphic imaging system.

Technical Framework

Neuromorphic imaging with Super-resolution.

Neuromorphic imaging is an emerging technique that imitates the human retina to sense variations in dynamic scenes. It responds to pixel-level brightness changes by asynchronous streaming events and boasts microsecond temporal precision over a high dynamic range, yielding blur-free recordings under extreme illumination. Nevertheless, this modality falls short in spatial resolution and leads to a low level of visual richness and clarity. Pursuing hardware upgrades is expensive and might cause compromised performance due to more burdens on computational requirements. Another option is to harness offline, plug-in-play super-resolution solutions. However, existing ones, which demand substantial sample volumes for lengthy training on massive computing resources, are largely restricted by real data availability owing to the current imperfect highresolution devices, as well as the randomness and variability of motion. To tackle these challenges, we introduce the first selfsupervised neuromorphic super-resolution prototype¹. It can be self-adaptive to per input source from any low-resolution camera to estimate an optimal, high-resolution counterpart of any scale, without the need of side knowledge and prior training.

Research Achievements

URL: Collobration

Related Publications