An adaptive solid-state synapse with bi-directional relaxation for multimodal recognition and spatiotemporal learning

The brain’s unique processing power, such as perception, understanding, and interaction with the multimodal world, is achieved through diverse synaptic functionalities, which include varied temporal responses and adaptation. Although specific functions in brain-like computing have been successfully realized, emulating multimodal recognition and spatio-temporal learning remain significant challenges due to the difficulties in achieving multimodal signal processing and adaptive long-term plasticity in a single electronic synapse. Here, we propose a purely electrically-modulated ferroelectric tunnel junction (FTJ) memristive synapse which realizes multimodal recognition and spatio-temporal pattern identification, through the integration of oxygen vacancies migration and ferroelectric polarization switching mechanisms, providing bi-directional relaxation and adaptive long-term plasticity simultaneously in the isolated device. The bi-directional relaxation enables multimodal recognition in our purely electrically-modulated FTJ device by encoding distinct sensory signals with different electrical polarities. The multimodal perception task is implemented with a multimodal computing system combining visual and speech pattern recognition. Moreover, the adaptive long-term plasticity allows spatio-temporal pattern recognition, which is demonstrated by identifying object orientation and direction of motion with a neural network incorporating our arrayed synapses. This work provides a feasible approach for designing bio-realistic electronic synapses and achieving highly intelligent neuromorphic computing.