Bimodal electroencephalography-functional magnetic resonance imaging dataset for inner-speech recognition

Foteini Simistira Liwicki; Vibha Gupta; Rajkumar Saini; Kanjar De; Nosheen Abid; Sumit Rakesh; Scott Wellington; Holly Wilson; Marcus Liwicki; Johan Eriksson

doi:10.1038/s41597-023-02286-w

Bimodal electroencephalography-functional magnetic resonance imaging dataset for inner-speech recognition

Foteini Simistira Liwicki, Vibha Gupta, Rajkumar Saini, Kanjar De, Nosheen Abid, Sumit Rakesh, Scott Wellington, Holly Wilson, Marcus Liwicki, Johan Eriksson

Research output: Contribution to journal › Article › peer-review

9 Citations (SciVal)

Abstract

The recognition of inner speech, which could give a ‘voice’ to patients that have no ability to speak or move, is a challenge for brain-computer interfaces (BCIs). A shortcoming of the available datasets is that they do not combine modalities to increase the performance of inner speech recognition. Multimodal datasets of brain data enable the fusion of neuroimaging modalities with complimentary properties, such as the high spatial resolution of functional magnetic resonance imaging (fMRI) and the temporal resolution of electroencephalography (EEG), and therefore are promising for decoding inner speech. This paper presents the first publicly available bimodal dataset containing EEG and fMRI data acquired nonsimultaneously during inner-speech production. Data were obtained from four healthy, right-handed participants during an inner-speech task with words in either a social or numerical category. Each of the 8-word stimuli were assessed with 40 trials, resulting in 320 trials in each modality for each participant. The aim of this work is to provide a publicly available bimodal dataset on inner speech, contributing towards speech prostheses.

Original language	English
Article number	378
Journal	Scientific Data
Volume	10
Issue number	1
Early online date	13 Jun 2023
DOIs	https://doi.org/10.1038/s41597-023-02286-w
Publication status	Published - 31 Dec 2023

Funding

This research was funded by the Grants for Excellent Research Projects Proposals of SRT.ai 2022. We would like to thank the Stockholm University Brain Imaging Centre (SUBIC) and, in particular, Rita Almeida and Patrik Andersson for giving us access to their facilities and for supporting us in this endeavour. We thank Petter Kallioinen and Christoffer Schiehe-Forbes for their valuable support during the acquisition of EEG data. Finally, we would also like to thank all participants for taking part in this study.

ASJC Scopus subject areas

Statistics and Probability
Information Systems
Education
Computer Science Applications
Statistics, Probability and Uncertainty
Library and Information Sciences

Access to Document

10.1038/s41597-023-02286-wLicence: CC BY

Cite this

@article{497bc430c09541d7af0fef86659442bf,

title = "Bimodal electroencephalography-functional magnetic resonance imaging dataset for inner-speech recognition",

abstract = "The recognition of inner speech, which could give a {\textquoteleft}voice{\textquoteright} to patients that have no ability to speak or move, is a challenge for brain-computer interfaces (BCIs). A shortcoming of the available datasets is that they do not combine modalities to increase the performance of inner speech recognition. Multimodal datasets of brain data enable the fusion of neuroimaging modalities with complimentary properties, such as the high spatial resolution of functional magnetic resonance imaging (fMRI) and the temporal resolution of electroencephalography (EEG), and therefore are promising for decoding inner speech. This paper presents the first publicly available bimodal dataset containing EEG and fMRI data acquired nonsimultaneously during inner-speech production. Data were obtained from four healthy, right-handed participants during an inner-speech task with words in either a social or numerical category. Each of the 8-word stimuli were assessed with 40 trials, resulting in 320 trials in each modality for each participant. The aim of this work is to provide a publicly available bimodal dataset on inner speech, contributing towards speech prostheses.",

author = "{Simistira Liwicki}, Foteini and Vibha Gupta and Rajkumar Saini and Kanjar De and Nosheen Abid and Sumit Rakesh and Scott Wellington and Holly Wilson and Marcus Liwicki and Johan Eriksson",

year = "2023",

month = dec,

day = "31",

doi = "10.1038/s41597-023-02286-w",

language = "English",

volume = "10",

journal = "Scientific Data",

issn = "2052-4463",

publisher = "Nature Research",

number = "1",

}

TY - JOUR

T1 - Bimodal electroencephalography-functional magnetic resonance imaging dataset for inner-speech recognition

AU - Simistira Liwicki, Foteini

AU - Gupta, Vibha

AU - Saini, Rajkumar

AU - De, Kanjar

AU - Abid, Nosheen

AU - Rakesh, Sumit

AU - Wellington, Scott

AU - Wilson, Holly

AU - Liwicki, Marcus

AU - Eriksson, Johan

PY - 2023/12/31

Y1 - 2023/12/31

N2 - The recognition of inner speech, which could give a ‘voice’ to patients that have no ability to speak or move, is a challenge for brain-computer interfaces (BCIs). A shortcoming of the available datasets is that they do not combine modalities to increase the performance of inner speech recognition. Multimodal datasets of brain data enable the fusion of neuroimaging modalities with complimentary properties, such as the high spatial resolution of functional magnetic resonance imaging (fMRI) and the temporal resolution of electroencephalography (EEG), and therefore are promising for decoding inner speech. This paper presents the first publicly available bimodal dataset containing EEG and fMRI data acquired nonsimultaneously during inner-speech production. Data were obtained from four healthy, right-handed participants during an inner-speech task with words in either a social or numerical category. Each of the 8-word stimuli were assessed with 40 trials, resulting in 320 trials in each modality for each participant. The aim of this work is to provide a publicly available bimodal dataset on inner speech, contributing towards speech prostheses.

AB - The recognition of inner speech, which could give a ‘voice’ to patients that have no ability to speak or move, is a challenge for brain-computer interfaces (BCIs). A shortcoming of the available datasets is that they do not combine modalities to increase the performance of inner speech recognition. Multimodal datasets of brain data enable the fusion of neuroimaging modalities with complimentary properties, such as the high spatial resolution of functional magnetic resonance imaging (fMRI) and the temporal resolution of electroencephalography (EEG), and therefore are promising for decoding inner speech. This paper presents the first publicly available bimodal dataset containing EEG and fMRI data acquired nonsimultaneously during inner-speech production. Data were obtained from four healthy, right-handed participants during an inner-speech task with words in either a social or numerical category. Each of the 8-word stimuli were assessed with 40 trials, resulting in 320 trials in each modality for each participant. The aim of this work is to provide a publicly available bimodal dataset on inner speech, contributing towards speech prostheses.

UR - http://www.scopus.com/inward/record.url?scp=85161923014&partnerID=8YFLogxK

U2 - 10.1038/s41597-023-02286-w

DO - 10.1038/s41597-023-02286-w

M3 - Article

C2 - 37311807

AN - SCOPUS:85161923014

SN - 2052-4463

VL - 10

JO - Scientific Data

JF - Scientific Data

IS - 1

M1 - 378

ER -

Bimodal electroencephalography-functional magnetic resonance imaging dataset for inner-speech recognition

Abstract

Funding

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this