Decoding Neural Activity for Part-of-Speech Tagging (POS)

Salman Ahmed; Muskaan Singh; Saugat Bhattacharyya; Damien Coyle

doi:10.1109/SMC53992.2023.10394253

Decoding Neural Activity for Part-of-Speech Tagging (POS)

Salman Ahmed, Muskaan Singh, Saugat Bhattacharyya, Damien Coyle

Ulster University

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

Abstract

Decoding Part of Speech(POS) tagging directly from electroencephalography (EEG) signals whilst user overtly spoke (voiced speech) sentences could improve direct speech brain-computer interfaces (BCIs) using imagined or inner speech. To the best of our knowledge, earlier work uses machine learning approach using 74,953 sentences/tokens recorded in 75 EEG sessions. The tokens can be found in 4,479 phrases consisting of terms from the English Online treebank which contains the record of weblogs, newsgroups, reviews, and Yahoo Answers. The results demonstrated the feasibility of POS decoding from EEG based on word class, word frequency, and word length with accuracy of 71%, 86%, 89%, respectively. We believe that there is significant room for improvement with more advanced artificial intelligence. In this paper, we further extend the existing work with end-to-end transformers. Our results presents transformer model outperforms benchmark traditional ML results with +20% in length, +13% for the open vs closed class and +12% in frequency. In our empirical analysis, we find the decoding performance was better when using multi-electrode recordings as compared to single-electrode recordings.

Original language	English
Title of host publication	2023 IEEE International Conference on Systems, Man, and Cybernetics
Subtitle of host publication	Improving the Quality of Life, SMC 2023 - Proceedings
Place of Publication	U. S. A.
Publisher	IEEE
Pages	3079-3084
Number of pages	6
ISBN (Electronic)	9798350337020
ISBN (Print)	9798350337037
DOIs	https://doi.org/10.1109/SMC53992.2023.10394253
Publication status	Published - 4 Oct 2023
Event	2023 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2023 - Hybrid, Honolulu, USA United States Duration: 1 Oct 2023 → 4 Oct 2023

Publication series

Name	Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
ISSN (Print)	1062-922X

Conference

Conference	2023 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2023
Country/Territory	USA United States
City	Hybrid, Honolulu
Period	1/10/23 → 4/10/23

Funding

ACKNOWLEDGMENT I would like to express my sincere gratitude to Alex Murphy for generously sharing the dataset and granting permission to build upon his work [1]. This work was supported by a research grant from the Department for the Economy Northern Ireland under the US-Ireland R&D Partnership Programme (USI-207) VI. CONCLUSION The study is focused to reproduced and improve the initial results reported in base paper[1]. We found that the classification accuracy for decoding part of speech from EEG signals was significantly higher than chance levels, indicating that the EEG signals contain useful information about the syntactic structure of language. The classification accuracy was highest for nouns and verbs, typically associated with more distinctive neural processing than other parts of speech. Additionally, the study found that the decoding accuracy was affected by various factors, such as word class, word frequency and sentence length. Our results indicate that the deep learning model transformer model outperformed traditional SVM for word frequency, length and class. Specifically, shorter words and more frequent words were associated with higher decoding accuracy, while longer sentences were associated with lower decoding accuracy. These findings suggest that EEG signals can be used to decode part-of-speech information in real-time, potentially enabling the development of novel brain-computer interfaces for language processing and communication.

Funders	Funder number
US-Ireland R&D Partnership Programme	USI-207
University of Bath

ASJC Scopus subject areas

Electrical and Electronic Engineering
Control and Systems Engineering
Human-Computer Interaction

Access to Document

10.1109/SMC53992.2023.10394253

Cite this

Ahmed, S., Singh, M., Bhattacharyya, S., & Coyle, D. (2023). Decoding Neural Activity for Part-of-Speech Tagging (POS). In 2023 IEEE International Conference on Systems, Man, and Cybernetics: Improving the Quality of Life, SMC 2023 - Proceedings (pp. 3079-3084). (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics). IEEE. https://doi.org/10.1109/SMC53992.2023.10394253

Decoding Neural Activity for Part-of-Speech Tagging (POS). / Ahmed, Salman; Singh, Muskaan; Bhattacharyya, Saugat et al.
2023 IEEE International Conference on Systems, Man, and Cybernetics: Improving the Quality of Life, SMC 2023 - Proceedings. U. S. A.: IEEE, 2023. p. 3079-3084 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics).

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

Ahmed, S, Singh, M, Bhattacharyya, S & Coyle, D 2023, Decoding Neural Activity for Part-of-Speech Tagging (POS). in 2023 IEEE International Conference on Systems, Man, and Cybernetics: Improving the Quality of Life, SMC 2023 - Proceedings. Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, IEEE, U. S. A., pp. 3079-3084, 2023 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2023, Hybrid, Honolulu, USA United States, 1/10/23. https://doi.org/10.1109/SMC53992.2023.10394253

Ahmed S, Singh M, Bhattacharyya S, Coyle D. Decoding Neural Activity for Part-of-Speech Tagging (POS). In 2023 IEEE International Conference on Systems, Man, and Cybernetics: Improving the Quality of Life, SMC 2023 - Proceedings. U. S. A.: IEEE. 2023. p. 3079-3084. (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics). doi: 10.1109/SMC53992.2023.10394253

Ahmed, Salman ; Singh, Muskaan ; Bhattacharyya, Saugat et al. / Decoding Neural Activity for Part-of-Speech Tagging (POS). 2023 IEEE International Conference on Systems, Man, and Cybernetics: Improving the Quality of Life, SMC 2023 - Proceedings. U. S. A. : IEEE, 2023. pp. 3079-3084 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics).

@inproceedings{a765fe3daba041f8b6182b2fb101e3e5,

title = "Decoding Neural Activity for Part-of-Speech Tagging (POS)",

abstract = "Decoding Part of Speech(POS) tagging directly from electroencephalography (EEG) signals whilst user overtly spoke (voiced speech) sentences could improve direct speech brain-computer interfaces (BCIs) using imagined or inner speech. To the best of our knowledge, earlier work uses machine learning approach using 74,953 sentences/tokens recorded in 75 EEG sessions. The tokens can be found in 4,479 phrases consisting of terms from the English Online treebank which contains the record of weblogs, newsgroups, reviews, and Yahoo Answers. The results demonstrated the feasibility of POS decoding from EEG based on word class, word frequency, and word length with accuracy of 71%, 86%, 89%, respectively. We believe that there is significant room for improvement with more advanced artificial intelligence. In this paper, we further extend the existing work with end-to-end transformers. Our results presents transformer model outperforms benchmark traditional ML results with +20% in length, +13% for the open vs closed class and +12% in frequency. In our empirical analysis, we find the decoding performance was better when using multi-electrode recordings as compared to single-electrode recordings.",

author = "Salman Ahmed and Muskaan Singh and Saugat Bhattacharyya and Damien Coyle",

year = "2023",

month = oct,

day = "4",

doi = "10.1109/SMC53992.2023.10394253",

language = "English",

isbn = "9798350337037",

series = "Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics",

publisher = "IEEE",

pages = "3079--3084",

booktitle = "2023 IEEE International Conference on Systems, Man, and Cybernetics",

address = "USA United States",

note = "2023 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2023 ; Conference date: 01-10-2023 Through 04-10-2023",

}

TY - GEN

T1 - Decoding Neural Activity for Part-of-Speech Tagging (POS)

AU - Ahmed, Salman

AU - Singh, Muskaan

AU - Bhattacharyya, Saugat

AU - Coyle, Damien

PY - 2023/10/4

Y1 - 2023/10/4

N2 - Decoding Part of Speech(POS) tagging directly from electroencephalography (EEG) signals whilst user overtly spoke (voiced speech) sentences could improve direct speech brain-computer interfaces (BCIs) using imagined or inner speech. To the best of our knowledge, earlier work uses machine learning approach using 74,953 sentences/tokens recorded in 75 EEG sessions. The tokens can be found in 4,479 phrases consisting of terms from the English Online treebank which contains the record of weblogs, newsgroups, reviews, and Yahoo Answers. The results demonstrated the feasibility of POS decoding from EEG based on word class, word frequency, and word length with accuracy of 71%, 86%, 89%, respectively. We believe that there is significant room for improvement with more advanced artificial intelligence. In this paper, we further extend the existing work with end-to-end transformers. Our results presents transformer model outperforms benchmark traditional ML results with +20% in length, +13% for the open vs closed class and +12% in frequency. In our empirical analysis, we find the decoding performance was better when using multi-electrode recordings as compared to single-electrode recordings.

AB - Decoding Part of Speech(POS) tagging directly from electroencephalography (EEG) signals whilst user overtly spoke (voiced speech) sentences could improve direct speech brain-computer interfaces (BCIs) using imagined or inner speech. To the best of our knowledge, earlier work uses machine learning approach using 74,953 sentences/tokens recorded in 75 EEG sessions. The tokens can be found in 4,479 phrases consisting of terms from the English Online treebank which contains the record of weblogs, newsgroups, reviews, and Yahoo Answers. The results demonstrated the feasibility of POS decoding from EEG based on word class, word frequency, and word length with accuracy of 71%, 86%, 89%, respectively. We believe that there is significant room for improvement with more advanced artificial intelligence. In this paper, we further extend the existing work with end-to-end transformers. Our results presents transformer model outperforms benchmark traditional ML results with +20% in length, +13% for the open vs closed class and +12% in frequency. In our empirical analysis, we find the decoding performance was better when using multi-electrode recordings as compared to single-electrode recordings.

UR - http://www.scopus.com/inward/record.url?scp=85187306146&partnerID=8YFLogxK

U2 - 10.1109/SMC53992.2023.10394253

DO - 10.1109/SMC53992.2023.10394253

M3 - Chapter in a published conference proceeding

AN - SCOPUS:85187306146

SN - 9798350337037

T3 - Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics

SP - 3079

EP - 3084

BT - 2023 IEEE International Conference on Systems, Man, and Cybernetics

PB - IEEE

CY - U. S. A.

T2 - 2023 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2023

Y2 - 1 October 2023 through 4 October 2023

ER -

Decoding Neural Activity for Part-of-Speech Tagging (POS)

Abstract

Publication series

Conference

Funding

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this