CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns

Peter Vickers; Rosa Wainwright; Harish Tayyar Madabushi; Aline Villavicencio

doi:10.18653/v1/2021.cmcl-1.16

CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns

Peter Vickers, Rosa Wainwright, Harish Tayyar Madabushi, Aline Villavicencio

Department of Computer Science

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

4 Citations (SciVal)

Abstract

The CogNLP-Sheffield submissions to the CMCL 2021 Shared Task examine the value of a variety of cognitively and linguistically inspired features for predicting eye tracking patterns, as both standalone model inputs and as supplements to contextual word embeddings (XLNet). Surprisingly, the smaller pre-trained model (XLNet-base) outperforms the larger (XLNet-large), and despite evidence that multi-word expressions (MWEs) provide cognitive processing advantages, MWE features provide little benefit to either model.

Original language	English
Title of host publication	Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
Place of Publication	Online
Publisher	Association for Computational Linguistics
Pages	125-133
Number of pages	9
DOIs	https://doi.org/10.18653/v1/2021.cmcl-1.16
Publication status	Published - 1 Jun 2021

Access to Document

10.18653/v1/2021.cmcl-1.16Licence: CC BY

https://aclanthology.org/2021.cmcl-1.16Licence: CC BY

Cite this

Vickers, P., Wainwright, R., Tayyar Madabushi, H., & Villavicencio, A. (2021). CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics (pp. 125-133). Article 2021.cmcl-1.16 Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.cmcl-1.16

CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns. / Vickers, Peter; Wainwright, Rosa; Tayyar Madabushi, Harish et al.
Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics. Online: Association for Computational Linguistics, 2021. p. 125-133 2021.cmcl-1.16.

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

Vickers, P, Wainwright, R, Tayyar Madabushi, H & Villavicencio, A 2021, CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns. in Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics., 2021.cmcl-1.16, Association for Computational Linguistics, Online, pp. 125-133. https://doi.org/10.18653/v1/2021.cmcl-1.16

Vickers P, Wainwright R, Tayyar Madabushi H, Villavicencio A. CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics. Online: Association for Computational Linguistics. 2021. p. 125-133. 2021.cmcl-1.16 doi: 10.18653/v1/2021.cmcl-1.16

Vickers, Peter ; Wainwright, Rosa ; Tayyar Madabushi, Harish et al. / CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns. Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics. Online : Association for Computational Linguistics, 2021. pp. 125-133

@inproceedings{e596719bf78842fbb1f9c71d339fd210,

title = "CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns",

abstract = "The CogNLP-Sheffield submissions to the CMCL 2021 Shared Task examine the value of a variety of cognitively and linguistically inspired features for predicting eye tracking patterns, as both standalone model inputs and as supplements to contextual word embeddings (XLNet). Surprisingly, the smaller pre-trained model (XLNet-base) outperforms the larger (XLNet-large), and despite evidence that multi-word expressions (MWEs) provide cognitive processing advantages, MWE features provide little benefit to either model.",

author = "Peter Vickers and Rosa Wainwright and {Tayyar Madabushi}, Harish and Aline Villavicencio",

year = "2021",

month = jun,

day = "1",

doi = "10.18653/v1/2021.cmcl-1.16",

language = "English",

pages = "125--133",

booktitle = "Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics",

publisher = "Association for Computational Linguistics",

}

TY - GEN

T1 - CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns

AU - Vickers, Peter

AU - Wainwright, Rosa

AU - Tayyar Madabushi, Harish

AU - Villavicencio, Aline

PY - 2021/6/1

Y1 - 2021/6/1

N2 - The CogNLP-Sheffield submissions to the CMCL 2021 Shared Task examine the value of a variety of cognitively and linguistically inspired features for predicting eye tracking patterns, as both standalone model inputs and as supplements to contextual word embeddings (XLNet). Surprisingly, the smaller pre-trained model (XLNet-base) outperforms the larger (XLNet-large), and despite evidence that multi-word expressions (MWEs) provide cognitive processing advantages, MWE features provide little benefit to either model.

AB - The CogNLP-Sheffield submissions to the CMCL 2021 Shared Task examine the value of a variety of cognitively and linguistically inspired features for predicting eye tracking patterns, as both standalone model inputs and as supplements to contextual word embeddings (XLNet). Surprisingly, the smaller pre-trained model (XLNet-base) outperforms the larger (XLNet-large), and despite evidence that multi-word expressions (MWEs) provide cognitive processing advantages, MWE features provide little benefit to either model.

U2 - 10.18653/v1/2021.cmcl-1.16

DO - 10.18653/v1/2021.cmcl-1.16

M3 - Chapter in a published conference proceeding

SP - 125

EP - 133

BT - Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics

PB - Association for Computational Linguistics

CY - Online

ER -

CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns

Abstract

Access to Document

Fingerprint

Cite this