Comparative judgments are more consistent than binary classification for labelling word complexity

Sian Gooding; Ekaterina Kochmar; Alan Blackwell; Advait Sarkar

Comparative judgments are more consistent than binary classification for labelling word complexity

Sian Gooding, Ekaterina Kochmar, Alan Blackwell, Advait Sarkar

Department of Computer Science

Research output: Contribution to conference › Paper › peer-review

5 Citations (SciVal)

Abstract

Lexical simplification systems replace complex words with simple ones based on a model of which words are complex in context. We explore how users can help train complex word identification models through labelling more efficiently and reliably. We show that using an interface where annotators make comparative rather than binary judgments leads to more reliable and consistent labels, and explore whether comparative judgments may provide a faster way for collecting labels.

Original language	English
Pages	208–214
Number of pages	6
Publication status	Published - 1 Aug 2019
Event	Proceedings of the 13th Linguistic Annotation Workshop: LAW XIII, ACL 2019 - Florence, Italy, Florence, Italy Duration: 1 Aug 2019 → 1 Aug 2019 https://sigann.github.io/LAW-XIII-2019/

Workshop

Workshop	Proceedings of the 13th Linguistic Annotation Workshop
Abbreviated title	LAW XIII
Country/Territory	Italy
City	Florence
Period	1/08/19 → 1/08/19
Internet address	https://sigann.github.io/LAW-XIII-2019/

Access to Document

https://www.aclweb.org/anthology/W19-4024.pdfLicence: CC BY

Cite this

@conference{20b40f27b17b4832a097beb6a7960cd4,

title = "Comparative judgments are more consistent than binary classification for labelling word complexity",

abstract = "Lexical simplification systems replace complex words with simple ones based on a model of which words are complex in context. We explore how users can help train complex word identification models through labelling more efficiently and reliably. We show that using an interface where annotators make comparative rather than binary judgments leads to more reliable and consistent labels, and explore whether comparative judgments may provide a faster way for collecting labels.",

author = "Sian Gooding and Ekaterina Kochmar and Alan Blackwell and Advait Sarkar",

year = "2019",

month = aug,

day = "1",

language = "English",

pages = "208–214",

note = "Proceedings of the 13th Linguistic Annotation Workshop : LAW XIII, ACL 2019, LAW XIII ; Conference date: 01-08-2019 Through 01-08-2019",

url = "https://sigann.github.io/LAW-XIII-2019/",

}

TY - CONF

T1 - Comparative judgments are more consistent than binary classification for labelling word complexity

AU - Gooding, Sian

AU - Kochmar, Ekaterina

AU - Blackwell, Alan

AU - Sarkar, Advait

PY - 2019/8/1

Y1 - 2019/8/1

N2 - Lexical simplification systems replace complex words with simple ones based on a model of which words are complex in context. We explore how users can help train complex word identification models through labelling more efficiently and reliably. We show that using an interface where annotators make comparative rather than binary judgments leads to more reliable and consistent labels, and explore whether comparative judgments may provide a faster way for collecting labels.

AB - Lexical simplification systems replace complex words with simple ones based on a model of which words are complex in context. We explore how users can help train complex word identification models through labelling more efficiently and reliably. We show that using an interface where annotators make comparative rather than binary judgments leads to more reliable and consistent labels, and explore whether comparative judgments may provide a faster way for collecting labels.

M3 - Paper

SP - 208

EP - 214

T2 - Proceedings of the 13th Linguistic Annotation Workshop

Y2 - 1 August 2019 through 1 August 2019

ER -

Comparative judgments are more consistent than binary classification for labelling word complexity

Abstract

Workshop

Access to Document

Fingerprint

Cite this