TY - UNPB
T1 - ComplexityMT
T2 - Benchmarking the Interaction Between Text Complexity and Machine Translation
AU - Imperial, Joseph Marvin
AU - Liang, Junhong
AU - Shoer, Belal
AU - Barayan, Abdullah
AU - Wilkens, Rodrigo
AU - Mussa, Omar
AU - Knight, Dawn
AU - Ribeiro, Eugénio
AU - Kochmar, Ekaterina
AU - Vajjala, Sowmya
AU - Alva-Manchego, Fernando
AU - Madabushi, Harish Tayyar
PY - 2026/6/3
Y1 - 2026/6/3
N2 - When a text is translated, does the translation retain the complexity of the original? We introduce ComplexityMT, a new challenge for assessing how text complexity and machine translation interact with and influence each other, using the Common European Framework of Reference for Languages (CEFR) levels as the measure of text complexity. Across six languages, including Arabic, Dutch, English, French, Hindi, and Russian, we evaluate three open-weight models, one closed model, and a commercial machine translation system on two tasks: i) correlation of CEFR with translation difficulty, and ii) shifts in CEFR levels of the source texts. Our experiments show that higher CEFR levels make texts more difficult to translate, and that machine translation shifts the CEFR level of the target text compared to the original source, for most languages. These findings provide new insights for researchers and practitioners working on multilingual pedagogical content generation and machine translation difficulty estimation.
AB - When a text is translated, does the translation retain the complexity of the original? We introduce ComplexityMT, a new challenge for assessing how text complexity and machine translation interact with and influence each other, using the Common European Framework of Reference for Languages (CEFR) levels as the measure of text complexity. Across six languages, including Arabic, Dutch, English, French, Hindi, and Russian, we evaluate three open-weight models, one closed model, and a commercial machine translation system on two tasks: i) correlation of CEFR with translation difficulty, and ii) shifts in CEFR levels of the source texts. Our experiments show that higher CEFR levels make texts more difficult to translate, and that machine translation shifts the CEFR level of the target text compared to the original source, for most languages. These findings provide new insights for researchers and practitioners working on multilingual pedagogical content generation and machine translation difficulty estimation.
KW - cs.CL
M3 - Preprint
BT - ComplexityMT
PB - arXiv
ER -