LongEval: Longitudinal Evaluation of Model Performance at CLEF 2024

Rabab Alkhalifa, Hsuvas Borkakoty, Romain Deveaud, Alaa El-Ebshihy, Luis Espinosa-Anke, Tobias Fink, Gabriela Gonzalez-Saez, Petra Galuščáková, Lorraine Goeuriot, David Iommi, Maria Liakata, Harish Tayyar Madabushi, Pablo Medina-Alias, Philippe Mulhem, Florina Piroi, Martin Popel, Christophe Servan, Arkaitz Zubiaga

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

Abstract

This paper introduces the planned second LongEval Lab, part of the CLEF 2024 conference. The aim of the lab’s two tasks is to give researchers test data for addressing temporal effectiveness persistence challenges in both information retrieval and text classification, motivated by the fact that model performance degrades as the test data becomes temporally distant from the training data. LongEval distinguishes itself from traditional IR and classification tasks by emphasizing the evaluation of models designed to mitigate performance drop over time using evolving data. The second LongEval edition will further engage the IR community and NLP researchers in addressing the crucial challenge of temporal persistence in models, exploring the factors that enable or hinder it, and identifying potential solutions along with their limitations.

Original languageEnglish
Title of host publicationAdvances in Information Retrieval - 46th European Conference on Information Retrieval, ECIR 2024, Proceedings
EditorsNazli Goharian, Nicola Tonellotto, Yulan He, Aldo Lipani, Graham McDonald, Craig Macdonald, Iadh Ounis
Place of PublicationCham, Switzerland
PublisherSpringer Science and Business Media Deutschland GmbH
Pages60-66
Number of pages7
ISBN (Electronic)9783031560729
ISBN (Print)9783031560712
DOIs
Publication statusPublished - 20 Mar 2024
Event46th European Conference on Information Retrieval, ECIR 2024 - Glasgow, UK United Kingdom
Duration: 24 Mar 202428 Mar 2024

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14613 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference46th European Conference on Information Retrieval, ECIR 2024
Country/TerritoryUK United Kingdom
CityGlasgow
Period24/03/2428/03/24

Keywords

  • Evaluation
  • Information Retrieval
  • Temporal Generalisability
  • Temporal Persistence
  • Text Classification

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'LongEval: Longitudinal Evaluation of Model Performance at CLEF 2024'. Together they form a unique fingerprint.

Cite this