On the disagreement of forecasting model selection criteria

Evangelos Spiliotis, Fotios Petropoulos, Vassilios Assimakopoulos

Research output: Contribution to journalArticlepeer-review

1 Citation (SciVal)
34 Downloads (Pure)

Abstract

Forecasters have been using various criteria to select the most appropriate model from a pool of candidate models. This includes measurements on the in-sample accuracy of the models, information criteria, and cross-validation, among others. Although the latter two options are generally preferred due to their ability to tackle overfitting, in univariate time-series forecasting settings, limited work has been conducted to confirm their superiority. In this study, we compared such popular criteria for the case of the exponential smoothing family of models using a large data set of real series. Our results suggest that there is significant disagreement between the suggestions of the examined criteria and that, depending on the approach used, models of different complexity may be favored, with possible negative effects on the forecasting accuracy. Moreover, we find that simple in-sample error measures can effectively select forecasting models, especially when focused on the most recent observations in the series.
Original languageEnglish
Pages (from-to)487-498
Number of pages12
JournalForecasting
Volume5
Issue number2
DOIs
Publication statusPublished - 20 Jun 2023

Bibliographical note

Funding
This research received no external funding.

Data Availability Statement
The data used in this research are available at the GitHub repository of the M4 competition https://github.com/Mcompetitions/M4-methods (accessed on 16 May 2023).

Keywords

  • M4 competition
  • exponential smoothing
  • information criteria
  • model selection
  • time series

ASJC Scopus subject areas

  • Decision Sciences (miscellaneous)
  • Economics, Econometrics and Finance (miscellaneous)
  • Computer Science Applications
  • Computational Theory and Mathematics

Fingerprint

Dive into the research topics of 'On the disagreement of forecasting model selection criteria'. Together they form a unique fingerprint.

Cite this