Judgmental Selection of Forecasting Models

Fotios Petropoulos, Nikolaos Kourentzes, Konstantinos Nikolopoulos, Enno Siemsen

Research output: Contribution to journalArticle

Abstract

In this paper, we explored how judgment can be used to improve the selection of a forecasting model. We compared the performance of judgmental model selection against a standard algorithm based on information criteria. We also examined the efficacy of a judgmental model-build approach, in which experts were asked to decide on the existence of the structural components (trend and seasonality) of the time series instead of directly selecting a model from a choice set. Our behavioral study used data from almost 700 participants, including forecasting practitioners. The results from our experiment suggest that selecting models judgmentally results in performance that is on par, if not better, to that of algorithmic selection. Further, judgmental model selection helps to avoid the worst models more frequently compared to algorithmic selection. Finally, a simple combination of the statistical and judgmental selections and judgmental aggregation significantly outperform both statistical and judgmental selections.
LanguageEnglish
JournalJournal of Operations Management
StatusAccepted/In press - 23 May 2018

Fingerprint

Time series
Agglomeration
Model selection
Experiments
Information criterion
Efficacy
Choice sets
Experiment
Seasonality

Keywords

  • model selection
  • behavioral operations
  • decomposition
  • combination

Cite this

Petropoulos, F., Kourentzes, N., Nikolopoulos, K., & Siemsen, E. (2018). Judgmental Selection of Forecasting Models. Journal of Operations Management.

Judgmental Selection of Forecasting Models. / Petropoulos, Fotios; Kourentzes, Nikolaos; Nikolopoulos, Konstantinos; Siemsen, Enno.

In: Journal of Operations Management, 23.05.2018.

Research output: Contribution to journalArticle

Petropoulos F, Kourentzes N, Nikolopoulos K, Siemsen E. Judgmental Selection of Forecasting Models. Journal of Operations Management. 2018 May 23.
Petropoulos, Fotios ; Kourentzes, Nikolaos ; Nikolopoulos, Konstantinos ; Siemsen, Enno. / Judgmental Selection of Forecasting Models. In: Journal of Operations Management. 2018
@article{bd412ff7f4a54303ad4aa4745df65227,
title = "Judgmental Selection of Forecasting Models",
abstract = "In this paper, we explored how judgment can be used to improve the selection of a forecasting model. We compared the performance of judgmental model selection against a standard algorithm based on information criteria. We also examined the efficacy of a judgmental model-build approach, in which experts were asked to decide on the existence of the structural components (trend and seasonality) of the time series instead of directly selecting a model from a choice set. Our behavioral study used data from almost 700 participants, including forecasting practitioners. The results from our experiment suggest that selecting models judgmentally results in performance that is on par, if not better, to that of algorithmic selection. Further, judgmental model selection helps to avoid the worst models more frequently compared to algorithmic selection. Finally, a simple combination of the statistical and judgmental selections and judgmental aggregation significantly outperform both statistical and judgmental selections.",
keywords = "model selection, behavioral operations, decomposition, combination",
author = "Fotios Petropoulos and Nikolaos Kourentzes and Konstantinos Nikolopoulos and Enno Siemsen",
year = "2018",
month = "5",
day = "23",
language = "English",
journal = "Journal of Operations Management",
issn = "0272-6963",
publisher = "Elsevier",

}

TY - JOUR

T1 - Judgmental Selection of Forecasting Models

AU - Petropoulos,Fotios

AU - Kourentzes,Nikolaos

AU - Nikolopoulos,Konstantinos

AU - Siemsen,Enno

PY - 2018/5/23

Y1 - 2018/5/23

N2 - In this paper, we explored how judgment can be used to improve the selection of a forecasting model. We compared the performance of judgmental model selection against a standard algorithm based on information criteria. We also examined the efficacy of a judgmental model-build approach, in which experts were asked to decide on the existence of the structural components (trend and seasonality) of the time series instead of directly selecting a model from a choice set. Our behavioral study used data from almost 700 participants, including forecasting practitioners. The results from our experiment suggest that selecting models judgmentally results in performance that is on par, if not better, to that of algorithmic selection. Further, judgmental model selection helps to avoid the worst models more frequently compared to algorithmic selection. Finally, a simple combination of the statistical and judgmental selections and judgmental aggregation significantly outperform both statistical and judgmental selections.

AB - In this paper, we explored how judgment can be used to improve the selection of a forecasting model. We compared the performance of judgmental model selection against a standard algorithm based on information criteria. We also examined the efficacy of a judgmental model-build approach, in which experts were asked to decide on the existence of the structural components (trend and seasonality) of the time series instead of directly selecting a model from a choice set. Our behavioral study used data from almost 700 participants, including forecasting practitioners. The results from our experiment suggest that selecting models judgmentally results in performance that is on par, if not better, to that of algorithmic selection. Further, judgmental model selection helps to avoid the worst models more frequently compared to algorithmic selection. Finally, a simple combination of the statistical and judgmental selections and judgmental aggregation significantly outperform both statistical and judgmental selections.

KW - model selection

KW - behavioral operations

KW - decomposition

KW - combination

M3 - Article

JO - Journal of Operations Management

T2 - Journal of Operations Management

JF - Journal of Operations Management

SN - 0272-6963

ER -