Simple Regression Models

Jan Lichtenberg, Özgür Şimşek

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding


Developing theories of when and why simple predictive models perform well is a key step in understanding decisions of cognitively bounded humans and intelligent machines. We are interested in how well simple models predict in regression. We list and review existing simple regression models and define new ones. We identify the lack of a large-scale empirical comparison of these models with state-of-the-art regression models in a predictive regression context. We report the results of such an empirical analysis on 60 real-world data sets. Simple regression models such as equal-weights regression routinely outperformed state-of-the-art regression models, especially on small training-set sizes. There was no simple model that predicted well in all data sets, but in nearly all data sets, there was at least one simple model that predicted well. The supplementary material contains learning curves for individual data sets that have not been presented in the main article. It also contains detailed descriptions and source descriptions of all used data sets.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 665992
Original languageEnglish
Title of host publicationProceedings of the NIPS 2016 Workshop on Imperfect Decision Makers
Publication statusPublished - 14 Aug 2017

Publication series

NameProceedings of Machine Learning Research
ISSN (Electronic)2640-3498

ASJC Scopus subject areas

  • Artificial Intelligence
  • Decision Sciences (miscellaneous)


Dive into the research topics of 'Simple Regression Models'. Together they form a unique fingerprint.

Cite this