Why most decisions are easy in Tetris—And perhaps in other sequential decision problems, as well

Özgür Şimşek, Simón Algorta, Amit Kothiyal

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We examined the sequence of decision problems that are encountered in the game of Tetris and found that most of the problems are easy in the following sense: One can choose well among the available actions without knowing an evaluation function that scores well in the game. This is a consequence of three conditions that are prevalent in the game: simple dominance, cumulative dominance, and noncompensation. These conditions can be exploited to develop faster and more effective learning algorithms. In addition, they allow certain types of domain knowledge to be incorporated with ease into a learning algorithm. Among the sequential decision problems we encounter, it is unlikely that Tetris is unique or rare in having these properties.
Original languageEnglish
Title of host publicationProceedings of The 33rd International Conference on Machine Learning
PublisherPMLR
Pages1757-1765
Number of pages9
Volume48
Publication statusE-pub ahead of print - 22 Jun 2016

Publication series

NameProceedings of Machine Learning Research
ISSN (Print)2640-3498

Keywords

  • sequential decision making; reinforcement learning; statistical structure of decision environments

Cite this

Şimşek, Ö., Algorta, S., & Kothiyal, A. (2016). Why most decisions are easy in Tetris—And perhaps in other sequential decision problems, as well. In Proceedings of The 33rd International Conference on Machine Learning (Vol. 48, pp. 1757-1765). (Proceedings of Machine Learning Research). PMLR.

Why most decisions are easy in Tetris—And perhaps in other sequential decision problems, as well. / Şimşek, Özgür; Algorta, Simón; Kothiyal, Amit.

Proceedings of The 33rd International Conference on Machine Learning. Vol. 48 PMLR, 2016. p. 1757-1765 (Proceedings of Machine Learning Research).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Şimşek, Ö, Algorta, S & Kothiyal, A 2016, Why most decisions are easy in Tetris—And perhaps in other sequential decision problems, as well. in Proceedings of The 33rd International Conference on Machine Learning. vol. 48, Proceedings of Machine Learning Research, PMLR, pp. 1757-1765.
Şimşek Ö, Algorta S, Kothiyal A. Why most decisions are easy in Tetris—And perhaps in other sequential decision problems, as well. In Proceedings of The 33rd International Conference on Machine Learning. Vol. 48. PMLR. 2016. p. 1757-1765. (Proceedings of Machine Learning Research).
Şimşek, Özgür ; Algorta, Simón ; Kothiyal, Amit. / Why most decisions are easy in Tetris—And perhaps in other sequential decision problems, as well. Proceedings of The 33rd International Conference on Machine Learning. Vol. 48 PMLR, 2016. pp. 1757-1765 (Proceedings of Machine Learning Research).
@inproceedings{162790aaba984ff7b8469cb232785175,
title = "Why most decisions are easy in Tetris—And perhaps in other sequential decision problems, as well",
abstract = "We examined the sequence of decision problems that are encountered in the game of Tetris and found that most of the problems are easy in the following sense: One can choose well among the available actions without knowing an evaluation function that scores well in the game. This is a consequence of three conditions that are prevalent in the game: simple dominance, cumulative dominance, and noncompensation. These conditions can be exploited to develop faster and more effective learning algorithms. In addition, they allow certain types of domain knowledge to be incorporated with ease into a learning algorithm. Among the sequential decision problems we encounter, it is unlikely that Tetris is unique or rare in having these properties.",
keywords = "sequential decision making; reinforcement learning; statistical structure of decision environments",
author = "{\"O}zg{\"u}r Şimşek and Sim{\'o}n Algorta and Amit Kothiyal",
year = "2016",
month = "6",
day = "22",
language = "English",
volume = "48",
series = "Proceedings of Machine Learning Research",
publisher = "PMLR",
pages = "1757--1765",
booktitle = "Proceedings of The 33rd International Conference on Machine Learning",

}

TY - GEN

T1 - Why most decisions are easy in Tetris—And perhaps in other sequential decision problems, as well

AU - Şimşek, Özgür

AU - Algorta, Simón

AU - Kothiyal, Amit

PY - 2016/6/22

Y1 - 2016/6/22

N2 - We examined the sequence of decision problems that are encountered in the game of Tetris and found that most of the problems are easy in the following sense: One can choose well among the available actions without knowing an evaluation function that scores well in the game. This is a consequence of three conditions that are prevalent in the game: simple dominance, cumulative dominance, and noncompensation. These conditions can be exploited to develop faster and more effective learning algorithms. In addition, they allow certain types of domain knowledge to be incorporated with ease into a learning algorithm. Among the sequential decision problems we encounter, it is unlikely that Tetris is unique or rare in having these properties.

AB - We examined the sequence of decision problems that are encountered in the game of Tetris and found that most of the problems are easy in the following sense: One can choose well among the available actions without knowing an evaluation function that scores well in the game. This is a consequence of three conditions that are prevalent in the game: simple dominance, cumulative dominance, and noncompensation. These conditions can be exploited to develop faster and more effective learning algorithms. In addition, they allow certain types of domain knowledge to be incorporated with ease into a learning algorithm. Among the sequential decision problems we encounter, it is unlikely that Tetris is unique or rare in having these properties.

KW - sequential decision making; reinforcement learning; statistical structure of decision environments

UR - http://proceedings.mlr.press/v48/simsek16.html

M3 - Conference contribution

VL - 48

T3 - Proceedings of Machine Learning Research

SP - 1757

EP - 1765

BT - Proceedings of The 33rd International Conference on Machine Learning

PB - PMLR

ER -

Access to Document