A game-theoretic analysis of the off-switch game

Tobias Wängberg; Mikael Böörs; Elliot Catt; Tom Everitt; Marcus Hutter

doi:10.1007/978-3-319-63703-7_16

A game-theoretic analysis of the off-switch game

Tobias Wängberg, Mikael Böörs, Elliot Catt, Tom Everitt^*, Marcus Hutter

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Citations (Scopus)

Abstract

The off-switch game is a game theoretic model of a highly intelligent robot interacting with a human. In the original paper by Hadfield-Menell et al. (2016b), the analysis is not fully game-theoretic as the human is modelled as an irrational player, and the robot’s best action is only calculated under unrealistic normality and soft-max assumptions. In this paper, we make the analysis fully game theoretic, by modelling the human as a rational player with a random utility function. As a consequence, we are able to easily calculate the robot’s best action for arbitrary belief and irrationality assumptions.

Original language	English
Title of host publication	Artificial General Intelligence - 10th International Conference, AGI 2017, Proceedings
Editors	Tom Everitt, Ben Goertzel, Alexey Potapov
Publisher	Springer Verlag
Pages	167-177
Number of pages	11
ISBN (Print)	9783319637020
DOIs	https://doi.org/10.1007/978-3-319-63703-7_16
Publication status	Published - 2017
Event	10th International Conference on Artificial General Intelligence, AGI 2017 - Melbourne, Australia Duration: 15 Aug 2017 → 18 Aug 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10414 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	10th International Conference on Artificial General Intelligence, AGI 2017
Country/Territory	Australia
City	Melbourne
Period	15/08/17 → 18/08/17

Access to Document

10.1007/978-3-319-63703-7_16

Cite this

Wängberg, T., Böörs, M., Catt, E., Everitt, T., & Hutter, M. (2017). A game-theoretic analysis of the off-switch game. In T. Everitt, B. Goertzel, & A. Potapov (Eds.), Artificial General Intelligence - 10th International Conference, AGI 2017, Proceedings (pp. 167-177). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10414 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-63703-7_16

Wängberg, Tobias ; Böörs, Mikael ; Catt, Elliot et al. / A game-theoretic analysis of the off-switch game. Artificial General Intelligence - 10th International Conference, AGI 2017, Proceedings. editor / Tom Everitt ; Ben Goertzel ; Alexey Potapov. Springer Verlag, 2017. pp. 167-177 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{64cadaf48c9c4fdeb2bfa24571ce7d1e,

title = "A game-theoretic analysis of the off-switch game",

abstract = "The off-switch game is a game theoretic model of a highly intelligent robot interacting with a human. In the original paper by Hadfield-Menell et al. (2016b), the analysis is not fully game-theoretic as the human is modelled as an irrational player, and the robot{\textquoteright}s best action is only calculated under unrealistic normality and soft-max assumptions. In this paper, we make the analysis fully game theoretic, by modelling the human as a rational player with a random utility function. As a consequence, we are able to easily calculate the robot{\textquoteright}s best action for arbitrary belief and irrationality assumptions.",

author = "Tobias W{\"a}ngberg and Mikael B{\"o}{\"o}rs and Elliot Catt and Tom Everitt and Marcus Hutter",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2017.; 10th International Conference on Artificial General Intelligence, AGI 2017 ; Conference date: 15-08-2017 Through 18-08-2017",

year = "2017",

doi = "10.1007/978-3-319-63703-7_16",

language = "English",

isbn = "9783319637020",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "167--177",

editor = "Tom Everitt and Ben Goertzel and Alexey Potapov",

booktitle = "Artificial General Intelligence - 10th International Conference, AGI 2017, Proceedings",

address = "Germany",

}

Wängberg, T, Böörs, M, Catt, E, Everitt, T & Hutter, M 2017, A game-theoretic analysis of the off-switch game. in T Everitt, B Goertzel & A Potapov (eds), Artificial General Intelligence - 10th International Conference, AGI 2017, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10414 LNAI, Springer Verlag, pp. 167-177, 10th International Conference on Artificial General Intelligence, AGI 2017, Melbourne, Australia, 15/08/17. https://doi.org/10.1007/978-3-319-63703-7_16

A game-theoretic analysis of the off-switch game. / Wängberg, Tobias; Böörs, Mikael; Catt, Elliot et al.
Artificial General Intelligence - 10th International Conference, AGI 2017, Proceedings. ed. / Tom Everitt; Ben Goertzel; Alexey Potapov. Springer Verlag, 2017. p. 167-177 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10414 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A game-theoretic analysis of the off-switch game

AU - Wängberg, Tobias

AU - Böörs, Mikael

AU - Catt, Elliot

AU - Everitt, Tom

AU - Hutter, Marcus

N1 - Publisher Copyright: © Springer International Publishing AG 2017.

PY - 2017

Y1 - 2017

N2 - The off-switch game is a game theoretic model of a highly intelligent robot interacting with a human. In the original paper by Hadfield-Menell et al. (2016b), the analysis is not fully game-theoretic as the human is modelled as an irrational player, and the robot’s best action is only calculated under unrealistic normality and soft-max assumptions. In this paper, we make the analysis fully game theoretic, by modelling the human as a rational player with a random utility function. As a consequence, we are able to easily calculate the robot’s best action for arbitrary belief and irrationality assumptions.

AB - The off-switch game is a game theoretic model of a highly intelligent robot interacting with a human. In the original paper by Hadfield-Menell et al. (2016b), the analysis is not fully game-theoretic as the human is modelled as an irrational player, and the robot’s best action is only calculated under unrealistic normality and soft-max assumptions. In this paper, we make the analysis fully game theoretic, by modelling the human as a rational player with a random utility function. As a consequence, we are able to easily calculate the robot’s best action for arbitrary belief and irrationality assumptions.

UR - http://www.scopus.com/inward/record.url?scp=85028471520&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-63703-7_16

DO - 10.1007/978-3-319-63703-7_16

M3 - Conference contribution

SN - 9783319637020

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 167

EP - 177

BT - Artificial General Intelligence - 10th International Conference, AGI 2017, Proceedings

A2 - Everitt, Tom

A2 - Goertzel, Ben

A2 - Potapov, Alexey

PB - Springer Verlag

T2 - 10th International Conference on Artificial General Intelligence, AGI 2017

Y2 - 15 August 2017 through 18 August 2017

ER -

Wängberg T, Böörs M, Catt E, Everitt T, Hutter M. A game-theoretic analysis of the off-switch game. In Everitt T, Goertzel B, Potapov A, editors, Artificial General Intelligence - 10th International Conference, AGI 2017, Proceedings. Springer Verlag. 2017. p. 167-177. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-63703-7_16

A game-theoretic analysis of the off-switch game

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this