Managing unintended consequences of algorithmic decision-making: The case of Robodebt

Tapani Rinta-Kahila; Ida Someh; Nicole Gillespie; Marta Indulska; Shirley Gregor

doi:10.1177/20438869231165538

Managing unintended consequences of algorithmic decision-making: The case of Robodebt

Tapani Rinta-Kahila^*, Ida Someh, Nicole Gillespie, Marta Indulska, Shirley Gregor

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

4 Citations (Scopus)

Abstract

In 2016, the Australian welfare agency Centrelink implemented an information system to automate the identification and recollection of welfare overpayments. Such algorithmic decision-making systems are increasingly leveraged to improve the efficiency of public administration. However, Centrelink’s scheme went horribly wrong: the system, branded as “Robodebt” by the popular media, generated debt notices that were inaccurate and based on insufficient evidence. Numerous vulnerable citizens who received the debt notices suffered a great deal of distress. While public controversy ensued, the ruling government continued to defend the flawed system until a court decision ruled it unlawful in 2019. This teaching case challenges one to analyze what went wrong in the implementation and management of the Robodebt system through the lens of sociotechnical systems.

Original language	English
Number of pages	7
Journal	Journal of Information Technology Teaching Cases
DOIs	https://doi.org/10.1177/20438869231165538
Publication status	Published - 2023

Access to Document

10.1177/20438869231165538

Cite this

@article{c884faf89dc842419096d9db7a38f911,

title = "Managing unintended consequences of algorithmic decision-making: The case of Robodebt",

abstract = "In 2016, the Australian welfare agency Centrelink implemented an information system to automate the identification and recollection of welfare overpayments. Such algorithmic decision-making systems are increasingly leveraged to improve the efficiency of public administration. However, Centrelink{\textquoteright}s scheme went horribly wrong: the system, branded as “Robodebt” by the popular media, generated debt notices that were inaccurate and based on insufficient evidence. Numerous vulnerable citizens who received the debt notices suffered a great deal of distress. While public controversy ensued, the ruling government continued to defend the flawed system until a court decision ruled it unlawful in 2019. This teaching case challenges one to analyze what went wrong in the implementation and management of the Robodebt system through the lens of sociotechnical systems.",

keywords = "Centrelink, Robodebt, algorithmic decision-making, automation, public agency, sociotechnical system",

author = "Tapani Rinta-Kahila and Ida Someh and Nicole Gillespie and Marta Indulska and Shirley Gregor",

note = "Publisher Copyright: {\textcopyright} Association for Information Technology Trust 2023.",

year = "2023",

doi = "10.1177/20438869231165538",

language = "English",

journal = "Journal of Information Technology Teaching Cases",

issn = "2043-8869",

publisher = "Springer International Publishing Switzerland",

}

TY - JOUR

T1 - Managing unintended consequences of algorithmic decision-making

T2 - The case of Robodebt

AU - Rinta-Kahila, Tapani

AU - Someh, Ida

AU - Gillespie, Nicole

AU - Indulska, Marta

AU - Gregor, Shirley

N1 - Publisher Copyright: © Association for Information Technology Trust 2023.

PY - 2023

Y1 - 2023

N2 - In 2016, the Australian welfare agency Centrelink implemented an information system to automate the identification and recollection of welfare overpayments. Such algorithmic decision-making systems are increasingly leveraged to improve the efficiency of public administration. However, Centrelink’s scheme went horribly wrong: the system, branded as “Robodebt” by the popular media, generated debt notices that were inaccurate and based on insufficient evidence. Numerous vulnerable citizens who received the debt notices suffered a great deal of distress. While public controversy ensued, the ruling government continued to defend the flawed system until a court decision ruled it unlawful in 2019. This teaching case challenges one to analyze what went wrong in the implementation and management of the Robodebt system through the lens of sociotechnical systems.

AB - In 2016, the Australian welfare agency Centrelink implemented an information system to automate the identification and recollection of welfare overpayments. Such algorithmic decision-making systems are increasingly leveraged to improve the efficiency of public administration. However, Centrelink’s scheme went horribly wrong: the system, branded as “Robodebt” by the popular media, generated debt notices that were inaccurate and based on insufficient evidence. Numerous vulnerable citizens who received the debt notices suffered a great deal of distress. While public controversy ensued, the ruling government continued to defend the flawed system until a court decision ruled it unlawful in 2019. This teaching case challenges one to analyze what went wrong in the implementation and management of the Robodebt system through the lens of sociotechnical systems.

KW - Centrelink

KW - Robodebt

KW - algorithmic decision-making

KW - automation

KW - public agency

KW - sociotechnical system

UR - http://www.scopus.com/inward/record.url?scp=85152397551&partnerID=8YFLogxK

U2 - 10.1177/20438869231165538

DO - 10.1177/20438869231165538

M3 - Article

SN - 2043-8869

JO - Journal of Information Technology Teaching Cases

JF - Journal of Information Technology Teaching Cases

ER -

Managing unintended consequences of algorithmic decision-making: The case of Robodebt

Abstract

Access to Document

Other files and links

Fingerprint

Cite this