A Scenario-Based Design Pack for Exploring Multimodal Human-GenAI Relations

Josh Andres; Chris Danta; Andrea Bianchi; Sahar Farzanfar; Gloria Milena Fernandez-Nieto; Alexa Becker; Tara Capel; Frances Liddell; Shelby Hagemann; Ned Cooper; Sungyeon Hong; Li Lin; Eduardo Benitez Sandoval; Anna Brynskov; Hubert Dariusz Zajac; Zhuying Li; Tianyi Zhang; Arngeir Berge

doi:10.1145/3716553.3750749

A Scenario-Based Design Pack for Exploring Multimodal Human-GenAI Relations

Josh Andres^*, Chris Danta, Andrea Bianchi, Sahar Farzanfar, Gloria Milena Fernandez-Nieto, Alexa Becker, Tara Capel, Frances Liddell, Shelby Hagemann, Ned Cooper, Sungyeon Hong, Li Lin, Eduardo Benitez Sandoval, Anna Brynskov, Hubert Dariusz Zajac, Zhuying Li, Tianyi Zhang, Arngeir Berge

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › peer-review

Abstract

Generative AI technologies are reshaping everyday environments by enabling multimodal interaction. As their ubiquity and agentic capacities grow, there is a pressing need to understand how these systems reshape human–computer interaction in relational, social, and systemic terms. We introduce a scenario-based design pack for investigating Human–GenAI relations. Grounded in assemblage theory and structured around a three-stage process—Prepare, Make, Reflect—the pack supports the prototyping, analysis, and critical reflection of emergent sociotechnical configurations. We evaluated the pack across three deployments: an ACM workshop (n=22), a multidisciplinary design session (n=20), and a university HCI class (n=260). Participants generated scenarios that surfaced relational issues of power, agency, visibility, and care. We contribute the design pack alongside an exploratory framework to advance relational enquiry into multimodal Human–GenAI relations, support more inclusive and socially responsive GenAI practices, and complement FATE approaches by grounding fairness, accountability, and transparency in lived, multimodal configurations.

Original language	English
Title of host publication	Proceedings of the 27th International Conference on Multimodal Interaction (ICMI ’25), October 13–17, 2025, Canberra, ACT, Australia.
Editors	Ram Subramanian, Yukiko I. Nakano, Tom Gedeon, Mohan Kankanhalli, Tanaya Guha, Jainendra Shukla, Gelareh Mohammadi, Oya Celiktutan
Place of Publication	New York
Publisher	Association for Computing Machinery (ACM)
Pages	145-154
Number of pages	10
ISBN (Electronic)	9798400714993
DOIs	https://doi.org/10.1145/3716553.3750749
Publication status	Published - 13 Oct 2025
Event	27^th ACM International Conference on Multimodal Interaction, ICMI 2025: Safe and responsible multimodal interactions - Canberra, Australia Duration: 13 Oct 2025 → 17 Oct 2025 https://icmi.acm.org/2025/ https://dl.acm.org/doi/proceedings/10.1145/3716553 (ICMI '25 Proceedings)

Publication series

Name	Proceedings of the International Conference on Multimodal Interaction (ICMI)
Publisher	ACM
Number	2025/10

Conference

Conference	27^th ACM International Conference on Multimodal Interaction, ICMI 2025
Abbreviated title	ICMI '25
Country/Territory	Australia
City	Canberra
Period	13/10/25 → 17/10/25
Other	ICMI is the premier international forum that brings together multimodal artificial intelligence (AI) and social interaction research. Multimodal AI encompasses technical challenges in machine learning and computational modeling such as representations, fusion, data, and systems. The study of social interactions englobes both human-human interactions and human-computer interactions. A unique aspect of ICMI is its multidisciplinary nature which values scientific discoveries and technical modeling achievements equally, with an eye toward impactful applications for the good of people and society.
Internet address	https://icmi.acm.org/2025/ https://dl.acm.org/doi/proceedings/10.1145/3716553 (ICMI '25 Proceedings)

Access to Document

10.1145/3716553.3750749Licence: CC BY

Cite this

Andres, J., Danta, C., Bianchi, A., Farzanfar, S., Milena Fernandez-Nieto, G., Becker, A., Capel, T., Liddell, F., Hagemann, S., Cooper, N., Hong, S., Lin, L., Benitez Sandoval, E., Brynskov, A., Zajac, H. D., Li, Z., Zhang, T., & Berge, A. (2025). A Scenario-Based Design Pack for Exploring Multimodal Human-GenAI Relations. In R. Subramanian, Y. I. Nakano, T. Gedeon, M. Kankanhalli, T. Guha, J. Shukla, G. Mohammadi, & O. Celiktutan (Eds.), Proceedings of the 27th International Conference on Multimodal Interaction (ICMI ’25), October 13–17, 2025, Canberra, ACT, Australia. (pp. 145-154). (Proceedings of the International Conference on Multimodal Interaction (ICMI); No. 2025/10). Association for Computing Machinery (ACM). https://doi.org/10.1145/3716553.3750749

Andres, Josh ; Danta, Chris ; Bianchi, Andrea et al. / A Scenario-Based Design Pack for Exploring Multimodal Human-GenAI Relations. Proceedings of the 27th International Conference on Multimodal Interaction (ICMI ’25), October 13–17, 2025, Canberra, ACT, Australia.. editor / Ram Subramanian ; Yukiko I. Nakano ; Tom Gedeon ; Mohan Kankanhalli ; Tanaya Guha ; Jainendra Shukla ; Gelareh Mohammadi ; Oya Celiktutan. New York : Association for Computing Machinery (ACM), 2025. pp. 145-154 (Proceedings of the International Conference on Multimodal Interaction (ICMI); 2025/10).

@inproceedings{dc4e847309b9484db0948028e45afbc2,

title = "A Scenario-Based Design Pack for Exploring Multimodal Human-GenAI Relations",

abstract = "Generative AI technologies are reshaping everyday environments by enabling multimodal interaction. As their ubiquity and agentic capacities grow, there is a pressing need to understand how these systems reshape human–computer interaction in relational, social, and systemic terms. We introduce a scenario-based design pack for investigating Human–GenAI relations. Grounded in assemblage theory and structured around a three-stage process—Prepare, Make, Reflect—the pack supports the prototyping, analysis, and critical reflection of emergent sociotechnical configurations. We evaluated the pack across three deployments: an ACM workshop (n=22), a multidisciplinary design session (n=20), and a university HCI class (n=260). Participants generated scenarios that surfaced relational issues of power, agency, visibility, and care. We contribute the design pack alongside an exploratory framework to advance relational enquiry into multimodal Human–GenAI relations, support more inclusive and socially responsive GenAI practices, and complement FATE approaches by grounding fairness, accountability, and transparency in lived, multimodal configurations.",

keywords = "Generative AI, Human-AI Relations, Scenario-Based Design",

author = "Josh Andres and Chris Danta and Andrea Bianchi and Sahar Farzanfar and \{Milena Fernandez-Nieto\}, Gloria and Alexa Becker and Tara Capel and Frances Liddell and Shelby Hagemann and Ned Cooper and Sungyeon Hong and Li Lin and \{Benitez Sandoval\}, Eduardo and Anna Brynskov and Zajac, \{Hubert Dariusz\} and Zhuying Li and Tianyi Zhang and Arngeir Berge",

note = "{\textcopyright} 2025 Copyright held by the owner/author(s).; 27<sup>th</sup> ACM International Conference on Multimodal Interaction, ICMI 2025 : Safe and responsible multimodal interactions, ICMI '25 ; Conference date: 13-10-2025 Through 17-10-2025",

year = "2025",

month = oct,

day = "13",

doi = "10.1145/3716553.3750749",

language = "English",

series = "Proceedings of the International Conference on Multimodal Interaction (ICMI)",

publisher = "Association for Computing Machinery (ACM)",

number = "2025/10",

pages = "145--154",

editor = "Ram Subramanian and Nakano, \{Yukiko I.\} and Tom Gedeon and Mohan Kankanhalli and Tanaya Guha and Jainendra Shukla and Gelareh Mohammadi and Oya Celiktutan",

booktitle = "Proceedings of the 27th International Conference on Multimodal Interaction (ICMI {\textquoteright}25), October 13–17, 2025, Canberra, ACT, Australia.",

address = "United States",

url = "https://icmi.acm.org/2025/, https://dl.acm.org/doi/proceedings/10.1145/3716553",

}

Andres, J, Danta, C, Bianchi, A, Farzanfar, S, Milena Fernandez-Nieto, G, Becker, A, Capel, T, Liddell, F, Hagemann, S, Cooper, N, Hong, S, Lin, L, Benitez Sandoval, E, Brynskov, A, Zajac, HD, Li, Z, Zhang, T & Berge, A 2025, A Scenario-Based Design Pack for Exploring Multimodal Human-GenAI Relations. in R Subramanian, YI Nakano, T Gedeon, M Kankanhalli, T Guha, J Shukla, G Mohammadi & O Celiktutan (eds), Proceedings of the 27th International Conference on Multimodal Interaction (ICMI ’25), October 13–17, 2025, Canberra, ACT, Australia.. Proceedings of the International Conference on Multimodal Interaction (ICMI), no. 2025/10, Association for Computing Machinery (ACM), New York, pp. 145-154, 27^th ACM International Conference on Multimodal Interaction, ICMI 2025, Canberra, Australia, 13/10/25. https://doi.org/10.1145/3716553.3750749

A Scenario-Based Design Pack for Exploring Multimodal Human-GenAI Relations. / Andres, Josh; Danta, Chris; Bianchi, Andrea et al.
Proceedings of the 27th International Conference on Multimodal Interaction (ICMI ’25), October 13–17, 2025, Canberra, ACT, Australia.. ed. / Ram Subramanian; Yukiko I. Nakano; Tom Gedeon; Mohan Kankanhalli; Tanaya Guha; Jainendra Shukla; Gelareh Mohammadi; Oya Celiktutan. New York: Association for Computing Machinery (ACM), 2025. p. 145-154 (Proceedings of the International Conference on Multimodal Interaction (ICMI); No. 2025/10).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › peer-review

TY - GEN

T1 - A Scenario-Based Design Pack for Exploring Multimodal Human-GenAI Relations

AU - Andres, Josh

AU - Danta, Chris

AU - Bianchi, Andrea

AU - Farzanfar, Sahar

AU - Milena Fernandez-Nieto, Gloria

AU - Becker, Alexa

AU - Capel, Tara

AU - Liddell, Frances

AU - Hagemann, Shelby

AU - Cooper, Ned

AU - Hong, Sungyeon

AU - Lin, Li

AU - Benitez Sandoval, Eduardo

AU - Brynskov, Anna

AU - Zajac, Hubert Dariusz

AU - Li, Zhuying

AU - Zhang, Tianyi

AU - Berge, Arngeir

PY - 2025/10/13

Y1 - 2025/10/13

N2 - Generative AI technologies are reshaping everyday environments by enabling multimodal interaction. As their ubiquity and agentic capacities grow, there is a pressing need to understand how these systems reshape human–computer interaction in relational, social, and systemic terms. We introduce a scenario-based design pack for investigating Human–GenAI relations. Grounded in assemblage theory and structured around a three-stage process—Prepare, Make, Reflect—the pack supports the prototyping, analysis, and critical reflection of emergent sociotechnical configurations. We evaluated the pack across three deployments: an ACM workshop (n=22), a multidisciplinary design session (n=20), and a university HCI class (n=260). Participants generated scenarios that surfaced relational issues of power, agency, visibility, and care. We contribute the design pack alongside an exploratory framework to advance relational enquiry into multimodal Human–GenAI relations, support more inclusive and socially responsive GenAI practices, and complement FATE approaches by grounding fairness, accountability, and transparency in lived, multimodal configurations.

AB - Generative AI technologies are reshaping everyday environments by enabling multimodal interaction. As their ubiquity and agentic capacities grow, there is a pressing need to understand how these systems reshape human–computer interaction in relational, social, and systemic terms. We introduce a scenario-based design pack for investigating Human–GenAI relations. Grounded in assemblage theory and structured around a three-stage process—Prepare, Make, Reflect—the pack supports the prototyping, analysis, and critical reflection of emergent sociotechnical configurations. We evaluated the pack across three deployments: an ACM workshop (n=22), a multidisciplinary design session (n=20), and a university HCI class (n=260). Participants generated scenarios that surfaced relational issues of power, agency, visibility, and care. We contribute the design pack alongside an exploratory framework to advance relational enquiry into multimodal Human–GenAI relations, support more inclusive and socially responsive GenAI practices, and complement FATE approaches by grounding fairness, accountability, and transparency in lived, multimodal configurations.

KW - Generative AI

KW - Human-AI Relations

KW - Scenario-Based Design

UR - https://www.scopus.com/pages/publications/105022289604

U2 - 10.1145/3716553.3750749

DO - 10.1145/3716553.3750749

M3 - Conference Paper

AN - SCOPUS:105022289604

T3 - Proceedings of the International Conference on Multimodal Interaction (ICMI)

SP - 145

EP - 154

BT - Proceedings of the 27th International Conference on Multimodal Interaction (ICMI ’25), October 13–17, 2025, Canberra, ACT, Australia.

A2 - Subramanian, Ram

A2 - Nakano, Yukiko I.

A2 - Gedeon, Tom

A2 - Kankanhalli, Mohan

A2 - Guha, Tanaya

A2 - Shukla, Jainendra

A2 - Mohammadi, Gelareh

A2 - Celiktutan, Oya

PB - Association for Computing Machinery (ACM)

CY - New York

T2 - 27<sup>th</sup> ACM International Conference on Multimodal Interaction, ICMI 2025

Y2 - 13 October 2025 through 17 October 2025

ER -

Andres J, Danta C, Bianchi A, Farzanfar S, Milena Fernandez-Nieto G, Becker A et al. A Scenario-Based Design Pack for Exploring Multimodal Human-GenAI Relations. In Subramanian R, Nakano YI, Gedeon T, Kankanhalli M, Guha T, Shukla J, Mohammadi G, Celiktutan O, editors, Proceedings of the 27th International Conference on Multimodal Interaction (ICMI ’25), October 13–17, 2025, Canberra, ACT, Australia.. New York: Association for Computing Machinery (ACM). 2025. p. 145-154. (Proceedings of the International Conference on Multimodal Interaction (ICMI); 2025/10). doi: 10.1145/3716553.3750749

A Scenario-Based Design Pack for Exploring Multimodal Human-GenAI Relations

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this