Why So Serious? Exploring Timely Humorous Comments in AAC Through AI-Powered Interfaces

Fig. 1: We explore how to design interfaces to support timely humorous comments in Augmentative and Alternative Communication (AAC). (a) Here an AAC user tries to make a joke in an ongoing conversation. (b) as the conversation continues, the user frantically tries to type the joke (c) unfortunately they trigger text-to-speech (TTS) too late to effectively land the joke causing confusion (d) we explore how to design interfaces to speed up the response through AI powered tools, while minimizing the cost of agency. (e) Leading to design that allows AAC users to meaningfully express themselves through timely humorous comments in ongoing conversations. Note: AAC input types and interfaces vary widely across users. The leading author of this work is an AAC user who commonly interacts with his phone as displayed here. This image is for illustration purposes only. Our interfaces were used on a range of devices in our studies.

Abstract

People with disabilities that affect their speech, often use speech-generating devices (SGD), commonly referred to as Augmentative and Alternative Communication (AAC) technology. This technology enables practical conversation; however there has been a growing interest in extending AAC to support more expressive forms conversations such as humor. In this paper, we study how to extend AAC technology to support a subset of humorous expression: delivering timely humorous comments -witty remarks- through AI powered interfaces. We conducted seven qualitative interviews with AAC users, and performed thematic analysis to gain in-depth insights about their experiences and challenges with AAC technology, and the role humor plays for them. We developed four simple AI-powered interfaces designed to support users in creating timely humorous comments during real-time conversations. Through a user study with five AAC users, we explored how to effectively support the delivery of well-timed humorous remarks. We conclude with a discussion of recommendations for interface design based on both studies.

Video

Citation

@article{weinberg2024whysoserious,
title={Why So Serious? Exploring Humor in AAC Through AI-Powered Interfaces},
author={Weinberg, Tobias and Kadoma, Kowe and Penuela, Ricardo E Gonzalez and Valencia, Stephanie and Roumen, Thijs},
journal={arXiv preprint arXiv:2410.16634},
year={2024}
}

Acknowledgements

Thanks to Zekai Shen, Samee Chandra and Maria Tane for helping in the develpment of the prototypes and Roy Zunder for helping analysing the data and wrinting. Thank you guys we couldn't have done without you!
The website template was borrowed from Michaël Gharbi.