ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality

Guande Wu, Jing Qian, Sonia Castelo, Shaoyu Chen, Joao Rulff, Claudio Silva

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Text presented in augmented reality provides in-situ, real-time information for users. However, this content can be challenging to apprehend quickly when engaging in cognitively demanding AR tasks, especially when it is presented on a head-mounted display. We propose ARTiST, an automatic text simplification system that uses a few-shot prompt and GPT-3 models to specifically optimize the text length and semantic content for augmented reality. Developed out of a formative study that included seven users and three experts, our system combines a customized error calibration model with a few-shot prompt to integrate the syntactic, lexical, elaborative, and content simplification techniques, and generate simplified AR text for head-worn displays. Results from a 16-user empirical study showed that ARTiST lightens the cognitive load and improves performance significantly over both unmodified text and text modified via traditional methods. Our work constitutes a step towards automating the optimization of batch text data for readability and performance in augmented reality.

Original languageEnglish (US)
Title of host publicationCHI 2024 - Proceedings of the 2024 CHI Conference on Human Factors in Computing Sytems
PublisherAssociation for Computing Machinery
ISBN (Electronic)9798400703300
DOIs
StatePublished - May 11 2024
Event2024 CHI Conference on Human Factors in Computing Sytems, CHI 2024 - Hybrid, Honolulu, United States
Duration: May 11 2024May 16 2024

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Conference

Conference2024 CHI Conference on Human Factors in Computing Sytems, CHI 2024
Country/TerritoryUnited States
CityHybrid, Honolulu
Period5/11/245/16/24

Keywords

  • augmented reality
  • large language model
  • text simplification

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality'. Together they form a unique fingerprint.

Cite this