TY - GEN
T1 - Satori
T2 - 2025 CHI Conference on Human Factors in Computing Systems, CHI 2025
AU - Li, Chenyi
AU - Wu, Guande
AU - Chan, Gromit Yeuk Yin
AU - Turakhia, Dishita Gdi
AU - Castelo Quispe, Sonia
AU - Li, Dong
AU - Welch, Leslie
AU - Silva, Claudio
AU - Qian, Jing
N1 - Publisher Copyright:
© 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM.
PY - 2025/4/26
Y1 - 2025/4/26
N2 - Augmented Reality (AR) assistance is increasingly used for supporting users with physical tasks like assembly and cooking. However, most systems rely on reactive responses triggered by user input, overlooking rich contextual and user-specific information. To address this, we present Satori, a novel AR system that proactively guides users by modeling both - their mental states and environmental contexts. Satori integrates the Belief-Desire-Intention (BDI) framework with the state-of-the-art multi-modal large language model (LLM) to deliver contextually appropriate guidance. Our system is designed based on two formative studies involving twelve experts. We evaluated the system with a sixteen within-subject study and found that Satori matches the performance of designer-created Wizard-of-Oz (WoZ) systems, without manual configurations or heuristics, thereby improving generalizability, reusability, and expanding the potential of AR assistance. Code is available at https://github.com/VIDA-NYU/satori-assistance.
AB - Augmented Reality (AR) assistance is increasingly used for supporting users with physical tasks like assembly and cooking. However, most systems rely on reactive responses triggered by user input, overlooking rich contextual and user-specific information. To address this, we present Satori, a novel AR system that proactively guides users by modeling both - their mental states and environmental contexts. Satori integrates the Belief-Desire-Intention (BDI) framework with the state-of-the-art multi-modal large language model (LLM) to deliver contextually appropriate guidance. Our system is designed based on two formative studies involving twelve experts. We evaluated the system with a sixteen within-subject study and found that Satori matches the performance of designer-created Wizard-of-Oz (WoZ) systems, without manual configurations or heuristics, thereby improving generalizability, reusability, and expanding the potential of AR assistance. Code is available at https://github.com/VIDA-NYU/satori-assistance.
KW - Augmented reality assistant
KW - proactive virtual assistant
KW - user modeling
UR - http://www.scopus.com/inward/record.url?scp=105005770087&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=105005770087&partnerID=8YFLogxK
U2 - 10.1145/3706598.3714188
DO - 10.1145/3706598.3714188
M3 - Conference contribution
AN - SCOPUS:105005770087
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - CHI 2025 - Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
PB - Association for Computing Machinery
Y2 - 26 April 2025 through 1 May 2025
ER -