TY - GEN
T1 - AdaptiveCoPilot
T2 - 32nd IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2025
AU - Wen, Shaoyue
AU - Middleton, Michael
AU - Ping, Songming
AU - Chawla, Nayan N.
AU - Wu, Guande
AU - Feest, Bradley S.
AU - Nadri, Chihab
AU - Liu, Yunmei
AU - Kaber, David
AU - Zahabi, Maryam
AU - McMahan, Ryan P.
AU - Castelo, Sonia
AU - McKendrick, Ryan
AU - Qian, Jing
AU - Silva, Claudio T.
N1 - Publisher Copyright:
© 2025 IEEE.
PY - 2025
Y1 - 2025
N2 - Pilots operating modern cockpits often face high cognitive demands due to complex interfaces and multitasking requirements, which can lead to overload and decreased performance. This study introduces AdaptiveCoPilot, a neuroadaptive guidance system that adapts visual, auditory, and textual cues in real time based on the pilot's cognitive workload, measured via functional Near-Infrared Spectroscopy (fNIRS). A formative study with expert pilots (N=3) identified adaptive rules for modality switching and information load adjustments during preflight tasks. These insights informed the design of AdaptiveCoPilot, which integrates cognitive state assessments, behavioral data, and adaptive strategies within a context-aware Large Language Model (LLM). The system was evaluated in a virtual reality (VR) simulated cockpit with licensed pilots (N=8), comparing its performance against baseline and random feedback conditions. The results indicate that the pilots using AdaptiveCoPilot exhibited higher rates of optimal cognitive load states on the facets of working memory and perception, along with reduced task completion times. Based on the formative study, experimental findings, qualitative interviews, we propose a set of strategies for future development of neuroadaptive pilot guidance systems and highlight the potential of neuroadaptive systems to enhance pilot performance and safety in aviation environments.
AB - Pilots operating modern cockpits often face high cognitive demands due to complex interfaces and multitasking requirements, which can lead to overload and decreased performance. This study introduces AdaptiveCoPilot, a neuroadaptive guidance system that adapts visual, auditory, and textual cues in real time based on the pilot's cognitive workload, measured via functional Near-Infrared Spectroscopy (fNIRS). A formative study with expert pilots (N=3) identified adaptive rules for modality switching and information load adjustments during preflight tasks. These insights informed the design of AdaptiveCoPilot, which integrates cognitive state assessments, behavioral data, and adaptive strategies within a context-aware Large Language Model (LLM). The system was evaluated in a virtual reality (VR) simulated cockpit with licensed pilots (N=8), comparing its performance against baseline and random feedback conditions. The results indicate that the pilots using AdaptiveCoPilot exhibited higher rates of optimal cognitive load states on the facets of working memory and perception, along with reduced task completion times. Based on the formative study, experimental findings, qualitative interviews, we propose a set of strategies for future development of neuroadaptive pilot guidance systems and highlight the potential of neuroadaptive systems to enhance pilot performance and safety in aviation environments.
KW - Adaptive user interface
KW - Aviation in virtual reality
UR - http://www.scopus.com/inward/record.url?scp=105002718212&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=105002718212&partnerID=8YFLogxK
U2 - 10.1109/VR59515.2025.00088
DO - 10.1109/VR59515.2025.00088
M3 - Conference contribution
AN - SCOPUS:105002718212
T3 - Proceedings - 2025 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2025
SP - 656
EP - 666
BT - Proceedings - 2025 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2025
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 8 March 2025 through 12 March 2025
ER -