DrawTalking: Towards Building InteractiveWorlds by Sketching and Speaking

Karl Toby Rosenberg, Rubaiat Habib Kazi, Li Yi Wei, Haijun Xia, Ken Perlin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We introduce DrawTalking, a prototype system enabling an approach that empowers users to build interactiveworlds by sketching and speaking. The approach emphasizes user control and flexibility, and gives programming-like capability without requiring code. An early open-ended study shows the mechanics resonate and are applicable to many creative-exploratory use cases, with the potential to inspire and inform research in future natural interfaces for creative exploration and authoring.

Original languageEnglish (US)
Title of host publicationCHI 2024 - Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Sytems
PublisherAssociation for Computing Machinery
ISBN (Electronic)9798400703317
DOIs
StatePublished - May 11 2024
Event2024 CHI Conference on Human Factors in Computing Sytems, CHI EA 2024 - Hybrid, Honolulu, United States
Duration: May 11 2024May 16 2024

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Conference

Conference2024 CHI Conference on Human Factors in Computing Sytems, CHI EA 2024
Country/TerritoryUnited States
CityHybrid, Honolulu
Period5/11/245/16/24

Keywords

  • creativity
  • human-AI collaboration
  • multimodal
  • play
  • programmability
  • prototyping
  • sketching

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design
  • Software

Fingerprint

Dive into the research topics of 'DrawTalking: Towards Building InteractiveWorlds by Sketching and Speaking'. Together they form a unique fingerprint.

Cite this