Representations, affordances, and interactive systems

Research output: Contribution to journalArticlepeer-review


The history of algorithmic composition using a digital computer has undergone many representations—data structures that encode some aspects of the outside world, or processes and entities within the program itself. Parallel histories in cognitive science and artificial intelligence have (of necessity) confronted their own notions of representations, including the ecological perception view of J.J. Gibson, who claims that mental representations are redundant to the affordances apparent in the world, its objects, and their relations. This review tracks these parallel histories and how the orientations and designs of multimodal interactive systems give rise to their own affordances: the representations and models used expose parameters and controls to a creator that determine how a system can be used and, thus, what it can mean.

Original languageEnglish (US)
Article number23
JournalMultimodal Technologies and Interaction
Issue number5
StatePublished - May 2021


  • Algorithmic composition
  • Generative systems
  • Interactive systems
  • Representations

ASJC Scopus subject areas

  • Neuroscience (miscellaneous)
  • Human-Computer Interaction
  • Computer Science Applications
  • Computer Networks and Communications


Dive into the research topics of 'Representations, affordances, and interactive systems'. Together they form a unique fingerprint.

Cite this