Using Machine Teaching to Investigate Human Assumptions when Teaching Reinforcement Learners

Yun Shiuan Chuang, Xuezhou Zhang, Yuzhe Ma, Mark K. Ho, Joseph L. Austerweil, Xiaojin Zhu

Research output: Contribution to conferencePaperpeer-review

Abstract

Successful teaching requires an assumption of how the learner learns - how the learner uses experiences from the world to update their internal states. We investigate what expectations people have about a learner with a behavioral experiment: Human teachers were asked to teach a sequential decision-making task to an artificial dog in an online manner using rewards and punishments. The artificial dogs were implemented with either an Action Signaling agent or a Q-learner with different discount factors. Our findings are threefold: First, we used machine teaching to prove that the optimal teaching complexity across all the learners is the same, and thus the differences in human performance was solely due to the discrepancy between human teacher’s theory of mind and the actual student model. Second, we found that Q-learners with small discount factors were easier to teach than action signaling agents, challenging the established conclusion from prior work. Third, we showed that the efficiency of teaching was monotonically increasing as the discount factors decreased, suggesting that humans’ theory of mind bias towards myopic learners.

Original languageEnglish (US)
Pages1159-1165
Number of pages7
StatePublished - 2021
Event43rd Annual Meeting of the Cognitive Science Society: Comparative Cognition: Animal Minds, CogSci 2021 - Virtual, Online, Austria
Duration: Jul 26 2021Jul 29 2021

Conference

Conference43rd Annual Meeting of the Cognitive Science Society: Comparative Cognition: Animal Minds, CogSci 2021
Country/TerritoryAustria
CityVirtual, Online
Period7/26/217/29/21

Keywords

  • machine teaching
  • reinforcement learning
  • theory of mind

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Artificial Intelligence
  • Computer Science Applications
  • Human-Computer Interaction

Fingerprint

Dive into the research topics of 'Using Machine Teaching to Investigate Human Assumptions when Teaching Reinforcement Learners'. Together they form a unique fingerprint.

Cite this