Universal linguistic inductive biases via meta-learning

R. Thomas McCoy, Erin Grant, Paul Smolensky, Thomas L. Griffiths, Tal Linzen

    Research output: Contribution to conferencePaperpeer-review

    Abstract

    How do learners acquire languages from the limited data available to them? This process must involve some inductive biases-factors that affect how a learner generalizes-but it is unclear which inductive biases can explain observed patterns in language acquisition. To facilitate computational modeling aimed at addressing this question, we introduce a framework for giving particular linguistic inductive biases to a neural network model; such a model can then be used to empirically explore the effects of those inductive biases. This framework disentangles universal inductive biases, which are encoded in the initial values of a neural network's parameters, from non-universal factors, which the neural network must learn from data in a given language. The initial state that encodes the inductive biases is found with meta-learning, a technique through which a model discovers how to acquire new languages more easily via exposure to many possible languages. By controlling the properties of the languages that are used during meta-learning, we can control the inductive biases that meta-learning imparts. We demonstrate this framework with a case study based on syllable structure. First, we specify the inductive biases that we intend to give our model, and then we translate those inductive biases into a space of languages from which a model can meta-learn. Finally, using existing analysis techniques, we verify that our approach has imparted the linguistic inductive biases that it was intended to impart.

    Original languageEnglish (US)
    Pages737-743
    Number of pages7
    StatePublished - 2020
    Event42nd Annual Meeting of the Cognitive Science Society: Developing a Mind: Learning in Humans, Animals, and Machines, CogSci 2020 - Virtual, Online
    Duration: Jul 29 2020Aug 1 2020

    Conference

    Conference42nd Annual Meeting of the Cognitive Science Society: Developing a Mind: Learning in Humans, Animals, and Machines, CogSci 2020
    CityVirtual, Online
    Period7/29/208/1/20

    Keywords

    • inductive bias
    • language universals
    • meta-learning
    • neural networks
    • syllable structure typology

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Science Applications
    • Human-Computer Interaction
    • Cognitive Neuroscience

    Fingerprint

    Dive into the research topics of 'Universal linguistic inductive biases via meta-learning'. Together they form a unique fingerprint.

    Cite this