Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Number

Sophie Hao; Tal Linzen

Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Number

Sophie Hao, Tal Linzen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Deep architectures such as Transformers are sometimes criticized for having uninterpretable “black-box” representations. We use causal intervention analysis to show that, in fact, some linguistic features are represented in a linear, interpretable format. Specifically, we show that BERT's ability to conjugate verbs relies on a linear encoding of subject number that can be manipulated with predictable effects on conjugation accuracy. This encoding is found in the subject position at the first layer and the verb position at the last layer, but is distributed across positions at middle layers, particularly when there are multiple cues to subject number.

Original language	English (US)
Title of host publication	Findings of the Association for Computational Linguistics
Subtitle of host publication	EMNLP 2023
Publisher	Association for Computational Linguistics (ACL)
Pages	4531-4539
Number of pages	9
ISBN (Electronic)	9798891760615
State	Published - 2023
Event	2023 Findings of the Association for Computational Linguistics: EMNLP 2023 - Singapore, Singapore Duration: Dec 6 2023 → Dec 10 2023

Publication series

Name	Findings of the Association for Computational Linguistics: EMNLP 2023

Conference

Conference	2023 Findings of the Association for Computational Linguistics: EMNLP 2023
Country/Territory	Singapore
City	Singapore
Period	12/6/23 → 12/10/23

ASJC Scopus subject areas

Computational Theory and Mathematics
Computer Science Applications
Information Systems
Language and Linguistics
Linguistics and Language

Cite this

Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Number. / Hao, Sophie; Linzen, Tal.
Findings of the Association for Computational Linguistics: EMNLP 2023. Association for Computational Linguistics (ACL), 2023. p. 4531-4539 (Findings of the Association for Computational Linguistics: EMNLP 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Hao, S & Linzen, T 2023, Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Number. in Findings of the Association for Computational Linguistics: EMNLP 2023. Findings of the Association for Computational Linguistics: EMNLP 2023, Association for Computational Linguistics (ACL), pp. 4531-4539, 2023 Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, Singapore, 12/6/23.

@inproceedings{de1cdec8e16446ef905f8437dc13e4d8,

title = "Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Number",

abstract = "Deep architectures such as Transformers are sometimes criticized for having uninterpretable “black-box” representations. We use causal intervention analysis to show that, in fact, some linguistic features are represented in a linear, interpretable format. Specifically, we show that BERT's ability to conjugate verbs relies on a linear encoding of subject number that can be manipulated with predictable effects on conjugation accuracy. This encoding is found in the subject position at the first layer and the verb position at the last layer, but is distributed across positions at middle layers, particularly when there are multiple cues to subject number.",

author = "Sophie Hao and Tal Linzen",

note = "Publisher Copyright: {\textcopyright} 2023 Association for Computational Linguistics.; 2023 Findings of the Association for Computational Linguistics: EMNLP 2023 ; Conference date: 06-12-2023 Through 10-12-2023",

year = "2023",

language = "English (US)",

series = "Findings of the Association for Computational Linguistics: EMNLP 2023",

publisher = "Association for Computational Linguistics (ACL)",

pages = "4531--4539",

booktitle = "Findings of the Association for Computational Linguistics",

}

TY - GEN

T1 - Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Number

AU - Hao, Sophie

AU - Linzen, Tal

PY - 2023

Y1 - 2023

N2 - Deep architectures such as Transformers are sometimes criticized for having uninterpretable “black-box” representations. We use causal intervention analysis to show that, in fact, some linguistic features are represented in a linear, interpretable format. Specifically, we show that BERT's ability to conjugate verbs relies on a linear encoding of subject number that can be manipulated with predictable effects on conjugation accuracy. This encoding is found in the subject position at the first layer and the verb position at the last layer, but is distributed across positions at middle layers, particularly when there are multiple cues to subject number.

AB - Deep architectures such as Transformers are sometimes criticized for having uninterpretable “black-box” representations. We use causal intervention analysis to show that, in fact, some linguistic features are represented in a linear, interpretable format. Specifically, we show that BERT's ability to conjugate verbs relies on a linear encoding of subject number that can be manipulated with predictable effects on conjugation accuracy. This encoding is found in the subject position at the first layer and the verb position at the last layer, but is distributed across positions at middle layers, particularly when there are multiple cues to subject number.

UR - http://www.scopus.com/inward/record.url?scp=85183303460&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85183303460&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85183303460

T3 - Findings of the Association for Computational Linguistics: EMNLP 2023

SP - 4531

EP - 4539

BT - Findings of the Association for Computational Linguistics

PB - Association for Computational Linguistics (ACL)

T2 - 2023 Findings of the Association for Computational Linguistics: EMNLP 2023

Y2 - 6 December 2023 through 10 December 2023

ER -

Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Number

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this