Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants

Gustavo Sandoval; Hammond Pearce; Teo Nys; Ramesh Karri; Siddharth Garg; Brendan Dolan-Gavitt

Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants

Gustavo Sandoval, Hammond Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, Brendan Dolan-Gavitt

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Large Language Models (LLMs) such as OpenAI Codex are increasingly being used as AI-based coding assistants. Understanding the impact of these tools on developers’ code is paramount, especially as recent work showed that LLMs may suggest cybersecurity vulnerabilities. We conduct a security-driven user study (N=58) to assess code written by student programmers when assisted by LLMs. Given the potential severity of low-level bugs as well as their relative frequency in real-world projects, we tasked participants with implementing a singly-linked ‘shopping list’ structure in C. Our results indicate that the security impact in this setting (low-level C with pointer and array manipulations) is small: AI-assisted users produce critical security bugs at a rate no greater than 10% more than the control, indicating the use of LLMs does not introduce new security risks.

Original language	English (US)
Title of host publication	32nd USENIX Security Symposium, USENIX Security 2023
Publisher	USENIX Association
Pages	2205-2222
Number of pages	18
ISBN (Electronic)	9781713879497
State	Published - 2023
Event	32nd USENIX Security Symposium, USENIX Security 2023 - Anaheim, United States Duration: Aug 9 2023 → Aug 11 2023

Publication series

Name	32nd USENIX Security Symposium, USENIX Security 2023
Volume	3

Conference

Conference	32nd USENIX Security Symposium, USENIX Security 2023
Country/Territory	United States
City	Anaheim
Period	8/9/23 → 8/11/23

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems
Safety, Risk, Reliability and Quality

Cite this

Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants. / Sandoval, Gustavo; Pearce, Hammond; Nys, Teo et al.
32nd USENIX Security Symposium, USENIX Security 2023. USENIX Association, 2023. p. 2205-2222 (32nd USENIX Security Symposium, USENIX Security 2023; Vol. 3).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Sandoval, G, Pearce, H, Nys, T, Karri, R , Garg, S & Dolan-Gavitt, B 2023, Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants. in 32nd USENIX Security Symposium, USENIX Security 2023. 32nd USENIX Security Symposium, USENIX Security 2023, vol. 3, USENIX Association, pp. 2205-2222, 32nd USENIX Security Symposium, USENIX Security 2023, Anaheim, United States, 8/9/23.

@inproceedings{acb6fb36a9aa4be2a704fe0e0599b8dd,

title = "Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants",

abstract = "Large Language Models (LLMs) such as OpenAI Codex are increasingly being used as AI-based coding assistants. Understanding the impact of these tools on developers{\textquoteright} code is paramount, especially as recent work showed that LLMs may suggest cybersecurity vulnerabilities. We conduct a security-driven user study (N=58) to assess code written by student programmers when assisted by LLMs. Given the potential severity of low-level bugs as well as their relative frequency in real-world projects, we tasked participants with implementing a singly-linked {\textquoteleft}shopping list{\textquoteright} structure in C. Our results indicate that the security impact in this setting (low-level C with pointer and array manipulations) is small: AI-assisted users produce critical security bugs at a rate no greater than 10% more than the control, indicating the use of LLMs does not introduce new security risks.",

author = "Gustavo Sandoval and Hammond Pearce and Teo Nys and Ramesh Karri and Siddharth Garg and Brendan Dolan-Gavitt",

year = "2023",

language = "English (US)",

series = "32nd USENIX Security Symposium, USENIX Security 2023",

publisher = "USENIX Association",

pages = "2205--2222",

booktitle = "32nd USENIX Security Symposium, USENIX Security 2023",

}

TY - GEN

T1 - Lost at C

T2 - 32nd USENIX Security Symposium, USENIX Security 2023

AU - Sandoval, Gustavo

AU - Pearce, Hammond

AU - Nys, Teo

AU - Karri, Ramesh

AU - Garg, Siddharth

AU - Dolan-Gavitt, Brendan

PY - 2023

Y1 - 2023

N2 - Large Language Models (LLMs) such as OpenAI Codex are increasingly being used as AI-based coding assistants. Understanding the impact of these tools on developers’ code is paramount, especially as recent work showed that LLMs may suggest cybersecurity vulnerabilities. We conduct a security-driven user study (N=58) to assess code written by student programmers when assisted by LLMs. Given the potential severity of low-level bugs as well as their relative frequency in real-world projects, we tasked participants with implementing a singly-linked ‘shopping list’ structure in C. Our results indicate that the security impact in this setting (low-level C with pointer and array manipulations) is small: AI-assisted users produce critical security bugs at a rate no greater than 10% more than the control, indicating the use of LLMs does not introduce new security risks.

AB - Large Language Models (LLMs) such as OpenAI Codex are increasingly being used as AI-based coding assistants. Understanding the impact of these tools on developers’ code is paramount, especially as recent work showed that LLMs may suggest cybersecurity vulnerabilities. We conduct a security-driven user study (N=58) to assess code written by student programmers when assisted by LLMs. Given the potential severity of low-level bugs as well as their relative frequency in real-world projects, we tasked participants with implementing a singly-linked ‘shopping list’ structure in C. Our results indicate that the security impact in this setting (low-level C with pointer and array manipulations) is small: AI-assisted users produce critical security bugs at a rate no greater than 10% more than the control, indicating the use of LLMs does not introduce new security risks.

UR - http://www.scopus.com/inward/record.url?scp=85164468296&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85164468296&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85164468296

T3 - 32nd USENIX Security Symposium, USENIX Security 2023

SP - 2205

EP - 2222

BT - 32nd USENIX Security Symposium, USENIX Security 2023

PB - USENIX Association

Y2 - 9 August 2023 through 11 August 2023

ER -

Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this