Evaluating LLMs for Hardware Design and Test

Jason Blocklove, Siddharth Garg, Ramesh Karri, Hammond Pearce

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Large Language Models (LLMs) have demonstrated capabilities for producing code in Hardware Description Languages (HDLs). However, most of the focus remains on their abilities to write functional code, not test code. The hardware design process consists of both design and test, and so eschewing validation and verification leaves considerable potential benefit unexplored, given that a design and test framework may allow for progress towards full automation of the digital design pipeline. In this work, we perform one of the first studies exploring how a LLM can both design and test hardware modules from provided specifications. Using a suite of 8 representative benchmarks, we examined the capabilities and limitations of the state-of-the-art conversational LLMs when producing Verilog for functional and verification purposes. We taped out the benchmarks on a Skywater 130nm shuttle and received the functional chip.

Original languageEnglish (US)
Title of host publication2024 IEEE LLM Aided Design Workshop, LAD 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798350376081
DOIs
StatePublished - 2024
Event2024 IEEE International LLM-Aided Design Workshop, LAD 2024 - San Jose, United States
Duration: Jun 28 2024Jun 29 2024

Publication series

Name2024 IEEE LLM Aided Design Workshop, LAD 2024

Conference

Conference2024 IEEE International LLM-Aided Design Workshop, LAD 2024
Country/TerritoryUnited States
CitySan Jose
Period6/28/246/29/24

Keywords

  • CAD
  • Hardware Design and Verification
  • LLM

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Graphics and Computer-Aided Design
  • Computer Science Applications
  • Software
  • Control and Optimization

Fingerprint

Dive into the research topics of 'Evaluating LLMs for Hardware Design and Test'. Together they form a unique fingerprint.

Cite this