TY - GEN
T1 - Evaluating LLMs for Hardware Design and Test
AU - Blocklove, Jason
AU - Garg, Siddharth
AU - Karri, Ramesh
AU - Pearce, Hammond
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Large Language Models (LLMs) have demonstrated capabilities for producing code in Hardware Description Languages (HDLs). However, most of the focus remains on their abilities to write functional code, not test code. The hardware design process consists of both design and test, and so eschewing validation and verification leaves considerable potential benefit unexplored, given that a design and test framework may allow for progress towards full automation of the digital design pipeline. In this work, we perform one of the first studies exploring how a LLM can both design and test hardware modules from provided specifications. Using a suite of 8 representative benchmarks, we examined the capabilities and limitations of the state-of-the-art conversational LLMs when producing Verilog for functional and verification purposes. We taped out the benchmarks on a Skywater 130nm shuttle and received the functional chip.
AB - Large Language Models (LLMs) have demonstrated capabilities for producing code in Hardware Description Languages (HDLs). However, most of the focus remains on their abilities to write functional code, not test code. The hardware design process consists of both design and test, and so eschewing validation and verification leaves considerable potential benefit unexplored, given that a design and test framework may allow for progress towards full automation of the digital design pipeline. In this work, we perform one of the first studies exploring how a LLM can both design and test hardware modules from provided specifications. Using a suite of 8 representative benchmarks, we examined the capabilities and limitations of the state-of-the-art conversational LLMs when producing Verilog for functional and verification purposes. We taped out the benchmarks on a Skywater 130nm shuttle and received the functional chip.
KW - CAD
KW - Hardware Design and Verification
KW - LLM
UR - http://www.scopus.com/inward/record.url?scp=85202181762&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85202181762&partnerID=8YFLogxK
U2 - 10.1109/LAD62341.2024.10691811
DO - 10.1109/LAD62341.2024.10691811
M3 - Conference contribution
AN - SCOPUS:85202181762
T3 - 2024 IEEE LLM Aided Design Workshop, LAD 2024
BT - 2024 IEEE LLM Aided Design Workshop, LAD 2024
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2024 IEEE International LLM-Aided Design Workshop, LAD 2024
Y2 - 28 June 2024 through 29 June 2024
ER -