OpenROAD-Assistant: An Open-Source Large Language Model for Physical Design Tasks

Utsav Sharma, Bing Yue Wu, Sai Rahul Dhanvi Kankipati, Vidya A. Chhabria, Austin Rovinski

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Large language models (LLMs) have shown significant potential in serving as domain-specific chatbots. Recently, these models have emerged as powerful tools for chip design, providing both natural language responses and script generation for domain-specific inquiries. Previous work has demonstrated the effectiveness of LLMs in assisting with physical design automation; however, these approaches often rely on proprietary tools, APIs, technologies, and designs. As a result, access to these models is extremely limited, particularly for new chip designers who could greatly benefit from a design assistant. This paper introduces OpenROAD-Assistant, an open-source chatbot for OpenROAD that relies only on public data and responds to queries in either prose or Python script using the OpenROAD APIs. OpenROAD-Assistant leverages the Llama3-8B foundation model and employs retrieval-aware fine-tuning (RAFT) to respond to physical design-specific questions for OpenROAD. Notably, OpenROAD-Assistant outperforms other foundational models such as ChatGPT3.5, ChatGPT4, Code Llama, Claude3, and other ablation study baselines on the measured metrics (pass@k for scripting and BERTScore/BARTScore for question-answering). OpenROAD-Assistant achieves a 77% pass@1 score, 80% pass@3 score for scripting, and it achieves a 98% BERTScore and 96% BARTScore on question-answering.

Original languageEnglish (US)
Title of host publicationMLCAD 2024 - Proceedings of the 2024 ACM/IEEE International Symposium on Machine Learning for CAD
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9798400706998
DOIs
StatePublished - Sep 9 2024
Event6th ACM/IEEE International Symposium on Machine Learning for CAD, MLCAD 2024 - Snowbird, United States
Duration: Sep 9 2024Sep 11 2024

Publication series

NameMLCAD 2024 - Proceedings of the 2024 ACM/IEEE International Symposium on Machine Learning for CAD

Conference

Conference6th ACM/IEEE International Symposium on Machine Learning for CAD, MLCAD 2024
Country/TerritoryUnited States
CitySnowbird
Period9/9/249/11/24

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Graphics and Computer-Aided Design
  • Control and Optimization
  • Modeling and Simulation

Fingerprint

Dive into the research topics of 'OpenROAD-Assistant: An Open-Source Large Language Model for Physical Design Tasks'. Together they form a unique fingerprint.

Cite this