PDAGENT-BENCH: Characterizing, Grounding, and Architecting LLM/VLM Agents for VLSI Physical Design

TL;DR

PDAGENT-BENCH is the first comprehensive, multi-dimensional benchmark for evaluating LLM/VLM-based agents across the VLSI physical design stack. Frontier models handle the concepts well but stumble badly when they have to actually drive EDA tools.

Key contributions:

• 353 curated problems spanning the physical design stack, combining conceptual questions with real-world industrial artifacts, each with expert-validated references and executable solutions.
• A systematic evaluation of 11 frontier LLMs/VLMs, revealing a substantial gap between conceptual understanding and tool-grounded execution — e.g. only 42.2% on Innovus script generation.
• An agentic workflow framework that improves performance through human-skill integration.
• Open-source release with process design kits and EDA tools for standardized, reproducible evaluation.

BibTeX

@misc{li2026pdagentbench,
  title={PDAGENT-BENCH: Characterizing, Grounding, and Architecting LLM/VLM Agents for VLSI Physical Design},
  author={Qiufeng Li and Rongqian Chen and Quan Cheng and Chengxuan Wang and Sizhe Tang and Chia-Tung Ho and David Z. Pan and Tian Lan and Weidong Cao},
  year={2026},
  eprint={2606.17253},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2606.17253},
}