YC is wrong about LLMs for chip design

I worked on the Qualcomm DSP architecture team for a year, so I have a little experience with this area but not a ton.

The author here is missing a few important things about chip design. Most of the time spent and work done is not writing high performance Verilog. Designers spent a huge amount of time answering questions, writing documentation, copying around boiler plate, reading obscure manuals and diagrams, etc. LLMs can already help with all of those things.

I believe that LLMs in their current state could help design teams move at least twice as fast, and better tools could probably change that number to 4x or 10x even with no improvement in the intelligence of models. Most of the benefit would come from allowing designers to run more experiments and try more things, to get feedback on design choices faster, to spend less time documenting and communicating, and spend less time reading poorly written documentation.

Author here -- I don't disagree! I actually noted this in the article:

> Well, it turns out that LLMs are also pretty valuable when it comes to chips for lucrative markets -- but they won’t be doing most of the design work. LLM copilots for Verilog are, at best, mediocre. But leveraging an LLM to write small snippets of simple code can still save engineers time, and ultimately save their employers money.

I think designers getting 2x faster is probably optimistic, but I also could be wrong about that! Most of my chip design experience has been at smaller companies, with good documentation, where I've been focused on datapath architecture & design, so maybe I'm underestimating how much boilerplate the average engineer deals with.

Regardless, I don't think LLMs will be designing high-performance datapath or networking Verilog anytime soon.

Thanks for the reply!

At large companies with many designers, a lot of time is spent coordinating and planning. LLMs can already help with that.

As far as design/copilot goes, I think there are reasons to be much more optimistic. Existing models haven't seen much Verilog. With better training data it's reasonable to expect that they will improve to perform at least as well on Verilog as they do on python. But even if there is a 10% chance it's reasonable for VCs to invest in these companies.

I’m actually curious if there even is a large enough corpus of Verilog out there. I have noticed that even tools like Copilot tend to perform poorly when working with DSLs that are majority open source code (on GitHub no less!) where the practical application is niche. To put this in other terms, Copilot appears to _specialize_ on languages, libraries and design patterns that have wide adoption, but does not appear to be able to _generalize_ well to previously unseen or rarely seen languages, libraries, or design patterns.

Anyway that’s largely anecdata/sample size of 1, and it could very well be a case of me holding the tool wrong, but that’s what I observed.

> With better training data it's reasonable to expect that they will improve to perform at least as well on Verilog as they do on python.

There simply isn't enough of that code in existence.

Writing Verilog code is about mapping the constructs onto your theory of mind about the underlying hardware. If that were easy, so many engineers wouldn't have so much trouble writing Verilog code that doesn't have faults. You can't write Verilog code just by pasting together Stack Overflow snippets.

Look at the confusion that happens when programmers take their "for-loop" understanding into the world of GPU shaders or HDLs (hardware description languages) where "for-loops" map to hardware and suddenly are both finite and fixed. LLMs exhibit the exact same confusion--only worse.