YC is wrong about LLMs for chip design

> If Gary Tan and YC believe that LLMs will be able to design chips 100x better than humans currently can, they’re significantly underestimating the difficulty of chip design, and the expertise of chip designers. While LLMs are capable of writing functional Verilog sometimes, their performance is still subhuman. [...] LLMs primarily pump out mediocre Verilog code.

What is the quality of Verilog code output by humans? Is it good enough so that a complex AI chip can be created? Or does the human need to use tools in order to generate this code?

I've got the feeling that LLMs will be capable of doing everything a human can do, in terms of thinking. There shouldn't be an expectation that an LLM is able to do everything, which in this context would be thinking about the chip and creating the final files in a single pass and without external help. And with external help I don't mean us humans, but tools which are specialized and also generate some additional data (like embeddings) which the LLM (or another LLM) can use in the next pass to evaluate the design. And if we humans have spent enough time in creating these additional tools, there will come a time when LLMs will also be able to create improved versions of them.

I mean, when I once randomly checked the content of a file in The Pile, I found an Craigslist "ad" for an escort offering her services. No chip-generating AI does need to have this in its parameters in order to do its job. So there is a lot of room for improvement and this improvement will come over time. Such an LLM doesn't need to know that much about humans.

LLMs only reach the performance they do because of the sheer scale of data they ingest. Training them on less data doesn't work as well, or at least you will overfit like crazy on anything the size of current models. So the question is where are you going to get anywhere near the volume of verilog code as is present in The Pile? The total amount of verilog ever written is almost certainly a few orders of magnitude less.